LaunchPod - Parul Goel
===

jeff part 1 of paruls video: [00:00:00] All right, Parul, welcome to the show. Thanks for joining today.

4 minutes of parul pt 1: Thank you, Jeff. I'm very excited to be here

jeff part 1 of paruls video: I don't know quite where to start here. You've had a great career spanning,, most recently senior director of product over at Indeed.

You were at PayPal for, for like almost nine years. The real thing I'm excited to do though is talk here about the PM operating system that you and a bunch of collaborators actually built, and, and we're gonna do a first for, uh,, Launch Pod. We're going to do a screen share and actually have you demo it and show it.

, But before that, maybe just give, , the audience the TLDR, like how'd you get here, you know, background, all that kind of fun stuff, just to, you know, , let people know who you are.

4 minutes of parul pt 1: Also about 20 years in product, and as you mentioned, I spent many years at PayPal. I Was at PayPal during a very interesting time when we were splitting from eBay, which was a huge loss, but it also cracked open new possibilities. And I was one of the fortunate ones, who got to capitalize on that opportunity.

We built a zero-to-one payments platform for marketplaces, so a lot of complex problems to solve, [00:01:00] like for compliance, for scale at risk, multi-party payments. But we also got some very impressive clients under our belt, like Facebook Marketplace, uh, AliExpress. So, so that was a very fulfilling, , experience.

And then I moved to Indeed, where I was leading the payments and, uh, billing platform. The challenge at Indeed was different. It was more scaling. Our billing platform was about to, , crash under the, the volume pressure. But the, the part that I enjoyed more was really a cultural transformation, taking my team from being order takers to real owners.

was a more interesting challenge for me, and that's where I really grew as a leader. , And then, you know, when AI showed up, I was like everybody else. I would read every LinkedIn post. I would read every article obsessively. And then I realized it's not giving me conviction, it's giving me anxiety.

It's giving me FOMO. So that's about a year ago, I decided that I am going to lean into the builder [00:02:00] persona. And so almost every day I spend some time building, and I built the, the product manager OS that we will talk about, a few other tools for myself. But my main takeaway is this, that the only way to really feel what AI can do, where your job would stay versus what parts would disappear, is to have hands-on experience.

That's my main takeaway.

Jeff: Awesome background. Clearly you have experienced, , product at, at fairly successful orgs across a couple companies . this AI thing has been coming in and really, I mean, I agree, building is kind of the only way you can learn it.

But you and, and a group of your colleagues took it to the next level and found, you know, a set of problems that you shared, and you all actually built a PM operating system, if you will, that solved a bunch of your problems.

So let's just jump on in. Can you give us a little context on like, what was the background?

Why'd you build this thing? And then, then we can get to demoing.

Parul: Sure. So I, I am really [00:03:00] excited to show off our PM OS. I built this with the three other product leaders, Bhagya, Vidya, and, , Priyanka, who also happen to be excellent builders. , And I learned a lot from them during this process. , they proposed this idea of building a PM OS, and the reason I was motivated to participate was, you know, as, as I have through my product career, I started as a product manager, then I was a senior product director.

I have realized there is a lot of grunt work involved that actually takes away from your high leverage tasks, right? And what I was doing in, in my last role was just doing the same status update for different audience. And I realized that I am just doing the thinking once. A lot of it is just re-projecting, changing the tone and the length and just going and talking about what they care about.

, And once I framed it that way, I was like, "Maybe I could build my way out of this." So , when my co-builders proposed a PM OS, I was like, "Yes, I want to do an exec update skill." so that, that's how it started. [00:04:00] we built it, the first draft over a weekend. We have tweaked it over time. It's built using Claude Code.

I use it via terminal, and what it does is we picked four of the core PM functions, and we automated them to a large extent. So exec update, a cross-functional update, synthesizing customer interviews, and PRD

Jeff: Nice. , You're gonna show us, , the one you made, right?

Parul: Yes. I'm going to show the exec update, but I would highly recommend check out the other skills, especially PRD draft, I think is my favorite one.

Jeff: And is this open sourced and online or just available for you, for you four?

Parul: No, it's open sourced. , Maybe we can include the link for my GitHub.

Jeff: So we're gonna start screen sharing, so if you're listening on sound, jump over to YouTube, , to see the video, or it's on our Substack. , But the link to the OS itself and the open source code so you can use it yourself is also, , just down below.

So check it out

Parul: So before I show you the system, I wanted to quickly show you the architecture. It has a [00:05:00] four-layered architecture, and in my opinion, this is kind of the soul of the system. So it has a perception layer, which is what the system knows. Before it generates anything, it goes through all the context we have given it about the company, people, how we talk.

, Execution, what it does, the four skills I talked about. Critique, before I see any of its output, it evaluates it on its own. For example, from the perspective of a designer, of a perspective of an engineer. So these are your coworkers weighing in before you send something out, and then continuity, what it remembers.

, I do want to quickly show you the context layer because I feel like that's the one people usually miss, and then the, output ends up sounding generic. this is my AI PM OS demo folder. I simplified it for the purpose of the demo. It's basically a collection of markdown files. Context library is where... Like, I think this is the most important layer. This is where my context lives. We had made up a fictional company, so it has information about the company, like the [00:06:00] business model, target clients, some decisions that have been made in the past, stakeholders who, you know, who are the people we interact with.

This is an interesting one because it's beyond just the name and role. It's what do they care about? What are their pet peeves? Things you actually learn about people as you work for them, things you actually think about before sending them something. And then there is also users, the main customer personas, and then the voice, which is kind of the culture of the company codified.

Jeff: so you, you took time before you kind of were able to really output anything from this to document this stuff and, and go through and, and kind of write out or I assume work with Claude to kind of write some of these out and build , these context docs.

Parul: in our case, it was one of my co-builders, Bhagya, who did this, who built the context library. But yes, I would say if you're building something like this, do not skip it. This is the most important So let me now show it to you. I will go to my-- the right folder And then I am going to start Claude 

Jeff: And I feel like terminal is one of those that a lot of [00:07:00] people, shy away from because you just hear terminal , I'm not an engineer, I don't wanna touch that. , I, I've built quite a few things, and I, I don't have a comp sci background. I've been able to operate with Cloud Code without going in the terminal. It just, it, it will just do the terminal stuff for you. , So kind of either way, whatever is nicer for you, I, I feel like can work really well.

Parul: , I completely agree. Don't let the terminal intimidate you. It just a few basic commands that you might have to learn, but other than that, Claude does everything for you. So I'm going to sh- to demo the exec update skill.

So I made up a fictional initiative, Fast Track.

The goal is to improve conversion, went live three weeks ago. There's some movement, but there is an engineering risk, This is really the, the thinking that I am doing, where I am applying my judgment. I have done enough of these at this point, enough of these status updates to know what matters.

Okay. So I have my raw notes, and I have told the system, "This is for Sandra Lee," who's the CEO. So I am going to call my skill and paste my [00:08:00] notes here, and then let's see what it does. Now it's asking me a few questions. This is the skill executing to the standard that I have defined, and it's telling me, "You have not given me enough information to reach that bar."

So I missed the color. So I'll say yellow because of end risk. And then I've also told it that don't be wishy-washy about the timelines. So it's asking me what does end of week mean? So I'll just give it a... Just August 11th see. So now it's again going back. And, you know, when you are in the process of working something, the last thing you want is to get into a back and forth with an AI assistant.

So we limited this. We said, "Okay, you can ask questions only once, and then you have to draft your status update with whatever information you have." You can still point out gaps, but no more, back and forth.

Jeff: And all the notes you gave it were that kind of like three lines that we saw right there. Basically, conversion increase was this, we don't really have enough [00:09:00] time to gauge it, and there's a little bit of an engineering risk.

Parul: Yeah. So now it has come up with the status, again, based on the structure I gave it.

So the name of the initiative, what it is, status, bottom line. Now, if Sandra stops reading at the bottom line, after bottom line, she has enough context. But if she has time, she can go through highlights, decisions, top risk. , You might say that this is not how I do status updates, or this is not how we do it.

The-- It's too long, the, the tone is wrong. But that's the whole point. You can change all of it to fit your context. It's all in your hands. N-now, here is where it gets interesting. So as I mentioned, it critiques its own work. In this case, it checked with-- I have defined three sub-agents, one from perspective of an engineer, a designer, and an exec to go through my status update.

If I were to send it to them, what would they say? So it's actually flagging to me that engineering is saying that they have missed this date twice. And I am telling the exec, I'm telling Sandra that we [00:10:00] have a high confidence in the August eleventh date, but I'm not saying why. So that might come across as overstating confidence.

But the other designer and exec had no, no concerns. , And that's a good point. It's actually flagging questions that I might get if I send this out. So I will correct it. I'll say theee, the data was in an unusable format that has been fixed.

I know, I think it will redo it

Jeff: And there you go. So there you

Parul: go. It listens to me. It gave me the updated one. okay, so I think it gives me another one, but I'm going to ignore it. I want to show you one more thing, which was the-- my whole reason of doing this. so I, I generated the status for Sandra, who's the CEO. Now, I will take the whole thing, what it gave me, so it doesn't ask me for the same information again, and I am going to ask it to do the same thing for a different, , stakeholder.

So I'll paste the same status that it gave me, and then I will [00:11:00] say for Kevin Park, I think who is the CTO co-founder. So a more technical persona. So let's see. Oh, no. Let's see. It said API error, but it went away

Jeff: and so, Kevin, it knows who Kevin is because that's in the context layer

Parul: It's in the context. In fact, and hopefully it will show us, sometimes it does. It updates the, the tone. It doesn't just relabel it, it updates the tone and it tells you why it did it.

Oh, in this case, in this case, it did not do it. But Kevin is the CTO, he cares about... Okay, let's see. Fulfill awareness, needed decision asks. Kevin Park, engineer capacity. Okay, so yeah, see it's, drawing who Kevin Park is. So what is he protective of engineering bandwidth, stakeholder. So In this case, for Sandra, this was mostly an FYI, and because, uh, Kevin is the CTO, it has actually asked him for help

in, uh, in meeting the August 11th date, right? And then I can go and leave. It's not right, I can go, I can go correct it. But it [00:12:00] actually draws from the context and tries to re-project my status in the right tone, what should be the ask, et cetera.

Jeff: And it's kinda, it's kinda blowing up Jake's spot right there, uh, in decisions and asks. This is the third estimate, and Jake has carried it alone. Do you want a second

Parul: Engineer duality. 

Jeff: I'm hope, I'm hoping that these are fake people at this point, 'cause Jake's getting his spot blown up real bad.

but it's a good ask. Like, that's a real,

that's a real

thing I would be asking, you know, our, our head of engineering here I hope there's not a real Jake going 

Parul: yeah. This is, this, this is all fake still. Also, you know, the, the reason why I thought, uh, you know, there are many reasons, but one of the reasons why I thought this was a good problem to solve via AI is it's low risk. I am always going to check before I send something out. It's never going to automatically sent out, right?

So something like, like sometimes if it doesn't get the tone right, then I can fix it and I can go to my markdown file and also change the rules that I have given it,

Jeff: So that's the use case you built. Can you give us just maybe the outline of, like, what are the other, , ones that the other folks 

Parul: Yeah. So there is one, , [00:13:00] synthesizing customer interviews. So if you have transcripts from an interview, you just feed it. It tells you the insights, it tells you the , key, quotes that you can pull from it. And actually, the next one is PRD draft. You give it a one-line idea, and it-- like I said, this is probably my favorite one.

It also gives you a prototype. It actually draws from user interviews, to draft the PRD. A cross-functional update, so not for an exec, for a working team to a broader audience, and then the ex-exec update. were the four things we built.

Jeff: that's actually great 'cause those things Work off of 

each other. 

so you can kinda see, you know, the exec update going out where have the PRD of what was supposed to be done, the original user info, you see where this all kinda blends together.

So, this is where I feel like the thing I ran into, and I, I don't know about you, is it turns out actually building code to execute just like go do this, go do that with, with AI now is not very hard. Getting it to do it right and do it well, That's the really, really hard part. can you talk to us a bit about how did you actually build that context layer?

Like, what was the [00:14:00] process to ensure that that was right and you documented the right things, that it used it correctly? 

Parul: for context layer, it was a little bit of trial and error, the main thing that we focused on as we shape product decisions, we think about product decisions, we think about communication, like what are the things that we consciously and subconsciously think about, so that was one criteria, like it is relevant to, product decisions, communication. And number two was that it doesn't change on a day-to-day basis, because context layer is commitment, right? If you put something in there, you have to keep it updated. So that was really the criteria that we applied in defining this context layer.

And then, you know, as we used it, we went and tweaked it a little bit here and there.

Jeff: and when you're kind of process-wise putting together the, you know, stakeholders set and all that, did you actually go in and, you're physically writing the markdown? Or is it more conversation with Claude saying, "Let's talk about who the stakeholders are and I want to describe them.

Let's talk about who the customers are. I want to describe them," 

Parul: Yeah, it's the, it's the [00:15:00] latter. So for m- most of these, it was conversations with Claude, , where we brainstormed and then came up with, okay, what we should include making of these fictional stakeholders.

Jeff: Obviously, there's kind of only two ways it goes. Either you mostly write it yourself or you mostly work through Claude to write it. And I feel like I see a lot more problems when you try and physically write it yourself versus if you work through Claude, it's gonna write it the way it wants it.

It's faster, I tend to go through and I'll read it afterwards. 

I think that's an important part lot of this stuff. You can just dive in and, and start to ask, "This is what I wanna set up. What's the best way for you to understand

it?" 

Parul: It's so interesting you say that because when I first started building AI tools, what I would do is e-either chat with ChatGPT or Claude and then come and copy-paste the spec into Claude code. And at one point, I was like, "What is the skill that I am using?"

Because anyone can do it. Like it's basically, it's just telling me what to do, and then I copy-paste. But as I start using those products, I realize that yes, the engineering part of it, the build part of it [00:16:00] has gotten easier, but the judgment part has not. one Time I was using, I think it was Claude generate an email reply, and it told this person, the meeting was more useful than I had anticipated, And thankfully, I did not send it. But like that's where the judgment is so off. That gave me a lot of comfort that my job is not going anywhere soon if that's what, what it wants to put in the reply.

But, I think this is where building hands-on is important because it shows you where AI is really powerful, but it also shows you where it falls short, where you will have to continue making decisions.

Jeff: Right. And, and how do you kind of iterate on that?, The things I've seen work the best, if you just wanna go in and do it real quick and then have it push output, it's probably gonna be garbage.

And where I've found it gets really powerful is I'll sit there and I will obsessively go through results and look through and go, "Is this right? Is this wrong? Why don't I like this?

It's not, "This was wrong." It's always going back and saying, " why did you get this [00:17:00] answer? Like, what led you to this? How do we prevent that from ever happening in the future?" , How much of that did you find here? Like, Was this kind of a one weekend build and good, or, or did you find it kind of took iteration over time?

Parul: Oh, it took a lot of iteration over time. , the first version is almost never good. in fact, I would say that after you launch something, the first few months should still be considered development because you use it, it falls short, you go tweak it, and then you try again.

Even with this, we built it over a weekend, and then we were testing it for several days. And I realized that when I had built it, I had over-indexed on a large in-flight initiative because that was what was top of mind But then when I tried an OKR update on an early-stage initiative, it was way off.

So I had to continue making these small tweaks that then eventually then added up, and you could see a huge improvement. But I do say that you have to build something you use every day because that's how you discover these edge cases, not by just building it.

Jeff: It reminds me of, I I wanna say it was [00:18:00] OpenAI. , It's 50/50, right? It was Anthropic or OpenAI, I don't know which, but let's go with OpenAI. and, and rolling out one of the newer products over the past. This is, this was a H1 this year kind of thing, I

believe. but the story stuck with me, where basically they were building something and, and the engineering lead said, "You're not allowed to code at all.

You're not al- allowed to handwrite any code. You just have to, if it, if something leaks through, if there's a problem or an error, you have to build the test cases or the loops all those kind of things

Parul: The 

evals and 

yeah, 

Jeff: to, to stop it from happening in the future." and the, the way the story goes is basically it started out with progress really, really slow, but it then basically slowly grew, but then it was compounding growth, and it hockey sticked up.

, And, and everything got much faster after that because it knew how to do that. And I've found in almost anything I've done where AI is evaluating data or that, it's the first step is you build it, and - it's gonna be way slower the first round because I have to obsessively go through every one of them.

But the second time or the third time or the fourth time, [00:19:00] until you finally realize, like, this is really good, and now I can just do this going forward like that.

but it brings up a question. , I'm curious to hear how you approach this. It's one thing if, you know, some of my use cases are, are evaluating SDR phone calls or something like that.

uh, is what it is. But for you, it's, it's, you know, in product, the project is changing. You might have a different set of stakeholders or different goal quarter to quarter. How did that context library stay up to date for you? Like, how did you hang on that where you were able to keep a context library, not just static going, but

Parul: Yeah.

Jeff: date over time?

Parul: . , there is a, a real tension there, because more context is better, but it does not come free. so like I said, our criteria was we did not want to, do daily maintenance of our context. So we made a conscious decision that we will only document things that are super important and do not change every day.

So, like the company identity, stakeholders, if you're lucky, and then users, they don't change on a day-to-day basis. because stale context is worse than missing context, right? Stale conte- context will mislead the system. [00:20:00] But the downside of this approach is that, yes, I mean, there are some lower level details or some maybe if you work with a large group, some of the names are probably not going to be in the stakeholder, document.

So yes, that does mean that you will probably have to revise , its output a little bit more. But that was the decision we made in our case.

Jeff: think this is a really interesting use case, and thank you for coming on and showing this. it Does kind of really point out , the gains you can make, but also where you need to be careful, right? Like, if you're writing updates for five different people, it needs to be five different kind of framings of the same data.

I don't think PMs, you know, good PMs don't really need to worry about their jobs. It shows, like this alone, what you showed, shows how important the judgment and what's actually important and, and picking the right direction to head in is, is not going away.

It's, it's even more important 'cause you can build anything.

Building the right thing's important. Looking forward,, what does this say to you about, you know, where PM is going and how you feel about, the role and the, the team kind of outlook?

Parul: Yeah, I, I completely agree with you. I don't think the product manager role [00:21:00] is going anywhere. , Even though product managers, product leaders spend a lot of their time on the grunt work, right? the value that they provide still comes from the judgment of, , what should we invest in?

Which customer problem should we prioritize? How do we solve that problem? What does the experience look like? And I think AI can maybe give you options, it can give you ideas, it can tell you the trade-offs. But I think the actual judgment or the actual decision still needs to live with the person.

because a lot of times these, these decisions are made based on the company values or an individual values, and AI is not going to be able to replace that. 

I do think product teams will get smaller. I think teams overall will get smaller. I think people will do more, a higher quality, higher leverage work.

A lot of this like, you know, writing the same status five times would go away. so the nature of work that we do would change, but I, I think product manager as a role would evolve, but it would still be there.

Jeff: well, , Parul, thank you so much for coming on, for [00:22:00] showing this, walking us through, showing us the context layer, walking through how you built it. you know, like I said, the link to the open source, pieces are, are below on the YouTube. They're on the Substack, so check them out. you know, we got links to the Collaborate as well, but you're also on LinkedIn, so I'm sure if people have questions, they're happy to ping you and, and pick your brain a little bit.

Parul: Absolutely. Find me on LinkedIn check out e- Exec Update and Other Skills, and if you have comments, I would love to hear them. So 

Jeff: Awesome. User feedback, right? it's always helpful. 

Parul: Yeah, Yeah, 

Jeff: Great. Well, have a good rest of your day, April. Thank you so much for coming on, and, uh, yeah, looking forward to chatting again soon hopefully

Parul: Thank you, Jeff