Logan Kilpatrick part 1 
===

~Test, test, test. Oh, I can fix that. I can fix that. Um, let's go. How about test, test, test?~

~Yeah, test, test, test.~

~Let's go.~ [00:00:00] Hello and welcome to Pod Rocket, the podcast brought to you by Log Rocket. Log Rocket helps software teams improve user experience with session replay, error tracking, and product analytics. Try it for free today@logrocket.com. ~Um,~ I'm Tej Kumar, a developer relations consultant, and ~uh,~ today I'm joined by,~ um,~ Logan Kilpatrick from OpenAI, ~um,~ Logan's,~ uh,~ developer advocate at OpenAI.

And today we're gonna dive into all things OpenAI, including G P D four, the Whisper, APIs, Lang chain, and so much more. ~Um,~ Logan, welcome.

Thank you for having me. This is gonna be a ton of fun.

~Yes, I, I fully agree. Um,~ just to get started, I was wondering if you could maybe introduce yourself and tell the listeners, ~um,~ about your role at OpenAI, ~um,~ how you got it, what you do on the daily and more.

~Yeah, sure.~ Happy to. So my name's Logan Gilpatrick, joined OpenAI back in December to help lead ~and,~ and build our developer relations function at OpenAI. ~Um,~ Before that was doing a bunch of other developer advocacy type work in various ecosystems. Was a machine learning engineer before. ~Um, ~so yeah, it was ~a, a,~ a lot of natural [00:01:00] fit to, to join OpenAI.

~And, um,~ originally when I joined, ~we,~ we really had ~the,~ the challenge of, ~uh,~ top of funnel awareness about what AI could actually, what our a p I was actually capable of. ~Um,~ So that was the historical context from me getting hired. And then my first day was actually, ~uh,~ chat gt hitting a million users. So we ~sort of ~didn't have ~that,~ that top of funnel awareness problem anymore because there are so many people who are now excited about what we were building.

So it, my, my roles really evolved into helping our product teams. ~Uh,~ better understand what developers want, ~um,~ how we can deliver on what the expectation is from developers so that people can continue to build their companies, their cool projects. ~Um, ~all that stuff on top of our A p I. ~Um,~ I also help do a bunch of Chatt p t stuff these days.

~So I,~ I help run our Chatt p t plugin store, have a team of folks who, who do all the reviews ~and,~ and policy and,~ uh,~ take down stuff related ~to,~ to running~ a,~ a store, ~um,~ as it stands today. So it's been a ton of fun, ~lots of,~ lots of different stuff. ~But yeah,~ my daily, ~my day, ~my day-to-day is very different, I think on any [00:02:00] given day, just because of how quickly we're.

Moving and shipping stuff and yeah, ~there's,~ there's always ~a,~ a fire to be put out.

~Yeah. Um, as a,~ as a dere professional and consultant myself, ~I,~ I've just been so curious about what this would look like at Open ai. 'cause as you mentioned, ~you,~ you went from zero I. ~Or, or,~ or a few to, to a million over a million users. So quickly, ~um, I, I was wondering, ~you mentioned, ~you know,~ is the work you do today is buffering, ~um,~ some of ~the,~ the feedback about the product ~and, and,~ and relegating that to the product teams.

I'm curious, ~um,~ if you could speak a little bit to how exactly you, that, I'm assuming the volume of feedback you get is tremendous. ~Um,~ and so there must be some type of process to filter a sort and then, ~um,~ delineate that.

~Yeah, it,~ it's a good question. So there's ~lots of like's,~ lots of feedback coming in about very different things, and I think that's actually the hardest piece of this problem is. ~Uh,~ not only like filtering from the different places that's coming in, but also understanding like what are the things that like we have direct control of ~and, ~and we can make a short-term impact on.

~So like~ there's people who are giving feedback about chat, G B T as like a product experience. There's people who are giving feedback about like [00:03:00] the models in chat, G B T and then it's the same thing for our a p I like the actual a p i models, people are sending feedback on. They're sending feedback on like the actual experience of using our a p I.

~Um,~ and I think it's like oftentimes the context in which someone is saying something about one of those four things is not clear. ~Like~ somebody will just say, Hey, this isn't working for me, ~uh,~ in X, Y, and Z use case. And then I spend a little bit of time trying to understand like, who is this person talking to?

~Like~ should it be coming to me? Should it be to some of these other teams? ~Um,~ so we don't have a super formal process. We do have. The ability for end users to send us feedback about models. I think it's like openai.com/model feedback or something like that. ~Um,~ but this is like generally ~a,~ a muscle that has to be built.

Chat BT has a little bit better because they have ~like~ built in UI flows that ~like~ enable this for the a p i. We don't have that. ~Um,~ so it really is like a lot more going and talking to developers, ~um,~ trying to help them build evals. ~Uh, ~and then ~sort of~ trickling that feedback up ~to,~ to various teams from a product experience [00:04:00] standpoint.

That, that's so interesting to hear that this function of devereux, of spending ~a ~a bunch of time understanding user feedback, ~um,~ and getting context is so universal. ~Um,~ be it at OpenAI with chat G p t, the models or, ~you know,~ at Twitch or Twilio or whatever, like ~this, this,~ this work is ~kind of~ the same. I love that.

~Um, I,~ I would be remiss if I didn't ask, ~um,~ on behalf of also the listeners, if. ~Um,~ you'll employ the use of AI ~in,~ in, in doing that and gathering context and understanding. ~Right.~ I ~mean,~ you would have to, ~um,~ use a model to classify feedback about a p i about models and then also add maybe missing context or something.

I.

Yeah, a hundred percent. So the, we do have a bunch of, ~um,~ Internal tools that help us do stuff like that. ~It's,~ it's actually very simple to ~like~ set models up to, to do this. ~Um,~ we also use some, like off the shelf products that have, that, those products actually also use our models to power those types of features.

So we have ~sort of~ both of those. But yeah, ~it's,~ it's definitely helpful. ~I,~ I do think that, You lose a bunch of the nuance when you ~sort of~ take people's perspective and ~like~ filter them through a bunch of models. ~Um,~ [00:05:00] so it is ~like,~ for me it's oftentimes more valuable to ~like,~ see the raw feedback than like process 20,000 pieces of feedback and then ~like,~ look at the aggregate overview because like I ~kind of,~ I, I generally have a sense of.

Like the 20,000 combined together opinion perspective, it's ~like~ a little bit easier to~ like ~generalize and understand that perspective. But I think it's often like the nuances that are, ~uh,~ that are really difficult to capture and those nuances are extremely important in the context of the way that our models work.

~So that's the,~ that's the challenging part is like everyone has like such a unique use case and they're, they want the model to be better for that use case, but it's oftentimes like hard to articulate how the. Yeah, how the model has changed for that use case,

~Right. So what I'm,~ what I'm hearing is, ~um,~ AI is not replacing the role of Dere work anytime.

it could definitely help. ~I,~ I do think that AI plus devel ~is,~ is gonna be like an exciting future, but, ~um,~ yeah, ~I,~ I think there's still a lot of human work ~that~ that has to happen.

~Fantastic. You know, I,~ I would be remiss if I didn't mention, ~um,~ and I think a lot of people listening [00:06:00] also have this image of ~like,~ OpenAI folks and even like Sam Altman's house being like this thing full of Jarvis systems. ~Like,~ hey, open the blinds, or, but it's~ like,~ it's a smart home, but on steroids, ~you know?~

~Um, so I'm glad,~ I'm glad ~you're,~ you're able to also emphasize that the human side of actually ~like.~ Doing stuff and understanding context ~is,~ is, ~um,~ pretty valuable still.~ Um, ~I.

~that~ that's actually the most compelling part about open AI is like the people, I think people over index right now and like the actual. Value add ~of,~ of having all of these models. Like the models are very helpful, but like having an incredible team of humans ~who~ who do this work ~is,~ is probably much more useful than~ I would bet,~ I would bet on that team than a team, A subpar team that's using models today.

~Fantastic. I, I'm,~ I'm really glad,~ um,~ you said that ~you,~ you mentioned in your previous answer,~ um,~ the use of eval and the community submitting evals, ~um,~ to OpenAI. I'm curious, ~um,~ especially for those listening, if you could clarify, ~um,~ an eval, what it is, how it works, and how OpenAI addresses, ~um,~ such submitted evals.

Yeah, so we, we have a, we released it a few months ago. ~Um,~ I think around the G P T four launch time. ~Um,~ but essentially the [00:07:00] general idea is we have this structured way of writing, ~um,~ prompt completion pairs. So you can essentially say ~like,~ here's some given input, here's the output that I would expect. ~Um,~ and then the model can essentially, ~uh,~ it will try to generate an output based on the.

Prompt inputs and then look at the delta between the provided output that you gave and the provided output that the model gave, and it'll score this. ~Um,~ and essentially over time you can, or not over time, but just like in the moment, you can run the eval and see ~like,~ here's what's the delta between what the model is outputting today and what you would actually want the model to output.

~Um, and there's,~ there's a bunch of other like types of. Evals that you can write. ~Um,~ but essentially like encompassing all of these different use cases where I have a very specific output that I want the model to create. ~Um,~ and being able to assess how good it is at actually creating that output on a consistent basis.

And you can add like hundreds and hundreds of different examples. And this is super useful. So you can imagine ~like, You know,~ you have some, [00:08:00] you're an e-commerce shop and like you have a very specific way that you want the model to output responses to users in some specific format, including like specific information.

~Um,~ and you can just put in like hundreds and hundreds of those examples, either from like your real world data or just like synthetically generated. ~Um,~ and then you can run that eval on like old models that we have on the newer models and you can see like how is. How are the models progressing over time on my specific use case?

~Um, and we run, uh,~ we run ~our, the,~ the public facing evals when we release new models to get a signal for, ~um,~ how yeah, how well the model is doing.

~Right.~ So part, this is about to be a new question. I apologize. ~Um,~ in advance, ~um,~ this sounds a little bit to me, like supervised learning, right? Where ~you,~ you ~kind of ~guide ~the,~ the machine learning process through, ~um,~ Feedback. Am I close here or what's the delta?

~Yeah, that, that is close. The,~ the caveat here is that we're not training off of the evals, so it's not improving the model performance. I would say that it's more akin to like in, ~uh,~ like a unit [00:09:00] test suite where like you're essentially giving, ~you know,~ here are the inputs and outputs that you would expect.

And then when that isn't the case, you essentially, like the model eval accuracy gets dinged, and then at the end of it you can say ~like,~ oh, this eval is like, 50% on G B T four and like maybe, ~uh,~ G B T 3.5. It's ~like~ 60% or something for some reason. So you can ~sort of~ see how well the model is doing on that, like unit test, if you will.

Why do y'all not then train on evals? Would that not give a cue to the model to ~like~ respond more this way?

~We,~ we could train on the models, the problem or on the evals. The problem would be that then we would over fit, so then all the evals would ~like ~look it, it would essentially be like, ~Uh,~ it would be like fake. Precision where it looks like, oh, we're really good at these models and like the reason we're so good is because we actually have that data directly in our sets.

Like you would actually want these intentionally to be like a standalone set of data that's not present. Like in a perfect world, you would actually ~like~ check the training data to make sure that it's not [00:10:00] contaminated with any of the data that's in the models. ~Um,~ like theoretically there could be contamination across that data, depending on what information people are putting in.

~Um,~ So yeah, you, in that case, you would get like false accuracy that the model is like really good at this use case.

That's fantastic. ~And, ~and thank you for clarifying that. 'cause my naive mind was just like, oh, you can just train with this, ~but,~ but ~you,~ you mentioned, ~um, ~exactly why that's a bad idea. ~Um,~ you also use ~this,~ this term overfitting. ~Um,~ and I know for the listeners of the podcast, they'd love to know more about that.

So if you could say a few sentences about Overfitting, why it's a problem and how it works, ~um,~ we'd appreciate that.

Yeah, a hundred percent. So you could, I, you could imagine, for example, somebody let's use, ~um,~ fine tuning as an example. So fine tuning is the process of taking like one of our models that exists today ~and,~ and making it, ~uh, you know,~ specialized for some specific use case. The problem is,

I interject right here. ~So,~ so an example of fine tuning would then be I take the base, ~um,~ stable diffusion model from ~like~ hugging face and then fine tune it using, ~you know,~ some photos of me. So I get these more specialized models that I could be like, make me Spider-Man swinging through Manhattan and it [00:11:00] would draw something.

Okay.

Yeah, exactly. ~And, and,~ and the challenge with overfitting is if you put in a bunch of pictures of you, ~um,~ if then the next user shows up who doesn't look like you and tries to use that same model, the problem ~is,~ is that you've given it all of this signal that it should be doing something. ~Um,~ and then that doesn't generalize well to~ like~ the next user.

~And,~ and this is real, super important when you think about these use cases and like the breadth of how our models are being used today. ~Like~ if we were to take a bunch of data from ~like,~ One specific group of people and, ~you know,~ flood the model training process with that data, ~um,~ you would then generally ~like~ not have a great outcome for people who like, aren't represented ~in that,~ in that use case group of people.

~Um, so yeah, it's,~ it's interesting ~and that,~ and that's as a broad perspective comment. ~Um, it,~ it's why like having. Data about like different groups of people all around the world is so important because you need to make sure that these groups ~are,~ are represented. And ~like~ if all of the training data is about, like from people in the United States, for example, like the [00:12:00] models are going to end up having this like very US-centric worldview ~of,~ of how they should be doing things and how they should be interacting.

And like you actually really want the model to have the context of ~like~ yeah. Different people's perspectives all over the world so that it, it doesn't. ~Uh,~ get pigeonholed ~into that,~ into that mindset.

~Right.~ So then I'm wondering if you're at Liberty, you speak at how, or rather speak to, excuse me, ~how um,~ How training does work on the models, if it's not through evals or if it's not through, ~um, you know,~ things that can be overfit into one specific domain. How you all, ~um,~ train such that it is more inclusive and diverse across the world.

Yeah, there's a, so~ I,~ I don't, I'm not deeply involved in the model training process by any means. ~Um,~ generally the way to augment ~and,~ and ~sort of~ help in situations where you might not have access to. ~Uh,~ diverse enough data is ~like~ to go and procure data so you can, ~like, you know,~ there's tons and tons of different companies ~and,~ and entities where you can either ~like~ buy pre-existing data sets or work with organizations, ~um,~ to actually like, have net new data created.

~And, um,~ yeah, having net new data created ~is,~ is something that [00:13:00] is, is super common. When I worked at Apple and we were, and I was a computer vision, ~uh,~ working on computer vision problems, we. Did the exact same thing where we had like a very specialized computer vision problem that we were trying to solve, which was taking an image and then make some decision based on that image.

~Um,~ and the data set that we needed essentially didn't exist in the world, so we had to go off and, ~um,~ pay people to like annotate hundreds and hundreds of thousands of these images so that the model could perform well. ~Um, and that's,~ that's generally what you need to do. Like the most interesting data, ~um, Is~ is probably like human created, manually curated data like that.

~All right. So,~ okay. That, that clarifies a lot of things. I, I wanna pivot a little bit and talk about G P T ~um,~ four. ~Uh, it's,~ it's this new model I've seen. I'm sure we've all seen~ the,~ the diagrams of, ~you know,~ G P T 3.5 has this many parameters. And then G P T four is this big circle next to it with ~like,~ The billions more.

~Um,~ and it's just these two circles that are just, ~you know,~ they look like the sun and the earth effectively. ~Um,~ I'm curious if you could speak to that, particularly clarifying to the listeners, ~um,~ parameters and their role. What do those even mean? What is even a parameter and what [00:14:00] makes G P T four so much bigger than g?

PT 3.5?

~Yeah. So a,~ a couple of caveats. One that image diagram that you're referring to, while exciting to look at, is not, ~um,~ I is not proportional to like the correct orders of magnitude. So I think Sam, ~our,~ our c e o mentioned before that ~it's, um,~ it's just not right. ~Like I,~ I think it's. It, it's not representative of the actual parameter count in, in general, I do think that ~like~ people overindex on the parameter count as like a simple mental model heuristic for the capabilities of the models.

~Um, I don't think that, that's~ my intuition is that maybe that you can do this indexing and using that heuristic today, but long term it's~ like~ not actually~ like~ fundamentally. The right ~like~ way of looking at it. Like just 'cause if some new model comes out that has a hundred trillion parameters, like it doesn't actually mean that it's going to be ~like~ bigger is not always better.

~Like~ it actually depends a lot on the data and the training process and the architecture and stuff like that. But generally parameters are just the number of, ~uh,~ of neurons and the network. I don't know if there's a, if there's a good way to simplify this, but if people have seen [00:15:00] visualizations of ~like the.~

~Um,~ how a model looks, and there's like different layers and there's these different parameters and the different layers, which essentially the idea is if the parameters activate in a certain sequence that represents ~like,~ essentially ~like~ the thinking process or like the ability to ~like~ answer a question essentially.

And so generally, ~you know,~ the more of these neurons you have, the more broader the problem set is of things that you could potentially answer. ~Um, That's my,~ that's my hopefully high level overview ~of,~ of why the parameters is people are talking about this. And generally, again, the idea is you have a bunch of parameters and therefore you can potentially answer more questions ~in a,~ in a deeper way.

~Right. It's a,~ it's a broader pool to be inspired from, so to speak. ~Um,~ great. ~Um,~ there's also talk now of, ~um,~ this new code interpreter feature ~in,~ in chat G B T, ~um,~ that people are calling G P T 4.5. In disguise. ~Um,~ I'm curious if you could speak to that at all, ~um, and, ~and either, ~you know,~ clarify, is it in fact G P 4.5 or are people just making stuff up like they did with the parameter size?

~Um,~ discrepancy.

~Yeah,~ [00:16:00] PE-people ~are definitely, um, like to,~ like to speculate, which ~is~ is always exciting, ~um, for,~ for people to speculate. Yeah. I think ~when,~ when G P T 4.5 ~is,~ is available, ~we'll, we'll,~ we'll release it to people and,~ uh, um, you know,~ after we can make sure~ it's,~ it's being released in such, in a safe way. ~Um, I,~ I do think that the excitement about code interpreters specifically, like highlights how.

Useful. These tools are specifically to like the developer persona. ~I, I think if you Yeah. The,~ the fact that the output for engineers ~and,~ and people in the software industry is text in a lot of cases, ~um,~ just ~like~ bodes so well with ~like~ the capabilities of these models. ~Um,~ if you think about ~like~ other roles where it's ~like~ much more.

~Not,~ not that engineers don't also have to do these things, but like your output is ~like~ tied to things that are just ~like~ a little bit, like less text-based. ~It's,~ it's just like much more difficult, ~like~ if you have to interact with people, ~like~ having a model that can like generate text, ~like~ doesn't help you a ton.

But I think for engineers, ~like,~ because the core deliverable is oftentimes ~like~ code that you type with your fingers, it's just so easy to get excited about this possibility [00:17:00] and the fact that the model can run. ~Uh, the,~ the environment of code interpreter can actually run the code and not just generate it and has that ~like,~ iterative loop.

So if the code, ~like the,~ the real magic part is ~like~ if the code doesn't work in code interpreter, it actually generates, ~um,~ it, it tries to regenerate the code ~and,~ and based on the error messages, do it Yeah. Regenerate it successfully, which is so exciting.

That is really exciting. ~Um,~ while we're talking about, ~um,~ chat, g p and OpenAI for developers, there was this. Beautiful presentation, ~um, when, you know,~ when GPD four, I believe, was announced, ~um,~ where, ~um,~ the presenter showed it ~this,~ this picture of ~like,~ here's a napkin, sketch of a web ui, ~um,~ make it happen. ~Um,~ but since then I haven't been able to use this feature. So ~what,~ what's up with that? The

~Yeah, I was,~ I was sitting there with Greg while he was doing that demo. ~Um,~ and it was, yeah, it's such an exciting demo. ~I,~ I don't think I had good perspective on like how excited people would be about that demo until after the fact. ~Um,~ but yeah, so the ability for G P T four to take in image input and then,~ um,~ take some action based on that is, is [00:18:00] coming.

~Um,~ it's generally, ~it,~ it's ~a,~ a very computationally. ~Um,~ difficult task to do and historically we have been extremely limited by compute resources, which is why ~like~ chat g BT used to like, ~you know,~ go down and not be available. It was ~like~ literally because there wasn't enough GPUs in the world to ~like~ run chat G B T for how many people were showing up to use it.

~Um, and uh, yeah,~ we've since expanded capacity and all that good stuff. But yeah, G P T four with image inputs, it will be coming. ~Um,~ Just the, ~it's the,~ it's the timeline itself. ~That's a,~ that's a little bit fuzzy,

Did the Microsoft acquisition have something to do with the availability of chat gpi? I assume it did, but.

~so,~ Open AI is an independent, ~um,~ entity from Microsoft. We have ~a,~ a multi-year, multi-billion dollar partnership with them, but we are, ~uh,~ strictly speaking ~like~ an independent entity that's governed by a nonprofit. ~Um,~ so~ we're not,~ we're not owned by Microsoft, ~um, or,~ or controlled by, by Microsoft. The OpenAI, ~uh,~ capped profit entity is run by the OpenAI [00:19:00] nonprofit entity, which, ~um, is,~ is wholly independent.

~This is,~ this is something I think worth discussing. I appreciate you clarifying that. 'cause this is a question that a lot of, so as you can imagine, I speak to a lot of people, ~um,~ attending a lot of conferences, et cetera. And this is the thing that skeptics come to me and they're like, oh, come on. It's just capitalism all over again.

~And,~ and I often have to do the work that you just did. So I appreciate you sharing that. ~Um,~ I also appreciate you highlighting. This interplay nonprofit and the ~um,~ and this is I think something a lot of people don't about, and. From, at least what I see on X or Twitter is people just show up with half knowledge and complain.

So I wanna spend a few minutes, if we can, ~um,~ talking about those things and clarifying them, ~um,~ just ~to,~ to not necessarily silence the skeptics because I think that's also wrong, but to inform the skeptics about the actual truth. ~Um,~ so what I wanna do is give you an analogy, a comparison, ~um,~ and close that is, Mozilla is the example I'm using.

'cause mozilla.com has a for-profit entity, ~um,~ that ~they,~ they, ~you know,~ try to get profit [00:20:00] from things like search, et cetera. But there's also the Mozilla Foundation that is fully open source and fully nonprofit, ~um,~ from day one. And there's this interplay between companies where Mozilla dot com's trying to, ~you know,~ make profit somehow.

But really the overseeing governing Bo Body is the nonprofit entity. ~Um,~ how close is that to the OpenAI model? ~Um,~ and. What does the nonprofit do?

Yeah, this is a really good question. So I'll say generally, based on your description, it sounds like ~there's,~ there's some resemblance. I don't wanna, ~um,~ I, I'm sure it's much more nuanced, like from an actual structure perspective. So I don't wanna make blanket statements without having ~like~ a good understanding myself of like the actual structure of Mozilla.

But in general, ~the,~ the interplay ~seems to be,~ seems to be similar in the sense that, ~um,~ open AI's capped profit entity has. ~Uh,~ yeah. ~Is~ is again, governed by that nonprofit entity. And I think the biggest~ like~ difference in general between this relationship and ~like~ the traditional company sense, ~uh,~ like a traditional for-profit company is that traditional [00:21:00] for-profit companies, and this is always so fascinating to remind ourselves of in, in, in the capitalist world, is that for-profit entities have like a legal.

Fiduciary duty to their shareholders to maximize profit no matter what. And you would actually like potentially be put in prison if you like, break ~that,~ that legal obligation that you have to your shareholders. And that's~ like~ a fundamentally different. ~Um,~ obligation that, that OpenAI has, which ~our,~ our obligation, you can read about this under, in our charter is, ~um,~ to actually ensure that ~the,~ the benefits of this technology one, actually benefit all of humanity.

~Um, and,~ and I think that's like such a different, I. Perspective on what our role is as an organization versus ~like~ a traditional for-profit, ~uh,~ entity. And ~like,~ I think a lot of the details about like how we distribute those benefits to society, ~um,~ still have to be worked out. ~But it's,~ it's important to note that ~like~ if you think about ~like~ Google doing AI and Amazon doing AI ~and,~ and all these other companies doing AI ~like.~

At the end of the day, their goal is to maximize profit for their shareholders, because that's the [00:22:00] structure of the entity that they're governed by. OpenAI does not have that goal. ~Um, ~and yeah, I think this, ~it's,~ it's nuanced, but it, in the end of the day, especially given how powerful AI technology ~is,~ is going to make ~a~ a huge difference.

So you mentioned the cap profit company. ~The,~ the for-profit company is governed by the nonprofit. ~Um,~ but at the same time, any for-profit company has to, has this legal obligation to maximize profit for shareholders. So ~how, how,~ how is that balanced then? Because it would seem, the governance by the nonprofit would nullify this legal requirement for the for-profit, but the for-profit would still be bound by that.

'cause it's a for-profit company, so I'm not seeing how they fit together.

Yeah, this is a really good question. I actually, I, it's unclear to me how much of ~the,~ the nuance structure of this ~is,~ is public facing. ~Um,~ so I, ~I don't,~ I don't wanna comment too deeply. My, my understanding ~of,~ of the structure is that, ~um,~ Because of the for-profit governance, there is no, ~uh, we,~ we [00:23:00] don't have that traditional obligation even

Sorry, ~you~ you mean ~just to,~ just to jump in here. I think it's important. You mean the nonprofit governance, not the for-profit governance?

Yeah, yeah. Sorry. ~The,~ the nonprofit governance ~is,~ is what mandates this, like ~the,~ the very unique structure ~and,~ and again, ~I,~ I don't know the specific details of like how that's worked in, but ~um,~ yeah, ~I,~ I think. ~I,~ I'm not sure. We definitely have some blog posts and things like that, so I'll, we'll see if we can link something for the readers.

~Um, yeah, it's worth, I, I just don't wanna.~

~Sure,~ sure. And we will put that in the show note captions. I think that's really important. 'cause I, I'm sure you, you have a front row seat ~to,~ to the skeptics as do I, but ~all,~ all I hear is, oh my gosh. They're coming to take our jobs and take our lives and ~it's,~ it's capitalism's gonna rule the world and it's gonna be like horizon zero down all over again.

~So, um,~ I ~fully,~ fully respect that. ~Um, I wanna be sensitive to, um, your, your microphone. There's a little bit of thunderstorms coming through. Um, so if there's~

~there are. It is. It is storming outside.~

~No, no, it's fine. It's fine. Um, great. All right. Let's, let's move on. Thank you, by the way, lo, um, Logan, this is, this is so good. Um, the people are gonna really enjoy this. Um,~ great. We've talked about GPD four, we've talked about 4.5. We've talked about the, ~um,~ napkin sketch and the image recognition coming, ~um,~ among developers.

~Um,~ what I hear a lot is, Talk about the whisper APIs. Everybody's talking about the whisper APIs, which is probably a good indicator of success ~on,~ on your part ~as~ as the dev develop person. But I'm curious, ~um,~ if you could quickly walk us through the [00:24:00] whisper APIs and what that means for developers.

~Yeah, great question. So, uh,~ whisper is actually one of the models that we open sourced, I think back in maybe 2019, something like that. ~Um,~ and ~it's a,~ it's a speech to text model. So essentially you can put it in audio and it'll transcribe it into text. ~Um,~ we made that, ~you know, the,~ the feedback that we heard from developers was, Hey, it's awesome that this is open source.

~Um, you know, it's,~ it's great. We can use it. ~Um,~ but it's actually really difficult. Like it takes a lot of work to ~like~ spin up. The correct resources to actually like, use this in a production setting or for my hobby project. ~Um,~ so we made it available ~in our,~ in our api, so now developers can have a simple interface to just, ~um,~ do ~speech,~ speech to text.

~Um,~ yeah, and ~we have,~ we have both transcription and translation capabilities, ~uh,~ which are, ~um,~ ex extremely capable from what I've seen. Like you can really do a lot of stuff. ~Um, and,~ and this is actually the capability that's powering. ~Um,~ I believe we have ~the,~ the speech to text capability available today in our iOS app.

And I don't use Android, so I. But I'm also guessing it's available in the new Android app as well. ~Um,~ and people have said [00:25:00] that they think that the speech to text capability is so powerful that they just literally use the chat G P T iOS app in some cases just for this transcription capability, ~um,~ to take their, yeah, because it, it really,~ I,~ I think the, for people who are, ~um,~ non-native English speakers as an example, like it just has a really good understanding of ~like~ some of ~the different,~ the different ways that people speak.

~Um,~ More so than some of~ like~ the out of the box models ~that~ that other companies have. So that's been ~the,~ the coolest thing that I've seen a bunch of like great threads on Twitter, people talking about this and, ~um,~ yeah, ~and,~ and the APIs allow people to ~like ~build this into their own stuff if they want to.

That's awesome. I might, ~um,~ I might, I. I can't wait for the YouTube content to show up of people comparing ~like, you know,~ Google translate, ~um,~ the whisper translate, et cetera, and ~see,~ see the real difference there. ~Um,~ that sounds really exciting. ~I,~ I personally have not used it yet. ~Um, but you know,~ after listening to you, ~I,~ I wanna give it a go and see ~how,~ how it works, especially as I learn.

So I live in Germany and I'm trying to learn,~ uh,~ German, and so this might help with that. ~Um, great. Yeah. Touch quickly on.~

Logan Kilpatrick Part 2
===

~Great. Logan,~ [00:00:00] I wanna touch quickly on, ~um,~ plugins and ~how they,~ how they work. ~Um,~ I understand they, they follow the open a p i, ~um,~ spec. So if you could speak to that, but also, ~um,~ the return from a plugin once a plugin response, is that, can that be used in training data or is that then also presenting the overfitting problem?

~Uh,~ if you could speak to some of that.

Yeah, great question. ~So the,~ the basic idea is that you can take an a p. Already exists or make a new a p I. ~Um,~ define the structure of way, of the way that, that a p I work using the open API I file. ~So like the,~ the very interesting thing is if you think about this technology versus like the app store, for example, on, on iOS.

~Like~ in the iOS case, you actually have to ~like~ send Apple, ~like~ all of the code, like the entire like package of code to be reviewed in the context of.

Control these.~ Um,~ and they essentially just~ like~ point to the, ~um,~ and point to [00:01:00] the, of the open API file, which is the, again, the structure ~of,~ of what. Essentially what functionality ~your,~ your A p I has. ~Um,~ so ~you,~ you can imagine, for example,~ like~ if you have an a P I that tracks sports scores~ or,~ or something like that, ~um,~ you can also, which is super helpful and gives developers a ton of flexibility.

You can have, ~um,~ an a p I that has~ like~ thousands and thousands of different endpoints, but you could only expose~ like,~ Three or of those through this specific open a p i file, ~uh,~ to chat. Bt if you only want to do ~like~ some very basic things, ~um,~ so you have a ton of flexibility. ~Um,~ again, essentially chat bt when you have a plugin installed, it sees that open a p i file so it understands like what actions are available to it.

And then based on the user query, like if the user's query was, ~you know,~ what's weather plugin? Plugin installed? ~Um,~ look at that. Open a p i file. See ~like,~ oh, here is how I have to formulate a request to this a p i. It'll actually write the code to send that request. It'll send [00:02:00] that request. It'll get the response back.

It'll interpret the response ~from,~ from the a p i,~ um,~ and then it'll provide the output to the users. ~Um,~ and then as far as the training question, ~um,~ so the thing that's actually trained on is the response to the users. ~Um,~ so if your request is like, what is the weather like in San Francisco? Send the request to the a p I get this, j ss o n object back has a bunch of metadata that has all this information.

If the response to the user is, the weather in San Francisco is 86 degrees, like that is the piece that's trained on. ~Um,~ yeah, and ~there's,~ there's also the ability to, ~um,~ yeah, there, ~there's,~ there's so much interesting things that can happen ~with,~ with plugins. I. Happy to answer more questions, but yeah, I get, ~uh,~ I.

For real, I could. I could speak to you for another. Hour and a half, honestly. But we, I do want to be respectful of your time, and there's two, ~um,~ burning questions. I don't know if we'll get to all of them, so I'll ask the first one and then maybe the second one. But ~I,~ I do want to not leave this podcast without talking a little bit about Lang [00:03:00] Chain.

~Um, I've,~ I've used, ~uh,~ Lang Chain at length. I absolutely love Lang Chain. ~Um,~ and ~I,~ I think it has a really promising future. ~Um,~ I'm curious if you could speak to one Lang chain ~for,~ for the listeners. ~Um,~ two, the, I believe line chain has this, ~um,~ this actor model as well where, ~um, you can,~ you can process the output of LLMs ~and, and, and,~ and add really multiple processing steps, almost, ~um,~ building like autonomous agents.

That's what I call it agents. ~Um,~ so if you could speak to that, but then also, ~um,~ share a little bit about the interplay between Lang Chain and OpenAI. 'cause as far as I know, Lang Chain is just an open source project. It's not owned by OpenAI. ~Um,~ so if you could, ~um,~ elaborate on that a little bit, I think we'd appreciate that.

Yeah, sure. So Lang Chain ~is a,~ is a independent, ~uh,~ for-profit entity. ~Uh,~ they have an open source, I don't actually know what the name of like their for-profit entity is. It might be Lang Chain. It might not be,~ um,~ Lang Chain as people commonly refer to it as the open source project that Harrison and, ~um,~ the rest of the chain team actually created.

~Um,~ And it's right, right now. I think the general sort of way of looking at it is it's a wrapper on top of [00:04:00] large language models, ~um,~ that essentially makes them easier to use. Like it has a bunch of built in tooling and things like that to make them easier to use.~ I,~ I will say that like in general, ~it's,~ it's all things that like, Open, AI open for our users would ~like,~ want them to have the ability to do.

~Um, so it's,~ it's wonderful that the Lang Chain folks are like willing to do this work and create something that, that works so well for people who are building on top of our models. ~Um,~ Lang Chain also has ~like a,~ a new,~ like,~ I think it's like a Sass product that they're building called Lang Smith, which ~um, ~does like more.

Helps people in more production use cases with monitoring and debugging and, ~um,~ all that type of stuff, which is again, super, super helpful. And like all things that make a ton of sense. They're very much like the rough edges of working with large language models. ~Um,~ so yeah, I, I think I. People should a hundred percent be looking at lane change.

Should a hundred percent be looking ~at~ at other tools that are out there. And also like thinking about where the opportunities are to build more tools like ~the,~ the space in general. The ecosystem of large language models is like very much in its [00:05:00] infancy and there's such an opportunity if you're somebody who likes building things for developers to actually~ like~ build some.

Really and ~like~ get some meaningful market share. And I think that's what's happened with, is they essentially have this massive market penetration, which has.

Yeah. Fantastic. All right. Unfortunately, we're out of time and there's a lot more to discuss. That just means we have to have you back on the podcast at some point again soon. ~Um,~ Logan, but before, before we wrap up, I'm curious,~ um,~ about, ~uh,~ what I wanna do is ask you three questions and then you pick the one you want to answer.

~They're,~ they're vastly different, ~um,~ but I wanna give you that, choose your own adventure. ~Um,~ number one, ~um,~ what are you generally ~like?~ Perhaps disproportionately excited about at OpenAI that the rest of us aren't. ~Um,~ two, ~um,~ anything on the roadmap that's coming up that you can share, I guess that kind of ties into one or three.

~Um,~ for anyone looking to penetrate the market and get into ai, ~um,~ more so now than before, ~um,~ what advice would you give them? So that's your choice of 1, 2, 3 questions and you're welcome to answer one or more.

~Yeah, I've got,~ I've got [00:06:00] hopefully an answer that will touch all three points, which is the thing that ~I'm,~ I'm most excited about is, ~uh,~ is fine tuning. I really think that fine tuning is not only going to, ~um,~ unlock so many use cases that were not possible before, but also give people what they've been looking for since day one with this technology, which is how build a differentiated business problems.

~Um,~ I'm not ~sort of ~stuck using the same things that my competitors are and have to ~sort of ~differentiate through UX and some prompting layers and things like that. So I, I really do think that fine tuning is gonna, is probably going to ~like~ 10 x the impact that, ~uh,~ g BT four ~and,~ and this AI technology has had so far.

~Um,~ and I'm really excited that. The way that OpenAI does fine tuning, ~um, is,~ is like so simple from an interface perspective and is also~ like,~ we don't require that you give us your, ~uh,~ like we don't train on the data that you give us through our a p i, all that type of stuff, which just ends up being like, great for [00:07:00] developers.

~So, um,~ it, yeah, it's gonna be so crazy ~to see,~ to see this impact and,~ um,~ I think we'll look back in two years and be like, wow, we were really excited before, but now people are,

All these fine.

~Fantastic. Uh,~ and of course, ~we'll,~ we'll have everything in the show note captions about how people can get started. ~Um,~ with that, Logan, this has been an absolute pleasure. Thank you for coming on the podcast. Thank you for personally, ~um,~ teaching me so much about things that I didn't know,~ um,~ from, ~um,~ what it means to do derel at OpenAI, ~um,~ to how.

~The, the,~ the non-profit entity governs the for-profit entity and all the way through plugins whisper Lang Chain. It's been an absolute pleasure. Thank you for joining us on the podcast.

Yeah, this was wonderful. A ton of fun and would love to be back. We need to check in every year and see how, ~uh,~ the predictions and.

Fantastic. Let's do that Once again, thank you for joining us on Pod Rocket and thank you for your interest in tech.