David Khourshid
===

Paul: [00:00:00] Hi there and welcome to Pod Rocket, a web development podcast brought to you by Log Rocket. Log Rocket helps software teams improve user experience with session replay, error tracking, and product analytics. Tried for free@logrocket.com today. My name is Paul and joined with us as David Khi. He's founder of Stately ai and you might also know him from being the creator of X State, the ever so popular state management,

and we're gonna be talking about his recent conference talk from AI to agi. And for people listening, you probably are inundated with AI content out there and how you can use it and this and that, but this is a developer's podcast. So I really look forward to digging in with you today, David, and talking about how can we actually integrate these tools into products into the new software that's coming out today.

So welcome to the podcast. It's a pleasure to have you.

David: Thanks. Happy to be here.

Paul: So before we hop into things really quick, you were on Pod Rocket? Two years ago [00:01:00] now. Yeah. Right. Wow. So since then much has happened, I mean, you're the founder , of Stately ai. And has that been your main focus since the last recording 

David: It absolutely has been my main focus. We've been around for two years now. Have a lot built out and we have a team of just under 10 people, so we're keeping it small.

Paul: So stately ai, you're the creator of X State, they probably have something to do with each other. If you go on the website, you're gonna see. My first thought was like Lucid chart. I'm seeing Lucid chart. I'm seeing Clicking and Dragon and all sorts of, my brain feels good. That's how I diagram my stuff out.

So stately AI helps you organize your XD and your applications at a very high level. But could you dig in a little bit for us and talk to us just for a few minutes about what State Lead does?

David: Sure. So in shorts Daily makes your application logic, visual and collaborative. So our main goal is that we want to take. Whatever lives inside developer's heads, whether it's application, business logic, features, how things work, like what's actually inside the code, [00:02:00] take that outside of developer's head and make it something that can be shared with the entire team.

And of course, traditionally diagrams help with this, but the problem with diagrams are once you write a diagram, it's written in stone, it doesn't really match reality with what's going on in the code. And , this is a problem with documentation too, where it's just so hard to keep it up to date.

So our solution is actually a really old solution called executable diagrams. And so these are diagrams which are basically always in sync with the code. They reflect the code and the logic that's actually written instead of you having to do two different things. And so we think that enables teams to collaborate on application and business logic a lot better.

Paul: And this endeavor that you're embarking on really is this powered and lifted up to new heights because of the tools that are coming out that we're talking about today, like the AI and transform models that are of the day.

David: I would say it enhances it. Yeah. Just a little bit of background. First of all, I'm not an [00:03:00] AI expert or anything more like an ai amateur. Practitioner, enthusiast whatever you wanna call it. But state AI that, came to be because. When I was creating the companies, I realized that state machines are actually the most basic form of ai.

So if you've ever played a game n b behavior is modeled as a state machine, or sometimes a behavior tree, which is just a fancier state machine. And yeah the most basic form of AI you could find is an estate machine. But with these new technological advancements like LLMs and deep learning and things like that I began doing a bunch of research and learned more and more that state machines and things like LLMs and deep learning actually fit together really well and can

Really superpowered ai.

Paul: So where do you find that intersection? Being most harmonious between the intersection of LMS and all this new stuff and like a traditional diagram. Cuz like when I'm thinking about drawing a lucid chart, it's a pretty[00:04:00] unique process that goes on in my head for every box that I draw in every arrow.

Is it straight? Is it not straight? They're so where do you find like a good harmony between those two intelligences coming together?

David: . I think that where it's most applicable and I guess harmonious is in describing how to do a certain tasks. For example, let's say you're asking a robots to pulling in an example f then they're like, make a peanut butter sandwich, right? You might just say, as a human, make a peanut butter sandwich.

But the agent needs to translate that into a series of steps where it's, okay, get the bread, get the peanut butter, open the peanut butter, put it on the knife, et cetera. And eventually the outcome is you've made a peanut butter sandwich. And so as humans, we might think, okay, so that's step one, step two, step three, step four, step five.

But we know in reality, it doesn't really always work that way. Because it's okay, we're add peanut butter now what? So now we have to [00:05:00] augment our process and say we have to go to the grocery store. We have to buy peanut butter, get all the ingredients basically handle anything that can happen and can go wrong.

So it's almost never just a straight like sequence of events and states there's always going to be branching. And if this happens, then do this. And. At the end of the day, you get a large state machine, which describes that process of even something as simple as making a peanut butter sandwich.

And so I think with intelligence AI is basically that, but represented at a much, much bigger scale where these state machines can be giants. And so that way you could tell some robot in the future, make me a peanut butter sandwich and no matter what the situation, it will know how to traverse that state machine in order to achieve that goal.

Paul: , if, I may try to distill a little bit. It's really, you're finding that the AI. Is complimentary to this fundamental idea of state machines and that if we're finding some way of like [00:06:00] from the human side right now at this point in time from the human side to create the fundamentals of the state machine and to define some constraints about the goal you find that AI can use that and really branch off and complete not only the task that was pre-described, but like any other like ifs and buts along the way.

David: Yeah, exactly. So I recently gave a talk at React Athon and it, it was titled making state management intelligent.. But in that talk I gave sort of a demo of this idea where I. Basically I created a state machine based application for an espresso machine.

And this espresso machine knew how to do certain functions like heat, the water, grind, the beans foam, the milk, all of these sorts of things, and put those ingredients together. However, I did not program anything in that espresso machine that said, here's how you make a cappuccino, or a latte or an Americano, or anything like that.

And so that is where something like LLMs come in because [00:07:00] LLMs can translate something like, make me a cappuccino and then figure out the steps in our given state machine how to do that. , that's where I find the intersection really useful.

Paul: When you're playing with these LLMs and with the state machines, so I gather you use X State a lot because it's it's your creation, it's your natural way of representing state. Do you find that you can just copy and paste a state machine into an L L M and have it understand it because I know it can read code and json and sort of, parse out needed things or I gather you use like.

Playing chain or some sort of library like that. I'm not really sure, like how do you go about passing these states and that knowledge and context into the LLMs and interacting with how deep can you go or how basic can you go rather,

David: Yeah, so I haven't dived too deep into Link Chain Auto, G p T, all these chain of thought based l l M agents. But the way that it essentially works is actually fundamentally different than things like link chain because, so just a little bit of background with things like link chain, [00:08:00] baby a, g I, auto, G P T, all of these tools that have come out.

The premise is that you give it a prompt where you're like, here's the goal I want to achieve. And then the l l m is basically asking itself in a cycle, okay, what are the tasks I need to do , to achieve that? So it goes task by task and it says, here's the first task. It executes it. And depending on the results, it's okay, I'm either going to do the next task or maybe I have a new set of tasks or some sub-tasks to get to that.

This approach though, is fundamentally different, where instead, and this is not for every application, but instead the tasks are written in stone more or less, As the state machine. So it becomes like the railway for some sort of agents trained to travel on, if that makes sense.

Like basically roads in a city. And so the benefits that gives you, Is that, the agent is not going to go off the path and try to find the solution where there exists no solution. Instead, [00:09:00] it's basically gonna just say, I'm traveling this well-known path and I know that I could just take a series of transitions to eventually reach.

A goal states that I need to reach. So it doesn't need to do the task of asking itself what to do. It only needs to figure out what is the final states that I want to go to. Much in the same way that you and I would use Google or Apple Maps where we're not planning like which streets we wanna travel.

Instead we're just asking it, we wanna go to this destination. How do I get there? And then Apple or Google Maps will say, here's the path you need to travel, and if something like there's a detour or something like that, then it will reroute us and take us in a different way. And so agents can be smart about that too, when they realize, Hey, I took this path, but there's an unexpected state.

Then it could figure out how to reroute given that same state machine and reach the end goal a different way.

Paul: So what's your state machine sort of approach? This is well suited for like an [00:10:00] agent to execute on top of and it can take decisions and make it versus like a self state producing agi almost. I don't want to call it an agi cuz it's baby it's not, it's just, it's calling itself like you said, in a loop.

But yeah, it feels like this is like at the agent level and you're really traversing the train tracks, like you said of the state machine.

Can you feed like a state machine that I might write with X State into one of these LLMs and get any sort of compelling results? Outside of an application boundary, like I want the l l m to do this and this and this.

Like I have this idea for how it's gonna generate a piece of text or a blog post or

Is that something appropriate to maybe play with X State with.

David: If I understand the question or the idea correctly, I'll admit that, like when chat g p t first came out I've been playing around with G P T three and 3.5 for a while before that. But chat g p t came out and it was like leaps and bounds above what, previous G P T three, even though they were actually really good.

And so something I was experimenting with early on [00:11:00] was having G P T given some examples of Here is some application logic that I wanna create. Make me a state machine from this. I was feeding it a few examples to do that, but I realized that with chat G P T, it actually didn't need any examples at all.

You could ask it and you could even try this now. You could ask it for a state machine for basically anything that you could think of and you could ask it to write an X state machine that represents that and it gets really creative. But the state machines are. Most of the time completely correct.

Like you could copy paste them, run them in your code. And so I thought that was pretty exciting. And furthermore, what it does is you could give it a state machine or use a state machine that generated, and it will explain it in human terms. And so that I find really fascinating where it understands the concept of here's a state machine of all my seats and transitions.

Let me put it into human terms and summarize for you what this state machine means and what it does.

Paul: Do you feel like X State could be a. Very [00:12:00] universal layer of lubrication between people using these large language models and their desired like, courses of action. I wanna hop off of this podcast and generate a blog post by making a state machine and then saying you generate this, then you check this, then you go to this and feed, , chat and say, go try it out.

Do you think this could be like a way we interact with fellow alums moving forward?

David: I think that's in short, yes, I might be biased because I love steam machines, but whether it's X State or some just base layer, just a level of abstraction lower than X state, some standard j s o format for describing these directed graphs yeah I think that could be a really good substrate for things like this.

For example, your blog post, if you want to make one. I know a lot of people just go on chat G P T and ask Hey, I need to make a blog post for this. Write it for me instead, you and I would probably wanna say, okay, first make an outline. Then research each of the sections, then write it, then question yourself on each of those sections, and then refine it.

If anything seems [00:13:00] off, then change the tone. There's a bunch of steps in order to make this essay, actually read really well. Cause admittedly chat, g p t can produce things that are sterile sounding, where after reading a lot, you could tell yeah, this was written by chat G P T.

It's really good, but it's, just very, flat sounding.

Paul: So I'm just gonna take a moment to remind our listeners that this podcast is brought to you by Log Rocket. Log Rocket helps you catch errors and improve your user experience with session replay, error tracking and product analytics. 

Spend less time debugging. And more time building your app. So head over to log rocket.com today and try it for free. David, if we could step a little bit into talking about how you're actually integrating LLMs into the product. Cuz we mentioned your product at the beginning of the podcast, stately and

just to recap really quick, cause it was like 15 minutes ago at this point helping us. Declare these state machines, build them. Please chime in if you want to add anything [00:14:00] in here, but you can click and drag like lucid char and collaborate with teams and such. I'm really curious how you're using this idea of state machines on rails for AI in that product or hoping to integrate it in the future.

So what are you working on right now?

David: Yeah. There's a short term, in a long term set of features, I guess plans you could call it. The short term is basically what every company is doing, and it's something that I believe in strongly as well, which is using things like um, G P T four Chat, G P t, LLMs in general to enhance the existing product instead of.

Being a product this is something that Notion is doing really well and so many of our other products are doing. So in the short term, sense of things we're doing things like the ability to create a machine summary or generate descriptions of states and transitions and even further, one of the most exciting things that I've played around with internally, Is generating machines.

So just what I told you where you could go to chat t p t and say, make me a machine for. Like [00:15:00] an espresso machine, just all the functionality, you're going to be able to do that in the stately studio as well. So you could even say, make me a simple checkout flow that has these states or whatever, or make me an authentication flow something like that and it will be able to generate that.

And furthermore, modify the machine based on natural language prompts. It will also do things like co-pilot where it will suggest transitions and actions and states for you and basically make it a lot easier and act as well your co-pilot or assistant in actually creating really robust application logic.

And so the long-term plan is what I alluded to earlier on where basically if you take LLMs plus state machines and use them together, then you form really powerful ai, which is actually deterministic, explainable, and inspectable, which means it's not gonna do anything unexpected or random, and you could actually test it because it's based on that.

State [00:16:00] machine rather than having it go off on its whims and figuring stuff out itself. And so we see stately AI as basically the platform for creating and augmenting these kinds of models. And, these models can actually get really huge, like you could just imagine the state space of anything really.

Even the chess game hasn't. Almost uncountable number of different states that it could be in. So just, thinking about things like that. That's more of our long-term plan, and it fits really well into a lot of the research that I talked about in my talk with playing Atari games and things like that.

Paul: So if you wanted to, for example, generate a state machine for logging in and logging at a user, if I were to go to chat g b t, I'm sure it could gimme something. Implementation wise, not so great. Maybe cuz it needs the libraries, my project structure and lots of context. But if I asked for a state machine, I reckon it could gimme something pretty decent on O I D C or like OAuth or something like [00:17:00] this.

When you wanna take those principles and integrate it into one of your customers products in stately, what are the sort of tools or fine tooth combs? That you use to take that output and actually make sure it's prim and proper and well-suited for the project at hand that somebody might be using, like your cleaning and filtering process.

David: Yeah, so I'll tell you from experience, because this is not something that we actually have released yet. It's more of a short term and, long term plan. But it is something that we're actively thinking about and let's say that you've generated a state machine either from Staley or chat g p t you're right, where it's going to basically form the structure of, your states and transitions and actions.

And then it's up to you as the developer to fill in the details later. And that's actually what, in my opinion, state machines are great for, is providing that structure where you could later fill it in with code. So it's not full code, it's not low code, it's more like, Code structure that, like scaffolding for a house or something where you could [00:18:00] just fill it in with any furniture, walls, paint, whatever later.

And so what I've found with co-pilot at least, is as I'm writing my state machines, or, even if I'm copying and pacing them from chat, G P T. Co-pilot is actually really good at inferring what exactly you want to do in things like actions and invoked actors and guards and things like that. Long story short, it's going to fill in the implementation details for you based on your actual project.

So let's say that you're using gonna say a really old library like passport js and you have a state where it's like logging in and you have an invocation where it's like, initiate OAuth request or I'm just making things up, but something like that. Then when you put your cursor inside that action, it's going to know, all right, I know exactly the code to fill based on the libraries you already have imported what you're doing in the rest of your project.

And it's really good at that. So we see the potential for just expanding the capabilities of the studio to [00:19:00] both provide that higher level structure and to also fill in the implementation details.

Paul: Is that a co-pilot strength that you've identified in particular?

David: Yeah. Yeah. And again, with LLMs and co-pilot it could get things wrong sometimes, which is why as a developer you should always, verify you know what it's producing. But the idea that like, you could use LLMs to basically number one, form that outer structure. Number two, just fill in the parts It makes things a lot better. I'll do a quick aside over here because I've seen a lot of demos where people would present a prompt where it's make me a project that does this and it has these requirements and it's in next or something. And what it will do is it will scaffold the entire project.

It will write all the code and do everything like that. Those are really impressive examples, but, Something that I'm really afraid of is the maintainability aspect, where it's okay, can you actually tell that l l m, let's change this feature. Let's switch out this framework. Let's do something else here.

Let's add a different page [00:20:00] in between this or a different step in this multi-step form and. I really believe it's gonna struggle because LLMs are good at producing, but in terms of trying to ingest a large amounts of data and make sense of it, that's something that even developers struggle with.

That's why I think, and I'm really bullish on things like using a state machine, because you're always going to have that exact same structure. Everything is just dates, events, and transitions. And so you could always add states compose states together, add, remove, transitions, and so it just, in my opinion, scales a lot better for doing things like application generation maintenance and also enhancing or changing features.

Paul: I really loved to hear what you had to say about maintainability because I can say on a personal front, if I used chat BT to make a bunch of functions to act. Access my database, and then I go use those functions, and then I come back a week later to expand on a new branch. I'm probably gonna [00:21:00] be writing overlapping code if I just continue to auto-generate and I'm not reading it myself and really understanding.

And that's like maintainability. The mental model that you have as a developer is your mental model. It's David's mental model, and it's never gonna change. Maybe it'll fade as you forget it but it's never gonna change. Every time you feed an l l m through, it's gonna think of it possibly a little differently, and the mental model will be different.

And I personally have experienced drift in my code, in , how they overlap. So when you call us out, I'm like, huh. He's right. I've totally seen that problem already crop up personally. So I like how you identified it too. It's like an input problem. It's a context and. It's almost like how it mentally models things like these LLMs. . The problem here is how it takes in context and mentally models, for example, a repo. Where are you drawing those abstractions it makes it unmaintainable over time.

David: Yeah. Yeah, exactly. , you often hear people think of like LLMs as just these large text predictors, like a fancy auto complete. Of [00:22:00] course it's more than that because there's a lot of layers, but at the end of the day, it's just a really superpowered pretty int intelligence, text auto completion tool.

And That's why if you played around with chat e p t, there's one thing that it can't do, and that's like when it's giving a response, it can't go back in time and modify like a beginning or an intermediate part of its response. It's like whatever it's said, it's out there. So you can't just go back and post and edit it and sliced things up or anything like that.

And that's. Not exactly how humans think. So as humans, we could actually write code and then we'll go back and say, okay, no, this needs to be changed. Or we could do it at a larger scale and say, this needs to be refactored rearchitected. Or we could say, I'm going to form a mental model of the code where these services are calling these functions, and these files are arranged this way, and based on that mental model, we could make decisions.

Whereas that doesn't exactly come naturally, at least. From what I [00:23:00] know to LLMs. So that's why I really believe in like just providing that additional structure whether it's like a state machine architecture or any other sort of architecture to LLMs to help guide it and say this is my mental model of our code base or anything else.

Please work based on that mental model and, do these tasks.

Paul: Yeah, and based on that mental model, it's this question of how you stuff all that information in and generate that mental model. And I agree with you, it does feel a little overlooked and it's a very complex question.

David: Yeah.

Paul: David, in the next coming quarter or two quarters, what can maybe you broadcast to listeners about things coming to state Lee, whether they be AI related or non AI related that you might be excited about?

David: Yeah, so our AI powered features are, coming this year, I wanna say it's not our main focus because our main focus is on basically the premise of state machines, which is something that's explainable, deterministic, [00:24:00] allowing you to collaborate with your entire team on them. And we think that no matter how advanced AI gets, like just having that solid foundation of here is exactly how my app works.

I know that based on, the mathematic principles of state machines, it's never going to deviate from if I'm in this state and this event happens, I know that this state will always be the result of that. I think , that's. Really important. But in the future, we do wanna build AI powered features on top of that to both help you in modifying that logic, improving that logic, and also just creating new logic.

But in terms of like non-AI stuff, we do have X State V five beta that's being released, so that's gonna be announced pretty soon, and we're adding a lot more features to the stately editor. We have a new URL as well. states.new, it redirects to the stately editor, but it's the easiest way for you to you were mentioning the lucid chart, so if you're like, I want to create just a flow diagram really quickly, you could just go to the [00:25:00] state.new and start editing from there.

And we also have some collaborative features coming up too. So that's pretty much it.

Paul: Awesome. State.new I love the, that we're using different top level domains now. It makes 

it things easier to remember cuz we're creatures of pattern. David, if people wanted to keep up to date with you. And your updates, do you post on Twitter or any social things where people can learn about stately?

David: yeah, Yeah. So I'm at David k piano. On Twitter, on Blue Sky, if you have Blue sky GitHub, it pretty much everywhere else. And you could also follow stately at stately ai pretty much everywhere as well. YouTube, Twitter, all of those things.

Paul: Awesome. David, it was a pleasure having you on. Thanks.

David: you for having me.