Where AI Still Breaks Vibing with AI Code, Episode 10: Brendan & Baird Brendan: Today, uh, we, we thought we would talk about, uh, you know, just a check-in, I guess, and see kind of where we are. We've been working, Baird and I at least have been working with AI well over a year. Has it been almost two years maybe? I don't even know. Kind of losing track. But, Baird: exactly Brendan: you know, I, I thought it would be good to, to check in, but also, you know, see where the road bumps are, 'cause things have improved a lot, especially since, like, when we started this podcast. But we still run into things either… If it's not daily, it's weekly, that are still frustrating, still either gonna be ironed out or maybe it's just, like, the status quo. You know, I'm not too worried about exploring that part of it, just more about, like, let's kind of see where things are. Baird: Yeah Brendan: so pre-show you were talking about something that you've been doing recently that had some of these issues. Let's maybe start there and then, you know, I, I think we both have some stories we can share as we go here, so I'll hand it over to you Baird: Yeah, definitely. A- and I think, just to kind of set it up a little bit, you know, looking around in the world today and how people talk about AI, you know, so much of it is agents are taking over. You know, they're do everything, they're gonna replace everybody. And I mean, don't get me wrong, I love agentic AI. I think there's a lot of really cool things about it. But, you know, like you mentioned, I was just working on something through my job where it's like, "Hey, this is a perfect use case for, like, how could I automate this?", So, you know, I, I was l- at work thinking about, you know, we have to do image processing essentially of a lot of images for a website we're building. And kinda the goal was, like, how can you make the images look better, you know? And this isn't, like, full-on image generation. This is really just, like, how can you clean up images and make them look more appealing? And, know, the awesome thing is, I don't know if you've played with, like, both Nano Banana and, um, GPT Image 2, like, where they are these days compared to where they were even, like, six months ago is amazing. Like, it actually does a pretty good job, especially, like, GPT Image 2, I've really noticed a big jump where, like, it'll follow your instructions. It'll do a pretty good job of actually, like, taking exist- existing things and improving them or changing them based off of, you know, what you ask. And so you think, "Okay, well, this is a perfect place where you can automate the task." You know, you can up a workflow. You can let it run, and it'll get you what you want. And so I essentially, like, built out a little bit of, like, an application where I can do a canvas with some AI attached to it, and I can have it generate stuff. I can have it, you know, re-, uh, do, like, the aspect ratio of the image so that it can fit, 'cause get into it a little bit in a second here, and then, like, remove the background because I needed the background removed. And so set this up. I have this whole workflow. It's supposed to work. Run it through, and about every second or third image, all of a sudden, you know, it would do great for, like, two images. one, it cuts off like 10 pixels on the side. And, like, this is images of sandwiches where it's like they need to be the entire sandwich. It needs to, like, be the image. It just, like, is putting a layer of, like, polish on it essentially. so, you know, that's not a huge deal. Like, you can fix it relatively easy. Like, I even could go out to ChatGPT and just, like, have it do it again there, and eventually it would do it. But when you think about automation, like, the whole premise of automation is, like, you don't have to go in and manually do things. It's like you can just set it and let it run. And I think, you know, that's one of the things about AI that, that's been kind of interesting, and, you know, this is a little bit of a, a tangent on this, but it's just like I still see the automation part as A sticking point just because it isn't deterministic. You know, like, you can't Brendan: Yeah Baird: that it's gonna be right every time, and a lot of automation, you have to have that reliability. And so it, it's one of those things where going through this process, it's been amazing because I'm like, I built this really cool pipeline where it actually is, like, between, not just regular technology, but AI. I've built this really cool workflow, and it does everything I want it to do except for kind of the most important part, you know, like that actual AI intervention part where it's jumping in to something. It's like, it did a great job, except did something that, like, the AI couldn't then go back in and easily, like, fix because it's hard to explain, you know, like exactly what's wrong. And, like, if you give it to, like… 'Cause one of the things I did, too, is, like, went into ChatGPT, asked it to give me, like, the prompt of how to do it in the first place. Then I gave it the images to show it, like, what was happening and said, like, can I fix this? How can I make it consistently actually, like, keep the whole thing in shot?" gave me, you know, another prompt with some changes to it. I ran that, and all of a sudden, the same issue's there, but instead of, like, fixing anything, it stretched the image of the sandwich in the background as, like, almost a gradient trying to fill the space with the sandwich versus, like, trying to keep it all in shot. A- And so it's just one of those Brendan: W- just real quick, what, what were you, w- which one were you using? Was it GPG2? Baird: So th- this is GPT Image too. And i- i- it's funny 'cause I've also tried the Nano Banana with this, Brendan: Image G… Yeah. Okay Baird: like, this is all through the API Brendan: Yeah Baird: to do it in the chatbot. So, like, you have a little bit less control. like, for whatever reason, Nano Banana, when I was running it through this way, was, like, not even close to the real sandwich. And, like, I had done it in Nano Banana, you know, in actual Gemini itself, and it Brendan: Yeah Baird: so I might have had an issue with the actual model I was calling there, but I didn't push too far on it. Um, but it was just kind of one of those reminders because, you know, we talked about it a little bit before too. AI has come so far from where it was, but there's still these friction points like this where it's like it isn't quite there yet. You know, there's these times where it just, like, kind of blows up in some really weird ways that are hard to recover from Brendan: Yeah. Right. And I mean, it, it also, it makes me think too, some of it maybe has to do with the way we're wording the prompts and thing… You know, all the things that you deal with anyways, but I, I, I'm not saying that's what it is, and it probably isn't even that. But it's like, it, it does make me wonder myself when I run i- into these issues. I, I usually will look at it and be like, "Oh, did I word it weird?" Like, "Did I… Was I not clear enough?" But, like, an example I have, and, you know, this is a more recent one that actually worked that was interesting. But, like, I wanted a concert ticket made s- you know, and I, I told it, like, vague things like, "Hey," that… Or not vague, but, like, only a few specific things. Like, "Here's the venue. This is the date. Um, this," you know, "this is the band." And I, I think that might have been it. And then it, like, put together a really cool… And, and this was NanoBanana, by the way. Baird: Sure. Yep Brendan: you know, it put together a really cool-looking, uh, like, not a… It's not like a ticket you'd actually would've got, but, like, a commemorative type of ticket, but it was cool. And then the thing that it did that I didn't like, but it wasn't a big deal, but it was just like I didn't tell it, and so it just kind of picked a section and a row and a seat number kind of thing. And once I saw that, I was like, "Well, actually, if you're gonna put that, I might as well put the real one." And, you know, my experience in the past, even with Gemini NanoBanana, had been it would give me an image that I thought was a pretty good start, and then I would say, "Hey, could you just tweak this little thing?" And it used to tell me, "Nope, can't… And I cannot edit the image." Now, this time it didn't tell me that, and I gave it, like, all the exact seat information, and it actually replaced it perfectly, and it turned out great. So that was, like, the first time I've actually… Granted, I don't do a lot of image stuff, but I do sometimes. And that was the first time I actually ever got it to change the image that it just created just slightly. And it, and it's actually reminding me too that, like, we've had some thumbnails that I've made in, uh, NanoBanana as well, and those… You know, that wasn't that long ago. That was, like, a month or two ago. And I was running into that same issue where, like, the thumbnail would come out, and it would say something I didn't like or whatever, and rather than it being able to just take the one it initially created and tweak it, I would have to basically retype the whole prompt but then add the things that I didn't like. Baird: Yep Brendan: in some ways it's gotten better, but i- in general, like, images are very tough. Like, Baird: Oh, it is Brendan: um, you know, they're, they're better from, like, a, a starting point now, but, like, once you're in the mix of like, like you were saying, like, tweak it, that's what it gets really tricky. Baird: Well, yeah, a- and that, that's actually kind of a, a hilarious example on, on my side is, like, another example with the whole sandwich thing is just trying to get it to edit an image it already did. 'Cause, like, I gave it the instructions, like… And I, this is one where I went to ChatGPT, took the wrong one, like, it did most of it right, the one where it cut off. Brendan: Yeah Baird: original that showed the whole thing, and I was just like, "Hey," like, "go in and just, like, essentially make sure you show the full sandwich from the new one," you know, like, using the old one as a reference. again, like, 8 out of 10 times it actually does that well. But then one time it, like, doubled half of the sandwich. Like, you, and you could literally see where it took, like, a slice and then put it over here as well, and it's just like, it's just so weird how random it can be. And Brendan: Yeah Baird: I, I, I also remember with Nano Banana, like, when it first came out and was a thing, and like, again, like, this wasn't that long ago, but it was long enough ago in AI time that it's, like, ancient at this point. But I remember it would do things like, there was, like, a limit how many times you could ask for regenerations, and you would Brendan: Right Baird: give it to it, it, but it wouldn't tell you that. It would, like, render the image, and then you'd say, "Hey, can you do this now?" And it would come back with the exact same image, and it wouldn't tell you it failed. And then, like, I looked around one day and it's like, oh yeah, it just stops doing it, but doesn't tell you. Like, it just keeps regenerating it. A- and again, like, that's just kind of a funny thing, but it puts in perspective that this stuff is not, you know, concrete. This stuff isn't, like, and reliable for a lot of things, you know? A- and I think that's just kind of like looking around at the world today from my perspective. You know, is agent this, agent that, like, it's gonna take over everybody's jobs and all that type of stuff. And, and I can see some of that. But to the point of everything we've been talking about here, there's all of these relatively, I wouldn't even call them edge cases, 'cause it's not like this is some, like, weird, you know, logic bug where something happens where it's all the perfect storm and then it goes wrong. Like, this is speaking natural language, straightforward, that you can even do, like, 10 times and it fails randomly. And, Brendan: Yep Baird: are definitely a big one, but I see it on the textile stuff too, you know? And I, I think- I've always known this, but I, I kind- it kind of dawned on me over the last couple weeks here, too, like, we're prompting in English, you know? A language that us as humans struggle to communicate with each other sometimes, and, like, understand the actual intent and meaning of what we're saying. And so that communication to the AI, like, sure, I know we're getting more and more to, like, do agent loop, and the agent figures it out, and you just give it kind of that initial context. But even seen weird, like, translation issues when AI talks to AI. I don't know if Brendan: Yep Baird: that before, too. Like, if you go back and forth with it, can struggle with stuff as well. And, and so it's just a really weird state we're in, even though, like, things have gotten so much better. Brendan: Yep. Yeah, no, I, I, I mean, I do agent stuff, but, like, I, I guess I haven't really paid attention to them talking to each other, but I could totally see where that would be a problem. Baird: Yeah Brendan: Um, yeah, I was trying to think actually. There, there might have been one time when I did run into something weird like that, but… Oh, actually that, th- this might not be exactly the same thing, but it is another, like, weird issue I ran into one time. And, and, like, C- Claude Cowork, for as good as it is, like I've… I, I forget if I've mentioned it on this podcast or my other podcast, but, like, I've kinda moved away from Claude Cowork because, A, it felt like it was a little too heavy as a, as a piece of software. For, like, most of the stuff I wanted to do, I could just do it in Claude, you know, Code Baird: Yep. Brendan: in, in the terminal. Baird: Yeah Brendan: But back to the thing I ran into, I was parsing… What was it? It was like- I think it was, it was either I was parsing files. Basically, I had like 10 different things I wanted it to do. I forget specifically what it was. Maybe it'll come to me, but it-- That's kind of the irrelevant part because it was whatever I wanted it to do, I wanted it to do 10 different times, and I told it, like… Or that's what it was. I remember now. It was Rollbar. I might have ac-actually talked about this, but I said, "Okay, go find, like spin up sub-agents to do 10 different… Find 10 different Rollbar errors," or something like that, and summarize them and all this different stuff. And then it came back, and it c- it was like, "Yeah, I'm done. I'm g- Everything's good." And like th-there was something weird about it, I forget. Again, it's been a while since this happened, but I just remember it being wrong, and then I pushed back on it, and it was like, "Oh, well, that's actually because I didn't spin up extra sub-agents. I just did it all myself and then just told you I did it." Baird: Yep Brendan: And I'm just like, "Well, at least you told me, I guess." But, like, but, but to the… Baird: in the end Brendan: Yeah, and but, I mean, that alone is like another example of s- you know, it plays into the automation thing, right? Like if I had just let it go and it just did that, then, you know, again, like hallucinations have been around forever, and we still run into them. But like this was another level of it where it's like, "Yeah, sure, I'll spin it up." And I think it did spin up like three of the Baird: Sure. Brendan: f- or maybe it was five of the 10 or something. It, it spun up a few, but then it got to some point where it's like, "No, no, I can just do it myself, and I'll take care of it." It totally didn't. Baird: He's got a hero complex. Brendan: Right. Right. I don't, I don't need those other sub-agents. Yeah. But yeah, I mean, so that, uh, that's a good point too, just like in about, or, or I guess another point about automations are, are make it hard. And like I've s- found so many times too with like cowork scheduled jobs, Baird: Mm-hmm. Brendan: this is a little bit tangential, where it's like is it the AI or is it just the software or the harness? But like, you know, I will get the cowork scheduled job running, and then a week later or less, like I will go to check the, the output, and then it's like, oh, it failed the last five times. And, you know, did it tell me? Probably, and I just ignored it, whatever. But like either way, it does feel like you have to babysit it a lot more, and that's a li- again, a little less about the AI, but it is a, it's, it's a AI harness, so Baird: But, but it, it's actually a good point too, because, you know, more and more we're moving into agent loops are the solution to things. And, and, you know, couple weeks ago, a month ago, it was like agent harnesses are the thing that you care about. Now, it's agent loop. Like, there's always the buzzwords and stuff like that. And, you know, I, I've definitely done agent stuff, but even just today, like, I'm working on migrating one website from, you know, to Sanity CMS. Relatively straightforward, like, and I've actually done this process before, where you tell the AI to essentially, like, grab the model, like the content-type model from Contentful, pull it down, and then do a full It, it's automation essentially. But it's Brendan: Yep Baird: you don't have to write all of, like, the transforms. You don't have to write all that type of stuff. And it's relatively straightforward. I ran the whole process. It, you know, did everything, tells me it's all good. But then it's like I had to go through and find a bunch of, like, little errors where it just A- and a lot of it's context. You know, like, it can't pay attention to everything in context, so it sees stuff. And if you point it to the little stuff, it'll eventually find it. But when you're doing, like, big chunks of things, it misses pieces, you know? A- and Brendan: Yep Baird: agent loop might find it eventually, but that's the part that concerns me of just being like, "Okay, AI, you go off and figure it out. Just keep going and going and going until you figure it out." It's like I've seen too much of it, like, when I've let it do it in other contexts where it just goes off the rips, you know? Like, you Brendan: Yep Baird: it if it's gone wrong if it's just constantly running trying to figure stuff out. And it, just one of those, because of the things that we're talking about here, I'm still wary of just being like, "AI, do all of it." Like, do I say yes way more than I probably should sometimes? Yeah, absolutely. But I always go back in and check things, you know, and I always dig into things and see, like, what really happened. I found, like, trying to say no and trying to steer too early leads to a lot of friction that way, 'cause then you're just babysitting stuff. So, you gotta find that middle ground. And I just haven't seen enough where I'm like agent loops are solution, you know? Like, I Brendan: Yep Baird: are times where it might be, and I think it, it's definitely something where it can be interesting, but I just think that We're just so concerned about, like, productivity and, like, you know, you got the whole token maxing stuff going on right now where it's just like, "How many tokens can I burn?" Which is great for, you know, AI labs who are selling you the model and wanna make money. Like, of course, they're gonna want you to run it constantly. But Brendan: Yep Baird: I found that, like, I'm, I have c- like, you know, significantly increased my productivity, still ha- you know, playing a very big role in what actually happens, you know? Like, the AI does a lot of the work, but I'm still, like, very much into what's actually happening. Otherwise, it can go wrong really quickly Brendan: Yeah. And, and that's about where I'm at too. It's like I, I, I don't even have a desire yet to have it fully automated anyways because I, because I don't trust it and because of a lot of the things we've been talking about. But be- also because I am already way more productive by having it do, do the, the tedious things at, at a minimum, let alone like writing code. You know? It's like there, there's so many of the, the command center things that I have going on that save me time there. And, and then, you know, like I've got several different things now where it's like I'm using AI to write some code that basically interacts with an API, and then once, once that code's written, I don't really need the AI anymore. Sometimes I will still have AI just like run the same script, and mostly I do that so that it can be in the loop and know like what the output is if there is an issue. Baird: Yep Brendan: But at that point it gets around the deterministic problem because I've already written a script that doesn't need AI, but like AI's kinda helping me. Baird: For Brendan: But I, I'm still not letting it completely do it on its own. And, but, but I'm also like way more productive, so like- Baird: Yeah. No, a- and I found that too. Like, even just the image stuff I was talking about. Like, a lot of the stuff I built, I actually, like, built software to help with the automation, you know? And it was, like, Brendan: Yep Baird: UI and everything. And in the past I wouldn't have done it because it would've taken, you know, a week just to build the software piece, and it took me, like, two hours, you know? And a Brendan: Right Baird: two hours was, like, I would build something, I would actually test it and try to go through the process. I'd find something that didn't work, and then I'd extend it. Or I would, you know, be like, "Oh, what if I could do this feature on top of it?" You know? And that's actually a part that I really enjoy personally, is, like, iteration on things, you know? Instead of it Brendan: Yep Baird: "Okay, AI, you go iterate and finish it," I like having that involvement of, like, truly crafting something. I just don't have to write every line of code to get there, which has changed things for the better for me personally Brendan: Yeah. Yeah, it's funny. I, I agree with that too. But it, but it's funny though because I was listening to, uh, a different podcast, um, Remote Ruby, like just this past week, or it was this week I guess, and those guys are still, you know, they're… I think they're using AI stuff, but like they're still having discussions around like, you know, controller level issues or like rails issues, and I, I did kind of find myself being like, "I kind of miss that stuff a little bit." Like, it's awesome to be able to have the code completely written by the AI. Baird: Yep Brendan: There, there was like something about it where I was just kind of like, "Yeah, but, uh, like if I let the AI do all of it, then I'm not gonna be digging in and like figuring that stuff out." And it's like both good and bad. Baird: Oh, for sure. No, a- and I think that's, from my pers-… I mean, this is just AI in general and, but it gets to your point. It's like, the AI does everything, it's like, what's even the point? You know? Like, it, it Brendan: Mm-hmm. Baird: it loses, and, like, that's just a personal thing. Like, I get the point is to make money and to build businesses, and, like, yeah, that's awesome, but I enjoy the process, you know? Like, I- for me it is, and you touched on it too, it's like, how do you get rid of the boring stuff and focus on the high value and the stuff, where you can truly, like, make your vision come to life Brendan: Yep Baird: needing, like… A- and I think I've touched on this before, like, reducing the activation energy has been one of the biggest things of using AI for me, is just, like, in the past it was like, "Okay, I wanna do this, but I've got to install all the boilerplate. I've got to set up the packages. I've got to, you know, hook up the database, do all the crud stuff." And, like, now that's, like, minutes worth of work. You know, it's like, okay, go and do all this stuff. You've got a project that's up and running, and then you can go from there. And, Brendan: Yep Baird: really still the most exciting… And, like, this isn't anything new, obviously. Like, this, we've been doing this for, like you say, probably two years at this point. But it's still the most exciting part for me, is that getting things out that I want, that are, like, in my head, that I know I wouldn't build if I didn't have the AI, and, like, enabling that has just been such a huge improvement in how I've been able to work over the last couple years Brendan: Yeah, I totally agree. Um, and this feels like maybe a good place to kind of start wrapping up, I guess, you know, f- uh, for me, kind of in summary, and then I'll pass it back to you. But like, you know, I, I feel like e- everything you just said makes sense. I, I think overall we're in a awesome place. I'm still loving it. I love using it. I constantly have to remind myself that, like, we are in, what, the like 1% or less of people that are using it in this way. Baird: Yep, Brendan: so sh- you know, anybody else that isn't using it probably wouldn't run into these… I mean, they would certainly run into these things eventually, but like they're gonna have like a lower bar, right? And so we're pushing things, and so it- it's cool to be in the position we're in where we are f- running into some of these barriers, but like kind of know how to, to navigate them, right? So overall, I think it's great, um, to, to have it and I think, um, yeah, I, I think I can kind of just leave it there. I'll, I'll hand it back over to you to, to kind of summarize where you're at Baird: No, I, I agree. I, I think it's… I've never been a person who's liked hype, and so I always have this, like, dichotomy of things get too hyped up, it almost, like, makes me not want to do it. Brendan: Yeah Baird: time, and, like, this is going back and this is a bit of a tangent, but it makes sense. Like, a couple years ago when we were in the metaverse blockchain era, I never got on board 'cause I never saw the value, all the hype. Where with AI, like, there's the hype, but I still see the value, so I've stuck with it and really, like, it has definitely improved things Brendan: Yep Baird: So yeah, I'm still excited about it, but it's been an interesting experience of like, where are those roadblocks? You know? It's something that I like to be real about it and not just think like, "Oh, it's gonna be perfect," but where are those pitfalls so you know how to get around them Brendan: Yeah, exactly. All right, cool. Well, since you've got kids trying to break the door down on you, Baird: Yep Brendan: a good time to wrap up. Um, I'll just add that, you know, o- one thing that I've been thinking about and, or I guess we've been thinking about, we've talked about a little bit, haven't really made it official, but I'll just kinda throw it out there. Like, you know, where we're at does feel like we could bring our knowledge to other people. So if anybody out there is looking for help, you know, some consulting, you know, we, we are considering that as an option for, to, to help people. Um, but also we- we're, we're very busy with other things, so it- it's kinda like, let's see if there's some demand. So I just wanna throw it out there as we're kinda checking in, like that is something we both are, are interested in helping people with. Um, you know, feel free to, to reach out. You can check in the show notes. We'll have a link to our website and other places you can contact us. Um, yeah. And also, if you're liking what we're putting out here, you know, give us five-star ratings. You know, give us thumbs up on YouTube, five-star rating, I guess, in Apple Podcasts or wherever you get your podcasts. And, uh, yeah, we'll, uh, talk to you next time