[00:00] Welcome to episode 63 of Expanding Beyond. Again, we've had a long break and I don't
[00:15] promise we won't have one after this episode again. How are you doing, Monika?
[00:21] I am doing overall well, but we live in interesting times. Yes, and it probably doesn't have all
[00:32] that much to do with LLMs. But maybe as well. There are multiple fires.
[00:40] Yeah, that is true. So yeah, I mean, we have been busy with life and we have been busy with just
[00:51] the events. As far as I'm concerned, there has been a lot going on personally and work-wise. So
[01:00] not all these things are something I can share. So yeah, so that's why the long pause as far as
[01:10] me. Yeah, same here. It wasn't like anything big, but it's just sometimes everything is super busy
[01:17] and stuff happens and then this just falls off the wagon, I guess. Okay, so this will probably be
[01:25] a bit late, but this will be our LLM episode, I guess. I mean, is there something else to talk about
[01:34] these days? Yeah, I don't know. Not so much. That's true. Yeah. So for us at the company, we
[01:42] we dipped our toes in with GitHub Copilot at first because, I don't know, it's you already pay for
[01:50] GitHub. So why not try that? That kind of worked, but it was a bit, I don't know, mediocre.
[01:58] Mm-hmm. Given that it's now owned by Microsoft, it's probably no surprise. But then we tried out
[02:07] Cloud Code and then essentially we gave up on GitHub Copilot. And that's essentially now the tool that we
[02:13] use and personally haven't handwritten a line of code for a while now, actually. Interesting. Yeah.
[02:21] Yeah. The move over has been pretty quick. On the other hand, I'm always keeping an eye on the Cloud
[02:28] status page. And it's amazing how in this day and age, it's like 98% availability is fine.
[02:40] Yeah. Yeah. Yeah. I mean, as a manager for me, it's slightly different.
[02:43] Yeah. So we had something similar. We started with a GitHub Copilot because I mean, we live in the,
[02:52] we don't use GitHub. We use Bitbucket. But we are still very much into the domain of Microsoft for
[03:02] other tools. So that was, of course, the natural move. And I mean, yes, it was great. So there was a huge
[03:13] difference, huge difference between Microsoft Copilot and GitHub Copilot. Microsoft Copilot,
[03:21] this is basically your, your meal of the day, chatbot. You know, it's like, you go there,
[03:30] you ask some questions. The good thing is that it's connected to all your ecosystem. So you can pull
[03:36] information from emails and whatnot. It's still absolute crap. Like how bad it is. I can't even
[03:44] start to describe. Um, GitHub Copilot, uh, instead, uh, was amazing at that point in time. I felt like,
[03:54] okay, this is very often. I was asking stuff to, uh, GitHub Copilot directly. I was like, okay,
[04:01] you know what? Forget about it. And, um, and then someone, uh, thanks Thomas lobbied really,
[04:09] really hard to at least try out cloud code that I was already using for my personal projects. And I knew
[04:17] it was definitely better, even for someone that is, let's say vanilla, like I am. And so as soon as I
[04:26] had the opportunity, I switched to a cloud code and yes, true. You don't have all the models that you
[04:32] have available with, uh, with Copilot, but the hardness, you can already tell there is a difference.
[04:38] And, uh, since then I'm a, I'm a happy girl. What I find it most useful for in my, in my use case is that
[04:48] it, it shortens the time between getting some instruction. Either I have to prepare
[04:55] I don't like I have to learn about something in the code that I'm not familiar with because of a,
[05:02] I don't know, like a meeting or something, uh, or there's a bug to be analyzed. Um, and in the past
[05:09] I had to defer to the engineers, uh, in, in the team. And now instead I can just fire it up, um, and,
[05:18] paste the content of the ticket because we're not allowed to hook the harness into the, not yet,
[05:24] but it's legal is looking into it. Oh, come on. I know. I know. But if you have to be compliant
[05:30] and you have to be especially rigorous, that's the, that's the deals. The thing is that sometimes
[05:37] it might happen that our customers, our users are putting data into, uh, into some tickets because we
[05:45] need, uh, that information to, uh, to debug something and, and that's not data that you can
[05:53] send out. Uh, right. So even if it's the commercial, even if it's the one where we have a special DPA
[06:00] with, uh, Anthropic and whatnot, we're still not allowed. So you have to copy and paste. And, um, but then
[06:08] it gets all the context and it's, uh, so much easier to, again, go from zero to one, uh, rather
[06:15] than just, you know, like waiting and, and, and looking around for, for someone that might have
[06:20] that information or not. I mean, even as, as the engineer in your example, even then as I find that
[06:28] is the actual, even more useful than actually writing the code is when you have, I don't know,
[06:34] I have this epic, it is kind of not super well specified, but on the other hand, as the developer,
[06:41] I'm also not a hundred percent sure what already exists. So I just say, Hey Claude, look at this
[06:48] Jira ticket. Um, and this is your repository. Um, just give me an idea of what's already there and
[06:58] what could be done. And then it's just so much faster. And then you go back
[07:04] and forth a few times and say, Hey, how would we split this up? And then yes, the stories or the
[07:10] text that comes out is always super verbose and no one's going to read it. And then you just,
[07:15] that is very true because it's always so much, but on the other hand, it's still, I mean,
[07:20] the alternative is that it's like you have, no one has looked at the code before you started, right?
[07:25] Mm-hmm.
[07:26] It is already a bit better than that. And for me personally, the barrier to entry is always
[07:33] also so much easier if I just tell it to start and then we're getting into it. And then in the end,
[07:40] I'm probably spending about as much time on it. I would have done before, but it's just so much
[07:46] easier to get started instead of procrastinating and doing, not doing it. Right. Yeah.
[07:51] For me, sort of the big, big thing.
[07:53] A hundred percent. I feel the same. There's that, there's the fact that, I mean, if you discard the
[07:59] fact that every time you're asking something, you're burning, I don't know how many trees,
[08:05] the fact that, so we always said that documenting things was hard because it takes time, right? It
[08:14] takes time. So people don't like to do it particularly. Also documentation gets stale.
[08:21] So it's better to have it as close as possible to the code. But then it's not searchable. There's a whole
[08:28] bunch of stuff in there. Right. And these kinds of tools actually allow you to extract the domain
[08:37] knowledge out of what's in the code, what's been codified by the code. This means theoretically,
[08:45] hopefully that everything that you get out of the, of the code is as up to date as possible.
[08:53] Information are as fresh as they, as they can be. And this is as close to the actual behavior as you
[09:01] can get. So that's also an advantage, right? Because you can just, you fire up a pod and it was like,
[09:07] oh, what can you tell me about this particular feature? And then it turns out something for you.
[09:13] Right. And that is on the spot. You don't have to have it done programmatically, not yet at least.
[09:21] And that's just great. This is something that we didn't have available before.
[09:25] Unless it's too complex and then it just doesn't work. Yep.
[09:30] And then it produces something that sounds correct, but actually isn't.
[09:34] And that's the big risk. It sounds plausible. It sounds plausible, but it tells you things in a way
[09:41] that look like it's certain. So very often we have to, um, we have to exercise the benefit of, uh, uh,
[09:53] sorry that we have to exercise the doubt. I was like, okay, maybe not, but you have to do it consciously.
[10:00] Okay. I'm going to digress, but let's start with this. So one of the things that I always looked for,
[10:08] uh, in, uh, um, and I was also given as an advice when interviewing, I was like, can this person admit
[10:16] that they don't know something? Because if you don't know, and you say that, you know, you're sending
[10:22] me completely off track. How can I trust you if you're not able to, to commit to the fact that,
[10:29] sorry, I don't know about this. Right. Yeah. And that's not something that by definition,
[10:35] LLMs do. So you can get this habit into a human, but in order to have it into an LLM, you have to,
[10:44] again, you have to use it wisely and you have to have maybe some, some lines in your skills.md file,
[10:52] or you have to set the tone of the conversation at the beginning, uh, of the, uh, of the chat.
[10:57] It's like, if you don't know something, let me know. But again, how can you tell that the machine
[11:06] really understands when they don't know something?
[11:09] Yeah. I mean, this is also totally against how these things work, right? They just find the most
[11:15] likely continuation of the text. Right. And if it's something unsure, then it becomes even less.
[11:23] Yes. Yeah. I'm not even sure how reliable it would be to, for it to say, I don't know. Right.
[11:29] Yeah. Yeah.
[11:30] It's like a restriction of the whole technology.
[11:33] Yeah. One of the engineers in my team said something, I mean, it feels obvious, but he has
[11:41] a very good point. It was like, we have been trying to shove these tools into our current work
[11:48] process, the way we currently work. Our ways of working were born out of a specific necessity that
[11:56] humans had, you know, like you have to have dailies because at the very least there's one touch point
[12:03] during the day. We talked about this in the past. If the team communicates well, you don't actually
[12:11] they are just a tool to get you to a certain state. Right. So, or we had code reviews because we wanted to
[12:19] have someone else verifying the code, challenging the idea, or you have, I don't know, like you have
[12:26] linters, you have a bunch of all these things in a team, the way the team behaves, it behaves in a
[12:32] certain way because it was out of the way humans were interacting. In this case, these tools are not
[12:40] are not necessarily useful in this context. So it might be worth rethinking the way we work so that
[12:49] then we build new processes around these tools. I was recently reading something. Yeah.
[12:56] Yeah. That's the thing I noticed, right? If you start with, I don't know, a bigger chunk of work and you
[13:02] go back and forth with the LLM and then it produces sort of the artifact is, hey, here are all the stories
[13:08] that need to be done. Then usually they are so far well specified that basically you just have to
[13:14] point the LLM at the story and it does it. 100%. So the work sort of moves to the stuff before and after
[13:22] more than it currently did. Right. Unless obviously there's these things where it just gets too complex
[13:30] and it just horribly breaks down. But then I'm going to argue for something else. But yeah,
[13:36] but then it is true, right? Then you probably need to work on the other thing no one likes to do is the
[13:42] documentation. Because then what I'm thinking is like, okay, how many of us go down to the assembly
[13:51] code and look that the compilers is actually translating correctly all of the instructions
[13:57] that we are giving in high level language? Yeah. Bear with me. I mean, I know compilers are
[14:02] deterministic, blah, blah, blah, whatever compared to LLMs is slightly different, but still the idea
[14:08] is the same. You don't control moving bits from one register to the other. You are giving instructions
[14:16] at a very high level that then are translated to something that makes sense for the machine that
[14:23] actually runs the code. In this case, then I would argue that there might be a point in which
[14:29] do we really care what's in there as long as it's behaving the way it should behave.
[14:37] I mean, eventually we might get to the point where our verification part is strong enough
[14:43] to do that, right? But if you have the LLM write the code and the tests for it, then it's basically,
[14:48] you have to look over it again, at least as a sort of the current status.
[14:52] I mean, right now, a hundred percent, you have to go over it. But I'm thinking that more than before,
[15:00] it becomes really important defining the behavior that you want to get to. Not like we were talking
[15:07] about specifying well what you want to have in terms of behavior is even more important than before to,
[15:15] in my opinion, to monitor for performance. It doesn't really matter how the code is written,
[15:21] as long as it's performing to a certain degree, that's fine. Yeah. Right. And furthermore, then
[15:29] how fast can I figure out when something is broken? How fast can I recover a bug from an incident?
[15:36] No, like all those things that in our industry are signs of degrading performance. Then at that point,
[15:47] whoever is reviewing the code or writing the code, who cares? How do you write it? Up to you. But right
[15:54] now we're not set up for that. And I mean, you can already tell. It's so, the thing is that it's so easy.
[16:04] It's so easy to delegate. You were saying before, delegating coding to this other thing that does
[16:11] everything that you would do. I mean, to be honest, I did something in two hours the other day, I'm
[16:16] creating some dashboards for me to figure out the classical metrics for my team. I was looking into
[16:25] that and I just bi-coded it with cloud code and in two hours it did what I wanted to do.
[16:31] Yeah.
[16:32] Wrongly, but it did it. Right? So, and I didn't touch a line of code. I just reviewed,
[16:39] I asked for the plan, give me the plan, let me know what you are intent to doing. So like, okay,
[16:44] this doesn't make sense. Let's do it this way, blah, blah, blah. Then I checked. Okay, does the code
[16:49] do what I wanted to do? So I look at the dashboard. Okay. This number doesn't look right. Let me check
[16:54] JIRA, whatever I'm using. So I checked there. That's not correct. Why is this the, why there is
[17:00] this, this crampancy? And then the element was like, oh yeah, because the API is doing this, this
[17:06] and that. And instead it should do this other thing. And I'm like, so it's very easy to delegate
[17:12] completely the work to this, to the, to the machine. But as you said, more than before,
[17:21] it's very important to check for what you want. Is that actually correct?
[17:27] It is currently very easy to also delegate the thinking to the machine, right?
[17:33] It's very hard to always stay on top.
[17:36] Yep. And, uh, our processes will have to, uh, be revisited because at that point, do you even need
[17:46] a review? Who knows? Especially if you count for the fact that you can tell it, or don't know what's
[17:53] your experience, but in my experience, whenever a ticket is written or story is written by LLMs,
[18:00] it's so verbose. There's just so much text everywhere. Same for when it codes, it adds so much
[18:09] stuff on top of what you would be doing. Like if I were to do something, I would make it much leaner.
[18:15] Why? Because there's friction. It costs me energy and, and focus to do those things. So I want to do the
[18:22] bare minimum, but these machines, they're not optimized for that. What they're optimizing for is
[18:27] token consumption. Yeah. It is a constant struggle to only get it to implement what you asked for
[18:34] instead of all the totally unrelated and unlikely edge cases that will just blow up, bloat your code
[18:42] and will make it unmaintainable super quickly. Yeah, that's true.
[18:47] Ah, so what do you end up having? If you apply no control over any of this, you end up having PRs.
[18:54] Recently something like this happened with 20,000 lines of code in a language that nobody really is
[19:01] an expert of because it's more efficient to do this in Python. The machine tells you like, okay,
[19:07] let's do it in Python and, and build some, um, some dependency on top of this for the whole company.
[19:16] And then you're like, oh, what we're going to catch it at Revvue. I was like, who is going to
[19:21] Revvue 20,000 lines of code in a language they're not expert of? It's like, I've been trying to do
[19:27] Revvues in C Sharp. It's very fine. You can't imagine how well that went.
[19:31] It looks good to me territory. Yeah. 20,000 lines. Yeah.
[19:34] So, and yeah, theoretically you could pit, you could pit different, different, uh, tools against each
[19:41] other. That's what you do. Like, you know, there's plenty of stuff like code rabbit, uh, or whatever
[19:47] else is out there. And right now there's another, that's called something like,
[19:51] Mm-hmm .
[19:51] Povo, something like this. Anyways. And it's true. Yes, of course they can catch each other's mistakes
[19:56] and whatnot. But again, then my question, uh, I was talking with a friend of mine. He's also a software
[20:02] engineer today. And, and I was like, then what's the point? Why do you want me there? What am I supposed to
[20:08] be doing? So there's also a philosophical question underlying all of this. It's not just a matter of
[20:14] tools and efficiency, but it's the very nature of what we do and how we do it as humans and as, uh, as
[20:22] engineers. Yeah. I mean, I don't know how true it is, but I always think of this. I mean, assuming this
[20:30] is not all just a bubble and most companies go bankrupt because it's all just too expensive and we are not
[20:37] paying enough. Um, assuming it stays around in some form, it is like, yeah, it is our industrialization
[20:45] point in time, right? Where I don't know, you had carpenters and then suddenly now there's Ikea,
[20:51] right? And the job is just completely different. And I guess we will have to adjust and see where we go.
[20:58] And then, I don't know, maybe there is a space for artisanal coding. You probably need to know the
[21:05] tools, right? I mean, even if you, if you, if you build, someone needs to build the factory and
[21:11] understand how it all works, right? But the day to day work is probably going to change. Yeah.
[21:16] Yeah. But we are now in the middle of it and we don't quite yet know where we end up, right?
[21:22] Oh, you're very much right. So, um, everything is interesting as I said at the very beginning.
[21:29] And, uh, it's interesting also because everything is happening very, very fast and it's not just LLMs.
[21:39] It's, uh, it's a little bit of everything. Like I was talking with a friend of mine. She is, uh,
[21:44] she's an educator and she was telling me how now whenever they had, they have to like,
[21:50] remember when we were in class, it would sit for like, what? 45 minutes, 50 minutes,
[21:55] no breaks, no, nothing. The teacher would speak. They would write something on the,
[22:00] on the wall or on the blackboard. And then, uh, maybe there was some exercise. There were some
[22:05] questions here and there, but you had to stay focused. And now, especially with kids in primary
[22:11] school, they have every 10, 15 minutes, they have to have some sort of like activity of some sort,
[22:17] keeps the kids engaged because otherwise they, they, they cannot follow. They just, they're not there.
[22:24] And I'm not saying this in, in the usual, uh, way of what old people say, oh, my times,
[22:31] it was so much better. Um, my attention span is gone as well. Um, I always had problem with my
[22:40] attention span, but this is a different story. Um, this is a completely different degree, right? So
[22:45] what I'm trying to say here is that everything, or since this is our, well, it's not monthly anymore,
[22:52] but this is our chat. How is it going in your life and, and, and how, uh, your work reflects on,
[22:58] on your life as well. Maybe it's age because I'm 45 at the, this year. I mean, that's, there's that.
[23:04] So there might be some, you know, like natural, uh, change in the way you think, the way you approach
[23:10] problems. I don't know. But in general, I have the impression that decisions, conditions, um,
[23:17] environment, like the whole system is so hyper reactive that it's almost impossible to understand
[23:28] what's the best next move should be career wise, work wise, business wise as well. So can I really
[23:37] blame companies for not knowing what to do? Um, I don't know, given, even, and this was a conversation
[23:46] I was having with a very young, uh, friend of mine these days that is getting into, into the workforce.
[23:54] I was like, nobody knows, remember that they're all making stuff up.
[24:00] Yeah. I mean, that's probably more important to understand today, but it's always been like that.
[24:05] Right. Yeah. But you know, like if you accelerate something, then the effect, a wrong decision are
[24:13] much bigger and much less, um, uh, that's the wrong English word. You cannot course correct.
[24:21] Yeah. Just because it's so fast.
[24:23] Yeah. I mean, we have seen our fair share of outages recently.
[24:27] Mm-hmm .
[24:30] Yes, you can do all those, use all those tools and you're faster, but it doesn't really mean that
[24:36] it's going to be more reliable. And then sort of the, the, the question that I'm asking myself is
[24:42] always, are we just going to accept this new normal of stuff, not being as reliable anymore? Right.
[24:48] Yeah. Or will that just be a phase and go away once we understand how it works?
[24:54] I mean, I have the, my hunch is that it's going to be quite similar to what happened to crypto.
[25:02] At the beginning was like, you know, free for all. Yeah.
[25:06] And in this case it's like, I can, you can see in the eyes of the CEOs out there, it's like, we can do so
[25:14] many more things. This can provide us with so much, like such a big advantage compared to our competitors.
[25:22] It's like, oh, we can be so much faster. Right. So there is this hunger.
[25:25] Yeah. But then everyone can be, right?
[25:26] Mm-hmm .
[25:27] Mm-hmm . So that's the thing. It's, it's not an advantage because everybody has it.
[25:34] Everybody's playing the same game. Yeah.
[25:37] So I'm wondering like at some point when shit is going to really hit the fan, how is that going to
[25:44] look like? So we're going to start holding back a little bit again. We're going to probably again,
[25:50] look at, it's not really safety, but you know, like being a little bit more conservative in, in what we do.
[25:58] Because as you said, these are tools that are extremely useful in the right context.
[26:03] You want to hack up something. You want to do something that, you know, it's like,
[26:08] it's only going to be used by you and a couple of other people in your company.
[26:12] Just fire it up, make it work. That's fine. But putting something in production, what I find
[26:19] extremely interesting is the combination of being fast in executing, also being conservative when it
[26:28] comes to making decisions. So you have this long live decisions that then are implemented super fast.
[26:36] It still takes months to get something out there. But again, and that's what I really loved about the
[26:43] concept of lean and, and of actual agile, and then completely lose the focus of the fact that you can
[26:53] iterate on the solution. Like the whole point of those, all those experiments was not only to be there
[27:02] before your competitors, but also to validate your idea. Am I going in the right direction?
[27:10] Am I really bringing value to my customers, to my users? Am I really adding something that people
[27:16] are willing to pay for rather than just going to ship fast? And then it's the worst decision you could
[27:23] have ever made. Yeah. Yeah. I mean, this is going to be the deciding factor, right? I mean, it comes back to
[27:35] all the same processes still apply and all the same good behaviors still apply. It's just now it's framed a
[27:46] bit differently, right? It's framed, oh, we need to do this because the LLM needs to, needs this, right?
[27:53] Um, so I, I do wonder sort of, maybe not in this exact example, because yes, I mean, it would be cool to,
[28:01] um, just be faster, right? I mean, the LLMs will help you hopefully prototype quicker and then you can
[28:07] throw it out and do the real thing. But what I've noticed is that sort of from, from a developer,
[28:15] developer point of view, it is actually possibly another argument you can bring, right? And saying,
[28:22] hey, we actually need this type of process because otherwise the LLM won't work correctly.
[28:30] Because the more I look at it is like, if a person is doing it, or if the LLM is doing the work, it
[28:37] still needs to be well, well specified. Yes. And all that, it's just, and it, it is all the same in
[28:44] the way, right? We are just, now we have a second consumer and all the same processes apply and nothing
[28:52] essentially has changed in that regard. Yeah. Yeah.
[28:56] It's just lays it bare much quicker if there's a problem. It's not done correctly.
[29:02] Exactly.
[29:05] 100%. So yeah. Interesting times, as I said.
[29:10] All right. Is that it for today then?
[29:13] Maybe one last piece. I don't know if you have been hiring or, uh, if you have
[29:21] looked at the, at the hiring market recently. Not, uh, not personally.
[29:30] Yeah. I mean, I don't know. I've had experiences where I've had interviews and it was just,
[29:36] it took 10 minutes into the interview until I realized whatever question we were asking,
[29:42] the answer didn't come from the person, but from whatever they've put into JetGPT. Interesting.
[29:47] Interesting. Um, so there's that, but otherwise I think, I mean, I'm not doing the hiring right now,
[29:53] but it seems to be super hard to find people at this point. So there's that. So one of the thoughts
[29:59] that I was having around hiring lately was this, because we, we are hiring not for my team, but in
[30:05] another area of the company. And someone, uh, decided that they're, the candidates are not allowed to use AI at
[30:15] all. I do remember this was very, very long time ago when I started looking for a job, uh, in 2009,
[30:25] 2010, there were a couple of interviews in which the recruiter, the, the, the, yeah, the hiring manager
[30:33] was actively encouraging me to like, you're allowed to search on Google. Yeah. Um, cause there were
[30:41] people that the expectation was you should be able not to, uh, so to code, not to, not by looking at
[30:48] Google, right. In search for a solution. On one hand, this is one of the, my pet peeves lately. Like
[30:56] if we are gonna be using these tools, then I would also expect to be able to use them while I interview.
[31:04] So then again, this ties back to, we need to change the way we, uh, we work to adapt and make sure that
[31:14] these tools can be integrated. Cause I want to know how you're using these tools. Otherwise,
[31:19] what am I testing you for? Yeah, that's true. I mean, knowing how to use an LLM is a skill set
[31:26] that you need to learn. That's true. Yeah. And of course, I don't want to select for someone that knows
[31:31] how to ask a couple of questions over, over a chat interface. That's not the point, right?
[31:35] But how do you build your own, how do you navigate a code base that you don't know about? How can you
[31:42] extract that information from there? If you were to write a specification, how would you write it using
[31:47] these tools, right? Rather than, uh, speaking of, or how would you, how would you deal with, um, having to,
[31:56] uh, build a brand new feature? How would you do it? Right? So asking like, can you, can you, uh, recently
[32:05] I had an experience like this. It's like, uh, can you code, um, a, um, a method, a function for
[32:14] extracting duplicates from an array? Like kind of question is that, what are you testing for?
[32:19] I am the, I can't understand the angle because you want to know that I am able to do something
[32:28] that simple, but then, and this is my second, uh, my second, um, argument for, uh, hiring lately,
[32:38] really seriously need to change. Do we really need that? Because again, if we go back to our,
[32:44] what we said at the very beginning, at some point, does it even make sense to understand
[32:50] what's going on inside the code? I'm being extreme here again. Yeah. Yeah. I mean, the,
[32:57] the way I've looked at those is like, this is a super low bar, but there's enough people that can't
[33:04] cross it. Right. Yeah. Although, and then that's why, right. And you can't give it as a, any kind of
[33:10] homework because I mean, that's LLMs now and maybe there's still that. Yeah. I mean, somehow we still
[33:19] need to somehow verify that they more or less know what a, what a code is. Right. I don't know.
[33:30] In the end, right. If our, our job is shifting towards, I don't know, more text and more specifications
[33:38] for LLMs, then I guess hiring would need to evolve there as well, but we don't know what's coming
[33:47] next week by whoever, whatever provider. Right. So it's, it's hard to always adjust.
[33:53] I mean, especially, so we were saying at the very beginning, for me as a manager, it's super useful
[33:59] to use these tools because I can, I can quickly go through someone else's code and figure it out.
[34:06] Um, I can ask for context of a specific PR, for example, being, uh, being created in a language
[34:13] that I'm not an expert of. So it gives me a little bit of a hedge compared to, um, an edge compared to
[34:22] what I, uh, what I was doing before or what I was experiencing before. Or it allows me to quickly
[34:30] hack something, fix a bug and, uh, and put it out there. Then yes, I have to have the critical
[34:37] thinking to go through the plan and look, cause like, okay, this makes sense. This doesn't,
[34:41] maybe I need to add a test for this. Uh, or, um, can we verify that those numbers are actually correct
[34:48] somehow blah, blah, blah. So there's all, all this part. And especially as a manager, when I go into
[34:56] interviews, uh, for, uh, uh, for this kind of stuff, if you look out there now, most of the job posts are
[35:04] like, Oh, but you need to be hands on. Okay. What does that actually mean for you?
[35:10] Yeah. It is because essentially it's the other way around, right? Because the actual managing of
[35:17] things is now becoming more important. Exactly. So I was telling Sarah again, extreme, uh, uh,
[35:26] extreme take here. So very hot take. Um, but I was telling the engineers in my team, I was like,
[35:32] you are all going to become engineering managers in the sense that what you're going to have at your
[35:38] disposal is a certain amount of agents that you can spun and you can tell them to even,
[35:48] you know, like orchestrate between each other and each one does a different thing. Or you have like
[35:55] four or five different windows with different problems open. And while the machine is turning
[36:01] out the solution, you jump to the next one. That's literally what you, what I do on a daily basis.
[36:07] I'm just telling someone like, Hey, we, we need this problem fixed, go and fix it.
[36:11] Mm-hmm . That's what happens. And then I can check back on the state. I'm like, okay,
[36:16] but this has created a problem on production or we need approval from this or the requirements are
[36:21] not correct. That's literally what I do. Um, I, I saw some people very scared after.
[36:30] Yeah, but it is true, right? If, if you really think it through, that's where we will end up or we
[36:42] might end up with.
[36:43] 
[36:43] Where we might end up.
[36:45] Yeah.
[36:46] I, again, as I said, I, I was being extreme in my, uh, in my take. There is, uh, in my opinion,
[36:54] there's definitely a lot of value. I mean, from, from what I'm seeing, uh, I'm, I'm following a
[37:00] bunch of people on YouTube or blogs or newsletters and, and that kind of stuff. And there's a lot
[37:07] that you need to think a bit laterally to get there. Um, because it's not just about optimizing
[37:16] what we're already doing. That's what most of us are doing right now. We're optimizing what we do
[37:21] already. But if you think a little bit outside of the box, you get to do stuff that you weren't,
[37:32] you really weren't. Uh, there's this guy, uh, it's a great, uh, engineering management newsletter,
[37:38] uh, that I'm following. And, um, uh, and, uh, he created a set of skills for, uh, engineering
[37:47] manager work specific for that. So right now we're focusing a lot on, you know, like what,
[37:53] what you need to do to be a good developer using these tools, but can you use these tools to do
[37:58] other things that otherwise you wouldn't? Um, and I'm not just talking about generating a one-on-one,
[38:04] you know, but if you put all of this together, for example, you could have something that tells you
[38:10] on a daily basis, like, oh, there are these many pull requests. This person is showing, uh, some
[38:16] interest in this other area, or they're spending a lot of time working on this rather than that.
[38:22] Then you can prepare an agenda, uh, for, and this is just very basic stuff.
[38:27] Yeah. I mean, that's essentially what our CTO is doing, right? He's sort of taking in
[38:34] all the data from everywhere, right? Our Slack, our GitHub, Jira, and then it sort of generates in the
[38:42] morning, a sort of what's notable type of document, right?
[38:48] Yeah. On the other hand, that looks like a Norwillian nightmare. No, no, I'm not, I'm not
[38:54] dissing your CTO specifically, right? But if you are looking at all this data all of the time, and like,
[39:03] you need air quotes to be controlled in order to really gauge how well the whole system is behaving,
[39:12] and you're even more of a cog. Yeah. Yeah. I mean, that's one way to look at it. The other is like,
[39:21] hey, that person actually needs that information and should know these things. And it's just
[39:27] often not happening, right? The communication isn't happening. So if it, if this is a tool that they
[39:33] can use to actually sort of retrieve this from the day to day work of, hey, certain PRs were merged,
[39:40] certain things were, were done, then maybe that is actually a useful thing. I don't know.
[39:47] Depends on the use case, I guess.
[39:49] Depends on the use case. So yeah, I still stand by what I said in the last episode we recorded on AI,
[39:59] where our jobs are safe. But yeah, we really need to change the way we look and our jobs and the way
[40:08] we live our jobs. And that might not be for everyone. Yes, our jobs are safe in the, on the
[40:14] assumption that this was always a job where you had to always learn and learn about the new,
[40:22] next new tool. Right. And if you look at it like that and say, okay, that's the LLM. And now it isn't
[40:29] just, I don't know, the new programming language or framework, but this is just the new tool to
[40:36] learn about and get acquainted with. Then yes.
[40:39] Yes. Otherwise, yes, probably not, not your job or your find your niche. Right.
[40:46] I mean, because there's always the things that these tools can't do, right. There's always
[40:51] on the edges, there's always other ways of working, but I guess the mainstream is just,
[40:57] I guess in some form this, this will stay, even if I don't know, the most powerful models at some
[41:02] point might be out of reach and you run whatever you can on your machine. But even that is probably,
[41:09] I don't know, a more powerful refactoring tool, if nothing else. Right.
[41:14] Yeah. And a friend of mine had a very good point. We were talking about this and it was like,
[41:22] if you consider anti-economic right now running these models is you can already see that as an
[41:34] amateur, you are in maybe a year or two years, you're going to be left with the older models,
[41:41] the cheaper ones in order to be able to afford them. Right. But the, um, it's going to be probably
[41:49] tools like the, the edge models, the, the, all the new things that are coming in harnesses and so on,
[41:57] they are going to be dedicated to, um, to, uh, professionals. So what looks like, what, what right
[42:06] now looks like it's democratizing this, this word that everybody was using a few years ago, uh, was
[42:12] democratizing programming. I mean, I have my partner, he has created something like, I don't know, it's
[42:18] like 10, 12 applications, games, whatever. And he has absolutely no idea how to code. Yeah. And this is
[42:28] good stuff, right? It's stuff that he's useful for him, that he was never able to find out there because
[42:35] it's niche, um, or it's just something that he does for fun. So amateurs and professionals will,
[42:43] it's not, it's not going to be so democratic in, in a few years from now, because these things are
[42:49] expensive and they don't scale, at least not yet the degree we need them to scale. Yeah. I mean,
[42:57] the question is how economical will it be for companies in the end, once they actually have to pay
[43:04] what it costs. Right. So possibly there's, I mean, I, this is always the pendulum right now, we're on the
[43:11] LLM far end of the swing. And I assume at some point that we will go back and balance out because
[43:20] we should ride the wave now as long as we can, I guess. Yeah. So for our friends out there,
[43:27] I have a recommendation. This is an Italian newspaper that has, um, uh, has a great offering
[43:34] of podcasts as well. And every now and then they have, um, guests that don't speak Italian. So they
[43:41] have the episodes in, in English. And this one is about, um, how AI is, um, is creating another
[43:54] version of the, uh, industrial revolution and how it's changing the relationship between people and
[44:02] their work. It's in English, of course. Um, so I'm gonna, uh, I'm gonna find it and, and share it,
[44:10] uh, with the, in the, in the episode notes, there's a lot to digest there. And it was very interesting to
[44:20] observe this historian, uh, in talking about how detaching people from the meaning of their work
[44:32] and their ability to control what the, what the work was about was what caused the riots, uh, back,
[44:41] uh, back in the days. And out of that, then we got over time, you know, like limitation in working
[44:50] hours and so on and so forth. And his point was like, these people didn't hate the steam, um, uh,
[44:58] the steam engine. They were hating how the steam engine was used to put them in, uh, in a position
[45:07] of, um, uh, of weakness. Yeah. Right. So that would be my recommendation for today.
[45:15] And I mean, that is true, right? So as programmers, we've been in a very strong position until very
[45:22] recently, and that's sort of slowly shifting. Although we probably don't even have to go
[45:27] back as far as that. Yeah. Yeah. I mean, the dot-com bubble afterwards, it was probably
[45:33] completely different. Yeah. But from an economical perspective, absolutely. Yeah. From the, or what I
[45:43] remember, the coming of internet was never, never really put in danger. The, um, the, the, the,
[45:55] the job market. At least that's what I remember. I was much younger, so I might be wrong, but that's
[46:02] what I remember. There was a bubble. So that was what endangered people's livelihood, but it wasn't
[46:10] changing substantially the value, the worth of, of, uh, certain people. I mean, nobody's was run,
[46:19] I assume people out there, correct me if I'm wrong, but I don't remember people going around and, and, uh,
[46:27] having countless meeting to tell people to use Excel, you know, like sometimes I'm like, can we stop?
[46:35] Please. Um, so yeah, so that's my rant for today. All right. Let's start with that. Uh, stop with that
[46:46] then. Yeah. Okay. So no promises from our side that the next episode will be, uh, timely, but maybe this
[46:55] year, who knows, who knows, maybe you're going to get a Christmas gift. Yeah, exactly. All right,
[47:03] everyone, uh, no proper outro from my side. I don't know if you remember Monica, which one I said,
[47:08] but like where, where can people find you? If you want to find me sketching and using watercolor,
[47:16] you can follow me on Mastodon. How about you, Monica? Um, you can find me on LinkedIn, I guess.
[47:25] Are you playing all those games now on LinkedIn? I mean, that's basically the only place I look at
[47:32] every now and then these days. Um, um, write me an email. Okay.
[47:39] Okay. That's better. Yeah. Yeah. Because LinkedIn is the new Facebook. Yes. Yeah. Please. Please. So
[47:46] monica.jambito at gmail.com. That's fine. All right. Um, all right then have a lovely time. Bye. Bye. Bye people.
[48:09] Bye.