PodRocket - Russ Miles - AUDIO EDIT
===

Noel: [00:00:00] Hello and welcome to Pod Rocket Web Development Podcast, brought to you by Log Rocket. I'm Noel, and today I'm here with Russ Miles. ~Uh, ~he's a technical product owner. Uh, the internal platform at Clear Bank, ~uh,~ is the author of 10 books,~ uh,~ most of them published by O'Reilly Media Here to talk about how developer platforms fail and how yours won't.

How's it going, Rus?

Russ: It's going wonderfully ~and, uh, apologies again for dragging you out on an early on a Friday morning, whereas I'm looking at mid Friday afternoon and feeling a little more relaxed.~

Noel: ~that's okay. It's all good. It's all good. I, uh, you know, I'm, it's, it's a good, it's a good morning coffee substitute. It's like, okay,~

Russ: ~You say that, but I could be so relaxed. I could, I could just bring your Friday down a notch.~

Noel: ~I, I, I dunno. I'll try to, I'll try to coast right where I'm at and we'll, we'll see where, we'll see where we wind up. Um. Let's talk a little bit, let's kind of, ~ let's set some terms first when we're talking about developer platforms and platform engineering. What are we talking about?

Russ: We are talking about primarily delivering a experience to engineering teams where they can,~ uh,~ produce, if you like, flow of value. ~Um, ~and so what you're doing is you're spending a lot of your time asking what is getting in the way of that flow you are asking, because one way of looking at it, the way I tend to look at it is that,~ um,~ what a platform is there to do is to support a [00:01:00] habitat.

This is a term that Richard p Gabriel uses,~ um,~ in some of his work. And I love it because it's not a factory, it's not pretending that we are just trying to produce, we're actually trying to create a habitat that supports how people need to work when they're creating software or creating value through software is a better way of looking at it.

And so. Because that is an incredibly broad remit. A platform, an internal developer platform is often composed of some thwarts, some things you always expect to see. And then it can be really interesting at the edge cases. It can be fascinating, the things that you try to innovate around. But yeah, the platform is there.

And actually I, I,~ And actually I, ~I, the reason I really like the word platform. Is that idea of it being lifting something up, you're trying to increase the flow. You're trying to help people do more with their. Incredible [00:02:00] skills. Their attention, their lives really, if you want to get dramatic about it, that's what a platform's supposed to be there to do.

And of course, there's the patterns that come out, team topologies that also amplify and extend these sorts of things and say, this is what a platform team is there to produce. ~Um, ~one of the things that isn't as obvious, I think in much of the literature is that it's a product proposition. When it's done really well.

So you look at all of these different pieces of that experience that are trying to enable this flow, and you are trying to distill a really compelling set of products from it. And you take a product philosophy about how you bring those different capabilities and that innovation to the teams, which again, is an area that,~ um,~ can really hinder,~ can,~ can be one of those things that causes these things to fail.

Noel: ~Mm-hmm. Yeah, I guess I'm, I'm, ~I'm curious ~where, ~where you ~kind of ~draw the line on ~what, like, um, ~what is more just strictly, ~you know, ~developer tooling choices for like an [00:03:00] internal team versus what is the platform? Because I feel like for some people, ~you know. ~There's a slew of make files and that's ~kind of ~all you got.

And like there, there's not a ton of ex, like in, in like at minimum, what, when do you think that there is like an internal platform? How do you define that? ~Like what, where,~ where is that line in the ~like ~this is just the tool we use versus this is our platform.

Russ: It is a really great question because I tend to be very clear in my talks that you've already got a platform. If you are delivering software, if your company is doing that and seeing value from that exercise, then you already have a platform. But what you've got is a platform that will have evolved naturally,~ uh,~ possibly in disjointed multifarious ways.

~Um, ~and what you'll be missing is potentially the economies of scale. Argument that can go with it. You are missing the opportunity to cross pollinate and learn from,~ uh,~ how other teams use these things. And so ~if you've got, ~if [00:04:00] you've got an incredible cottage industry of everyone ~kind of ~doing it their own way, then you miss that opportunity to optimize how people can move from team to team.

For example, onboarding is interesting and, ~uh. ~Innovative for every single team. 'cause it's slightly different. ~ um,~ so it's kind of a scaling pattern to say that ~you're, ~you're moving from a platform that you already have that could be. It's maybe optimal in the pockets. It's not optimal at the large.

And frequently I will see that,~ um,~ companies get to a certain size when they go. I think we can benefit from a platform as a product now. They don't often say that last bit unfortunately, but they will say, I think we need a platform now. And then there's me going, you already got one. ~Um, ~but yeah, to take a product view on it is basically saying we are going to treat it as a product.

We're going to try and make it consistent where that consistency is beneficial to the teams [00:05:00] and we're going to,~ um,~ potentially. Look at where it should be customized and different in different areas. We're gonna, context is gonna be a very important part ~of the, ~of the game, but we are going to focus on building it as a set of products.

We're gonna take a product approach ~to, ~to this piece of the puzzle that we thought was just tools. We thought it was just ways of working, but actually when we view it as a product set that we can tune. To get these bigger benefits from then we're at that size, we're at that step of maturity where that just makes sense.

Noel: ~Yeah, I think I just, anecdotally~ I've spent most of my time at smaller companies, smaller firms, wherein it, it always feels to me like ~we end, ~we end up with something that I think one would define as like the developer platform internally, but I don't, it, it feels like there was seldom. An intentional decision to ~like ~go out and build this thing.

Kinda ~like, ~like you're saying, and it's more ~a as we as,~ as we, ~you know, ~plot along, someone's like, Hey. This is like a thing that's helpful for us. We should like, [00:06:00] make this part of the tool chain that we use for, ~you know, ~whatever builds or deploys or, ~you know, whatever, publishing,~ whatever it may be. ~Um, ~and it ~kind of, it kind of ~has this just like organic thing and certain pieces of it end up being used by certain subsets of the organization, but maybe I've just been fortunate and then I've never felt that there's like ~a ton of, ~a ton of strife there.

And you mentioned that ~like, ~like you said. ~We're all, ~we're all on a platform to some extent, but often you're not the one that chose to ~like ~use the specific tools that end up being this platform. ~Do, ~do you, I guess ~have, ~have you experienced that all or do you think it, do you think it at larger organizations, it ends up just ~kind of ~inherently being more prescriptive and organized.

Russ: ~It's, ~it's a really interesting point you make about ~the, ~the sorts of context you work in and what that might mean to ~whether, ~whether a platform's even recognized or how it gets tuned. ~Um, ~I'm ~very, ~very fortunate in my career. Almost from the first day of it was I was working on developer tooling and platforms.

~Um, ~back then I, it was [00:07:00] nothing glamorous. I should point out, I was working in configuration management ~was the, ~was the job I was doing and it, and that was because it was so painful. To get a version of the code on somebody's machine that was accurate and working that there were people like me who were building systems on top of SCCS and other UNIX tools for version control.

And then there were these things called as it build stacks and things like you could, lots of different ways that code could be managed so that a team could work together on some things. And ever since then. For one, in one way or another. I've been working on, ~you know, how you, ~how you craft these platforms where commonality is useful, where difference is important and what I've seen there is these moments where a,~ um,~ consistency in one area is good because it's no longer the most innovative area.

It's like, be different when different is valuable. ~And, um,~ and that is context sensitive, but in some [00:08:00] things it's really obvious. ~Um, ~a good example is if you have all of your teams debugging your pipelines, and is that really valuable? ~Uh, ~is that really what you want to be encouraging? ~Um, ~or do you want them to be doing something else with their attention and their abilities?

And it's then that you might go, okay, some standardization will take some pain away here. ~Um. ~There's a phrase that's used a lot in the literature around platform engineering, which I think is important, is that we're there to optimize the cognitive burden on software developers.

Noel: ~Mm-hmm.~

Russ: And it's soft and misunderstood as we're trying to reduce cognitive load on the developers, and we are not really, it's much more important to think of it as a tuning exercise rather than a complete reduction.

And it comes back to something I learned when I was working on. The spring framework, and I was working in SpringSource and Interface 21 with the teams there. One of the things that was constantly on their [00:09:00] minds that I could see was, what are we asking people to hold in their heads? What are we, what are they able to do really well because we are, we're saying to this other thing, you don't need to worry about it so much. And the balancing act of allowing the right thing to be easy. And by easy we mean low cognitive

overhead and ~you know, ~no le less friction in doing it. Making that really easy and then making other things that are acceptable in certain contexts possible, and not just possible, but again, probably reasonably low friction was a balancing act ~that, um.~

That we bring to platforms. We're not trying to hide away the complexity of platforms so that they become surprising to you. ~Uh, ~'cause you obviously you have runtime facets to a platform, build time facets to the platform. And then of course, developer tooling, local development as an experience as.

Coupling all that together is what we do in platform engineering, internal developer, platform engineering. It's [00:10:00] all of that. And what we're constantly asking ourselves, well, I'm always asking myself is, ~are we taking,~ are we hiding something that actually increases friction

or are we,~ uh,~ or are we taking something away from that moment where the person will want their head in a different space anyway.

Noel: ~Mm. ~

Russ: And the interesting thing,~ uh,~ just to share with you is that the majority of the big gains that I've seen in platform engineering tend to always come back to ways of working. ~They, ~they tend to come back to ~how do we, um,~ how do we make decisions as development teams?

we have the cognitive space to do so?

Can you know, it's thinking about that is, makes a big difference. There's a lot of emphasis at the moment on AI assisted coding and all of the, and the five coding community and everything else. ~And I think the, I ~I find the, um, the storm around it interesting because really the question isn't can we produce code in larger quantities or,~ uh,~ even faster?

It's, can we make good decisions? About what we're producing. ~Um, ~and that doesn't come back to [00:11:00] can you write code fast? It comes back to usually where are your feedback loops when you've made a mistake.

Noel: Yeah. Yeah. ~Mm-hmm.~

Russ: And so, you know, amount of times we've worked with teams where we've gone, your feedback loops are right at the end of everything you've done.

They're way over here when you've deployed to production or they're way over here in staging, or they're way over here in your pipelines. Could we shift that into local dev? Can you see that when you are coding and. I'm not alone in this perspective. One of the people I respect hugely in this industry ~is, ~is Rod Johnson, the creator of the Spring Framework and the spring framework only existed.

Apologies to anyone listening who's going Spring framework spring, what? 'cause if you haven't come from a Java background, you may not have touched it, but it. What it did was a really simple value proposition that got completely lost to some degree in everything else it could do. The value proposition it did.

The thing that I believe,~ uh,~ from talking to Rod in the past is, was the reason he created it is he wanted to do TDD and doing it in the enterprise flavors of Java at the [00:12:00] time was almost impossible.

Noel: Yeah.

Russ: The feedback cycles were super slow. And so what he said is, can I build the confidence in the code I'm writing right now in front of me that everything else can be slow.

It can take time to get to production, it can take time to go through approvals. It can get time to load up into containers and stuff down the line,

Noel: ~Mm-hmm.~

Russ: but can I get those feedback loops right now that make me feel I've done it right? And everything down from here is probably just mechanism.

Noel: Yeah.

Russ: ~Um, ~so yeah, a lot of the.

Feedback loop tuning, and I, on stage, I call these things ood loops. I say design your platform with ood loops in mind. When I'm doing the training course, I talk about two or three things, OOD loops, habits. What thing habits do you want to encourage? Those ood loops are something you can do that with. It's saying, can we help to establish the confidence and decision making loops as close to the developer's moment? Rather than pushing it out into the periphery [00:13:00] where we're waiting and it's coming back later on to ask us to context switch to something else.

Noel: Yeah. ~Right, right, right. I feel like this, like inherently, then again, you're kind of, kind of, um, ~like you said, this like kind of ~lack, ~lack of surprises as things go down the pipeline ~ like having, um,~ having a good grasp on why something has broken as it has gone through. ~Um, I feel like that does seem, ~that does seem like ~kind of the, the, ~the crux of productivity here, but ~I'm, ~I'm curious how one balances that ~ or I guess your perspective on how one balances that, ~because ~it, ~it almost feels to me like the pipeline will be the least surprising if you are familiar with what it's doing.

Right? Like as a developer, if you're familiar with the internals and what it's trying to do. But I think at that point then. You are in this mode where like the entire engineering team ~has kind of has this um. Like they,~ it is part of their cognitive load all the time of what the pipeline itself ~is going like ~is doing, like they are probably contributing to it or looking into it or debugging it.

Like it feels to me like inherently there'll be the most empathy there if everyone ~kind of ~knows what's going on. How do you like balance that? Like we want to abstract this and simplify it [00:14:00] so you don't have to know, but when something breaks it's like easy to go in and. Figure out, ~you know, ~why whatever you've just done has caused a new problem that wasn't there before.

Russ: ~It's, ~it's a great question again. ~Um, ~so one of the things we, I do is I use a,~ um,~ concept, ~I suppose, ~of development time thinking, and then operational thinking. So not to say that that's two different people because we're not trying to go back to Dev and s, we're trying to keep that responsibility where the feedback loop is tightest. ~Um, ~but. Something I used to say a lot to folks is shift left is sounds great in principle, but you are, if you are shifting left with no help, support with the decision making processes, you're shifting left, then what you're doing is not shifting left. You are dumping left. ~Um, ~so there is a very real danger that if you don't understand the things that people want their heads to be thinking about at a given moment in time, then you could shift all the concerns of pipelines left.

With no [00:15:00] support and no help. And just say, right now you've gotta think about all this because you're now in a, one way of thinking of this is you are now in an arms race with the pipeline. Okay? ~And, ~and that doesn't help, right? ~That does, ~that doesn't help with things. So what, what.~ what, ~I tend to do is very closely ask and interact with, and we use the DX tool as well to survey this and get numbers on it as well.

~I, ~I go and ~I, ~I try to get as close to the lived developer experience as possible.

Noel: ~Mm.~

Russ: And then begin to solve what is in their head when they're trying to do the work they're doing. And yes, I will shift left. We will bring things and we'll bring that awareness of what the pipeline's looking for. 'cause the pipeline is just a mechanism.

It is just a technical choice that we feel the best place to, to surface Certain things happens after you've finishing developing. ~I, ~I love that as a question by the way. It's just to say to people, when would you want to know about this? Do you want to know about it now or much [00:16:00] later? And,~ And, ~and usually it's not later.

Okay. Don't tell me when I've done my work that my work isn't right.

Noel: ~Mm-hmm.~

Russ: ~Um, ~in fact, that was how TDD originally was introduced to me, and it wasn't really as a TDD exercise. It was,~ um,~ I remember I was developing some code at the Financial Times and incredible boss there who will have to forgive me, I cannot remember his name off the top of my head, but I had the best engineering manager because I finished my code.

And I did the usual thing of the time, which is throw it over the wall and wait for testers to go tell me it was wrong. ~Uh, ~I thought it was right. It was classic. Worked on my machine kind of mentality. And,~ uh,~ I remember doing this at the ~end of, ~end of the day, I think it was only ~a, ~a Thursday. And I remember a boss just telling around and going, are you sure it works?

And I was like, I, yeah, absolutely. I wouldn't be saying it's done if I wasn't sure it works. He said, are you sure? If someone was to say to you that, ~you know, your, ~your home will be bombed ~if, if your, uh,~ if the tester comes back and says It doesn't work, do you feel you could do anything more to be confident?

And I was like. ~I, ~I, he caught me. I was [00:17:00] there till I think 7:00 PM that night, because he's right. ~I, I, ~I had accepted doubts and I accepted those doubts because the pipeline had my back because ~the, ~the tester I had in my back.

Noel: ~Mm-hmm.~

Russ: And so I learned that,~ um,~ the contract between you as a developer and the rest of the world is to be as confident as you can possibly be.

That what you are creating is the right thing. When you are writing it. Any feedback loop that doesn't surface at that moment to help me do the right thing ~is, ~is something of friction and frustration. And so when you look at it that way, there's an awful lot that we've incorporated into our pipelines where we could say, ~you know, ~you're battling the pipeline, just like I was battling the tester.

I was in arms race with the tester at that point. And,~ uh,~ so we, if we put more in the pipeline, we're making the pipeline more complex. We're making it hard to maintain, but we're also,~ um,~ then introducing this toxic relationship with the pipeline.

~ ~So [00:18:00] take it. The first question is, can you move it out the pipeline, move it left.

Sometimes you cannot. Some, a lot of times you can, with work, you can,

Noel: ~Mm-hmm.~

Russ: ~um. ~The other challenge is to try to implement the whole Luda loop.

Noel: ~Hmm.~

Russ: So a good example of this is, say in your pipeline, you surface security vulnerabilities.

Noel: Yeah.

Russ: ~Um, ~and it might even block your PR continuing. It might even block what you're trying to do.

Okay. ~Um, ~that is in OODA language. That's the observant orient covered.

Noel: Okay.

Russ: ~I, ~I'm very specific about the OODA loop of entirety because does it help you make a decision? Does it help you take some action? If it doesn't do those things, then it is naturally frustrating.

Noel: ~Mm-hmm.~

Russ: ~Um, ~now I used to work on fast jets. I used to work ~ um,~ on things like the Joint Strike Fighter and ~uh, ~the Euro Eurofighter is now called the Typhoon.

And one of the things you learn when you are creating the warnings panels, and you are creating the cockpit equipment is don't tell me if I can't do [00:19:00] anything about it. ~You know, ~if it's not pertinent to this moment in time, don't tell me. Which is really all we're saying with developer experience. And tuning this cognitive load.

So again, it's great to move left and to stop that arms race with the pipeline or arms race with production, whichever,~ uh,~ source that information's coming from. But where the real potential, I think, actually in AI assisted coding comes in is where you can make those ood loops complete.

Noel: Mm-hmm.

Russ: you know, don't wait for me to create a PR and A and get A-C-I-C-D build going before you tell me I've gotta make some changes.

That's not helpful. I'm not gonna like you for that, but I am gonna like you if I'm in the IDE and I'm working on a class, or I'm working on a function and. Literally my as assisted coding pair programmer or mob programming crew are turning around going while you're there, this is going to flag something down the

Noel: [00:20:00] Mm.~ Mm. ~

Russ: and, but here's your fix. Now ~here's a, ~here's a good candidate for your fix Now. That's an experience that I've been trying to engineer with the Genie as Ken Beck tends to call it,~ um,~ AI assisted coding. ~And so, ~and go,~ what,~ what could be surfaced then? Not just as, here's a decision to make, but here's an action to take, and then how do we tune it so that action is super close to what you want to hear and do at that time.

Noel: ~Yeah. Yeah. I, I, I like, I, um, I mean, I think, I think I, I, ~I empathize a lot with this idea of ~like, ~yeah,~ like, like ~push things as close to development time, ~you know, ~like ~when, ~when you're in, when you're hands on code as you can. And,~ um, I guess, I think, ~I think for some of these things it's like~ it's like we have, um,~ we can kind of run with your,~ uh,~ I think ~like, um, ~vulnerability or ~like, you know, ~like security kind of tools here.

And I think depending on what it is like. As engineering teams. I think the industry standard, we've ~kind of like ~gotten better at the tooling, just ~kind of ~doing this out of the box for us a little more. It's like, hey, we'll have ~like ~a, ~you know, ~an NPM package scanner or whatever you install a package and ~like ~as part of the NPM install process, you have ~like ~a post step that goes and does a sweep and make sure like nothing's broken, but that [00:21:00] feels very different than ~like, you know, ~some kind of ~like ~CICD tool.

And maybe this is more what you're talking about that's like. Going in and making sure, like testing against some, or use, using some like pen test, like suite that's like running all this automated stuff against your front end trying to break something. ~Um, ~so is that, is I guess, is that what you're talking about?

Like the second half of that being more like automated and just happening all the time while you do local dev?

Russ: Yes. ~I mean, ~I think if it, if the pipeline's silent. You're doing things right. So if the pipeline is fixing things for you, what is really the problem that's going on? Because clearly ~what you, ~what you've actually declared is a human in the loop is not necessary.

Noel: ~Right, right.~

Russ: But for anything, and this is what, this is the sort of question I like to ask ~when I, ~when I hear,~ uh,~ we're gonna introduce a new phase to a pipeline

~ ~because we're gonna check this other thing.

A new plugin, something is

gonna come 

Noel: Yeah. ~Mm-hmm.~

Russ: The question I ask, which it sounds crazy in terms of why isn't everyone asking here is, okay, what's the decision in action that developer's gonna take at that [00:22:00] moment in time? And timing is everything. In cognitive load terms, timing is absolutely everything. ~So.~

~You know, ~if we surface good information and signals from our CICD bill,~ that's,~ that's great, but does it actually help anyone do anything with that? ~Um, ~again, back to my cockpit analogy, if you like the, when I was working on cockpits,~ um,~ one of the things you'd I worked on was 3D audio. I didn't work on 3D audio 'cause they were keen on listening to music and thinking that, ~you know, ~queen was playing in front of them.

~Um, ~I was working on 3D audio 'cause they're trying to list to listen to where a missile is coming

from~ ~and that would help 'em take maneuvers that would be more, more appropriate, faster. Classic oodle loop thinking. ~Right? ~And the oodle loop being something that came out of dog fighting initially. ~Um, so. ~It's the same thing. It's like, when are you raising the signal? When are you, are they? It should be the moment that is closest to when the person needs to make a decision and perform an action of value. And when you look under that lens at [00:23:00] all of the different signals that you are hoping your development teams get, are they getting them at a moment when the team are best?

Tuned to do something about it,~ um,~ sensibly and quickly,

Noel: Yeah.

Russ: and so you might find some things. Yeah. Raising a ticket, sticking it on a backlog, that's the right thing to do. It'll have all the information they need to do it later on, but most of those signals aren't expecting to be dealt with that way.

They're signals where, ~you know, ~for example, the SEC Eng team are going, we're gonna raise this thing and we're gonna expect you to take action soon. And then they'll be shocked when I turn around and say, you've told them there's a problem. You've given 'em a vague reference to some docs to help them with it.

Do you know what's gonna happen next? They'll say,~ well,~ yeah, they're gonna fix it. Like, no, no, no, no. It goes into a backlog somewhere.

Noel: ~Mm-hmm.~

Russ: And they go maybe one day. ~Right. ~And then they wait for you to moan as to when it needs to be done, or maybe the tool, the tooling moans. But we're trying to get away from that nag

Noel: or whatever. ~Yeah, yeah, ~yeah.

Russ: ~yeah, ~yeah. Exactly. So we're trying to, [00:24:00] we're trying to nag wear by definition. ~Is ~is something that hasn't considered the loo well enough.~ well ~Uh, and because we can be sloppy with people's attention,~ uh,~ and their lives sometimes,~ um,~ it's different when someone's in a cockpit and they've gotta make decisions that will save their life immediately.

And everyone isn't aware of the consequences and the importance of those decisions.

Noel: Yeah.

Russ: But most people aren't aware of those that when it comes to their development teams,

Noel: ~Right, right.~

Russ: ~uh, ~even though they're some of the most expensive resources you've got that do incredible work and they are potentially your differentiator in the marketplace, tuning their experience is something we don't really do much of or is quite, it seems very early doors

Noel: Yeah. ~Right, right. ~Yeah.

Russ: yeah.

Noel: So I think, I guess ~what's, what's, ~what's coming to mind for me here is like in, in,~ um,~ in the notes from your talk here, ~I've got like, ~you use this like term Franken platform and ~like ~the bureaucracy of ~like ~the development pipeline. So like ~it's, it's, ~it's hard for me not to see there being kind like two sources of most of this, most of [00:25:00] these steps or components maybe that are ~in, ~in the typical like.

Kind of pipeline development flow. It feels like there are those that are added truly for the sake of improving productivity, like of the development team inherently. And then there's ~like ~the external factors that some other concern in the organization has decided needs to be part of this process.

Like security auditing for something doesn't feel like we're not doing that because we think a developer is more productive. If they're, ~you know, ~worried about these security concerns, it's because ~like, ~we know it's a necessary thing to have this check in place, and ~it, ~it feels like that those, that subset where it's like this isn't being done inherently for productivity at production time sake, but there's some other concern, like those are the ones that I think are always gonna be more difficult~ to like, make, to,~ to make a decision that optimizes on.

~Uh, ~the least amount of developer pain, right? Because it's just like, no, we just, we ~need, ~need this check here.~ Like how, how do you approach those, I guess, ~do you have this kind of dichotomy in your head? Do you approach [00:26:00] these kinds of tools differently or is it the same? Is it the same decision making process regardless of that,~ uh,~ source?

Russ: ~That's such a good question. Uh, they've been all good questions today. Thank you for~

Noel: ~No, of course. I know I'm completely off my outline just 'cause now I'm like picking your brain, but yeah.~

Russ: ~no, and it, and it's, and it's fantastic, right? It's where the best questions arise. Um, so yes. ~Do I have that dichotomy in my head? No. ~Um, ~do I think it exists? Yes. ~Uh, but, ~but,~ uh,~ but no, I don't. So ~this is, ~this is something that I think is,~ um,~ an interesting thing in the software development industry and it goes back to something that's a personal theme of mine, which is the metaphors we use that don't service.

Noel: Okay.

Russ: So the metaphor of productivity,~ um,~ comes from the metaphor of production. Which comes from the industrial age and factories.

Noel: Yeah,

Russ: I don't think software development is a consistent factory based exercise, and even our terminology like pipelines and

Noel: I, yeah, I know. ~We, ~we use all these terms. ~Yeah,~

Russ: Yeah. We use so many terms from.

The factory metaphor, or even the foundry metaphor, I was thinking about this earlier on today. ~Uh, ~foundries,~ uh,~ exist because enormous force is necessary to turn one thing into another, which is, again, if you think about it, not the best metaphor [00:27:00] for what software development is supposed to look and feel like.

~Um, ~so yes, I do. I see this dichotomy. ~Uh, ~everywhere almost. Yes, I see people going. There are security concerns that need to be considered. There are regulatory concerns that

Noel: Right, right. Yeah.

Russ: ~Um, ~there is usually a host of non-functional, and I hate that term by the way, but non-functional things to think about, which are incredibly functional in nature, but they're considered nonfunctional because they're not on a feature list.

Noel: ~Right, right.~

Russ: ~One of the, ~the reason I don't carry ~the metaphor, sorry, not, not the metaphor, ~the dichotomy in my head, or at least it's there, but I put it on the same shelf as suspicious metaphors like factory,

Noel: Yeah.

Russ: is that as a developer, my responsibility is to produce value. Productivity of software is the wrong metaphor for me.

~It's the, ~it's the wrong analogy. ~Um, ~it, I know that Kelsey Hightower, I think, has been talking about this recently, saying The best code you write is the code. You don't have to write at all.

Noel: ~Mm-hmm. right, right.~

Russ: ~just. You know, ~you, there's something that you can do that doesn't require [00:28:00] more lines of code, and we all know that, right?

So we've known that for many, many decades now, that source lines of code, terrible measure of productivity. ~Um, ~and yeah, isn't it really tenacious that thought that we are trying to be productive? And it's a, it's really tenacious~ to, um,~ to boards of companies because they see we have nothing and then we have something so surely.

Something's being built.

~ ~Therefore, to make the building really consistent and factory eyes, it, then we lose any dependence on people. We can use them and we can swap them in and out. And it, it all feels very attractive ~to, ~to somebody who doesn't or has never engaged in the activity of creating any software. And I'm very careful with that. I'm not saying coder. 

Noel: Yeah. 

Russ: saying designer or architect. ~Um, it's a, it's a, ~it's a wonderful creative exercise trying to solve a problem, trying to create something that might, that didn't exist before, [00:29:00] that solves something and adds value to something. But the thing I find tragic is that that's not really what we teach anyone.

Software development. In universities particularly, ~and it's, it's not what we, it, it's,~ I would say that the closest I've seen to understanding this and ~uh, ~and really grappling with it is Dave Farley's,~ uh,~ work on modern software engineering because it, it calibrates it back to what are we here to do?

We're not here to produce code. We're not here to produce anything. We are here to enable value.

Noel: ~Mm-hmm.~

Russ: And to do that,~ we,~ we need a disciplined approach, which is ~the, ~the scientific method. And my favorite explanation of the scientific method is A, people can check it out online. It's Richard Thyman talking about how science works.

It's a brilliant video and it starts with, I've got a guess, and hopefully it's an educated guess.

Noel: ~Right, right. ~Yeah.

Russ: And then you go, okay, what I compute the consequences. And I go and compare the consequences against reality. And does it play out? I look, I use falsification, I use Carl [00:30:00] Popper's, falsification. I go,~ right,~ yeah.

~Is it, ~is it not? Is it doing what I expected or, and is it doing things that go against that? And that's all good learning. And I go back and I change my guess.

Noel: Yeah.

Russ: Okay. That's software engineering. And at no point have I said you code anything

necessarily.~ ~So this is where I think that we. I'm hoping that over time Dave's message is gonna land.

I'm doing my best in my own blog to try and continue to amplify this message,~ uh,~ because when I'm creating platforms, I'm creating platforms to support that work

Noel: Yeah. Mm-hmm.

Russ: and, and more often than not, the think the platform is trying to increase flow of code or flow of, ~you know, ~function points or whatever, and it's flow of value.

Noel: Yeah.

Russ: And when you know that, you go, okay, how do we judge value? ~Well, ~we judge value when it's in someone's hands.

Noel: Yeah. Yeah.

Russ: so you've got to shorten the feedback loops. To the point where someone can, you know, a user developer are taking responsibility for something that's [00:31:00] a long way down the loop

Noel: ~Mm-hmm.~

Russ: And so the more feedback loops we can bake into a platform that help the flow of value,

Noel: ~Mm-hmm.~

Russ: ~uh, ~land well and the flow of experimental value land well and you start to test your guesses the better.

Noel: ~Right, right. Mm-hmm.~ 

Russ: Because the problem is that, as I see it, we've built platforms and we still naturally, because of these metaphors that we've bought into that are still extremely tenacious,~ we,~ we,~ um,~ we're still building factories. That implies that we know exactly what we're building when we're building it,

Noel: ~Mm-hmm.~

Russ: because then it's just gonna go through and think it's gonna be right.

But if you take Dave's approach, Dave, sort of what comes across in the modern software engineering book is it's experimental. We are having to guess, we're working with complexity ~and ~and complicated environments that we're dropping change into and judging value is. Not trivial. So yeah,~ it's,~ it's really important I think to calibrate and if there's one thing we can do as an industry, I think,~ um,~ [00:32:00] is realizing that the feedbacks are everything because we are performing the scientific method when we are trying to create value.

Noel: Yeah.

Russ: If we can let go of thinking code is the point. Thinking that the. That, that all of the technical nuance is the measurable stuff that matters. ~It's, it's, ~it's important,~ um,~ to when you go, when you're an experience working with a great developer. And I can't hold my hand up there, I'm an okay developer as far as I'm concerned, but I've worked with some great ones.

Noel: ~Mm-hmm. Mm-hmm.~

Russ: ~When, ~when you work with a great one, ~um. ~It's stunning to see how they think about software. ~Um, ~I will go back to Rod Johnson on this simply because ~I've, ~I've paired with him a couple of times in my life and they've been the best experiences I've ever had because he has. No respect for the code beyond the fact that it should be useful and should be doing something valuable.

He deletes with impunity. He's,~ he,~ he's resting heavily on TDD,~ uh,~ on there being great tests, unit tests, [00:33:00] integration tests, and accompanied with very fast feedback loops. And, ~you know, ~the version control or Gith GI being the main one, he's leaning into that so hard ~with, and he, ~and he cannot accept if it's not there because he can't be.

The incredible developer and engineer that he is without it.

And so when you see that, you go, oh, that's what it's all about. And everything else then goes under the lens of how do we get this person's guesses of value ~to to, ~to be val validated as quickly as possible.

Noel: Yeah. Yeah.

Russ: ~Um.~

Noel: ~I, um, I, I, I, ~I think that's an interesting insight. So ~I, ~I want to keep, like applying that or ~kind of, you know, um, ~layer that over my question from before, where I guess ~when I, ~when I'm talking about these two types of concerns, I guess in like the pipeline, in the whatever, I know I'll do my best not to use these terms, ~these two, these two, ~these two types of priorities or sources for,~ um,~ concern within our thing we're using to produce value.

Russ: ~Hmm.~

Noel: ~Um, ~it's, it still feels to me ~even, ~even if we're not, [00:34:00] one of them, isn't specifically tuned to writing more lines of code, but like this is a thing where the source of it is the creator of value that's doing ~this, ~this. Thing like I'm watch watching a really good developer just like they're internally motivated to have a good test suite 'cause they know they can make changes quickly and hit run and see the feedback versus an external team coming in and being like, Hey, we have this full like front end integration suite we wanna run all the time.

Or we have this like big security thing we wanna run all the time to make sure there's no regressions here. And I think like those still feel like. If you are the, ~you know, ~the value creator, that's the owner of one that's self implying this versus the organization saying ~like, ~no, we wanna do this just to make sure, just to double check that everything's okay.

Those still feel like they need to be like, handled a little bit differently. Or ~the, ~the implicit relationship as the value creator between those two is always gonna feel a little different.

Russ: ~It, ~it, it does. ~Um, ~but ~it's, ~it's really, I found it really interesting to get in the head ~of a, ~[00:35:00] of a great value creator

Noel: Yeah.

Russ: because one of the things that's on their mind is value doesn't exist in a vacuum.

Noel: ~Mm.~

Russ: It has to land well. In the context and environment that it's gonna land. So it's no good me writing the world's best feature if it lands and is completely

Noel: Rife with security vulnerabilities. Yeah. Yeah.

Russ: ~So, ~so that ~a great, ~a great developer is thinking ~is, ~is able to consider those things and,~ and, ~and integrate and collaborate. ~ ~as part of the value creation and it's there where the tuning of the cognitive load is important is because you are carrying an awareness of several things, and so you want the concrete signals from the experts perhaps on those things to rise at the right times.

Because you wanna create a value, but it's no good you being completely distracted from the code, business code you're trying to write by someone popping in and going, by the way, you've got a library somewhere else in your code base that is a bit insecure. Can you go look [00:36:00] at that instead? No one's trying to create clippy for code bases.

~Right. Um, ~yeah, ~I see, ~I see you are writing an application. Have you thought about your security concerns in your Docker container? ~Um, ~it, we're not trying to achieve that, but a great developer. I think learns to hold an awareness and maintain a focus.

Noel: Yeah.

Russ: And so, um, what you look to do is produce a development environment that allows that focus to shift.

Noel: ~Mm-hmm.~

Russ: And keep that awareness still there. It gets very cockpit like, you know, if you look at ~a, ~a cockpit on a fast jet, it looks like a medley of noise, but it is not. There are things that are very carefully organized to help you focus depending on the thing you're doing. So when you're in a dog fight, there's a thing to look at and

~work with.~ When you are landing, there's another thing to look at and concentrate on, ~and they, ~and they're very carefully crafted and it's the same as a developer. You, your job as a developer is to deliver value into context.

Noel: ~Mm-hmm.~

Russ: And if that context is [00:37:00] surfaced badly, then you are gonna see it as in conflict with what you do. we're trying to surface it in a, an augmenting way

Noel: ~yeah, ~yeah.

Russ: what you are trying to do. So you can flip focus at the right moment. To other things, and you should, and you can know that you're not trying to be surprised later on in the game. ~I mean, ~you mentioned surprise earlier. One of the things that,~ um,~ I think it was Juergen,~ uh,~ Juergen ler,~ um,~ in the spring framework.

He, the principle of Lee Surprise. Very forefront in every piece of code ~I ~I work with there.

Noel: ~Mm-hmm.~

Russ: ~Um, ~so you're trying to not surprise people late. The worst time to tell someone their spring application is poor is when they're trying to launch it into a container.

Noel: Yeah. Yeah.

Russ: So you wanna do is you wanna surprise them.

You don't wanna surprise 'em at all, but you want to surface to them much earlier if you can. And I remember that was one of the big reasons to have all of the plugins in back in the day into Eclipse to help development with Spring. It wasn't the spring itself had a problem. It was just that there were things that could be noticed and [00:38:00] surfaced at development time.

That would be good to know before you've got anywhere near running it.

Noel: ~Right, right. ~Yeah.

Russ: and that's the tuning that we do. So you're definitely not ignoring those things and you're definitely not trying to, this is interesting actually. We mentioned abstractions and simplification earlier. Abstractions don't always simplify,

Noel: Yeah. ~Mm-hmm.~

Russ: something as simpler if I am able to disentangle things in my head and focus on one thing with an awareness of other things. ~Uh, ~Richard Hickey does an amazing talk on this. He talks about,~ uh,~ simplicity and,~ um,~ easiness and simplicity being something you can measure and is consistent across humans.

And easiness is very subjective to the human.

~So, um, ~we can. Tune things so ~that ~that awareness is still there and you can switch your focus to the right place to, to pay your attention. And I do like the, the phrase paying attention. ~Um, ~it's one of the most valuable resources you've got as a company that relies on software, is where [00:39:00] are people paying their attention to?

And I wish people asked that question. When they introduce teams, and I don't mean teams as the groups, I mean teams as the tools. And when you introduce these incredible noise generating engines in the sake of supposed collaboration, and what they do is they arrive with an enormous amount of distraction and attention drainage.

~Um, ~so yeah,~ when,~ when we're looking at our platforms. These are the things we're trying to bake in is how can we help people focus at the right moment, the right time at the right thing, and still have an awareness of other things? Back to being a pilot, you're aware that you've got a plane and you're gonna land it later and you're, but right now you're trying to dodge a missile.

Noel: Yeah. Yeah,~ Yeah. Yeah, yeah, yeah. There's a, ~there's a specific set of the, of concerns at any given moment that,~ uh,~

Russ: There is,~ and,~ and when you look at it that way, it's, it is not noise and it's not badgering and it's not, ~you know, you, ~you've got a re a system that's gonna land into a [00:40:00] regulated environment.

Noel: ~Mm-hmm.~

Russ: That's a privilege,

right? That's a friction and a privilege. Let's be clear. ~Um, ~but ~I don't, ~I don't even like it being characterized as a friction.

It's just that value is gonna be judged a certain way when it lands.

Noel: Yeah.

Russ: doesn't mean that you shouldn't be able to develop and experiment and explore the value in the way that we have to in software engineering.

~Um, ~but ~you are gonna do, you know, ~judging the value is going to be against the mirror of those contextual factors.

Noel: ~Yeah, I guess so. Like, I kind of just, just, we're getting low on time here, but I kind of, ~I wanna ~close this, ~close this loop a little bit on these kind of, again,~ like, um, like, ~like developing, deploying software into a regulated environment. I think it's as ~good a ~good a phrase as anywhere I think. At the end of the day, most Des ~like ~understand that ~I, ~I want a tool that helps me make sure that I'm not like introducing some security vulnerability or breaking a regulation or ~whatever, ~whatever it may be.

But I, I do, I guess anecdotally, time and time again, I always ~see that, ~see ~that ~that process is ~always kind of. I guess~ often seen as like this kind of nuisance, right? Like it's this annoying thing. We've gotta go make this happy. ~And I, I think that it's like there's off, it's, ~it's often because those systems are so particular, there's a lot of [00:41:00] like false positives going on.

~Like ~there's exceptions that need to be added to these rules. ~Is there like an organizational kind of burden here that you, um, a framework for the organizational burden to help determine like how much of a role, like how do, how do we determine, or how do we. ~How do we ensure that the individuals that are saying that this piece needs to be in our pipeline, ~like ~need ~this, ~this security check, this audit thing needs to be there, are actually like doing the stewardship that should be done to make it so ~this, ~this piece of our developer platform doesn't feel like a pain point.

Like how do you ~kind of ~organizationally propose that ~and, and, ~and ~ensure that, ~ensure that at a given entity is like the one. Stewarding their,~ um,~ piece of the pipeline.

Russ: So ~the, ~the framework that I use,~ uh,~ for these is. Every single source of information to help a team deliver value is important. It starts from ~a, ~a position of importance. It's not a nuisance, it's not a nag. It's only a nuisance and nag when it, when we try and deliver it. ~Uh, ~so ~it, it starts, ~it starts with it being,~ um,~ sources of information to help developers make better decisions.

Noel: Yeah.

Russ: Okay? So we start from [00:42:00] there and then we. ~The, ~the sort of framework of questioning I bring is, do we need the feedback loop at all?

Noel: ~Mm-hmm.~

Russ: Start there. ~You know, you, you, ~you want to ensure that there's, ~you know, um, I mean, ~a good example is a recent feedback loop around critical vulnerabilities. We want people to be fixing their critical vulnerabilities within.

Say 10 days,

Noel: Yeah. Yeah. ~Mm-hmm.~

Russ: Okay, so let's, we could start by going, yes, we could check that at CICD level, and we could then graduate it into being a issue or an incident, or maybe we could alert on it. We could. The thing is we've, we haven't asked the first question, which is, how did we get there in the first place?

Noel: Yeah.

Russ: ~You know, ~is there any way we can avoid that? Feedback loop at all. That's the first case. So a ~bit ~bit Sun Sue, at this point we're saying, ~do we, ~do we question whether we have to enter the battle at all?

Noel: right. ~Yeah, yeah, ~yeah.

Russ: ~Um, ~and then, and by asking that question, which doesn't get asked often, but by asking that question,~ we,~ we start to not just shift left, we shift it off the board if we can. And then if we find, okay, no, it is. [00:43:00] At the moment in time, perhaps not able to remove from the board. Okay. ~Um, ~we can use a guardrail to do this. We could use observability to produce such a guardrail. Maybe. ~Uh, ~there are mechanisms we can do to do the o and the O part. We can observe and orient people towards these things.

Noel: Yeah.

Russ: ~Um. ~The question I still ask at that point, which isn't obvious from the a, from the,~ uh,~ the acronym or the initialization,~ uh,~ is that ~when,~

Noel: ~Mm.~

Russ: when do people wanna know this?

Noel: Yeah.

Russ: If you just say we need a N loop, or even if you just say, we need an ooh, then ~um, ~you are not even asking half the question. The important point is when.

When should this be brought to people's attention? When can they prioritize it? When can they do something about it? And then you have to have that attention as a scarce resource. The scarcest resource we have is human attention. So how do we tune this now to bring the right information at the right time to the right [00:44:00] attention?

Of the teams,

Noel: Yeah.

Russ: ~um, ~you notice that I'm still not talking about pipelines. I'm still not talking about ~any, ~any of the other bits and pieces. ~I'm, ~I'm saying there's something valuable that needs to be brought to an engineer's attention. ~Um, and then we, well then, ~then you craft the surfacing. The observation, the orientation, and you cry and then you, so that it at least arises in a helpful place, a non-destructive place, back to your nuisance.

You try to avoid the nuisance,

Noel: They're feeling like a nuisance. Yeah. Yeah,

Russ: the feeling ~and, ~and then the worst. That ~right. ~It then escalates from nuisance to flood, to overwhelm, to burnout, you know, being very, I mean, these things we could smile about, but burnout, no one's smiling about.

Noel: Yeah. ~Mm-hmm.~

Russ: What we're doing is we're then crafting it such, and we'll get it wrong.

It's all experimental, it's all gonna be science, but we'll craft that feedback loop and that OODA loop essentially, that then brings the observation orientation to a place where the teams can find it valuable rather than a nuisance. And that's where DX can be very [00:45:00] useful. So developer experience surveys and things like that.

You can say we, ~you know, ~we've surfaced this to you. What do you think about it? Honestly, what's it doing for you? ~Um, ~and then you can start to look for patterns of decision and action that you could offer them.

Noel: Yeah.

Russ: ~Um, ~and that often comes a little later, but it shouldn't be lost because if we can take the decision and action and we can support the decision in action, ~well ~then we start to have a proper order loop.

That is definitely not a nuisance. Now it's turning into valuable response.

Noel: Yeah.

Russ: And then you can start to ask some really entertaining questions. Like we're back to, can we get it off the table?

~Uh, ~do we need a human decision here?

Noel: Yeah.

Russ: And sometimes you don't and sometimes you really do and that's okay. But we should be asking those questions.

'cause again, the most valuable resource we have in software,~ um,~ value delivery is human attention. And it always has been.

Noel: Yeah. Yeah. Nice. ~Um, ~man, again, I feel like we, we ~kind of ~went, ~we went, ~we went completely off of my,~ uh,~ outline here, but I feel like we,~ uh,~ we covered a lot and ~we, we, ~we touched on some stuff a [00:46:00] little bit ~as we, ~as we ~went, ~went through the talk. ~Um, ~you mentioned ~your, ~your,~ um,~ blog earlier, Russ. Is that like, where can people find, where can people find that?

Russ: ~Uh, ~they can find that on Substack. So I,~ um,~ some time ago I realized I had about 30 years worth of stuff I'd noticed. Noticed and noted down some things were half completed stories. Some things were rules and recipes for how to do things. Some are, ~you know, ~incubatory ideas at best. And so what I've been doing for the last five years is trying to figure out if there's anything good.

That,~ uh,~ compost heap of ideas, and I seem to find a few that are,~ um,~ often prompted by what I notice round and about in the industry. ~Um, ~I have gen AI to thank for a lot of the stuff I'm writing because it does seem to be we should go back to basics in sub circumstances, which is always a lovely thing.

And so, yeah, my, my blog is the software in Sheridan. That's the, ~uh. ~The Greek ancient Greek word for handbook,~ um,~ it was inspired as well because I bought a copy. Of the Epictetus in [00:47:00] Sheridan from 1578. ~Um, ~and I then realized I had to learn lat and possibly Greek to translate it. ~Um, ~but yeah,~ I, I, ~I loved it.

I was reading the title and the title of that book. Just very quickly for you, just a interesting little

tidbit, the title of that book,~ I'm,~ I'm gonna misquote it because it's,~ uh, it, ~it's essentially in Latin. ~Um, ~but it mentions,~ uh,~ PUO, which is a dagger. So that was when they grabbed me first, right? It's called Epic Titi,~ uh,~ in Sheridan.

And then in the next part of the sentence, there's this word puo, and it's big on this front cover. And I was like,~ what,~ what the hell is that word? And it's dagger. And what it is, is the rest of the, the subtitle, if you like, is ~a, ~a dagger for carving out the things you could do ~to, ~to. Live an artfully great life, human life.

Noel: Yeah. ~Mm-hmm.~

Russ: I thought what the subtitle to me is even better than It's a handbook. Literally. It's supposed to be small enough to go in the hand. Okay, great. And it's diminutive, right? So it's small handbook in Ceridian. ~Um, ~but the rest of it is, ~you know, ~a dagger. ~For, ~for the [00:48:00] art of human, of ~living a ~living a human life.

~Well, ~and I'm misquoting it, but it's there. And I just thought that's perfect. And then when I thought I'm gonna start to surface these different bit tidbits of,~ uh,~ lessons I've learned and things we've covered today are in there as well. ~Um, ~yeah, I thought I'd make it the, in Sheridan, the software in Sheridan over on Substack.

Noel: Nice. Yes. ~We'll, ~we'll get a, we'll get a link to the show notes as well for those listening. ~I mean, like, ~I don't know how to spell this Latin word. We'll have a, we'll have a link in the show notes. ~Um, ~but thank you ~for, ~for coming on and,~ uh,~ and chatting with me. Ross, it's been a pleasure. ~Um, ~this was thought provoking again.

I feel like I, this was the best cup of coffee I could have had,~ so,~

Russ: I'm glad I offered that coffee to you ~so early on a Friday,~ and ~uh, ~thank you for inviting me on.

Noel: ~Of course, ~