NOEL:  Hello and welcome to the Tech Done Right Podcast, Table XI's podcast about building better software, careers, companies and communities. I'm Noel Rappin. If you like the podcast and would like to encourage us to continue, please follow us on Twitter at @Tech_Done_Right or leave a review on iTunes. iTunes reviews really do help new listeners find our show. We also have a new newsletter where you can find interesting stories, podcast, reviews and some mini-essays and rants from me. You can subscribe at http://TechDoneRight.io/Newsletter. Thanks.

Today on Tech Done Right, we're going to be talking about Legacy Code and Code Abstractions with Michael Feathers. Michael, do you want to introduce yourself?

MICHAEL:  Yeah, sure. My name is Michael Feathers. I guess, the thing I'm known best for in the industry is the book, 'Working Effectively with Legacy Code' and I do quite a few other things but I tend to be the go-to guy for legacy code problems across the industry.

NOEL:  I always think of this is being kind of a be-careful-what-you-wish-for that you wrote.

MICHAEL:  Yeah. It really is.

NOEL:  You wrote this book about legacy code and then spent a tremendous amount of time staring at sort of face-meltingly bad projects. Is that an accurate perception?

MICHAEL:  It is but the thing that is funny about it is that I can tell just about everybody that I've seen worse code. When you go around the world and you see all sorts of crazy code you could say, "Well, you know, I've seen worse so don't feel like you're in a completely unexplored territory."

NOEL:  I'm trying to decide whether to give into the temptation to ask you what the worst thing you've ever seen is or to not.

MICHAEL:  I can't. Because of NDA, I can't really sort of broadcast the domain but I've seen some crazy things.

NOEL:  I just picture the end of Raiders of the Lost Ark over and over.

MICHAEL:  Yeah, fair enough.

NOEL:  What other things do you do now? What else are you doing?

MICHAEL:  Well, a lot of consulting around legacy code still. People reach out to me because of various problems that they have. I have two other interests that I'm digging into quite a bit now. One of them is really how we get in the situation in the first place, what are the things that lead to technical debt within organizations and how do we avoid those. The other is I'm working out for a while is writing a book about error handling and what I'm discovering was a broader topic than that. I've been digging into what it takes to make software simpler in terms of complexity. There's often a lot of complexity comes up when you're dealing with errors. Sometimes like normalizing cases, making sure that bad things can't happen is better than going to deal with them once they have happened so I'm in that direction.

NOEL:  That's funny because I definitely see some cases where it's easier to deal with the bad thing than it is to figure out what's causing the bad thing. 

MICHAEL: Yeah
  
NOEL: Those all kind of seem related to me but maybe we can talk about legacy code and talk about how teams can avoid getting into the problem in the first place. In the legacy code book, you define legacy code as code that doesn't have tests. Is that still the definition that you're comfortable with? Has that definition shifted over time?

MICHAEL:  It's funny because I think it's really more... it's a definition with a purpose. Who am I to go and actually change the definition of legacy code? Essentially, the traditional dictionary definition is it's a code you've got from somebody else or from sometime in the past. I came up with that definition of legacy code as code without tests. It really underlined a particular situation and give people a galvanizing direction to go in with what they're working on.
  
It came from the observation that at least at that time, the code bases I'd found that had been easiest to work with, had extensive test coverage. It was very easy to go and make changes in them and understand quickly whether you changed behavior or not. It seemed like it wasn't really a quantitative differences. It was a strong qualitative difference between those two different kinds of code bases, one with tests and one without. I stick by that. I think it's a good working definition to go and give people a direction to move in when they're working on complex code bases.

NOEL:  Do you still see that kind of split where the code with tests still tends to be a lot better than the code that doesn't have tests?

MICHAEL:  It's not that it's better. It's just that it's workable. 

NOEL: Oh, I see.

MICHAEL: It's something you can do something with. Once you have the test coverage, then you're like, "If I do something, to the code is it going to bite back if it doesn't like it? I'm changing behavior in a way I really expect to," and that makes all the difference in the world. Most of the work I do to help people get tests in place has to be, by necessity very focused because it takes a long time to get test coverage. I hope people go and get coverage in the areas where they're about to make changes. It's a tough thing to go and do but it is way better if you had it from the beginning.

NOEL:  A lot of those techniques are covered in the book. What prevents teams from getting in the trouble in the first place? Or maybe put it the other way, what makes a code bases go bad, I guess?

MICHAEL:  It's funny about this too. I have like a love-hate relationship with the term technical debt. The genesis of it came from Ward Cunningham back in 1990s and he was trying to find a way to communicate with people in the organization he was in. He wanted to go and communicate to them how development can be approached differently and strategically in order to achieve certain things. He said, "We can do the simple thing that gets things done right now and then we can come back later and fix it but until we do, it's like we're accumulating debt."

That definition is fine but I think the people that use that term technical debt quite often to go and describe something which is more like entropy in a way, that as you make changes, the code tends to get a little bit messed up and tests become harder to deal with over time. I think that some of that is really just a natural process. I think that if we're very diligent, we can avoid it but when you look at the behavioral incentives quite often, ending up with a sprawl in your code bases, it's pretty much what the incentives lead us to. I like to put things this way: it's easier to add code to an existing method or to create a new method. Right?

NOEL:  Right. It's easier right up until the moment when it would have been easier to create a new method two weeks ago.

MICHAEL:  Exactly. It's like we're human so we basically tend towards the simpler solution and then it catches up with us like most things in life, if you aren't diligent about certain things, that will catch up with you.

NOEL:  Yeah. Technical debt and legacy, both seem to be terms that have useful definitions that tend to in casual conversation both be used as a synonyms for bad. I think the other thing is that one of the things that I find is that building in the kind of structures that are going to help your code in the long term, building those in at the beginning of the project just feels weird. It feels like overkill and it feels like, "Why am I adding all of these objects to this simple workflow."

MICHAEL:  That's true. I guess it's funny because you and I we're talking the abstract here without looking at a particular example but it is possible to over-engineer things from the beginning. But I think that people's opinion of what is over-engineering changes as they develop in the industry. I've seen situations where someone is very new and he sits next to someone who's very experienced and the experienced developer goes and says, "Let's do this and this," and then the newbie who has not really been down this path a number of times before looks at this as being over-engineering.

That can happen and it really comes down to how many times have you basically sort of like painted yourself into a corner, that kind of thing. But having said that, the other thing too is that there are situations where you know you're going to have to go and rework something but doing the simpler thing is good for right now. It's just at some point, you have to make a real shift in your structure. It's good to know that those things happen as opposed to going and just assuming that you'll get the proper structure in place from the very beginning. Quite often, that doesn't work.

NOEL:  Sometimes, it's a case of if I had known this code base was going to live this long, I would have taken better care of it. Is there a way to get around that? What do you see that successful teams do? Is it that they're just better engineers, they have a better plan, do they have better support? Is there a difference between what a successful team looks like over time and an unsuccessful team looks like over time?

MICHAEL:  Are you talking about the code or the team practice some stuff?

NOEL:  Are there practices that encourage better code like by the time you already see the team, it's too late. You can't --

MICHAEL:  Well, there is that thing. Naturally, I get called in when people have trouble. It's like I can see some good code bases with a strong culture around them. It's kind of rare relative to the other things that I see day-to-day. I noticed that the teams that get better are often ones where the central value is basically making sure that everything is going to work well durationally, in the code base. They approach it with a degree of care. They don't really assume they have the answers. They're just constantly looking for, "If we do things this way, will this be better what we've been doing so far," and that becomes a topic of conversation across the team.

NOEL:  So continual improvement?

MICHAEL:  Yeah, continual improvement. I guess, continual improvement in the team domain is really more like we have this conversation over and over again. We don't get tired of it because we just know that there's always something to learn. There always like we reflect upon the things we done and seeing how they have developed over maybe the past year or so or several years. It's like, "Well, we tried this. This wasn't working so well. Let's try this. What happens when we do this?" That kind of thing. That's a topic of conversation across the team.

NOEL:  What are the first things that you do when somebody calls you in and says, "We think we have a problem. We think you can help us."

MICHAEL:  The first thing for me is listening. Pretty much let people do sort of a complete brain dump on me and just tell me everything that's going on, what they see as the problem, history of the code base, history of the people, everything along those lines. But for most part, people quite often diagnose very well where the pain points are. Aside from any objective criteria that you can use like cyclomatic complexity or commit count or something like that, people know where it hurts. So listening first and then it comes down to doing my own investigation and understanding, are the things that are missing are the things that basically -- based on my experience -- I would see that they wouldn't necessarily see that could become problematic in the future. That's pretty much the approach.

NOEL:  Has the approach or the tools or techniques that you've used changed any in the 10 years or 12 years since the book came out? Does it make a difference? For a language like Ruby that wasn't popular, does that make a difference in your approach?

MICHAEL:  Yeah, there is a bit of a schism between dynamic and static. There are things you can do in static. You can leverage support from the compiler in some ways that are not really available to in dynamically-typed languages. It's true in the book. I don't really cover dynamically-typed languages all that much. I find in dynamically-typed languages, I'm more often putting probes in the code to understand what's happening with it, as opposed to going and just relying upon certain constraints that are forced by the type system. There are things like that. Yeah, dependency-breaking techniques there are quite a few.

I'm a bit more proactive these days about going and getting people to really consider doing enough analysis and be able to delete certain portions of their code. I find that pretty valuable on running into teams. My eyes weren't really open to it earlier on but teams that basically have a very false view of what their code base is because they're looking at code that isn't really being executed at all and they don't even realize that. I'm kind of emphasizing that a bit more of these days.

NOEL:  Yeah. I find that that could be a really big problem. I find that that actually also becomes a problem isn't stuff that is, at least in web projects the styles and things that are not necessarily code directly can tend to get really cluttered. Nobody wants to delete anything because nobody knows where it's used.

MICHAEL:  Yeah, it is a thing and it's really funny about this because it's like hygiene in a way. It's important but the thing is like you said, in absence of information what do you really do? You basically sort of bias towards not breaking anything and then you're in that strange situation. I kind of hope that tooling develops a bit more to going to help out with this. I created a project up on my GitHub account called 'Scythe', which is like that big ax that the Grim Reaper uses.
  
It's very simple. Basically, it goes and codifies something I've done for years with logging. What you do is you place calls in your code of a particular format. All the call does is basically just set an attribute on a file in the file system so when the code executes, essentially that happens. You put these calls in your code and a command line utility will go and rip through it and basically go ahead and record everything you place in the code as a call and periodically, you check and find out whether the call has been made in the past month, past week, past day.

With that information, it doesn't necessarily tell you that an area code is dead but it gives you a hint about whether it might be and you can do further investigation and decide whether it's was something that should go. I think that we really need more, in the way of coverage in production to be able to make assessments like that.

NOEL:  Yeah and that becomes a real problem, especially like the team turns over and nobody remembers there's no institutional memory. Do you find it to be a significant problem -- lack of institutional memory in the kinds of things that you look at?

MICHAEL:  Yeah, definitely. I won't mention the countries involved because it might betray the client in a way but I remember going and visiting a team in one country years ago and their code base was just dropped on them from a team in another country. It was a recent acquisition. There was actually nobody to help them to make any sense out of the code base at all. Essentially then, everything is scary. Your fear level is high because you have no clue, no roadmap about what's going on in the codebase.

I think that there's something bigger in this and a lot of organizations aren't really paying attention to this yet but there's a question about how much turnover can you have in a team and still maintain a degree of institutional memory. Some turnovers are unplanned and then there's other cases where you might -- because you're aware of these things -- choose to turn over. Choose to go and actually have people leave one team and join another. But I think there must be -- and I don't know how we would actually determine empirically -- some turnover rate past which it just gets ridiculous. There can be some turnover that's healthy. It gets different perspectives in your code base but past a particular rate, you probably fall apart and you end up in that massive fear situation.

NOEL:  Yeah, I think it's about context. All of the code was written at a particular time, to solve a particular problem, a particular request, like nobody tripped over their keyboard to put a line of code in the code base, like everything was there to solve what somebody thought was a problem at some point. Either you are that person and you remember it or you don't remember it or you don't have access to that person and then you don't have a sense of why the code was there, especially if it's something that is unclear or you can't tell what problem it's trying to solve. That to me is where the turnover becomes a problem. You just lose access to the context of the problem that was being solved at the time things were written.

MICHAEL:  I wish I would see more teams doing this but it's just making short videos at the end of an iteration or periodically and go and say, "This is what we did and why we did it." The why is the most important thing. Imagine, joining a project and you can go back and look at three years' worth of videos where basically the developers just briefly within five minutes or so, explain what their design choices were, for the changes that they had made recently.

If you had a history like that go from the beginning of a project, yeah, that can be extremely galvanizing. I've seen some teams do it but I haven't really seen it as a consistent practice but video is so cheap and so as memory these days so why not stand in front of a camera for four or five minutes and explain this things.

NOEL:  I could see that would be really helpful. What do teams do or what specific things happen in the code to create technical debt or things that need to be paid off in the longer term?

MICHAEL:  As I said earlier, I think that the incentives in a way are kind of odd. When I say incentives, I mean it's easier to make changes in a bad way than it is to make changes in a good way. But as professionals, we should be able to jump over that hurdle and make changes in a good way. 

I have been centering on a different idea there recently. Well, let me give you a background, a little story. One of the first companies I worked with the industry was a biomedical company, FDA-regulated. We had extremely long product cycles because software had to wait for hardware and we wrote some damn good code but we weren't under any time pressure at all and there's deliberation over everything. We'd sit in the room and argue about things and come up with a good structure.

Now, there were other things that happened that were kind of awkward because the fact that we had long development cycle it's easy to over engineer things and stuff like that, but we ended up with some really good code. Since then of course, we've had Agile and the entire thing that's happened with it. I think that there's some aspect of technical debt that that comes from our early history with Agile in a way that we see a team as being a team of developers and then there are the business people in another room and we talk to them and we go back and forth with them. But I don't think the communication is rich enough. I think that's an issue for us.

What I mean by this is in many teams I visit, it's like the features come from the product team and they give them to the developers to get a sense of whether they can be done within the next two weeks or so and there's estimates involved and they select what they want and you go forward with this. It happens usually in the planning meeting. But the thing is kind of funny is that when you have these meetings, there's never any information about what the readiness is of the code base. It's like how easy will it be to add in this particular feature.

It seems like because that discussion is off the table, it's easy for things to fall apart with no real recognition. The developers may notice some things are kind of falling apart but they feel responsibility. They feel like, "I should be more diligent and since I wasn't diligent, this is falling apart a little bit," but this is always something that people in product need to be aware of.

NOEL:  On some teams that plays out slowly as the developers slowly start giving higher and higher estimates.

MICHAEL:  Yeah. Essentially then of course you know --

NOEL:  Almost without even noticing noticing it.

MICHAEL:  Yeah, why are the estimates are taking longer, that kind of thing. I think it's because we feel a sense of responsibility that we feel like, "I can give the people in product this bad news. What exactly are they supposed to do with it?" I think this comes back to -- you've heard of The Law of Leaky Abstractions --?

NOEL:  Yes.

MICHAEL:  -- by Joel Spolsky, any non-trivial abstraction leaks and the example that he goes and uses for this is say, a piece of software that can operate across a network and you try to go and write the code so that code on either side of the network connection doesn't have to be just a network connection at all. We know that isn't going to work because networks can fail in strange and mysterious ways. Essentially, a lot of the software needs to be aware of the fact that the network is involved so you can just have a simple read and write method but it's going to fail over network in ways that would never fail if you run a single processor, for instance. So the abstraction kind of leaks.

The reason I bring this up is because this system principle like many systems principles doesn't apply just to software. I think it applies in teams as well that if we have the abstraction in place then all we have to talk about our features and estimates, that's leaky because the fact that we have this code base. If we're not talking about how it's being changed, essentially the leak is going to occur and it just takes longer to do things: the estimates become longer and longer and longer.

NOEL:  The velocity starts dropping nobody can quite tell why.

MICHAEL:  Exactly. The alternative to this is to go and make sure that business understands a simplified view of your architecture and you can go back and forth with them and discuss, "You want these features. These features that you want are going to impact these three components out of seven, maybe," and it's like, "That's great but understand that this one is a lower quality level right now or lower readiness level because of some work we did over the past three months. We can do this, it's going to be tougher going."

If you select a different feature, maybe things won't be impacted quite as much but the real thing which is nice about this is doing this over a period of time with business people because they start to go and see the impact of their choices and how that affects readiness in the code base. You just can make different choices sometimes. It's not like going up to the business and saying, "I need extra time for refactoring," or, "Give me weeks to refactor things and things will get better." There's no context on that.

NOEL:  Right because that's one of the failure modes, right?

MICHAEL:  Yeah.

NOEL:  Like everything gets bad and bad and bad and at some point, you go back to the business owners and you say, "We need two weeks to do refactoring." "What's that?" "Well, it's not going to change anything but things are going to get better."

MICHAEL:  You're giving them an absolutely no context at all. I think the core thing with this is that people in business are using the software. The software has particular characteristics to the degree that we try to hide that, we can get ourselves in a bit of trouble.

NOEL:  You started talking about how Agile might be a team into technical debt, that's not what I thought you were going to go. I thought you were going to say that Agile leads teams to think in terms of very short sprints and that encourages... Maybe not encourages but guides teams towards a more short term...

Michael:  There is an aspect to that too. I'd like to see whether there's a kind of measurement along those things. I gave this experience with working with this biomedical company and I feel that there is an aspect of technical debt which is really just an aspect of piecemeal growth. I have this feeling that software becomes like biology over time. It's one of those things where you can look at like a sprawling forest and jungle and stuff like that and get that feel in a code base. That's a typical thing.

NOEL:  I guess what I'd say is that there are certain teams that I have been on that have been Agile where the iteration boundaries become such strong artifacts in of themselves, that teams cut corners to try to hit an iteration boundary, which I think leads to like the loan shark of technical debt, like a short-term, high-interest technical debt.

MICHAEL:  It's funny because when you go back to the beginnings of Agile, that's like there are two different strains. One was Scrum, which it's like saying, "Make a commitment to this backlog and do what you can to and get it done," and with extreme programming, the idea was like that there was a moment of truth, maybe a day or two before the end of an iteration where you're able to go and say, "You can't do this right way so you're just not going to have this be part of. This is going to get done, this iteration and you can schedule for the next iteration if you want to. We can schedule something else." But that again, that comes down to a bit of an issue of balance of power, I think within an organization, if it is split in that product versus development way.

NOEL:  I also think it's funny that we're talking about something like legacy code which would seemed to be on the face of it, to be just a code problem, that it doesn't take us very long before going back to team communication, interaction with stakeholders, planning structure, all these communication issues. They have really, really strong effects on what code gets produced and how good it is.

MICHAEL:  Yeah. It's an odd thing about this. I have done some writing recently. Something that I present this point of view as something I called as symbiotic design practice. The idea behind this is we have people in organization and then there's the code that much like symbiosis in biological systems. We depend upon the code and in a way, the code depends upon us. I think that this is like taking the lesson of Conway's Law and extending it. I think that Conway's Law is like the tip of the iceberg when it comes to understanding how code and people interact in an organization.

NOEL:  Conway's Law is the idea that the structure of the code reflects the structure of the organization.

MICHAEL:  Yeah, that's a brilliant summary. When you state it that way, it's almost all encompassing in a way. There are lots of things that impact structuring code that we don't really think about all that often and it's a shame because we're all about going and trying to write better code and form better solutions in a context. That means being hyper aware of all the other things that can end up being reflected in the code that we don't intend.

NOEL:  Yeah. I think that even things I find, for instance the projects that tend to work -- I work on a project that has a very, very strong annual cycle. They have a particular date every year that they need stuff done by and they generally don't have enough work to fill the time so we work on them for a fairly intense four-month break and then generally very little work is done in the code for eight months. I find that that structure has a strong effect on the quality of the code, if for no other reason that it makes it much, much harder to maintain context over time. All kinds of things I think can get in the way.

MICHAEL:  A story I heard from someone at a very big company that I won't name years ago it was kind of fascinating. He was talking about how they kept adding more and more checks to the build, things beyond your typical automated test. It actually caused quality to go down and I thought, "What's this about?" They got to the point where actually a developer could go ahead and check something in or put something into the build and only get notification the next day, whether that was accepted or not.

What you think is, "All these additional checks must mean something to quality. They must help out in some way," but the thing that he pointed out with this, was that imagine being a developer and you're doing this and you don't want start bunching up your commits. You have this thing you're trying to add a feature in the code base and if you're doing some refactoring and you put it all in this one big commit, that you are going to pass in or one big submission, how do you really feel about doing a lot of refactoring when you're doing that because if would feel so bad to have your submission rejected the next day and realize it is because refactoring you did as opposed to the thing you're supposed to be doing -- the feature change. It's like you skimp on the refactoring because you're risking too much by going and doing it.

NOEL:  Yeah, I definitely find that long feedback loops have pretty tough effects on code quality that start I think much faster than most people realize that people start to do things like bunch commits that it starts to affect the way that you code. It's a long standing mini-rant of mine that Ruby developers that never experienced test driven development in the Smalltalk system are missing something about test driven development because they've never had the experience of running something and having instantaneous test feedback that Ruby and Rails developers are used to thinking that one or two or five or 10-minute test feedback is instantaneous and it's just not. It actually does make a difference. This is one of my cranky old-man rants.

MICHAEL:  Oh, I have that too and I think it's funny because DHH -- we can get into that also -- historically doesn't care for TDD or he's, I guess a couple years ago was making the point that it can be like a religious thing. But you're working in Rails, if you want to test a model and you do that consistently, you're going to have these really, really long test cycles.

NOEL:  Which don't feel long to him but do feel long to somebody. It was very interesting because I was just at RailsConf last week as I tape this and DHH did an opening keynote and among other things -- he said a bunch of different things -- he said that.. I want to get the working right... he said that he didn't regret saying TDD was a matter of faith but he regretted making that sound pejorative because a lot of what programmers do is taken on faith, which I thought was an interesting way to put it.

MICHAEL:  I think the other aspect of this and I don't want to get on his case at all about this because I think it's not productive and everybody has their different experiences within the industry but I had an experience years ago with someone who is prominent in the industry -- not him -- who was looking at a piece of code in a completely different domain that what he was used to and really just dinging it for the way it was handling errors. He was saying, "You would never do things this way, da-da-da-da-da-da," but this guy was a systems programmer. My assumption was that he hadn't really done very much application-level programming so it was a completely different world to him. He looked at it and just thought it was foolish. That does happen.

It happens less so now because more developers are using both static and dynamically-typed languages but it's not uncommon to go and find people who basically worked in static that basically think that dynamically-typed languages are the hight of foolishness and it's impossible to do good work in them. We are all like our world view is informed by what we do. It's rare to have done enough things in the industry where you can trade off many of them and go and recognize. It's like, if this looks odd to you, take a look at the context. Is your context different from the people that are doing it? That's not a hard lesson but it's also something that it's hard to go and step outside the bowl and half of the time we don't know that we're in that particular fishbowl, for instance.

NOEL:  Yeah, I definitely remember being at a company that started off being mostly web developers and ended up hiring a bunch of enterprise-y level backend people that they continually -- you may remember this too -- was some sniping back and forth, as to who were the real programmers and who were just --

MICHAEL:  Yeah and so much of this culture. It's bizarre but that's what people do. Essentially, we try to generalize from our experience and then the thing is that we have different experiences and then you get these clashes.

NOEL:  Yes. Not all of us had the experience of going in and looking at terrible code all around the world.

MICHAEL:  The thing is that comes with its own issues also. As I mentioned a little bit earlier, you asked me about like the best code I've seen and it's like, "I've seen some good code but it's also I'm not the best person to give you that perspective." I used to call this as 'consultants' disease'. Essentially the idea that they only call you when there's a problem. I'm assuming it's much the same way that if you're a lawyer or a psychologist or a medical doctor. Medical doctors probably walk around just like, "Oh, everybody is sick and dying."

NOEL:  But the difference is that I would tell a doctor that they're probably are healthy people. I don't know, are there healthy code bases?

MICHAEL:  Like I said, I don't get to see too many of them but --

NOEL:  Speaking on things that you take on faith... there are healthy code bases out there.

MICHAEL:  Yeah, and the thing that is improvement is a very real thing. You can work in an older code base and make things better over time. It's definitely --

NOEL:  Yeah and it's a process, it's not something you do overnight and trying to do it overnight, I think causes more problems.

MICHAEL:  Yeah. It's almost like triage. There are always lots of problems but then the question becomes what's hurting the most. Based on your product plan, what's going to be impacted over time, which areas of code that you need to go and do some work to? The answers are all questions that come up. I guess also there are people to realize that if you have a code base that has been around for seven years or more, seven plus years, you're not going to go and fix the entire thing. There's always going to be code that you're not going to touch again. This just the way of distribution of code change in the industry so you want to basically get good practices in place going forward and deal with things which are really impacting you.

NOEL:  Make sure you make things a little bit like a Boy Scout camp site rule. You leave things a little bit better than you found it every time.

MICHAEL:  Yeah and also just recognizing what good is. It's really rough when I do this to realize that you're sometimes work with developers who have not really had the experience of a good code base. Then you're like, "What do we do here?" For the most part, the thing I find very valuable to recognize is that even in the most difficult to deal with code base, there are usually always opportunities to introduce new code without modifying system code all that much and you build those pieces up TDD then delegate to them from existing code. You can have the greenfield experience within an older code base and that helps.

NOEL:  You slowly redirect functionality away from the worst parts.

MICHAEL:  Strangler applications, a pattern that...

NOEL:  One thing that's funny is I don't think it's specific to long running Rails code bases but I bet it's much more endemic to long running Rails code bases is you kind of see forensically various fads pass through the code base. This is the year we all liked doing form objects this is the year we all did... and some of that comes from a good place like that comes from an opportunity and an attempt to try to improve things. Is there in a sense in which like not sticking to that kind of improvement or trying to change things too fast? Is that becomes a problem? Is that something that you see a lot?

MICHAEL:  It's definitely not just in Ruby. I saw in C++ community years ago, Java, definitely. I mean, particularly with web technology in Java, there's been like a constant parade of, "Here's the way we do things now. Let's go and move the next thing," type thing.

NOEL:  Today, everybody will wear blue.

MICHAEL:  Exactly, yeah and the thing is there is a wheel of progress with all this stuff and you have to move along with it. The thing I always hope is that you're able to segment your transitions in such a way that you don't mix two styles in the same place all that often because that can be kind off difficult and jarring. I used to have this question that I always asked rhetorically that is like, "How many different directions can your code base support?" It's like an archaeological dig. You go back and it's like, "People invented fire here. Oh no, they discovered the wheel here," and you see all these pieces of older technology.

I think there are two aspects to this. One is basically supporting older technology which can have its cost and you have to decide whether you are going to rewrite for bigger pieces because that's problematic. If that isn't the case, then you can keep older technology and your technology in production but my hope is always going to sort of like reduce the jarring nature of that by not mixing them too much.

NOEL:  Yeah. One of the most frustrating things I've ever seen in a code base that I ever had to work on for any length of time was a code base that was in JRuby and half the database access was through Rails Active Record. And half the database access through Java Hibernate --

MICHAEL:  Oh, my God. Yeah.

NOEL:  -- and they just mixed kind of willy-nilly and you look at that and you think, that seems like a poor...

MICHAEL:  Yeah. Essentially, it's terribly opaque. It's very hard to go and tell looking at things where those interfere with each other.

NOEL:  They both fed off the same database, just going in different directions and it was maddening. It was very, very hard to unravel too because things depended on various minor corks of the different access methods in ways that were hard to predict.

MICHAEL:  I think it's great with that to actually consider what would it be like if you were able to go back in time and talk to the developers who basically made the decision to do that. It's like either they were very inexperienced and don't realize that's problematic or they were more experienced that they hoped they would have the time to fix it. Then you get past a certain point and it seems like the amount of time we would invest to fix this is just astronomical so we just have to live with it.

The most fascinating areas for me with legacy code bases is there's this space where you make a reversible decision and then you transition into a point where it's not that it becomes irreversible, but it becomes something you would never really want to reverse because it's expensive. I think that that transition point is something we don't really know enough about software development.

NOEL:  In my experience is that you almost never see that except in hindsight.

MICHAEL:  Yeah and half of it really comes down to experience also.

NOEL:  And that's right because if you avoid something like that, you never notice. The other problem with a lot of these techniques is that if they're done well, they almost never get credit for it because it just seems like things happened easily and you only notice the stuff that that is problematic. It's something that becomes a pain point, almost sneaks up on you and you don't even realize it that it's undoable until it's already passed.

MICHAEL:  You know, I haven't really talked to people in depth about this as like a practice or something but I think that it is beneficial for most organizations to have somebody else look at the code periodically and like you don't know what you're missing if you can't see it and somebody else might be able to go and see where things are problematic in a way.

NOEL:  A lot of times, a team can do this by hiring juniors.

MICHAEL:  Yeah, that's true.

NOEL:  They won't have the experience and sometimes that's a good thing. Sometimes you need somebody to go, "That Emperor is missing some clothes."

MICHAEL:  Yeah.

NOEL:  I don't see it.

MICHAEL:  And then they see it. Another aspect to this, like a deeper thing in Conway's Law is that if you have a stable team working out in an area of code, the code can start to go a bit sour without maybe realizing it because they see it day-to-day and getting juniors to come on a team when they look and if they actually have the gumption to be able to go and step up and say, "The Emperor has no clothes."

NOEL:  Right and that has other organizational implications like you have to set an organization where people feel like they can do that.

MICHAEL:  Yeah and the other half of that too, not just novices but getting experts to come in periodically also. A health check is a nice thing to be able to do. But I'm really amazed by how all this stuff comes down to the socio-technical. It's also in conjunction with the code base. The people aspect is so very important and it impacts the code at the deep level.

NOEL:  Michael, do you have some another kind of example of how Conway's Law plays out in how teams legacy code base forms?

MICHAEL:  Yeah. One thing that I've been digging into quite a bit recently is error handling in applications. It's one of those areas that people don't really talk about all that much, which is unfortunate because the things you don't pay attention to often become problematic. But I've noticed that quite often when you break a code base into separate components, for instance what it does is it tends to lead to excessive error handling at the boundaries, particularly in cases where the people creating the code aren't really aware of who's using it and vice versa.

It's easy when you're making decisions in a particular area of the code to say, "I've got this and I don't know what this really means that this thing is. There's something wrong here and I don't know what that really means to the person that is requesting something from me," so they will throw an exception, for instance or try to give as much information back to the caller as they can and that's okay. But it's also is not as good as if you really knew who the caller was.

Some of this is actually like an aspect of excessive generality in a code base. If I'm creating something where I have like 15 or 20 different users, I have to basically have some mechanism for dealing with the unexpected, that's going to work for all of them. Whereas, if I know who's using me, I can be in a situation where I can do something much more appropriate to the context.

I guess, relating this back to Conway's Law, this comes down to if you're breaking your code into separate pieces, how good is the communication between the different groups. The other thing is how many callers do you have? That step in particular can affect the structure of your code. People hardly ever think about that.

NOEL:  What would you recommend the team does to try and avoid that kind of problem?

MICHAEL:  Some of it really comes down to understanding whether the social structure you put in place, like the team boundaries are really, whether there's enough communication across them. Also, whether the layering that you have is actually useful. Sometimes people will go and have a layer in the software like here's a particular component, here's another component. They basically have that there because they see these things as different domains. That's fine but maybe if it's only like a single-user situation, like this code is being used by just a single user, maybe that boundary isn't really worth anything at all. It just really comes down to a lot of reflection upon the problem and the team structure, whether the team structure is appropriate for the problem.

NOEL:  Well, thanks Michael for letting me talk to you. It's been too long and I should figure out a way not to have it be like five years until the next time we talk. Where can people get a hold of you if they want to read the things that you're writing or contact you in other ways?

MICHAEL:  The easiest way is Twitter. My handle on Twitter is just @mfeathers but also, I have a website, R7KRecon.com and a blog which is through the website. You'll be able to go and get access to the blogs.

NOEL:  Great and you're still working on another book which may come out.

MICHAEL:  It's about error handling and reducing conditionality in code. I'm actually working on that right now.

NOEL:  Great. Thank you very much for being on the show. Tech Done Right is a production of Table XI and it's hosted by me, Noel Rappin. You can find Table XI on Twitter at @TableXI and me at @NoelRap. The podcast is edited by Mandy Moore. You can reach her on Twitter at @TheRubyRep. Tech Done Right can be found at TechDoneRight.io or downloaded via iTunes or wherever you get your podcasts. You can send us feedback or ideas on Twitter at @Tech_Done_Right or subscribe to our new newsletter at TechDoneRight.io/Newsletter.

Table XI is a UX design and software development company in Chicago with a 15-year history of building websites, mobile applications and custom digital experiences for everyone from startups to story brands. Find us at TableXI.com where you can learn more about working with us or working for us and we'll be back in a couple of weeks for the next episode of Tech Done Right. Thanks for listening.