Amos: Whoa. So zoom has two windows that open, full screen as I have multiple ones. And one of them is just me. So I can see my video on a separate screen. Chris: Oh, fancy. Amos: Yeah, that's weird. Chris: That's called the vanity monitor. Amos: I don't know that we're going to be able to record I'm so infatuated with myself here. Amos: Welcome to Elixir Outlaws. The hallway track from the Elixir community. Chris: My brain is a pudding right now. Probably a Figgy pudding. Amos: I just started the recording at the perfect time. Chris: I'll be right back. I've been standing in here for a bit. Amos: Waiting on my slow, butt yeah. Chris: Where'd you go? Where have you gone? He's gone. Amos: Oh, you know you're a father. I don't mean to tell you what was going on. My son has joined us down in the basement and he's over working on his homework. I think he's gonna make Juliet's job extra difficult. Actually, once he gets all his books out, I'm sure that he'll be quiet doing some math. I think, you know, those things where you have discrete units that then you use substitution after you've solved them to test them. Chris: Oh, that's a segue right there. Sorry. I'm running on such little sleep. We're really deep into the no sleep territory of this child. Amos: Well, we can pack this up and fill it in some other times. Chris: Think we've given the people what they want at this point? Amos: I think so. I think that, uh, we've had, uh, a segway that's so wide that it probably can't get through the door. We've mentioned parenting, and Anna's not here. Chris: So we've fulfilled our, uh, contractual obligations. This is what the people want. This is what they, this is the hard hitting dad-cast that they show up for every week. Amos: Do we need to talk about software? I got stood up this morning for my second business meeting with somebody. First time, the guy's like, I didn't even look at my calendar this morning. This time I get a message 10 minutes after I'm there that says, I look forward to meeting you and has a calendar thing and I click on it and it's exactly the same as the ones there. And I'm like, okay, whatever. He just replied and left the attachment or whatever. So I said, yep, I'm already here looking forward to it. And then I get a message almost immediately with a calendar invite for the next day. And I just emailed him back and was like, why don't we catch up after the new year. So I don't know how many times you you've rescheduled before you say, okay, you're not worth my time. Chris: Is this a potential client? Amos: It's just a local person. Cause I'm trying to get to know more people around here who does business stuff. And the other people have told me, I should meet with. Chris: He's hustling. Out there Hustling. He's crushing it. Amos: Yeah, maybe I'm sure that he's man, I don't doubt that. It's just that our time is equivalently Chris: Almost as though you value your time. As much as he probably values his time. Amos: I value his time as much as I value my time. Also. I don't want to waste his time. Chris: There's an equality of time in your worldview, which, you know, probably would be shocking to him. I've definitely had those sorts of interactions with people where it becomes very clear that the way that you value your time is not the way that anybody else values your time. You know, your, your time is on like a black Friday sale. Amos: You only have a few hours or yeah, there's a million people after it? Chris: Both, both and, and occasionally people have fights that then get documented on the internet. Amos: Those are fun ones. Chris: Have I ever told you the story of when I was working at Walmart? Amos: You worked at Walmart too? Magician books, we both worked at Walmart. We're both programers. I mean, you worked in Arkansas, so apparently you worked at, oh wait, you were a developer at Walmart. Chris: Yeah, no, it was my very first programming job was a C developer for Walmart on the point of sale. Chris: I was a cashier at high school. Chris: Depending on the time frame, we may have interacted via software. Amos: No I'm way older than you. And it was my high school job. Chris: Oh, I wasn't even born. Amos: Does this leading to unit testing. Chris: We'll get there. This is a good holiday story. Every year, you know, I had to work black Friday and w w there was nothing we could do to be clear. Cause like shipping this code meant burning a physical gold master disc that took, you know, it had to run overnight. To compile the whole thing and like get the desk. And then we like physically shipped that desk to the stores in some cases. And so there's nothing we could do, but we're like on call anyway. So we're, you know, there I was, I think my shift was something like 10 at night to five in the morning or something like that. That's when that was when I was working black Friday so I could be there, but like the whole time we're in this big room. And then like all of like the CTO and the VPs of engineering or whatever from our department are all in there. Chris: And it's a bunch of like point of sale people and a bunch of people who are talking, who manage all the connections to the credit card processors and whatever. And they were walking around and for fun, cause it was largely just boring. We just found things to entertain ourselves. So one of the fun things was we would pick stores to see who had the highest sales. So we just pick random store numbers and be like, I'm going with this with this one. But the other fun thing I was doing was I would just had Twitter open and I just had it on the hashtag Walmart. And at one point, and this was still sort of, I think before normals were using Twitter, like this was in an era, you know, where it was still sort of like a weird, it was like kids, you know, still and like e-people and whatever, like just nerdy people. And so like the execs were like, what is this? And I'm like, it's Twitter. So that was already funny. Like I had to explain the concept of Twitter to people Amos: When Twitter was still trying to figure out what the concept was. Chris: That may still be today, who knows, uh, nothing's really changed. But in any case, you know, there's all these like links to stuff and they're like, oh, is this Walmart? Let's see what people are saying. This is going to be really exciting. There's going to be like all these nice things that people are saying. Like, this is like what these executives like think in their mind. And all of the links are to people like fighting or to like, you know, the cops having to come in and like break up fights and like arrest people and stuff. And so the more I start scrolling down, they're just like, oh dear. Oh no, what is happening in our stores right now? So you, people are real out of touch, aren't you? Amos: Did you break the ice by bringing up peopleofwalmart.com? Chris: Uh, no, it did not. This was like, I think some of these people's first experience with an actual physical Walmart. Amos: Is hilarious. I'm guessing that the high level executives at Walmart rarely actually shop at Walmart. They probably rarely shop for themselves. Chris: There's a whole interesting culture amongst all that. The other thing is that in Northwest Arkansas, there are like really nice Walmarts. So there's like a really nice Walmart in, in part of town where like all the executives live and that's where they all go shop. And it is unlike any Walmart you'll ever see. It's where all like the new fangled stuff gets tested out as well. Amos: I need to go for a drive. Chris: All the lights are like solar panel or wind powered. Wow. It's like a whole, it's like a futuristic Wal-Mart it's like the nice uppity one where they test out new fancy stuff. Amos: Did you have to do the Walmart cheer when you worked there? Chris: Absolutely. Oh, that is oh, absolutely. Amos: That was the worst as a high school kid. And they make you do the Walmart cheer at the end of the day, before you go home. Chris: We did not have to do it that often. We did it at like quarterly meetings. Amos: If you closed the store. Cause we, we were not at 24 hour store. Then there weren't very many of those when I was working at Walmart. But if you close the store down at the end of the night, they would gather everybody up by the cash registers before you went outside and you had to do the Walmart cheer. Chris: Give me a squiggly. Amos: Give me a squiggly. And so every time we would be up there and most of us are, you know, they're high school kids because all the old women worked during the middle of the day and all the high school kids work at night. And we're just thrilled high school kids during the school week. Chris: Yeah, that would, that happened at a lot of quarterly quarterly meetings. Those were dark times. Those were dark times, but I also had to get paid. You know, you got to get paid. That's true. You got to make a living. And that was the only place that wanted to like hire me as a programmer because I had no training as a programmer. And it turned out none of the other people who were writing C for Walmart had training as programmers either. Which was fine. I mean, it was interesting, like all the other C developers, mostly what I ended up working with were IBM subcontractors. And these IBM, some contractors had been programming this like point of sale system for Fork of OS2 nonsense with like pre-NCC for 20 years or something like that. And you know, what job they had before they were doing that? They're in a cement mixing company. And then they literally were like, you know, I think there's probably some money in these computers and they got some brother-in-law deal and came in and taught themselves C and became IBM. Some contractors, like they just knew some people who like, and they got a little bit of like training and they just like kinda went for it and taught themselves all this crap. And they just did it and they made it all happen. So there, there's your anecdote about whether or not you need a formal CS degree to become a programmer. You don't even need a high school degree to become a programmer. As far as these guys were concerned. Amos: I did have a guy that worked with me that he had quit high school to play chess full time. Chris: That is an interesting set of career choices. Amos: Now he's had quite a few years of development experience and he's, I think he's a developer at New York times. Now. I haven't talked to him in a few years because we had some moving around. But so it went from, you didn't need a degree to, it was almost impossible to get a job without one, two. You really don't need a degree again. Chris: I mean, it probably has more to do with like overall saturation of the market and just demand. Amos: Yeah. And availability of information too. Chris: Well, and just pipelines, like whoever's hiring has some pipeline. And if the pipeline is, I hire people from Stanford because that's where like all I went to school or whatever, then that becomes the way that you hire. And so though, not about the education and so much about the connections, it's all, it's all who, you know, so they say, you know, it's, it's who you know. Amos: You don't pull yourself up by the bootstraps. Somebody else pulls you up by the bootstraps. Chris: Yeah. Well, yeah. Amos: Yeah. All right. So. Chris: Oh, my brain is pudding. You wanted to talk about unit testing. I'm glad that your brain is pudding. Chris: I'm so not with it. You're gonna, you're gonna win all these arguments about unit testing because I'm not with it right now. Amos: I'm not any great debater. So. Chris: I don't disagree with you at all. Amos: You brought this up, so you, you need to frame this topic in here. Chris: All right. Um, how do I succinctly explain this? Really? It was, I think it was, was, it felt like it would've been too long before we talked about testing again, which is to say it had been like three weeks since we've talked about testing last. So it was, we were due to talk about testing more. I, I see. Um, no, I wanted to talk a little bit about the value or lack thereof of doing unit testing, because I feel like it's really nuanced and it's like a discussion that people are having right now. And I think it came up the other day. Cause I said something like, yeah, I don't even unit tests. So I don't know. Maybe don't take my word. Amos: I think I said something like, I don't like your face. Chris: Yeah. Yeah. The thing that was more or less how the conversation went, but it's, it's more nuanced than that. And so I thought it'd be interesting to talk about that nuance and also like what drives us to do these kinds of things and - cause I think I have a very different perspective on it than you do. And I think that would be interesting if the court will allow it. Amos: What is your definition of unit testing? I see varying levels of what that means to people. Chris: Yeah. I don't know. I think people, people use these terms like in really terrible ways cause they just throw them around and then like a lot of people, a lot of like frameworks let's say end up adopting those terms and then misusing them and then it skews, everybody's understanding of this stuff. It gets really tricky to define all this stuff. I think in a platonic ideal kind of way, I would say a unit test is a test that has no state and you're just testing inputs and outputs. Like you're, you're setting up the world and you're sort of - in Elixir. Let's also caveat this with Elixir. A unit test would be something like here's a function, pass it some values and assert some outputs on the other side of it. Amos: So you're passing data in and getting, getting data back out. Would you say that, uh, in order for it to be a unit test, it, it must be a pure function. Chris: I think strictly speaking, yes. I think colloquially, we don't do that. And we would even, you know, a lot of people would say that like, oh well I'm doing, I'm using Ecto inside of my unit test and that's totally fine. And they would be totally fine with that. Whereas I think like from a strict definition kind of sense, which actually I'm not interested really in debating these terms, uh, just to be clear. Uh, but I do think they're important to like discuss, but I don't care in terms of like what you call these things. Uh, but it is interesting to maybe define them. So just to get that out of the way, I don't really care what people call this stuff. Amos: That's why I asked for a definition. Chris: Some people would say that it's totally fine to like talk to the database like through Ecto. And I think, you know, Rails did, did that a lot. And lots of other frameworks have done that and made that distinction where it's like, we're we, we call these things unit tests or whatever. They're not, but in a purely strict way, they're not, but it's fine because people don't want to mock out things. People don't want to mock out the entire database connection just to run tests. Amos: It gets tedious and hard to maintain often when you have a lot of mocks and if you have too many mocks, then you're really just testing your implementation anyway, not the true nature of the function. Chris: This is actually to me where this starts to like super break down. And I think what you're driving at is, is actually the problem, right? Like testing against the database or, or rather it's my problem with unit testing in general. Like if you're testing against the database, you're doing it for a bunch of reasons. Number one is because your application is hopelessly intertwined with that database, whether you like it or not like storing data is a facet of what your application does because it turns out that every application is just competing against spreadsheets, right? Your application has to baseline be better than a spreadsheet at whatever it is that it's doing. What that means is that you have data, which you then turn into information to display to people. And you have to be better than a spreadsheet if your company wants to like survive because otherwise it'll just, somebody will just use a spreadsheet and they'll beat you. Like, like, like as the limit goes to spreadsheet, your company becomes bankrupt. That's the calculus, right? So you have to store data. And it turns out that these databases aren't actually agnostic. Like these databases are not interchangeable. These databases are finely crafted, interesting pieces of software. And we rely on a lot of the features that they provide. And they're not all one size fits all. Like they're not, they're not, you can't actually swap them in and out willy nilly. So you end up wanting to do a test against the database because it turns out that what your application actually does is put data in a database and then turn that data into information. And all of that is willfully just intertwined forever with the underlying logic of the database. Now, why do I say all this? Well, I say all this, because this is where unit testing actually starts to break down because there's actually very few things in your application that fall into this category of like just pure stateless data transformation. Right? Most of it is actually doing side effects. There's like probably this very, very tiny, small amount of actual underlying business logic that is stateless. Now, the more that you can get the better, you're always going to be that's best case scenario as you, as you do most everything, statelessly. But at the end of the day, the core promise of your application is that it stores data and then turns that data into information. So turning that data into information is probably a stateless thing that you can do. Like you have some arbitrary process that pulls data out of a database, and then you have the actual business problem of turning data into information, which is a stateless operation, which you could like pipeline together and make very nice. But most of what your application is doing is side effects. And so I think most applications, what they end up doing is side effects. Amos: Like, yes, at some point, yeah. I mean, without side effects, it's, it's useless application. Chris: Exactly. Yeah. But even, but even more to the point, like if your application can't talk to the database or if the, if the, the fundamental feature set of the database changes or isn't what you desire it to be, then the conveyance of that information also starts to go away. Amos: So why does that mean that unit tests are't good? Chris: So here's the thing, right? You have probably some logic that is stateless, that is just pure, just transformations. And those are really good. And again, you know, the more you can do as pure transformations, the better I don't mean to sit here and decry like that as a design philosophy, because I think that's really, really good. That's the point - we want to do that. We want to do pure transformations everywhere. I tend to think that there's actually very little of that that happens at any given application. It's typically often very constrained to basically one look or almost one location in the app, in the app. There's like a set of modules that are just pure. Amos: Then unit tests those. Chris: Sure. It's fine. But - Amos: You're really making this fight hard whenever you're like yeah, go ahead. Chris: There's very little of it to actually do. And here's the other thing, every test that you put in place is coupling. Can we agree to that? To those terms? Amos: Yes. You have to couple do something in order to test it. Chris: Right. Can we also agree that coupling is generally a bad thing? Amos: Yeah. Yes. Generally the less coupled you can be... Chris: We have entire design philosophies about how to decouple things. That's basically all design is, is decoupling things. Amos: Yeah. It's figuring out how to minimize the cup. Cause you have to have some level of coupling at some point. Right. But it's all of our design things are about minimizing that coupling so that we can hopefully modularize things to a point where a change in one place doesn't destroy everything else. Chris: So does it then not follow logically what you actually want is as few tests as possible. Amos: Yes? But to me it doesn't mean no unit testing. It means better thought out unit tests. Chris: Or as we've discussed ad nauseum on this podcast, before you do property-based testing, that's where the real power is. Right. And this is to me where I get down to this idea that it's like, if you're talking about strict unit tests, like things that actually test pure functions, you actually don't want to write them. What you actually want to write is property-based tests for those things. Again, caveated with you understand the API you're building, you understand the trade-offs that you're making and you understand the kinds of inputs that are valid. And examples are fine. If you have two examples. Amos: That's fair enough. But property based testing doesn't mean to me that that's not unit testing. I use property based testing for unit testing, as well as integration testing or what some people call it, functional level testing. So, so property-based testing to me is just an approach to implementing either one of those types. Chris: And so at that point, maybe we're just arguing semantics here, but there's an important distinction here, which is that if what you're going to do is write an example and then you're going to write 20 more examples in order to make sure that you're, you know, RGB to hex converter is working correctly. What you actually wanted to do was write a single property-based test with real generators that are going to help explore the state space and find faults. That's actually what you really wanted to do. And then you actually don't want to write any other tests at all in between that, you know, that stupid testing pyramid that's garbage. And I don't agree with like, like that testing pyramid is, is, is actually a total trap. The whole idea with the pro with this, the testing pyramid, right, is that you have full integration level tests up at the top and you want very few of those. And then at the bottom you have a lot of unit tests. And then in the middle you have just garbage terms that they made up, like of like of random, you know, other types of tests. Here's my take on that. You actually don't want anything in between the very top and the very bottom. You actually never want to write any of those. None of those other things inside that pyramid are useful. All they do is further couple your implementation to the underlying framework that you're using, or they coupled to your API, which lowers your ability to change and adapt. Amos: Okay. Part of the reason that I write the tests in the middle is that every single function is an API call and there's some kind of contract there. And so I use the unit test, we'll call it a unit test, even though it's in the middle and probably talking to things below and other modules, we'll get back to that. So I use it as a way to make sure that that contract is still in place because even at the functional level, at the top level, if I am doing property testing, I'm often going to miss a path, right? If I'm driving from Chicago to LA Chicago's, the front end of my application in LA is the back end. There are an awful lot of ways to get there and I'm probably not going to hit all of them, but I'll hit a few. And maybe I say, Hey, a lot of people like to drive down Route 66. So we're going to check and make sure that Route 66 is you can still get all the way down it. And I might check like a couple other bad paths. And I, and I do throw up a property test and property-based test with hopes of expanding that space of which I'm testing. And over time, not on every test run, but over time, hopefully I hit every path to, from Chicago to LA, but sometimes need to know, can I get from Chicago to St. Louis? And is that contract still in place and still what I'm expecting? Chris: I totally get that. And I empathize with the desire, so want to put those contracts in place cause you know, left to our own. We don't have those in elixir. There's no other way to enforce the contracts between things calling each other, other than tests. Right. And Dialyzer ish, you know, that's a mixed bag in terms of whether or not that's going to tell you that you violated it or not. And to be clear, you know, again, I think the unit testing stuff is fine. I think you should do it. But I think often you don't actually want a lot. I don't think the pyramid is right. I actually think you want very few of those and you want the, mostly as property-based tests because those are going to be the ones that actually like can help explore the state space, but let's talk about something concrete. And I think maybe this will help explain the point that I want to make a little bit better. So let's say that you have some API and it's a Phoenix application or plug, if that triggers your like whatever, like it's just some API with some thing. Amos: It works over Telnet. Because I like that. It's very nice. Chris: Let's use Phoenix as the example. So let's say that we're using Phoenix, Phoenix provides a Phoenix controller test thing that essentially just calls your controller action as a function and it sets up plug, it sets up like the plug con correctly. Amos: Yeah. I was going to say some kind of like mock connection. Chris: It just generates a plug con and then it calls it as a function. Right. Okay. So that's scenario one, you could write your controller test for your API as Phoenix controller tests and do it that way. I think there's not a ton of value in doing that versus writing a test that calls out to curl and actually curls the real API. And I think most of the time you would be better served by calling out to curl and curling the API and then asserting things on the response. And the reason is going back to this whole idea of like, well, you test things on the outside and you don't test things in the middle. If you're using Phoenix controller tests, you're testing things in the middle, you're testing, all that framework stuff. You're coupling to this implementation about routing layers. And you're helping this inflammation about plugs and cons and all this kind of stuff would you actually want to do is just curl the end point with some amount of data in the right headers and get all that stuff set up correctly and then watch it flow through the system. And then assert that like stuff comes out the other end that is the right layer to be testing at because it actually exercises the full system. And it's not coupling you to what controller this happens to be in. And it doesn't give you a chance to like mock things out, which you don't really want to do. Anyway. Amos: The nice thing about testing at that level where you're using curl is that I can really replace the entire application with another one and not have to worry about it. That happens way more often than I have to change my database at the bottom. Chris: I think about how many times you change, like control our actions or move to a different version or change the schemes that you're working with, right? Like you don't actually want to couple to any of those underlying things. You just want to set up the application and run it and then see that stuff comes out the other end. Like when you hit an API. Amos: So unit testing, I'm going to agree you here, it's, it's a form of coupling, right? I'm coupling my test to my implementation, but at the same time I'm getting the documentation-ish of my implementation. If they have good unit tests, I have a decent level of programmer documentation. Anyway, that is guaranteed that the contract is explicit here based on those tests. And for me, it helps me notice when maybe things are getting a little too big or a little out of hand. So I'm also a big proponent of TDD if, because if I can't hold enough of it in my head without staring at the code, that's currently there, that my code is doing too much. And also I feel like it allows me to check that coupling level between that piece of code and other stuff in my system, because it has to be slightly able to be decoupled if I can talk to it with a unit test pretty easily, right? Because that means that it's not so coupled to the piece up above it, that it can't function outside of that environment. I mean, that might be a different type of coupling than like, you know, you change a function name and now you've got to change everything. But I, at one point I was a big fan of mocks and I'm not so much anymore. Not that they don't have their place in aren't useful, but if you're trying to make things so decoupled that you're just passing in a bunch of mocks and then mocking out exactly your implementation, then your tests become extremely coupled couple by name, coupled by every little detail inside of the function. And now your tests become really brittle. Like if I have to test that, I'm adding one inside and I mock out adding one, this is, this is really naive. You would not mock out adding one I'm sure, but that I've changed by contract. So that up above I pass in one number less so that I don't have to have this one-off adjustment of adding one. Now I have to go through and change every mock to not add one. That's just crazy. Chris: I definitely went through a huge TDD phase. I went all in on that kind of thing. It's hard not to separate out a lot of this testing stuff from that journey for me of going through TDD and like really trying to like inculcate those skills in myself. It's tough. Like I still do that occasionally. I don't find the value in it that I found three years ago. I don't find that my designs are better because I wrote the test first anymore. I think I did at a point in time. And I think that has changed for me a lot. And I don't find that my design is often that much better when I do that now for me, for me personally, like I don't, I'm not here to like decry, like the use of TD or whatever, but I, I don't personally find the same benefits that I've found before, especially when it comes to like, oh, I'm going to mock out these 20 things and then like pass stubs in, or that's going to like force me to have a better design. I don't find that same value that I used to. Cause I just think, I just think that like, in terms of like breaking down pure unit tests, like there's actually very few of those that exist. There's very few things that you're doing in your application that aren't just side effects. And so like your integration testing anyway, and mocking any of that stuff out is lies because that's not actually how it works in the real world. And at the end of the day, I think the desires that I have of testing have changed, I don't test for design anymore. Cause I don't find that same value. I test specifically to find errors. Amos: Well, yeah. And the, the design stuff, you've probably internalized a lot of the lessons that you learned. Chris: I think so. I think that's fair. Amos: I've seen people that when they're not writing unit tests are not doing TDD, I can watch them code and say, oh, you used to do a lot of TDD. I can tell by the way that you just lay out your code, that it's just done in that way, because you've practiced over and over and internalized a lot of the design there. But I guess I test not only to find bugs, but to also to find done. And I, I'm curious about the side effect stuff. Like you say that most things are side effects and I'm wondering if that is required or are there, are there ways that we can push more to data and that this is probably a very different conversation, but are there ways to push more of that off to pure data? Like, you know, you can even remove a lot of business logic in certain places, if you, if you get your data model right. Chris: I think there are ways of doing that. I think it's hard. That's the other thing I'll say. I actually think that's a hard skill. I actually think it might be harder if you come from a school of doing TDD in an object oriented world, this is purely an dotes and intuition, you know, as is, as is everything that I say is all anecdotes. And in intuition, Amos: I was going to say our entire shows, anecdotes. Chris: I don't know when I started saying it. I say like, that's my intuition a lot to try to differentiate between the, these are things that I find to be facts. Like a lot of people state these things as facts. I try to really hard to make sure it's like, these are things that I feel. Amos: I understand that. But I think the longer that you do this, the more things become intuition and feeling in, and you do. If somebody asks you a question, I think you really do have to dig to why do I feel this way? Right. But your feelings and intuition, I think you have to start to trust those more. The longer you do this than you did when you started. Chris: I think most of design is feeling just generally. I got the feeling. I do think my anecdotal experience has been that doing functional design where you do more as data structures is harder to get used to just because a lot of TDD really pushes you towards data hiding, like in a lot of like, OO best practices really push you towards data hiding. They really want you to lean into data hiding. And I think a lot of us can really internalize that. I know I certainly did. I've watched other people do it and there is power there. When you're talking about a really high level API, like you really do want to hide the underlying data structures from the end user in a lot of cases because they don't want to have to care about them. Amos: The person on the webpage shouldn't know that you're using, uh, an array or a list. Chris: Right. They don't want to care, but when you're doing a lot of this design stuff, what you're really doing is passing data structures in and then getting like these rich data structures back out. Right. And that's an almost tautological the obvious thing to say, but I know for me, at least I really butted up against that and had some sort of, you know, reactions to that, that I had to kind of break where I had to become cool with the idea that I was just going to be passing this data around and not hiding. And any of my callers could just understand, could like dig into that data and understand what it was, right. That took getting used to because the natural inclination is to like hide all that. And the only way to hide it in Elixir and Erling is to start doing things like using ETS or keeping it in a process. And then you've got processes everywhere that you're dealing with. And that presents its own challenges. Let's say you have a piece of state that you've got in a process somewhere that you're, and you did this for design purposes. Cause you're like, I don't want anybody to know about this piece of data. Well now you just added a bottleneck into your system just because you didn't want people to see that data structure. So you've actually impacted the performance of the application because OO design told you to do that. Amos: I have a tendency to, you know, it's an open data structure, like a struct or a map or a list or whatever the anybody can dig into if they want to. But I still, I would say almost implement data hiding. Like if I have a struct, instead of using the access, I will create functions in the module that defines that struct to get to the data within the struct. That way the internals of it, I can change at any point in time and still give somebody that same contract, which is then the stuff that I would unit test, I would say, Hey, I have this contract here that says, if I initialized this user struct and asked for a full name that I get last name, that first name or comma first name or whatever. I mean, that's a very small, simple implementation that I'm talking about there. You can imagine bigger ideas there, but I still try to hide that. Chris: That's totally the right approach to take still, I think. Amos: But people could still have access to that data. And I guess for me it doesn't bother me because I did C, and a lot of times you're just passing data around and not really hiding much. And I did Ruby and Ruby - you can fake hide it, but it's still all available pretty easily. And then the only language that ever fought me on that a lot was Java. And I don't really enjoy doing Java in any way whatsoever. So I feel like I've internalized the lessons that data hiding have, but I don't try to really just completely block people from all the data, by throwing it into some obscure ETS table that they don't know about or doing all my work in the database. So unit testing helps me implement those ideas though. Chris: Thats reasonable, but I just think, and again, you know, it keeps saying this, but it's, it's not that I think that unit tests aren't good are important. I think people put so much emphasis on them. I think people are beholden to this idea that like lots and lots and lots of unit tests are good. And I'm not convinced of that fact just because in terms of finding bugs, those aren't the ones that often find the bugs for me, for me in my experience, unless you're talking about and making us, uh, maybe arbitrary distinction between property-based testing and normal example based unit tests. Amos: Even if you don't. I still think that for me, lots and lots and lots of unit tests doesn't mean anything. If I have a bug in my system in let's say one module, right? It's just pure data. If I have a bug in there, I should probably only have one test that fails. If I have more than one, I feel like my thought processes upfront. Weren't that great. Maybe I had make a bunch of examples because I didn't understand everything. But to get there for me is really the TDD thing is - one test with one assertion, really. Let it fail, use the minimal amount of code to make it pass. That's not just stupid code. Right? And I'm not building a lookup table. And then we'll add one more. And one more until I feel like it's pretty solid or until I'm like, oh, now I see what's going on. Let's go write a property-based test. Or if I can start with property-based test even better. Chris: That's hard to do a lot of times unless you really understand what it is you're trying to build. Amos: Right. I can do a property-based test for addition, but - Chris: This is the tongue-in-cheek example I gave in my property based testing talk forever ago. Let's TDD addition. And the correct method of TDD in addition is that you just have all these, like one-off nonsense things that you do because you're just trying to make tests pass. So you just discard numbers and you do whatever. Cause it doesn't matter. As long as the examples pass, you did it. Right. And of course, none of that is actually true. And that's the tongue in cheek, straw man version of that. But I do, I think there's a little bit of truth to it. Like I think once you understand the thing that you're trying to build, which edition you can easily sort of start out and you can be like mathematically, here's the four properties that make arithmetic arithmetic, or make addition, addition, here's the rules that addition has to follow. And we understand those very well. There's not a lot of addition as a service companies out there. Amos: So you have to determine what the properties of your system are. Chris: Although there probably are a bunch of like node modules, but whatever, most of the time you end up with this problem that you need to identify and sure if testing and writing test for it's, the way that you get there, then like good for you. That's not the way that I get there, you know, really anymore. But you know, if that's working out for you then Hakuna, Matata, I got no problems with that. Amos: I'm okay with it. As long as whenever I sit down to work on code with you, if I walk in and flop a conditional somewhere that there is some tests somewhere that fails, I don't care if it's functional or unit that's when I start getting scared, especially when I move into a system and they're like, hey, we need you to revamp this entire section. And I'm like, okay, but I just changed five conditionals and nothing's failing. So what's going on here? Chris: I mean, there's definitely those sorts of things. At the end of the day - I think there's two things. One is my view of testing is just not really to get good design out of it. For me. I haven't convinced myself that that's a, that's a path to good design. I have to see a lot of people who are really good at design who don't do that kind of stuff. And then too, I think just generally speaking, like I don't know that I care that much about a lot of the things that people care about, you know, like test speed. I don't give a crap about that. I just don't, like it's not a thing. And I get that. It screws up people's workflows and then other people like, you know, really care about it. I care about finding bugs. If my property-based like state machine thing needs to run for 15 minutes, I'm okay with that. Amos: I don't care about that on a build and sometimes locally, but I do care what I'm in the thick of things and trying to move forward because I like to move in such small increments that if it takes 40 minutes for it to run, then that just sucks. Chris: Defintely you can start to have real problems with getting queues, backed up for deployment. If every deploy takes two hours to run all the tests I get that that's a race to the bottom and there's probably some arbitrary limit where, and I do care about it. And that's a number that I just pick for the project. Amos: This is why I like property-based testing though is locally, I can have a configuration that maybe runs 10 examples, right? It generates tenant. And then on the build, it might run a hundred and then maybe I have a weekend build and overnight build that runs a thousand of each or 5,000 of each. Chris: I was watching somebody talk about their property based testing suite to try to find a bug. And they're like, yeah, we had to let it run for four days before I found everything. And that's totally cool. People definitely do four day long test runs. I seem to recall - Bacho used to do that. They would just run for like a week. They would just do property-based test runs for weeks at a time to try to find - suss out bugs. But also if you're, if you're having to do that, that probably says that you're not really searching the state space very efficiently and that's a whole different thing. And then at that point, you know, it becomes about how do you find bugs faster? Which really means how do you explore the state space faster. Amos: But I think that weekend, four day run or whatever, any errors that it does come up with, it gives you a little more information about what your input should be, and it should cause you to change your tests too. Not just your code probably. Chris: And there's lots of tools out there to do faster, searching or more intelligent searching to get farther out on the tail of the bell curve. Amos: We brought this all into property testing and how you should do it. Chris: We're contractually obligated at this point. We're now also contractually obligated to do a, uh, advertisement for a friend of the show. Fred Avers - really good book. I'm forgetting the exact title, but it's proper testing. It's really good. It's from Prag. It's really good disclosure. Like I was one of the reviewers on that book and I've really, really enjoyed it a lot. I learned a bunch of stuff from it. Me too. We get zero kickbacks for pitching this. There's no ads on this show. I don't know if anybody knows this. We don't make any money from this show. There's no ads. Amos: We just spend money on this. Chris: I do think we need to invent ads. At some point. Amos: I just ordered 200 more stickers and I sent stickers to somebody in London. Dominic. I hope he's giving them out at the local user group because I sent him more than one friend of the show Dominic and he's new to Elixir. So welcome to the community. Dominic. Enjoy your stickers. Yeah. And he already got them. He told me he thanked me the other day. The Fred's book is Property-Based Testing with Proper Erlang and Elixir: Find Bugs Before Your Users Do. Chris: I think it's one of the best books for any Elixir person. So absolutely a required reading for Elixir people. Amos: He spent so much time on input generation and annealing. Chris: Simulated annealing. Is that what it's called? Amos: Moving down to the valley and in your dataset and trying to zero in on certain, I think that just the attention to detail there, I learned so much and Fred has this knack for coming up with examples that they're... Chris: They're like Advent of Code challenges. Like they're just complicated enough, they're engaging, but they also help really demonstrate the point. And Amos: They're easy enough to actually digest too. Yeah. You don't get lost in the actual domain. Many authors have a hard time getting there, but Fred just, he just seems to get there all the time. You got to live there. It seems like maybe we should have him come on and talk about this book. That'd be good, Fred, if you're listening and if you're not listening, it's okay. Cause, uh, I'll just reach out to you. Chris: Anyway. Yeah. That's what I had. I think people should consider what it is that they're doing. Considering the fact that every test you write is coupling and question the things that we've been told because that's how we make progress in the world. That's kind of where I'm at. I'm kind of questioning the things that I've held true for a while. Amos: I think it's a good place to be. Yeah. It's worth doing. Stop thinking for yourself. Live an unscripted life. Don't follow the masses all the time. Chris: Put that on some sort of Hobby Lobby, white board with a tree painted on it, live an unscripted life. Amos: If you don't mind somebody using adult words, there is a book on business and just life in general. It's very thick though. It's pretty cool. Chris: Is it what they teach you at Harvard Business School? Is it everything else that they don't teach you at Harvard business school, which is apparently the sum total of all knowledge. The second one is much thicker than the first. Amos: Wow. It's called Unscripted Life, Liberty and the Pursuit of Entrepreneurship by M J DeMarco. And I don't think that you need to be an entrepreneur to get - Chris: Step one - disrespect everyone else's time. Only value your time. Your time is more valuable than everyone else's. Amos: Yep. Always. And he basically just says, don't fall into the convention of, and I'm not telling people to not go to college, but that you have to go to college. You have to do this. This is how you get be successful in life. Sometimes you just need to think is this right for me? And you know, if you want to be a surgeon and you want to be my surgeon, I really want you to have gone to college, but you don't need that for everything. And he's not just talking about college. I'm just throwing that as an example, but don't follow the herd. Be yourself. Learn. Chris: Swim upstream. That's another good Hobby Lobby quote. Chris: You've got to spawn some time. So, uh, we still have to give away cards real quick. So I say we do a drawing again, but we, we have to do something interesting so that we get to laugh at different things on Twitter. So how about - this is Christmas? Well that we are recording this. It probably won't come out until afterwards, but okay. So I say everybody Elixir ugly sweater is the hashtag and you have to send - you have to be following us on Twitter. Chris: You have to send us a picture of yourself in an ugly sweater. This is maybe the best idea you've ever had is as long as they send it all to you, I don't, I'm not getting on Twitter anymore. I think I'm done with Twitter. Amos: They have to send it to the Elixir Outlaws account, which is ElixirOutlaws. So it should be really easy unless you spell like me and then you'll send it to somebody else and they'll get a kick out of it anyway. And don't forget, #MyElixirStatus and #ElixirUglyChristmas sweater and following us. And then we'll run you through a little program and pick somebody. Alright, Merry Christmas, happy, new years, whatever you, whatever you celebrate or don't celebrate. Chris: For all you atheists out there, Merry Presents. Amos: Merry Presents. That's right. All right. Thank you. Thank you, Chris. Have a wonderful day.