*intro music* CHRIS: Welcome to Elixir Outlaws - the hallway track of the Elixir community. CHRIS: *coffee noises* AMOS: That coffee sounds good. CHRIS: *more coffee noises* AMOS: With this complex topic, I’m having a beer. CHRIS: *siiiiiip* Mmm…. That’s good… AMOS: I’m having a beer and caffeine. I mean, why not an upper and a downer at the same time? CHRIS: You gotta be careful. That’s how Chris Farley died. AMOS: Uppers and downers? Coke and beer at the same time? Excpet that his coke is different - I think - than mine… Is this to soon. CHRIS: I think, yes… Alright, are we all recording? ANNA: We’re all recording. CHRIS: I think we should get all of the preliminary things out of the way upfront, before we get onto the main topic. If anyone has any topics they want to bring up ahead of time, we should go ahead and cover those before we get into the main topic. AMOS: I think we should start with a dramatic reading of that forum. CHRIS: Oh my gosh. No. ANNA: Of every post? AMOS: Uh, yeah. I didn’t make it through all of them. We do have a winner for our book - the book that we’re giving away. And it’s in Sao Paulo, Brazil. And it’s, uh, Viny - I’m just gonna say Viny, because I don’t know if he wants his name sprawled out. Even though we said it on Twitter. So congratulations to Viny. I hope he gets his book before we get to do our podcast on the book. CHRIS: Nice. Anna, you’re about to leave - you’re about to, like, leave the country, to go to Sweden. ANNA: Yeah, I’m excited! CHRIS: I’m very jealous. That’s gonna be so much fun. ANNA: Yeah, it’s gonna be fun! Have you been? CHRIS: Yeah. I’ve been. That’s part of why I’m jealous. AMOS: Been to Stockholm, or… ANNA: Yeah. You’ve been to CodeBeam, right? CHRIS: I haven’t been to CodeBeam. I’ve been to Stockholm. ANNA: Any ‘must do’s? CHRIS: I would recommend all the different museums over there. They’re all really really fun. And they’re within walking of the downtown area. And they’re well kept… I mean, I dunno. Stockholm is just a beautiful city. ANNA: Yeah, I’m excited! Should fun! And I’m excited about ElixirConf! AMOS: Oh, yeah! Congratulations there too! CHRIS: It’s getting towards the time of year where conferences come up, and I’ll get to see a lot of people that I don’t talk to except through text or RCE or forum posts, ya know… All my Elixir friends. That’ll be fun. AMOS: Yeah, I picked up my tickets, and I think I’m gonna pick up some GigCity tickets, too. I might bring my son. It depends on whether or not he wants to go. CHRIS: That’s awesome. ANNA: Wait, so can we all maybe record in the same space? CHRIS: We have a pretty nice AV setup at our public library, so we could probably record at that. AMOS: Like a real studio? It’ll be like Good Eats on Saturday Night Live? CHRIS: Yeah, something like that. It’ll be just exactly like that. AMOS: Or a really good NPR show. CHRIS: Oh, you know, we all just talk breathlessly and at very low volumes? That kind of NPR show? AMOS: Yesss. CHRIS: (doing voice) I’m really happy that you could all come to the conference… ANNA: Alright. Anyway… CHRIS: Or are you talking more about, like, a high polished NPR show, which is more like a spoken word poetry jam where somebody has written this essay, then they just read it for an hour into… AMOS: And we definitely don’t have the budget of, like, Radiolab, so that’s not happening. CHRIS: Speak for yourself! ANNA: Well, maybe Chris does… CHRIS: No, it’s not like I have a lot of money, it’s just I don’t know that there’s a lot of funding for NPR these days. ANNA: That’s fair… AMOS: Well, Anna has a plane to get on, so… ANNA: Yeah… Should we just dive in? CHRIS: Yeah, alright… Just thinking about that forum, I get - bored is not the right word. Maybe just exhausted. AMOS: I only made it halfway through the thread, because there was a spinoff thread about rethinking AppENV, and there was Sosha’s post about rethinking AppENV, and then I had to read everything on Twitter, and… Everything about this is long winded. CHRIS: Should we describe what we’re actually - what the actual subject is, just in case people aren’t familiar? AMOS: I think we should let Anna describe it. ANNA: No, you - I think Chris has the most feelings. AMOS: Well, we were just talking about long winded. So I think Anna should describe it. I think Chris has some passion going into this. ANNA: I wanna hear how Chris describes it. AMOS: Alright. Let’s hear it. Let’s hear it, Chris. CHRIS: Alright. What we’re talking about is the proposed changes - well, it’s a weird proposal. It’s not even really a proposal. It was more like a request for comments. If that makes any sense… But, we’re talking about the ongoing discussions that are happening around config. And the main thread in all this is a thread in the Elixir forum called ‘Proposal: Moving Towards Discoverable Config Files’. And, I think the way to do this is probably to lay out the context, and then talk about the proposal. Because the proposal doesn’t make much sense without the context, if that’s fair. ANNA: Sure. AMOS: I’m afraid that there was a lot that went into that forum discussion that - it’s related, ‘cause it’s in the same area as the original post, but it’s not trying to solve the problem. But will solve the problem. Probably… There’s just so much going on in there that… CHRIS: That forum thread - this is how I have to imagine it must feel to be on the W3C and get proposals, or whatever the JavaScript board is, or - this is, like, TC39, or whatever that thing is called. Like, that’s what this has gotta feel like. People have just, like, dogpiled onto this one thing, just voicing their opinion, whether their opinion has anything to do with the proposal at hand. AMOS: I think that means that people - configurations has bitten a lot of us working in here, and it’s a hard problem. I think that says something about the topic area more than whether the proposal itself is good or not, or solves the problem at hand. CHRIS: I think that’s why we should cover the actual context of why this is happening, because it helps explain why people feel strongly about this. With your permission, I will now try to explain the current state of mixed config. ANNA: Yes, yes. CHRIS: So, mixed config. It’s existed for as long as I’ve been doing Elixir. I remember learning about it, and thinking ‘oh, this is cool. It’s got it’s own configuration stuff’. Mixed config does a lot of things. It allows people to configure different libraries, different applications that they have, those dependencies dependencies… all of it gets read out of your config. And the way that it does that - the config scripts themselves are just Elixir scripts. And they just get executed. They’re arbitrary code that gets executed. And you can put whatever you want in there. And they go through and they run that code. Well, one of the fancy pieces of that code is that little config macro. And what it does, is it - under the hood, is that it populates Application.imf. So when you say things later on, like Application.gitimf, that’s why you can do that. And mixed just runs that whenever you boot your application. And it knows whenever those files change, it needs to recompile things. So it can also recompile your project based on what happens in config, and whenever it boots, it just - it executes those files, and those end up populating Application.imf. Now, let’s talk a little about - what are the problems with that. And I think the real thing is, mixed config, and the reason this forum post is so ridiculous, is, it isn’t just one problem. It’s like a rat-king of problems. There’s like, a ton of problems that are all intertwined together. AMOS: So should we start with the one problem that Jose was kinda trying to solve? Or, I bet he was trying to solve. Which is releases versus... CHRIS: Well, right, so, the main problem - the problem that most people point to for being the biggest source of pain with config, is that they work differently depending on whether or not you’re running your application with mixed. So if you’re running with mixed, it always runs those scripts at boot. However, if you build a release, there is no more mix. And if you’re deploying with releases, those no longer get run at boot time. They’re only run at - compile time is kinda a weird word… They’re run when the release is created, is kinda a more correct way to say that. And they are executed and then compiled into a sis.config file, that can be used when you actually boot your release. But what that means is, if you have code inside of your config files - especially side effecting code - the things that pull value, like if you write system.gitimf inside your config file, that gets run once, and it only gets run once if you’re running a release, and it only gets run once when you build a release. So, it’s gonna be based on whatever the build environment is. So, if those environment variables don’t exist, in your build environment, then they don’t get populated with anything. AMOS: Or the wrong thing. Like, if you look at host - system host - then you have different addresses. CHRIS: So, that’s the main source of pain. Now, I think there’s - well, and to be clear, that is what this proposal is attempting to solve - is that specific problem. Now, I think it’s worth pointing out that there are a bunch of sub-problems with mixed config, that are largely - they’re more human problems and community problems. One is, config has allowed a lot of overuse in application environment, and - basically everyone agrees at this point that our overuse of application environment is not a good thing. That’s why it’s in the library guidelines as a ‘don’t do this anymore’ kind of thing. Literally, there’s a guideline that’s like ‘stop using application.imf for configuration.’ AMOS: Well, and to be fair, when people were first coming to Elixir, that was a lot of the only examples. And when you see something called config that allows you to pass these variables, or configurations down fairly easily, it made it seem okay. CHRIS: Well, and I would bet, most people don’t realise what it’s doing. Which is like, updating application.imf, at all. And all of the major libraries out there - to your point - they did this. This is like, how you configured… everything. ANNA: It was just how you did things. CHRIS: Yeah. And it’s how you did things for the most popular libraries, was you used this stuff. So that is - there’s a cultural problem in that we’ve overused the thing a lot. And it’s become the defacto standard way to do it. If you look at just about any Elixir library read me, the first step after install is ‘here’s what to go set in your mixed config’. AMOS: I guess my thing in reading the proposal is, that problem, like you said, it’s a cultural problem, it’s a people problem. Is it a people problem that can be solved with technology? Or is it a people problem that needs to be solved with dissemination of knowledge? CHRIS: So that’s a very interesting question. I’m not sure… I’m not sure. So, what has happened is, in an attempt to get people to stop using mixed config for what we’re now kinda universally calling ‘dynamic configuration’, where dynamic means, ‘I need to get values from the environment, or from some other source, and that’s gonna be how I configure my database connection, or my Phoenix in point, or whatever’. AMOS: Like if you need a host name. CHRIS: Yeah, anything like that. If you need to get dynamic configuration, there’s a push to get people to stop using mixed config for that. I guess… It’s hard to say that it’s the supported answer because it’s not that well supported, but the paradigm that has come out recently, is to use the init callback pattern. So this is stuff like endpoint enrepo. AMOS: Why do you want configurations to come at runtime versus compile time? CHRIS: The reason you would want to get that stuff at runtime versus compile time is - it depends on how you’re deploying your stuff. Maybe you’re deploying the same release but to different environments. Maybe you’re deploying to staging versus production. And you don’t want to bake in your database hosts and your database port and your database name, and all that stuff, ‘cause that means you have to compile a staging release and a production release. AMOS: Right. I want to compile once and run on ten servers, not have to compile on each server. CHRIS: Yeah. There’s a whole bunch of reasons why you don’t want to do it at compile time. There have been other attempts to fix this in the past. One of them is the system tuple… I don’t know the right word for it, but people would do this pattern where, if you passed in a tuple and the first thing in the tuple was an atom, that was system, you could pass a string as a second thing, and then that would be evaluated when the dependency booted, and then look into the environment to go get the value. That was like, a pattern that came out. It was definitely not a standard. Most libraries didn’t support it. And it was weird. Like, there was no documentation on why it worked. It was just a totally convention, and not everybody supported it anyway, so it was barely a convention. So there was that one. The other really common answer to this is to replace OS bars. Which, if you’ve built a release, you probably are aware of the ‘replace OS bars’ option, which allows you to use values in your config as strings. And you put a $ in front of them. And then, before a release boots, it runs a shell script, which I believe awks out of those strings, and looks for them as environment variables and attempts to expand them during that stage. And that’s part of distillery. Distillery builds that into it, and so it runs at expansion, as a pre-boot step, so that your config is correct before your step boots. So that’s another option that people have done. Both of those problems, I want to point out, only solve the problem of environment variables. They won’t solve the problem of ‘what if my config lives in a vault? What if my config lives in a database? What if I get my config any other way than environment variables?’ AMOS: For Bamboo - the email thing I created - the Bamboo config adapter, Bamboo didn’t allow dynamic configuration at runtime, so if you wanted to add a way for your application to change email hosts, you had to be able to compile that in. Which means, also, if my application is running, not on some website that I have, but let’s say I have something I give out to customers to run locally, I have to recompile for each one of them, instead of giving them a configuration page. And then if they make a change, they have to call me, and I have to recompile it again. ANNA: Yeah, that’s frustrating. CHRIS: That is the other important important subproblem with config. And I won’t say - it’s not correct to say it’s a subproblem. It’s a feature of config that is used to good effect in many places, but provides limitations overall. And that is, that the config is populated prior to mixed compiling your Elixir code. And so, your macros - you can get access to application.imf while you’re compiling. So, this is used in a couple different places. Thus the config has to be there. Like, for instance, Ekto. Ekto has to know at compile time if you’re using the mysql adapter or the postgres adapter, because it compiles different code based on that. So that has to be in your mixed config. AMOS: My thoughts on that were - why aren’t the compile time things configured within your mixed efs, instead of in some config - and even in your dependencies. Could you pass your config into your depth string, right? So you could pass something into that for each application that needs its own configuration. CHRIS: Well, so a lot of people brought up - I mean, okay. Let me try to start over. There’s… AMOS: We’re a ways into this for you to start over. CHRIS: So the compilation step stuff is really interesting. The really - if you understand that mixed config is compile time, then that’s fine. You can use that to good effect. The really nefarious stuff is when people start to bake in what should be dynamic configuration, at compile time. And that definitely happens. Most people get bit by the thing where they do application.gitimf with a module attribute, and then they change their config and it doesn’t recompile or whatever… that’s definitely a common problem, you know. AMOS: Oh, I have somebody knocking… I’ll be right back… That was hilarious. I have ‘on air’ written on my door. CHRIS: On your door? AMOS: Yeah… It’s glass. CHRIS: Oh, alright. AMOS: So I wrote on it with a whiteboard marker, and I put paper on this side of it so that everybody can see. But, he probably didn’t read it. He’s from Gana, too, so I don’t know if he read it or not, or knew what ‘on air’ meant. Sorry! CHRIS: So, to loop back to what you were saying about mixed, and I’m gonna get there in this, like, sort of circuitous route, but it’s that - some people just said, basically, why don’t you make mixed config run at runtime? Solves the problem, right? But you can’t actually do that, because it’s needed for a lot of dependencies, so that they can use it to compile. I think generally speaking, a huge part of this problem, at least culturally, is that people don’t think of Elixir as a compile language. And it 100% is. People don’t treat it like that. People are looking at it like ‘I’m gonna configure this like I would configure my JavaScript app, my Ruby app, whatever, and it’s not a language like that. It’s a compiled language. So you do need some configs that are at compile time, and you also need a way to get dynamic configuration. To your point, Amos, about ‘it’d be nice to put that stuff in just mixed.exs’, well, that was brought up in this thread, and sort of shot down. From what I can tell, it’s a taste thing… And… ANNA: Yeah, I was gonna say, why is that not a reasonable option? CHRIS: Taste-wise, it doesn’t feel good. Like, I don’t want to do it. It doesn’t feel like the right place to put it. So… and we already have config. Everybody already uses config, and we’re gonna just, continue to do that. So. Now, I will also say there is a lot of debate around ‘how willing as a community are we to break things?’ Now, I don’t want to speak for people, but I would say that the answers that have been given is ‘we’re not breaking anything’. So everything has to be backwards compatible. AMOS: And I think that - that’s not abnormal in a lot of communities. ANNA: Especially as they grow, right? And they become a little bit more robust. I think that makes it harder… but I think that begs the question of, when it seems like there’s a need, how do you solve - given that you don’t want to break anything, how do you actually go about solving that problem? AMOS: And not discouraging people in the process. CHRIS: I think all of that brings us to the actual proposal. And yes, I think to your point, this proposal is very much about ‘how do we solve the immediate problem of the discrepancy between running things with mixed locally, and running a release?’ And how do we solve that. And from what I can tell, just to summarize, the proposal basically boils down to, mixed config should support some amount of runtime something. Like, at this point the conversation has kind of devolved a little bit. But there’s two main threads to this - two main options, but both of them involve doing something that allows people to specify runtime configuration via mixed. So, with that, I’ll shut up for a little bit. I’ll have more to say later, I’m sure, but someone else can talk. ANNA: I mean, what do you think about that proposal? I know you also had thoughts, Amos. AMOS: So, I struggle, because I see the community going back and forth because, I think that the config problems they’ve run into - they want to solve all of them. And part of in solving the ones they’re having problems with, is very related to this release versus running with mixed issue. So… everybody is jumping in and saying, ‘hey, I’m very opinionated on config because I’ve run into these issues.’ I think that Jose’s proposal solves the issue that he’s after at the moment - like, what’s right in his face is, let’s solve this for releases in the quickest way possible to reduce people’s pain right now, in the moment. I feel like it’s like taking ibuprofen. It might be covering up a bigger problem, but sometimes you have to cover up a problem to focus on the next one. ANNA: Or at least find a place where you can start. It seems like there’s a lot of different things that people have strong opinions on, which is not unlike any programming community. But, I think, also, you have to start somewhere, and trying to solve everything at once will usually get you nowhere. AMOS: That’s what I worry about too, with the whole proposal - with how big this discussion has gotten. I really like how in the proposal, that the inversion where you include files, and putting that into the mixed.exs, I think that that is a good step. It does make things more complicated, and you might have to think through it a little better, but I think keeping - I think keeping the calls to mixed.emv out of your configuration files, is a step in the right direction, which that would do. It’s just a limit on what’s in there. I don’t think the on boot call back that people had is bad, because I think that it might make people think about what they’re doing in there, but at the same time, I don’t think that it solves anything other than putting something in somebody’s face. And I think that the inversion of control is a whole lot better way to do that But at the same time, I don’t think this solves the wider issue. I think this solves a very limited issue. ANNA: But the goal was to solve this particular issue, right? CHRIS: I think - my take on it - and this is probably wrong, but just based on conversations I’ve had on Twitter, and conversations I’ve had elsewhere, I think what happened is this - ‘cause I think - to me we have a solution already. And the solution is init callbacks. Like, that’s a really good solution. Ekto and Phoenix both support it. We’re not working on libraries - that’s what I’m doing, and I think that’s a really good way to do it. A really good way to do it is just freaking pass arguments to stuff when you start it. Like, just pass arguments to function calls. There is no reason you have to have this configuration thing to do all this. My understanding… If I had to guess where this is coming from, I bet this is coming from people…. I don’t want to be negative… AMOS: Well, before you end up negative, maybe I can ask this question init callbacks, if - or passing things in - how do we do that when we’re starting an application? CHRIS: Oh, it’s - the way we did it is, you just add it - if you have a dependency that needs configuration before it boots, or as it boots, and it doesn’t have a good way to manage that, is, the way that we solved this at Le Tote, was we would add that dependency to that list of included applications, which means it needs to be included in the release, but you’re gonna manage it’s startup time. We would populate the application.imf ourselves in the application start callback, and we would start the dependency ourselves. AMOS: You came up with a workaround. CHRIS: I mean, I would say it’s… To me, it’s not even a workaround as much as it seems like a viable way to do things. Two things - I don’t know why that’s a workaround, and why that’s not acceptable. I mean, I don’t know why people balk at that. AMOS: So, I’m doing an included application, but one of my other applications depends on that too, so like, FTP, and I have three or four things depending on that, and once I make an included application to one of them, I have to decide which one of them is in charge of it, and the other ones kind of assume that FTP is started if that application has started, and it might not be. CHRIS: Right, in which case, you do end up in a cascade of ‘well, now I have to start a bunch of that by hand’... And some of that stuff you don’t get to control. Like, Colonel needs a configuration, and it’s gonna start before any of your other applications start. So you have to set all that crap up. Some of the other libraries, if they’re harder to work around, you have to like - you’re gonna have to do more work in some ways. But I think for the vast majority of things, that’s okay… Like, I don’t know… I’m just not offended by that. It doesn’t feel wrong to me. It doesn’t offend my sensibilities. In many ways it’s like, ‘I understand these things because I set it all up’. I guess there’s an argument to be made that maybe it’s more error prone, if you set it all up yourself. But if you’re just starting apps and passing them values… I think, if I had to guess, I think this proposal is mostly coming from, I think the core team, which is to say, like - when I say core team, I mean, like, the Phoenix team, the Ekto team, and then the actual Elixir team, lumped all together, ‘cause they’re a lot of the same people. In general, just core-ish people. I think they’ve had to deal with a bunch of questions about this. I think they’ve had a bunch of issues that have been opened about… well, not to be too flippant, but I’d guess a lot of the questions are ‘why doesn’t this work the way Rails does? Why doesn’t this just work the way I think my Rails application should work?’ ANNA: Which is, I mean, it’s - we have conversations about the community defining itself differently. Like, it’s not Rails. CHRIS: Right! I think there’s a huge part of that in this. AMOS: This problem, like boot time versus runtime, a lot of times with the constants, people would assign them to something, and then think ‘every time I call, I’ll get the new environment variable’. And well, no. It happened at startup. So you kinda end up in the same situations. CHRIS: I think there’s a fundamental misunderstanding, which is that a lot of people coming into Elixir don’t think of it like a compile language, and that’s what it is. And I think that’s leading to a lot of these problems and these kinds of questions. You know, just generally speaking. ANNA: And how do we ship that? How does the community ship that? CHRIS: Well… that’s a really good question, because as some folks said on Twitter, we tried unit callbacks for a year and it didn’t work. We still have a lot of issues about it’ You know, Chris McCord was saying this to me on Twitter, ‘we still get a bunch of issues, people still open poll requests, people are still…’ So to them it’s a very very real problem. The fact of the matter is, all of the other people that I’ve talked to, that are doing Elixir in production, don’t have questions about this. I think all of us liked init callbacks, and we just did our best to avoid mixed config. AMOS: I think the init callbacks have a PR problem. I mean, I knew about them, but I haven’t really thought about them, because every library is application.config. Every piece of documentation talked about application.config. There’s very few places that are talking about the init callbacks. CHRIS: Right. And if they are mentioned - like, if you look at like the Ekto and Phoenix talks, they’re practically an afterthought. It’s like ‘oh, if you do need to set stuff, you can do it this way too’, but all the getting started guides, all the instructional materials - like you said, they all just look at mixed.config. AMOS: They’re trying to give the simple path to get going, right. That’s the same problem that you have with examples on any piece of software looking at stack overflow is that if you see these examples and they’re all - maybe not the ideal world example, but the ‘here, this is how I can quickly show you how to do it, after a while, after you’ve seen that a thousand times, that’s all you think about - is that quick way to do it. And you don’t think about ‘oh, what are the tradeoffs, and what am I dealing with here?’ But if you see the ‘what we would prefer’ as the main thing over and over, it might be harder for you to grasp at first, but if you see it a thousand times, then that’s the way that you think things are done. CHRIS: I think part of it too is just that - I mean, I wonder anyway - I wasn’t there, I wasn’t involved in the discussions, but I wonder if the init callback thing was more of a safety valve - like, it was like ‘we have to have a solution for this, so this is what we came up with’, and it wasn’t proven when they did it, and that’s why there wasn’t a big PR push to be like ‘this is the right way to do things now’. I mean, that’s me just completely speculating. I don’t know. But, it does feel like the kind of thing where it looks like it’s been added as a way to solve a problem, and the problem they were solving wasn’t the overuse of mixed.config. The problem they were solving was ‘we do need to give people a way to dynamically do this stuff, so this is the way we want to do it, and we’re gonna see if it works out’. My opinion would be, at this point, to just say ‘let’s just all ditch the way that we’ve been doing it, and lean into this - and lean into things like init callbacks’. Because here’s - I’m talking way too much - but, here’s the other question I want to pose for people - or, maybe not question, but here’s the other things, the other scenarios that I think about when I’m thinking about this kind of stuff. Dynamic configuration, by necessity or by definition, is configuration that you don’t have local inside source control, right? It’s like stuff that you can’t control. It’s something that you need to get from an external source. And that external source is most often environment variables. But, for many of us, it’s not. For many of us, it’s ETS or like, some global store somewhere, or it’s a database, or it’s vault. You know, it can come from any of the numerous sources. And when you do that, you have to do something that involves a side-effect, and as we all know, side effects are unreliable, they can crash, and bad things can happen. My core problem with all of these proposals is that I think it’s an attempt to bandaid over a singular problem, which is environment variables one, and it doesn’t take into account the kind of systems thinking and fault tolerant design that I would like to see people being pushed to do. ANNA: Is that actually what’s happening here, or is that like - maybe Amos already said this - is it a need to currently solve the problem so that we can move forward and think about how to solve it more holistically? CHRIS: I don’t want to lay a bunch of blame at anybody’s feet here. I think they’re solving a problem that they’re being presented with a lot, and as I told Chris on Twitter, the people who don’t have problems with this don’t open issues on GitHub repos. You know, they’re hearing from a lot of people who are complaining about config, and config has been a source of complaining in the community for a while. My response is that - I don’t think the problem is with config. I think the problem is with education, and the way that people have used config and overused config. Because if no one ever did anything that involved side effects inside of a config file, this problem would literally not exist. If you never called system.gitm, or did anything else with any sort of side effect, literally the problem goes away. AMOS: Right, but, that would be a difficult thing to build to force people not to be able to do that. You’d have to be able to go through and know every piece of code that could cause a side effect. CHRIS: You can’t actually determine that with any sort of… you know… AMOS: Right. CHRIS: In any sort of realistic way. But, as I’ve pointed out - I don’t know. I don’t know how I feel about this, but - I mean, this would never fly, because one, it’s a breaking change, and two, it’s the community would probably revolt - most of the community would probably agree with me on this - but, if you want to make them work the same way that releases work, literally make it only compile time, and stop running mixed.config and boot. Then they work exactly the same way. AMOS: Solved. ANNA: Yeah, I think that’s part of the discrepancy, right, is that you have these different dynamics depending on the environment that you’re in. CHRIS: But if what your goal is is to reconcile the differences between running things in development and running things in releases, then that’s the fastest way to do it. And it’s the safest way to do it, because you don’t add any more complexity. In fact, you start removing complexity. And then you can start to actually force people to go down a different path. And that for me - that’s where I start to - I don’t want to say ‘take issue’, but that’s where I start to disagree with the proposals, because I think the proposals are attempting to solve a problem about setting configuration at run time. That’s the territory that these proposals are starting to run into, and personally, I’m not sure you can solve that. I don’t think it’s a one size fits all issue. I think it falls squarely into the camp of ‘it depends’. Because, maybe the right thing for my system is ‘I want to boot my app first, then I want to get configuration, and depending on the success or failure of that configuration, I want to retry or I want to bring my app up into a stable state’. Maybe what you want to do is start the beam, get everything kind of like, working and be at state zero - like, the lowest baseline level of runtime. Then you go to state one, which is where I make call to vault and start a configuration that I need, and then I start up things like my repo, then I start up things like my web server, then I start up things like my kafka connection, or whatever. And that is predicated on whether or not that call to vault succeeds or fails. But if it fails, my app doesn’t crash - it just stays at state zero until it can successfully transition into state one, so I can actually get the configuration that I need. And maybe that’s the right thing to do for my app. Maybe that’s the right scenario. Maybe the right scenario is that I crash, right? Maybe the correct scenario is ‘oh, can’t talk to vault so shut it down. We’re not gonna bring an app up if we can’t get it’s configuration’. There’s a debate there, I think, about the right way to do that. And it’s going to be different for every single person in their applications. ANNA: Agreed. I was just gonna say - I totally hear what you’re saying, and it makes sense. I’m imagining the core team - imagining being in their shoes. They’re getting all of these requests, all of these complaints, right, like, how do you respond then, right? Like, you need to do something. Which is what I think they’re - maybe it’s not as holistic, but they need to do something to respond to what people are requesting. And it sounds like - and from what you’re saying, right, maybe there’s a shift in how we think about our applications, or a shift in how we think about Elixir as a language, but that - how do we do that? That takes longer, and it’s harder. CHRIS: Absolutely. I mean you’re fighting inertia. I mean, this is the way that people have done this for the entirety of the Elixir community’s life. And now we’re saying - now we’re rethinking that. Now we’re starting to question that. And no crap it’s taking people a long time to actually buy into that. AMOS: And if you change it - I mean, if you think the formatter got a lot of complains and poll requests and ‘this should change’ and ‘that should change’? If tomorrow you said, “Okay, config only runs at compile time and not at run time… It’s gonna be a nightmare. I’m going to have to unsubscribe from the GitHub list. CHRIS: Just to be clear, no one will ever do that, just because it’s such a breaking change. It would make the entire community furious. No one is gonna actually go for that. AMOS: If I was Jose, I’d do it now, just because you said it would be hard. ANNA: I mean, I think in general, as programming communities grow - or when we build our applications, right, its not any different - you build things one way, and then we have them one way for a long time, and then you’re like ‘oh, this isn’t working anymore. I need to rethink this’, and it actually does take a lot of time and a lot of effort, and it is hard. It’s not just Elixir - it’s any programming community. AMOS: So what if we can make two configs? A run time and a compile time, and then make the run time have a depreciation warning. CHRIS: Wait, which one have a deprecation warning? AMOS: Run time. So you could have two config.exs settings. One says look at this file for run time, one says look at this file for compile time. So you’re separating your compile time settings completely. Like, the run stuff won’t even have an option for at compile time. So it forces people to separate them into multiple files. And then after a while say ‘hey, we’re gonna remove the runtime config option so you gotta find a new way, whether thats init callbacks or whatever, ‘cause that option is going to go away’. CHRIS: Yeah, I dunno. That might be a way to do it. And to be clear, that’s one of the proposals that people have brought up now, like ‘well, let’s just have two different sets of files - one that’s specifically for compile time and one for run time. AMOS: I did see that. It didn’t look like there was a whole lot of happiness behind it. And I understand the pain there. It’s like ‘well, now I have to change it’. CHRIS: And now I have six things to change instead of just three. AMOS: And I gotta bounce between a bunch of files… All for what feels like the same thing. It feels like the same thing, whether you’re doing it at run time or compile time. It often looks similar, or feels similar. So you’re like, ‘it’s all configuration. Let’s just put it under this big umbrella.’ CHRIS: I mean, my take on it is one - I think that overall we use config way too much, especially compared to like - maybe the right thing to do is to just pass arguments to functions, and then you just skip this whole problem entirely. And the other thing is I can’t help thinking - maybe I should be the one to decide what to do if I can’t talk to my external data store or whatever, and decide how to bring my system up. And that’s not gonna be as interesting for everybody. I think people do want a quick solution because that’s what they’re used to and, ya know… ANNA: ‘Cause we’re human. CHRIS: I mean, there are totally valid reasons to want that. I’m not even saying that people are wrong to even propose this kind of stuff. It’s that these proposals feel like they’re not for me, in a lot of ways. Like, these proposals are solving a problem that I don’t have because I have a different way of thinking about this that I just do when I’m building systems of Elixir. So whether we add an on-boot second to configs that gets run with app-boots or whether we run the configs twice, that way we get them at runtime, I’m not gonna notice ‘cause I just don’t use mixed configs for anything that I can, and I just have different ways of working around it that I have in different ways in my application start. I say all that - I don’t think it’s wrong for people to want a solution to quickly solve this problem, but it’s also a request for comments, and that would be my comment. If it were up to me, it wouldn’t change anything. I wouldn’t change anything about how mixed config works, I would start to change the way we educate people on how to use it. But I also - my libraries aren’t popular, you know what I mean? I don’t - I’m not on the core team of anything. And you look at a guy like Jose - he’s running Ekto, Phoenix, and Elixir itself. He’s making those decisions. He’s dealing with a lot of those same questions, he definitely has more insights… I mean, all of the people who responded to me from different core teams were basically all saying ‘this is a problem that we experience that you probably don’t experience because your libraries suck so much’. *laughter* AMOS: That’s not fair. They did not tell you your libraries suck. ANNA: They did not tell you that. CHRIS: No. They’re all way too nice, which is why we’re having this problem. Which is why they’re even entertaining these ridiculous tickets. ANNA: You may not have this problem, but whether or not you think it’s reasonable, there’s clearly a large group of people who - or, at least a vocal group of people - who have this problem. And the core team is trying to do something. I don’t know right, I don’t have insight into how they’re thinking about this whole thing as a system. Again, they’re just going back to that larger stepping up a little bit more high level when we think about communities and we think about change and we’re building software communities, change has to happen in pieces. I think it’s harder for a community, as it gets more established, to make huge changes suddenly. So they’re trying to start somewhere. Whether or not there is also something around education. I mean, that’s also really important and I think it also needs to happen… I don’t know… CHRIS: Yeah, that’s definitely hard. And I think that gets back to the stuff that we talked about a couple weeks ago, which was - how much influence has the Ruby community had over Elixir, how much does marketing and the groups of human beings over marketing influence the design of Elixir, and what kind of decisions are we making? To me, this falls squarely into the camp of like ‘this is clearly based on trying to capture the market of people coming from - I don’t want to say Ruby exclusively, but languages like Ruby. How do we make this feel more like it felt when I was in this other paradigm and did this other thing. Which is not - I know it sounds really critical when I say that, but I’m not actually making a judgement on that. That’s - I just think it’s important to acknowledge that that’s probably where that’s coming from. Whether or not it’s a stated thing or it’s just subconscious. And ya know, like I said, I’m not… I’m not gonna leave Elixir over this. I’m not gonna be like ‘oh, they added on boot? Screw this, I’m out of here’. That’s not what I want to convey. AMOS: I’m gonna write Go ‘cause I love formatters. CHRIS: Go and I have an interesting relationship… But no, I mean, I think… I don’t know. I’m a little worried - I would be a little worried I should say, about adding more features to Mix because it will solve a specific problem that some people have, but it will also create new problems… And I think it’s going to make it potentially more difficult to solve those same problems in different ways. It goes back to the thing of like - what this solves really really well is environment variables. What this doesn’t solve in a holistic way is ‘what if I need to get my stuff from EtCD?’ And that’s a much more complicated problem. ANNA: And maybe this shouldn’t be trying to solve that. Like, is there another next step? I don’t know. CHRIS: It’s really tough. I just… Again, it goes back to this idea - I just think this stuff falls so clearly in the camp of it depends on your application and your needs that devising any solution that’s going to holistically solve all of it is just probably not that likely. I don’t think you’re going to be able to come up with one singular good solution that always works for everybody’s use case. AMOS: Then I think we can go back to the beginning of this conversation and say ‘Is this a problem that has to be solved with technology, or is this a problem that has to be solved with people and education?’ And I think that if the answer to config is ‘well, it depends’, then it’s definitely an education issue and not a technology issue. CHRIS: Yeah, I mean. That’s kinda where I land on it. I, you know, like I said, I’m not gonna leave the language over it or anything. I’m pretty in the camp of I think these proposals have the potential to do more harm than good long term, just because it does add a lot of complexity and it’s gonna add a lot more - it’s gonna do a couple things. One is it’s going to add complexity as it becomes the standard way of doing this. And the other thing is that it’s gonna paper over a thing that we’ve already agreed is a problem, which is the use of application.imf, right? It’s solving an immediate problem that people have - which is a valid problem. I don’t want to have to spend all my time answering questions about why people can’t do dynamic configuration from their environment variables. So it’s solving a legitimate problem. I don’t want to diminish that. But, you know, I think it’s a solution that has the potential to add problems. I don’t know. I’m saying the same things over and over again now. ANNA: No, but I mean, I think - again, we’ll see. You have to start somewhere. People have a problem. You’re trying to create a solution. And maybe you’re right, or maybe it manifests differently. Future-proofing is also really hard. We don’t know. I think what you’re saying is valid, but taking a small step forward rather than trying to drastically change things isn’t unwise. But I think - I keep coming back to this idea of ‘well, there’s a larger problem that isn’t necessarily any technical problem, but how we think about the language as a community. And using it to build systems, and what paradigm are we using or what paradigms are we bringing from other communities, and how do the effect this particular language. CHRIS: I think - to take a slightly different spin on this as well, I think a large contributing factor to this whole discussion is the fact that deploying Elixir is kind of a total crapshoot right now. It’s like - don’t - it’s very non-standard. And that’s not - I don’t want to slag anybody in saying that, but there are a bunch of ways to deploy Elixir right now. All of them are kind of equally valid. And it’s very very non-standard. And I think that’s part of where this problem comes from. Ya know. You can cut a release. You can actually build real release using distillery. And that’s what a lot of people do. But just as many people just use mix - just skip releases all together, often because of these problems, and they just run it with mix. Many people just run it with git and compile on their server. Some people build a release, and then put it inside a dopper container. There’s all these ways to do the same thing. And there’s pros and cons to all of it. And I think that contributes to all of these ways and different inconsistencies - is that there’s all these ways to go about solving the deployment problem. And this is obviously a thing that only becomes an issue when deployed. And so I think you run into some issues there as well. Or, that adds to the confusion. Whereas, if the entire community always built releases, we’d probably have better solutions to this. Or, I won’t say better, but we’d probably have a solution to this. We’d probably have a concerted, conventional solution to this if the de facto way to deploy it was always using releases. AMOS: I’m for that. CHRIS: I mean, we use releases a lot. We did it - that’s what we always did at LeTote, was we built releases with Distillery. But even if we were deploying with Docker, we deployed the release into a Docker container ‘cause it makes the container a lot smaller. So you can just throw it inside an alpine container and deploy the container. See, that’s part of the problem too, is because - and we didn’t even talk about the people who were doing code reloads. Which is not a lot of people. I know very few people who actually do that, but it is a feature of the language that you have to support, at least from - I don’t know about have to, but a lot of people choose to support. And like, the release tooling and that kind of stuff - it has to support hot code reloads because people do that. And so that adds a whole other layer of nonsense to this because now you’re like, ‘if I did a hot code reload, now how do I start to get the configuration again? Now do I need to rerun my runtime configs and repopulate my application.gitim? Is that the correct thing to do when I run a hot code reload? That’s such an open question - how do you do that? Oh, and let’s try this on. What if that doesn’t work and you want to role back? Should those environment variables have been versioned in some way? Should that break your entire system because you need to roll back for a config that you’ve pulled in from some external source? I mean, this is a hard problem. This is a really hard problem. AMOS: I think that’s one that we might have to discuss on a different day, because if we don’t deploy Anna to Stockholm soon, she’s not going. ANNA: Yeah, I have to get going soon. But I think this is worth continuing. We could maybe do it again. If y’all are up for continuing the discussion. CHRIS: It’s a really deep topic, and I think it’s really hard to - its multifaceted and it has all kinds of different problems. And there’s not a standard way of doing this in Elixir ‘cause there’s not a standard way of rolling out new Elixir code. Even if you are doing releases. You can do releases with immuitble deploys in Docker containers, or you do releases with hot code reloads. Or mix. There’s all kinds of problems that we have to solve for, and they’re complicated. They’re really really complicated. It just goes back to that thing where, you know, I just wonder how this one solution is going to play with all that. ANNA: It’s interesting for sure. CHRIS: I don’t know. I just. I’m sorry I talked too much on this one. ANNA: No… To be continued, right? AMOS: Sounds good. I don’t think we have a lot of ending thoughts. Unless someone has something they want to say. Chris, be quiet. *laughter* ANNA: No, I think. Yeah, to be continued. CHRIS: It might be kinda fun to get some friends of the show on the show and actually talk to them about it. AMOS: We didn’t mention any friends of the show…. I guess maybe Vinny. Vinny won the book, so I guess he’s a friend of the show now. CHRIS: I did have to bring this up - I did spend a good deal of time talking to Fred about all this. AMOS: Ah, I thought we were gonna get through without mentioning him. ANNA: No, we can’t. CHRIS: He had a lot of good points we could talk about on the show. I mean, he does the thing that I’m advocating. But anyway, I felt like we needed to bring up - he’s not even a friend of the show. He’s like the chief outlaw or something. AMOS: He’s Jesse James. CHRIS: Yeah, no, it’s an interesting topic. It’s fun. I appreciate everyone letting me ramble for so long. ANNA: No, it’s really interesting. And important. CHRIS: I do appreciate all the hard work that the team is doing. Even if there may be some differences of opinion, there’s no hard feelings. ANNA: And I think we’ve mention that - it takes so much time to sustain and run...Alright, I gotta run! AMOS: Have a great day! CHRIS: Bye!