Paul: Hello and welcome to PodRocket. Today, we are very fortunate to have Andre Eriksson on with us. Hi Andre, how you doing? Andre Eriksson: Hello? I'm doing great. Thanks. How are you? Paul: Good. Thanks for taking the time to come on the show. Andre is really a Jack of all trades, a full stack developer who's bringing us a very exciting project that he's been working on called Encore. We're going to delve into Encore, talk a little bit about the paradigms behind it, some of the decisions that are made, and who it's going to be empowering and helping in the Go Community landscape, because this is a Go... We can call it a framework, right? This is a Go framework. You don't hear that every day. Andre Eriksson: Yeah. Paul: I feel like there's not a lot of Go framework, so that's cool. Eric has also worked at Spotify for seven years, and if anybody's heard stories on Stack Overflow or Y Combinator, that's the real deal. You're working at Spotify. They do a lot of amazing heavy stuff over there. Thank you for coming on, Andre. Let's get right into it. Why don't we start with why did you found Encore? What is Encore if you were to do 30 seconds for anybody that doesn't preemptively know it, who's watching the show? Andre Eriksson: The very short version of what Encore is is we think of it as a backend development engine. What we mean by that is you could think of it as a framework, but something that does much, much more than a traditional framework. So Encore not only makes it easier to write code in your editor, and make it easy to build distributed systems and backends, it also takes that value in all the way out into production, so helping you deal with the cloud essentially and everything that that entails, whether it's boilerplate, or complexity, or infrastructure, or observability, or CI and CD and so on. It's a very end-to-end experience for backend development. Paul: What's one of the number one things in terms of the speeding up the backend process that you would like to brag to somebody about? Just so I can orient my head around what layer we're thinking at, because backend's huge, right? Everything's a backend. Andre Eriksson: If you look at the last, say, 10 years or 15 years, with the introduction of the cloud, which most people would say started with AWS in 2006, right? Before then, it was really simple to build a backend. You just rented a server somewhere, and then you put your software on it, and then it ran. These days, everyone is building distributed systems and microservices, and everyone is worrying about availability, and reliability, and uptime, and consistency, and scalability, not to forget maybe the most important thing. We're never going back to that world, because everybody expects so much more out of modern software than back then. Paul: Oh yeah. Andre Eriksson: With that shift to the cloud, what used to be very simple is nowadays much, much more complicated. It turns out most of that stuff that you have to do in exchange for the power and flexibility that you get with the cloud, you end up having to do a lot of fairly mundane, boring boilerplate work that isn't really about your product. Basically, every single company building backends are going through exactly the same challenges, and doing the same type of work over and over again. That's really what we're trying to change. Paul: It's good for the DevOps guys. Once you figure it out, you just build up your little script library. Just hit go when you get to your new company. It all works out. I mean, that's very true. When you're starting something new, there's a lot of boilerplates. What's one of the examples of Encore and how it reduces that boilerplate right from the start? Andre Eriksson: We took some inspiration from if you've seen tools like Terraform or Pulumi, this so-called infrastructure as code, where you're basically describing the infrastructure that you want as in a programming language, and then these tools analyze that and then provision the infrastructure for you. We felt this is very powerful, but why is it a separate thing from your actual application? What Encore really is all about is not having to write that additional step in the first place. Encore, instead, lets you just define your or backend with a regular programming language in Go. Andre Eriksson: You can define... In a very lightweight way, you just define functions that then become API endpoints, and you can then start a use infrastructure primitives like defining a database, or a queue, or an object storage bucket, or a Chrome job or whatever directly in your business logic. Then Encore analyzes that, and builds up a graph and essentially a model of your whole distributed system, and then in the same way that Terraform or Pulumi works can take that model and provision all the infrastructure for you. Andre Eriksson: Unlike other tools that try to make it easy to develop for the cloud, Encore isn't trying to get you to buy infrastructure from us. We deploy straight to your own cloud, so you get all of that flexibility of, "I want to use AWS, or GCP, or Azure, or whatever," and we will just work with your cloud provider to provision it straight into your own cloud, so you attain 100% full control over your backend, but still, you get all these productivity benefits that you might get from Heroku or whatever. Paul: So for the cloud interoperability, that's something that needs to be built into the framework, right? The framework needs to know how to talk to these guys. You mentioned GCP, AWS. Now, what if we want to self-host, is there OpenStack support? Is there any self-hosted or bare-metal support there? Andre Eriksson: Not something that we provide right now. There are people already doing that by themselves, but the official support, if you will, is limited to the major cloud providers right now. We definitely want to extend that to any server in the world that you want to use will be supported, but because so much is based on this idea of how can we understand how everything fits together, and use that understanding to make your experience 10 times better, we really want to be very deliberate about how we add that support so that even if you deploy to your own server that's running in a closet, we can still provide you all of those benefits. Paul: Right. That makes sense. When you try to abstract to any server, you really have... You're solving a new problem at that point. It's really like, "How do we abstract computing to a manageable, mapable space," which is that's a whole other thing in itself, I'm sure. I specifically was asking about OpenStack, because they do have some of this like, "Oh, I want an R5 Xlarge," but it's a whole other kind of worm, so totally understand that. Besides, the majority of deployments that we're going to find these days are all on the major cloud providers, so it's very fitting there. Paul: At Spotify, how would you use Encore? Is there some way when you worked there however many years ago that this would've been a lifesaver? Andre Eriksson: Our team basically comes from... It's a team of people from Spotify, and Google, and Monzo in the UK, all these really modern tech companies. From the outside, it often feels like, surely, these people have figured out, right? It's going to... If I just joined these companies, everything is going to be amazing. Then on the inside, it's not really the same way. At Spotify and basically all these unicorn tech companies, they dedicate about 30% of their engineering headcount to infrastructure and internal tooling, so 30% of the engineers are dedicated to improving the productivity of the remaining 70%. Spotify is several thousand engineers at this point. Paul: Not cheap. Andre Eriksson: They've been doing this for over 15 years. So surely, it's all perfect on the inside at this point. Paul: Of course. Andre Eriksson: But the reality is when you're this big company, you have these really difficult challenges because everybody has just been doing their own thing for 15 years, that there is no standards. There are guidelines and there are standards, but nobody is following them, so everything looks different. That means that it's almost impossible to build a really fantastic developer experience because it's all the wild, wild west basically. The reality is if you use Encore today, and you're an early stage startup or whatever, you will be much more productive than if you join Spotify, even though they've spent 15 years making themselves more productive. Andre Eriksson: That's really weird to think about, right? The same is true at Monzo and Google, where even though you have all of these investments into internal tooling, it's so difficult to really make a dent when you cannot build a cohesive experience. That's really what Encore is all about is how can we build a really fantastic experience that is more opinionated maybe than you're used to, but the trade-off is that you get an experience that you simply cannot get in any other way? Paul: I think it's really easy to quickly jump to the conclusion of the, "This thing is too opinionated, and therefore we're not going to use it." The thing is in the context of Go, if you're using a framework you're already too opinionated, because that's how Go rolls... On the other hand, I've listened to one other podcast you were on, where there was a long back and forth about how opinionated is Encore. One of the gentlemen speaking was coming from my backend, being DevOps oriented. So from my perspective, it's like, "Well, if we have standards that are in code, they'll actually be followed," because you can have as many standards as you want. Paul: Nobody's going to follow them unless there's an iron clad rule at the end, and they have to do it. If you're cornered a little bit into what you need to do, and you lay the foundation, from my perspective, my job will be 10 times easier. I'll know how the application's built. I'll know how to troubleshoot it. I'll know where to look for logs. Speaking on the topic of observability, visibility, whatever you want to call it, how does Encore handle that, because there's a lot of pieces we're talking about here? In my application, I can just say, "Boom, give me a cue." Paul: Where do I get logs? Where can I see the throughput? How can I troubleshoot that? Maybe we could start to talk a little bit about these abstractions that are made, and some of the magic for those concerned about it. Andre Eriksson: That's really interesting. I think in many ways this was a key driver behind Encore being opensource, which is that you can actually dig into what's going on inside. We have companies who are using it, who like you mentioned, and maybe this is a slight digression, but when you have these standards that are enforced, Encore is essentially a compiler. If you do something that violates the standards that Encore set, you will get a compile time error saying, "If you want to do this with Encore, you have to do it this way." That means that you have these guarantees about how things are structured. Andre Eriksson: So because it's open source, and the parser, and the static analysis, and all of that is available as just a regular Go library, companies are actually importing that, and then using this to get a really much more detailed understanding of their own internal systems. Then you can build your own tools on top of that if you want. If you want to figure out how are teams building their solution, maybe you want to build some internal documentation system that is maybe more specific to your use case, Encore already provides API documentation, but there's always an opportunity to do more. Andre Eriksson: You can do that because it's all open source, and you can just use that. The fact that Encore sets these standards for you can then become something that you can turn around on, and build more innovative tools on top of. Paul: Right. It's really... We can get down as deep as we want. It's open source. We can really pick it apart. What's an example of how this company you just mentioned, for example, anonymous that is using Encore to check and provide these bowling guardrails on their lanes? Could we dig into that a little bit, and talk about what value they're actually getting? What are they checking? Andre Eriksson: Every company you end up like we've talked about, you end up with these... You're building internal tools. You're building... You're setting standards and guidelines and so on. Just by virtue of building their backend with Encore, they came and found Encore and already had a backend. They built some prototypes, and realized this can be huge in terms of just how we as a company can enforce these standards, so they started building a fairly opinionated onboarding script that is very tied to their new Encore backend. They found that just by having these things in place, they essentially could cut onboarding time by 80% compared to their previous stack. Andre Eriksson: That came down to just having this understanding that came from the Encore parser, and then building a fairly small set of tools that were specific to their use case like, "Oh, we're constantly doing this sort of work. Let's write a small program that uses the Encore parser, and then spits out something, and then go from there." I can't go too much into detail about what exactly they're doing, but it was pretty night and day compared to their old solution in terms of just onboarding new people. Andre Eriksson: That also comes down to the fact that everything is set up in the same way that once you've seen one of them like, "Oh, this is how you do... This is how we define an API, or this is how you define a backend service." Then suddenly, you know for a fact that everything will look the same regardless of which service I'm looking at. Paul: Do you feel like Encore helps, I want to say, package Go into a more understandable, workable thing if you're approaching Go for the first time? I remember when I first did Go, I was angry and overwhelmed for about two weeks because there was no functional support, very little functional support. To actually get a web server working, the way you should with returning errors, returning logs, it's hard. So does Encore you think help speed up this process of new people getting into it? If you're new to Go, should you check out Encore? Is this something you should start with? Because usually, when people get into JavaScript, you don't tell them to go start with react. Andre Eriksson: I would like to say yes. Obviously, I'm biased here. There are a lot of people coming to Encore who are new to Go, and in their words, absolutely. They definitely feel this is a really easy way to get started with Go. But it's also I think... Our goal is certainly that it's a really easy way to get started with backend development period. Because I think in the last, say, 10 years, there's been incredible innovation on the frontend side with the state of- Paul: [crosstalk 00:18:30]. Andre Eriksson: As a long time backend engineer many, many years ago, it was really easy to look at JavaScript, and go like, "Oh, this is terrible. What's going on?" Then you can be a bit... You blink, and suddenly a few years later, they're the ones looking down on you, because frontend development these days is in a much better state than backend development. I think in many ways it's because the JavaScript being the standard has spurred a lot of innovation, because you can make assumptions about how things will work. On the backend side, you have, what, 10, 15 different programming languages that people use, and within each one, numerous frameworks and libraries and so on. Andre Eriksson: It's impossible for any tool that tries to support all of them to provide an opinionated, excellent experience. That's why Encore says, "Okay, we're just going to pick one," and we picked Go for reasons. It happens to be a really fantastic language for backend development. But I think in many ways, the most important thing was just picking one, because by saying, "We're going to standardize on this, and this is how things are going to work," that enables this innovation that you have seen on the frontend side. Paul: Right. It's really conglomerization and organization of what tools we're using and how we're using them. Is there anything that you would warn against people? Let's say I'm inventing some crazy IoT backend that needs to communicate with millions of sensors over IRC, because why not from the '90s, right? Is there anything you would warn me against using Encore right away? Is there anything that you might want to look out, or certain use cases that might require a heavy-handed, more experience in order to keep the ball rolling? Andre Eriksson: We're definitely working on supporting all different types of use cases, whether it's TCP, talking to IRC servers, or Encore itself, like an Encore application itself being an IRC server or whatever, or live streaming video, whatever have you. Today, we are focused very much on in services that are exposing APIs over HTTP. That's HTTP/1, HTTP/2, HTTP/3 whatever. That's where we're starting. So as long as what you're doing fits into a traditional HTTP API, it should be a pretty good fit. Andre Eriksson: I wouldn't use it if you're live streaming video, because the way Encore works is it's very much focusing on this serverless paradigm, where things can come up and down over time, so more stateless than long running service that is doing really... keeping a lot of things in memory, and it can't crash because then the live stream will go down or whatever. That's where we're focusing today, but we will add support for other maybe more unusual use cases over time, but it's important to start with the most common use case, and making that experience really amazing before moving on to other use cases. Paul: You got to make sure it's done right, so people stay. On the topic of what's going on in the future, what can we use it for? What can we not use it for? What's being worked on right now? What's actively in development? What type of issues are really interesting that are open on your GitHub? If we have listeners that want to go contribute, check it out, what should they look at? Andre Eriksson: The main things we're working on right now is adding support for what we consider the main cloud primitives. What I mean by that is what really changed with backend development with the advent of the cloud is in many ways, software isn't really being written anymore as much as it's being composed. You're taking these managed services, and you're plugging them together, so you're taking... You have your API, and that's receiving some data, and then you put that data on a queue, and then you need that queue to exist. Then you need something to listen on the other end, and then you're building up these abstractions with these Lego blocks, essentially. Paul: We're becoming practitioners, less engineers because it's just putting together the correct things correctly, I guess. Andre Eriksson: We want to get back to where back in development is much more about how can we spend our time and energy, and leverage our skills in building products? When you are building it with these Lego blocks, the state today is that you are spending all of that time, configuring all of these Lego blocks, so your job essentially becomes essentially just plumbing things together. That's why we're spending all of our time writing YAML files, and configuration these days as opposed to building excellent products, which is really where we should be spending our time. Andre Eriksson: To do that, that's very much what Encore is about is how can we take these concepts, put them straight into your editor so that if you want to define a queue, and then use it, that's one line of code in your program? You don't need to do anything else to hook everything together. Then when Encore deploys your application to production, it will then set up that queue, and it will set it up in a way that makes sense for your cloud provider and your needs. That's really what we're working on right now is adding support for all of these really critical building blocks that people want to build their applications with. Andre Eriksson: So coming up soon is things like object storage, queues, caching, and... I mean, there's lots more, lots of different types of databases, et cetera, et cetera. That's the very next thing we're working on. A big part of what we're doing, and all of this is being done in the open is figuring out what that API should look like. Like, how should we make that experience as fantastic as possible, and how can we make it... How do we find the right balance between ease of use and flexibility, and how can we make the common cases simple, but still allow for more advanced use cases, so you don't run into limitations? Andre Eriksson: We're doing a lot of work on figuring out what these APIs should look like, so feedback is definitely very valuable on that. Then looking a bit ahead, we want to spend much more time when we have these building blocks in place on innovating on the experience that is back in development. How can we make it easier to debug your software? How can we make it easier to observe it? How can we make it easier to not just go from, "Now I'm spending all of my time in your editor," but then how can we make that as productive as possible? Andre Eriksson: How can we add... How can we take insights from your production environment, and feed them back into you in your editor so that when you're writing code for an API, you should be able to get insights like, "Hey, this API is being called 300 million times a day. You really need to think about this, because you're about to call that service over there, and it's going to immediately fall over because it can't handle this load, because it's much slower than your service?" Andre Eriksson: Its SLA is that it will respond in 300 milliseconds, because it's doing this really compute heavy thing. You can't just go off and call that 300 million times a day. Paul: If I could have framework that could preemptively warn me like that, I'm done, man. That's all I need. That sounds beautiful. Am I calling this endpoint 300 million times? That can sneak under the rug? You have no idea that that's happening until you slap on a monitoring framework that's way too expensive that your company shouldn't be paying for, and then you find it out and you're like, "Oh, well, damn, let me go fix that." That sounds really cool. I want to go check out your framework right now. I'm currently building a little endpoint to just be a proxy, very simple. Paul: But if I could see how the memory's flowing through and what endpoints are being hit, that would make my life a lot easier. I guess to circle back on to what's going forward, what you guys are working on, it's a lot of legwork, right? It's interacting with these cloud providers, figuring out the right way to abstract them. Speaking of... You keep mentioning like, "Oh, if I want to make a queue, it's just really simple. I just throw it in my code." I want to do a really quick, deep dive if that's all right with you. Paul: I'm really curious about how that... What's the semantics of that happening? Is there a file where I define a Terraform esque thing? Because I'm thinking like, "Okay, there's a file, and it tells me what my infrastructure looks like." But the thought of defining a queue in one line in my program, is that stateless in some way, and it checks that it's there? How does that work? Andre Eriksson: So in many ways, Encore is a compiler, right? You have... People are used to compilers that essentially take your code, and turn them into a binary that you can execute. Encore is a different type of compiler, and that it takes your program, and compiles it to a distributed system. The output is not one binary, but here are 15 different microservices, and this is how they're deployed. This one needs a database, and this one needs a queue, and that one is connected to that queue, and so on. The way it works is exactly like a compiler, and that the API, like what you as a user of Encore, what you're interfacing with is just, "Here is a package." Andre Eriksson: It's a package called queue. Then to define a queue, you just create a variable in your program that essentially by using a construction like queue.new. Then you pass in the parameters into that function call, and it becomes an object in your program that you can just interface with. So if you want to publish things on that queue, it's just a method called publish. Then on the other end, you can interface with the same object to receive things on the other side of the queue. We're essentially providing this abstraction where infrastructure resources are just objects that you define in your program. Andre Eriksson: So it's just a regular Go-based API. Where in the same way, if you want to construct, I don't know, an HTTP request, you import the HTTP package, and then you fill in the fields. With Encore, you can do the same thing with infrastructure. Then it just becomes a native Go object with methods, and you just interface with them. Then Encore takes care of all the rest, setting up it to actually work. It works the same way for local development. It works the same way in GCP, AWS, Azure, whether it's a production environment or a testing environment or whatever. Andre Eriksson: It works whether it's one service or 15 or 50. Whether it's deployed in one region of the world or 20 regions in the world, it's all the same. Paul: So it will make that queue if it doesn't exist. If it already exists, they'll say, "All right, I'm deploying this thing. I can see these things are connected with the queue, and this queue, it's already there. We're not going to create it. We're going to hook into it. It takes care of all that for us, and beautiful. Andre Eriksson: Yeah. Paul: Beautiful. It's too good to be true. I haven't used Ruby on rails, but this sounds like, from my colleagues who have used, it's very much like you define these objects, and it will take care of you. Sounds great. If any of our viewers want to go check out Encore, we'll put it in the show notes, the GitHub link. Are there any other resources or things you'd want to point people to, or projects you'd want them to look at or anything, really? Andre Eriksson: The official project website is encore.dev. You'll also find that on the GitHub page, and then we're working on a bunch of example projects. So if you want to do something like, "Oh, I want to build a simple SaaS application, or, I don't know, a to-do list, I guess is the canonical example project- Paul: Of course. Andre Eriksson: ... we have a bunch of examples on how you can build things with Encore. We're actively working on adding more of that. You can find that on both our GitHub as well as in the documentation on encore.dev. Paul: Great, so we'll link the GitHub and encore.dev. All right, thank you so much, Andre, for taking time to come on the show. It was great talking about Encore. I know I'm going to definitely going to go use it just because it looks cool. I like organizing, because I always get too messed up in the things I write in the backend. Thank you for your time, and looking forward to meeting you around. Andre Eriksson: [crosstalk 00:33:58]. Absolutely. Speaker 3: Thanks for listening to PodRocket. You can find us at PodRocketPod on Twitter. Don't forget to subscribe, rate, and review on Apple podcast. Thanks.