Ben: Hello and welcome to PodRocket. We are live in our brand new podcast studio. We have all the fancy microphones and everything, it's pretty cool. To celebrate our first-ever recording in our studio, we have Luca Palmieri here. Luca is a principal engineer at TrueLayer and the author of Zero to Production in Rust. How's it going, Luca? Luca Palmieri: Hello. It's going really well. Thanks for having me. I feel kind of important now, knowing that it's your first episode in the new studio, but it's a pleasure to be on the show. Ben: Yeah. So I'm guessing today we're going to talk about Rust. Rust generally, I think of as more of a back, in the context of web development, something you do as more on the backend. But I do know a lot of folks starting to use Rust on the frontend as well. So what I'd love to talk a bit about both if that works for you. Luca Palmieri: Well, I'm mostly an expert on the backend, but I know a little bit of what's moving around on the front end, so I can provide you with what I know in that bit as well. Ben: Awesome. So can you tell us a bit about Zero to Production in Rust? Kind of what was your goal in writing it? I think after you purchase it, so why should folks purchase the PDF and learn about Rust? Luca Palmieri: Yeah, sure. So the book was born a little bit by chance. So we started to use Rust for us for the backend, we had it through layer. We realized that actually there weren't a lot of resources to go from, I've read the Rust book and to actually I know how to use Rust for backend development. That's because Rust wasn't necessarily associated with back end development in the first place. Luca Palmieri: There was a lot of buzz around Rust for system programming, usage in contexts, where usually you would go for C or C++. So Rust for backend development was an area of interest, but it was not mature from a documentation perspective. Furthermore, a lot of things that changed around that time because it was in Kuwait, which was released in 2019. So the whole ecosystem was going through very fast fast-paced changes. Luca Palmieri: So as I was onboarding different developers onto our projects, so the stuff we were doing it through that year, I realized that actually a lot of those things could be codified. So I just probed a little bit for interest. It turned out that actually there was demand for intermediate/advanced material in the backend ecosystem. Luca Palmieri: So I wrote down what I believed was the minimum curriculum of what you should be able to do to say you're able to use Rust in this space, and that's how the book was born. Originally it was just a series of blog posts until I got to the point where I actually decided, well, actually, this is serious, so we're going to make it a book. Luca Palmieri: That reception has been good so far. As you mentioned, 1000 copies, 1500 copies, which was much more than I was expecting, especially given the book is still in progress. Kaelan: Yeah, it seems on the reaction I found on the Rust community was very positive. I think people like to have more in-depth resources, especially on the Rust community. They really like their documentation. Luca Palmieri: That's one of the things that the community is proud of. There's been an effort, I mean, Zero to Production is not the only one. There are new books coming out in these months, which try to fill the gap in different spaces. Rust in Action was finished just a few weeks ago, I think was officially published, and that does it for system programming. Luca Palmieri: There's a new book from Jon, who does the same just for knowledge of the language. So going back into the internals. So it feels like the whole ecosystem kind of landed on the fact that beginner documentation at this point is very solid. Now the next step that needs to happen is actually making sure that people can be productive in the sub domains that they actually want to use the language for. So the challenge is one step ahead, so to say, in terms of documentation and just knowledge basis. Kaelan: I like the idea of like a real world guide. I also know that, what's the saying, at least in the JavaScript community, whenever you publish a book, it's obviously the moment that it gets published. Do you see yourself updating this, like every year or so? Or what do you think? Luca Palmieri: So there's always baby steps, first step is finishing it, then we can talk about updating it. I do feel like the book is going to be a leading product. Now, of course, once you finish it and you do a printed edition, the printed edition is going to have a life cycle. That's not something you can update every week. Luca Palmieri: But I do see myself keeping that up-to-date as we go forward year by year, which might mean also the driving parts of that as the framework changes and patterns evolve. That's something I took into account when I decided to embark on this specific journey and it has happened even while I was writing it. So large parts of it have had to be rewritten, updated to just style differently as things changed. Ben: So looking at Rust in general, why should someone who's maybe familiar with backend development and JavaScript or Python or Ruby, what advantages do you get by learning Rust and building your backends in Rust? Luca Palmieri: That's a very interesting topic and I think it's where a lot of mis-comprehension and misunderstandings usually lay. When people try to bounce the proposal using Rust in the backend, or just in web development in general, because people associate Rust with performance. So they say, "Well, actually in most cases, business applications and the price programmers do not meet performance. We don't need it to be fast." Luca Palmieri: That's actually when Rust excels. It's not about speed in backend development, it's about domain modeling. So it's actually being able to go and build a satisfactory domain, a satisfactory model of the domain you're trying to solve. Using the type system to make sure that as much of what you learned about the domain is actually encoded in the model. This is the number of tests you need to write and make sure that a bunch of things, which usually happen in the price projects, actually happen safely. Luca Palmieri: I'm thinking specifically about evolution of the model and deprecation itself. So algebraic types, okay. enums are extremely powerful to model finite state machines. Actually most of the entities that you have in the system are usually finite state machines. There are objects that evolve through a finite life cycle, which you are trying to model. Luca Palmieri: So for example, you might go for a user, user is a pending user. When he signs up, then he becomes an active user, then he might get suspended. You might have a newsletter just from a draft for publish, it's adapted to an archive use level. All those things, you try to model inside the code. You can model with types using Rust type system. You can move a lot of runtime checks to compile time checks. Luca Palmieri: Then you can make it so that when a new developer actually arrives on a code base and they're actually working on that code base, the compiler helps them in doing the right thing. So they don't feel overwhelmed for example, by picking up other [inaudible 00:07:12] code base and having to kind of reverse engineer in their head what goes here? And what depends on this one? Luca Palmieri: It becomes a lot easier as well to give instructions because that's a lot more can be encoded into the type signature, which makes it so that it's safer to build higher towers of instructions and on top of one another. Luca Palmieri: Now this is not unique of Rust, of course. Rust is not the most [inaudible 00:07:35] type system out there, functional language is asking a bunch of others, have done this before. But I think that Rust strikes a particularly interesting balance between expressiveness, so the things you can do. And pragmatic-ness, so actually being able to teach that effectively without getting lost necessarily into the theory that surrounds type systems and what they can do. Luca Palmieri: So you get options, you get results, you get a lot of things that usually are up for scale in mono terminology, actually distilled down to things that people can use for the value they provide. Then maybe once they've actually mastered them, they can go back and understand what that looks like. But the focus is on getting stuff done and I think that's very big in the Rust community, is a very good trait that I think would be beneficial if they maintain as they go forward. Kaelan: Yeah. Makes sense. Are there any domains specifically for the backend web area where you would not recommend Rust? Or I guess use cases that you would recommend another technology over Rust? Luca Palmieri: Backend is a gray domain, right? So it's that domain where most languages can succeed if the person using them is sufficiently capable in using that language. Even languages you might not expect being able to succeed in that domain. Luca Palmieri: So the choice is not necessarily around the language, unless you have specific requirements, but then it's very difficult to give indication if you have specific requirements, because we need to go and discuss the specific requirements. But generally speaking, your choice of technology would be influenced by the people you have on the team. So what they're knowledgeable into and what they can do. Luca Palmieri: The type of ecosystem you're integrating into, if you're trying to do something which is DevOps-y. So it goes into the cloud native ecosystem, but upscales as synergies with a lot of projects. So you might be able to leverage a bunch of libraries because the project you want to integrate into have been in that programming language. Luca Palmieri: So it's a case by case decision, so to say. I mean, I've done stuff, we're about to do stuff, thank god, where we had to integrate with so KPI's. .NET was a great fit because that tends to have a lot of the libraries that you need, which in Rust do not exist. Now we didn't actually do that, but we were fairly clear that it was not going to be Rust because we'd have to bootstrap all the libraries that we didn't have. Luca Palmieri: So for specific use cases, you always want to look at the ecosystem that you need to leverage in all maturities. Ben: In Rust, what is the state of kind of back-end frameworks and ORMs and all of the supporting tooling that helps make it so easy nowadays to write a backend in Python or JavaScript or Java? Luca Palmieri: Well, maybe it's not as easy. I think that that's a fair warning that people need to read before they deep dive. But I think the bigger blocks of the ecosystem system are there. Luca Palmieri: So from a work framework perspective, you have Actix Web, was been going on for a few years at this point in time. Has been used extensively in production applications, including ours. It's very stable. There's a large community, it's fairly well-documented in terms of examples. Some parts of could use a bit more love. But overall, if you want to get something done, you can get it done. Luca Palmieri: So you're not going to be blocked by the framework missing capabilities that you need to actually fulfill a use case. But it's not necessarily the easiest work framework to onboard on. There's a lot of type system knowledge that you need to have and running middleware is fairly complicated. Luca Palmieri: Then you have Rocket, which I think in many ways is the easiest to pick up. So is the one with the developer exceedance is the best. But it's been stuck for a very long time. I think more than a year at this point in moving between zero dot four, zero dot five. So moving between a synchronous first design to an asynchronous first design. Luca Palmieri: Of course, asking people to define the master is not necessarily a part of the proposition, especially in a business environment. Now they finally got the release, it looks very nice. The economics are still good, even if the moving over to a sync. So I've been looking over it and it's looking solid. So it's definitely an option. Luca Palmieri: It's the one probably that goes to the more rails like between the web framework and the Rust ecosystem. So the one that tries to be a tiny bit more opinionated and has a lot more building blocks that you can just put together to get something that works. But other frameworks that are a little bit more micro frameworks. Luca Palmieri: So they're more like Flask or FastAPI. So less opinionated, they just take care of the HTTP layer, and then the rest is pretty much on your own, which is something you might like or something you might not like, depending on your tastes and what you need to do. Luca Palmieri: I think last but not least, there's Warp, coming from the Tokio ecosystem. Once again, very experienced maintainers being used quite extensively, fairly unique design. So very similar to tower, and so modeling the entire application is basically a function of the request that outputs the response. Makes a button usage of the PAC system to compile check a bunch of things, which is useful. Luca Palmieri: On the flip side, it can be a little bit daunting and adders can be quite confusing for newcomers. So that's why we've steered away from it for example, for usage that's really. But depending on your maturity and what you're trying to do, all three are viable, it's mostly a matter of taste. Luca Palmieri: But I don't think we have yet Django or rails in the Rust ecosystem. I think we're far away from that level of productivity in just like shopping from a shelf and getting an application done in a couple of hours. On the RM side, probably not an ORM per se, I'm not sure we have at the moment. As it qualifies, for example, looking at active records or entity framework. Luca Palmieri: The closest you might find is diesel. Diesel is very interesting, very powerful library. It attacks a compile time most of your credits, so is able to build a model of the code and verify that it having type checks. You get that presentation in code of your schema. [inaudible 00:13:42] migrations. Fairly useful. Luca Palmieri: Now of course, once again, a lot of type magic in there. So from time to time, you might get errors, which are not the easiest to parse, if you are a beginner. So as you often have, we do a RAM like tools. If you actually get into complicated credits, they might get more in the way than they actually provide you. But I don't think this is specific of the library. This is more of the type of function it fulfills. Luca Palmieri: If you are more of a SQL type of guy, then you can go for something like a SQLx. When you get to that SQL, they have some fairly magical markers that are going to execute SQL queries against the database. Compile find, compare the SQL queries and compile time against the database. So they're able to define types, and they're able to check that you're quarrying columns that exist. They are parsing the right types. More or less that your queries are sensible. Luca Palmieri: It's not particularly opinionated and we found that it works fairly well for us. It's what I usually recommend people have a minimal level of familiarity with SQL. Because it allows them to shape the persistent layer as they see fit. But of course, they do SQL. Kaelan: That's an impressive list of features actually. It seems like you have a lot of options. Pretty much every category I can think of. But I guess, like you said, we're just missing the rails alternative. When you look at a new open source project in Rust, in the Rust community, at least, what do you look for in a Rust project? Luca Palmieri: It depends. I guess the first assessment that you usually make is, what do I need to use this for? And how difficult will it be to remove this if I need to? So that changes the way I assess the project. Luca Palmieri: Well, let's assume I'm shopping for something which I think is going to be important. So it's going to be something that I can't easily change later in the project, I usually look for adoption, as you usually do for libraries. So are people using this? Ideally they're using it in production context. So they've actually battle tested the solution. Luca Palmieri: It's not always possible to check this. So you look for proxy measures, which usually is are there libraries using this library? Or talks or articles written by businesses actually using the library? Then you check the owner, I guess that's the other one. So you'd go and actually check if they've done previous work in the Rust community. Luca Palmieri: If you've used libraries that actually were designed by them and you've used them proficiently. How they managed usually pull requests and issues in the past? Especially if it's something you're going to rely on. None of this, I appreciate is Rust specific, it's usually just good benefit management when you go and take on an open source dependency. Luca Palmieri: From a Rust specific perspective, there's usually a basic level of assessment of compile time dependencies. What are they bringing in? Is this going to make the project much heavier to compile? Unsafe is usually something we check for. Is this making extensive uses of unsafe? Is this something we should be concerned about? Should we audit the unsafe parts for security purposes? Luca Palmieri: This of course depends what the library is doing and is the library tested? That's usually another one. So quick checks of making sure that there's a CI pipeline. The test looks sensible. We're not going to find out that a new release broke and then we have problems to solve. So vendor management, like nothing more, nothing less than vendor management. Ben: One thing that caught my eye in the notes on chapter one of the book is the idea of security auditing. Some of, as I understand it, one of the advantages Rust can have when helping you build your backend is giving you some more kind of better security posture out of the box. Could you talk more about that? Luca Palmieri: Yeah, sure. I think you can attack the problem on the different angles. On one side, you have security coming from mismanagement of memory, right? That makes Rust ideal from an auditing perspective, because you can focus on a smaller subset of the code base you're actually auditing. So you can focus on the parts which go outside of safe Rust. That makes it easier, for example, if your internal function was auditing the libraries you're using to actually go in and make some checks. Luca Palmieri: The other thing is more cultural, I believe. In the Rust communities there is established culture around misuse of a system's APIs. So designing interfaces, which are difficult to misuse. So they're talked about with the beginner in mind and the person who's not an expert in the domain, who is actually trying to use the library. Luca Palmieri: That usually makes it so if you model your domain properly and you actually put the right constraints in, that there's a cascading effect on the security, because a lot of things that usually are possible, especially when you use primitive types, and you have a lot of leeway, so that it became impossible. Not because you're designing for security, but you get that as a side effect. Luca Palmieri: Now nothing is going to defend you from a mis-designed GWT validation routine. So logical letters, which lead to exploitable security within that are still possible. So you still need to look out for those. Kaelan: It's still pretty impressive that manages to do so much. A radical departure from other languages, and specifically important for the web community, where security is even more in the news. I guess that's one of the reasons why you see a lot of companies adopting Rust. Kaelan: I remember reading recently that Discord switched from Go to Rust. I guess as more and more and more people are starting to talk about Rust, we'll get more of that. What do you think the blocker is, I guess? The things preventing companies from switching to Rust? What can Rust do better to take over the world? Luca Palmieri: Wow, that's a bold mission, for sure. I'm married to that one. But in general, let's look at it from a company perspective. That's something we did, for example, we didn't start as the Rust company. We still aren't a Rust company, we're a polyglot technology business. Luca Palmieri: So if you're a company looking to adopt the language, you look into variety of factors. First of all, is some of the ones we already talked about. So you look into the ecosystem. Does this ecosystem contain the building blocks I need to do and fulfill the business function that I need to deliver? So that's step number one. Luca Palmieri: At this stage, we can say that Rust does, although from time to time, you might actually need to go out there and build some things yourself that you might usually shop for. So smaller libraries, utilities, upstream a couple of PRs. You need to be a tiny bit more hands-on into the ecosystem, but it's not massive and you're not adopting Rust pre 1.0, so it's doable. Luca Palmieri: The second one is talent. If you're building a business and you want to adopt a primary language, you either need to have expertise already. So you need to have employees who are familiar with using the programming language, and can up-skill new hires. Or you need to hire people who know the technology. Luca Palmieri: Now, just by the nature of grasping a young programming language, you're not going to find a lot of senior developers out there who've done Rust for five years in productions. Because nobody, or very few people were running Rust five years ago in production. Luca Palmieri: So as you see at this point in time, and that's a trend I've been seeing a lot for the past 12 months, there's a lot of companies actually training internal stuff. So going to consulting companies who have expertise in Rust, actually organizing workshops and up-skilling sessions for their own employees and then starting to hire externally. Luca Palmieri: Of course, when you're hiring somebody, you usually try to put them in a position to succeed. So if you're hiring them and they didn't know the language and didn't know the domain, that's going to be tough. But then if you can teach them the language and they know the domain, that's an easy sell. If they know the language, then you can teach them the domain, that's once again an easy sell. Luca Palmieri: So I think this is just going to solve itself quite naturally. As early adopters get into the market, they're going to train up a bunch of engineers. Those engineers are going to go somewhere else. Scarcity is going to become less of a factor and then hiring would become less of a factor. So you're going to hire for Rust, if you want to. Luca Palmieri: At this point in time specifically, it's I think quite the sweet spot to be on this specific technology, because you're getting that way where engineers are usually are fairly in tune with new technology. So you get a particular type of person that might be interesting for startups or scale-ups. Luca Palmieri: So somebody who is a little bit more into staying up with the trends and just try and get around a little into the open source community. So you might get developers who are a little bit of a different profile from what you find, for example, in more established ecosystems, but it depends very much on what type of qualities you're trying to select for. Luca Palmieri: What else do you have? Then you have also interoperability with existing languages. So am I going to be able to use this efficiently with the rest of my stock? There's usually not a massive problem, especially we're talking web and back ends. Usually we're running on a microservice architectures, so communication happens over the network. Not a big deal. Luca Palmieri: What's the last one I wanted to mention? I always forget but there's a last one. Kaelan: Right now we mostly do frontend. Exclusively do frontend, unless you count mobile, I guess. But telemetry is very important we know for backend services as well. It's an area that we have talked about in the past entering. Can you talk more about the state of telemetry in Rust and why that's so important? Luca Palmieri: Yeah, absolutely. I spent the last six months working in telemetry. So the state of telemetry in Rust is good. That boils down pretty much to the tracing create in the doc ecosystem. What tracing has done, tracing has made it extremely easy for people to do structured instrumentation. Luca Palmieri: So they'll be able to collect telemetry data structured response, which makes it very, very easy to then move from a tracing instrumentation to exporting your telemetry data. Which is usually where the industry is going in the backend. So that makes it fairly convenient if you have a Rust application to actually be exporting structure distributed traces to whatever you're using. Luca Palmieri: So that could be Honeycomb, that could be Jaeger, that could be Zipkin. But that's the kind of data that you want if you're looking to troubleshoot the microservice architecture. Because usually requests go to a bunch of different services, back and forth from time to time, and you want to be able to follow that call to out. Luca Palmieri: The more you can capture a structured data, which you gather through doing structured logging, the more easily you can actually search through that data, and the more easily you can actually pinpoint where stuff is going wrong. So if you combine tracing with the supporting libraries, and you combine it with drive markers, it actually becomes as easy as putting an instrument on top of a function. To actually capture something which is a logical unit of execution inside the program. Luca Palmieri: If you do that consistently, then when you look at your traces, you actually look at the logical sequence of steps that you're going through to fulfill some kind of business process. That makes it great for troubleshooting, but also great for onboarding, I found from time to time. It's very, very powerful to give some new developer joining a team, a complete trace request of the system and shows what it takes to do X. Then they can actually correlate that with the code base and easily build their own mental picture of the system. So it's good. Kaelan: Is that an area that you think Rust is uniquely positioned for? Or is it just that Rust allows you to do it in a nicer way? Or? Luca Palmieri: So yes and no. On one side, I think you get the benefit of arriving Rust. So you arrive to the telemetry problem after others have spent years thinking of the best ways of actually instrumenting applications. Then you have the unique window of opportunity to actually make that the default in an ecosystem. So then everybody benefits from using the best in breed at this point in time paradigm to do instrumentation. So that's one side of the argument. Luca Palmieri: The other side of the argument, I think, relates to the way Rust does error handling, which actually makes it slightly easier to capture in a stature fashion, some type of instrument, some type of information. The fact that Rust doesn't have exception, but Rust has added as values, makes it slightly simpler to keep in check what is being handled where? And what we're logging and how? Luca Palmieri: So that makes it easier to capture, for example, structured information about errors. That's something we do a lot for example. Is this service access failing? Okay. I want to know why is it failing? But the most important thing I want to know at step zero is, is this failing because something that happened inside the service? Is this failing because of something that's happening in another service? So is it failing because it's trying to make a call to something else that is not working? Luca Palmieri: You can collect that by actually attaching metadata to your error. You can make sure that all the errors have the metadata, because the top level handler, your web framework, actually forces the editor timed by the handler to have that metadata. So you know by actually building that framework internally, that all the applications are going to have that information. Luca Palmieri: That's just because if they're not going to compile, if they don't. It's much more complicated to do the same if you're using exceptions because they flow if propagation is implicit. You actually don't have a way to have those checkpoints, unless you have check exceptions, but that won't work with Java. From what I hear, they're not necessarily lovely. Kaelan: Yeah, that's quite convincing. Like a lot of the other problems that Rust solves, it solves it at a high level with the language end. Just makes everything easier, it seems, I think that's pretty much all we wanted to talk about. Thanks for coming on. Luca Palmieri: It's been a pleasure. Thank you for having me. Brian: Thanks for listening to PodRocket. Find us @PodRocketPod on Twitter. Or you could always email me even though that's not a popular option. It's brian@logrocket.