Brian: PodRocket is sponsored by LogRocket, a frontend monitoring and product analytics solution. Which is to say, it's not really sponsored by anyone. It's sponsored by us LogRocket, and we're giving it away for free. The podcast is free. The product is not free. There's a free trial. We could split hairs about whether or not that's free to you, but anyway, that's it. There are no more ads. If you're interested and you want us to know that you came from the podcast, please go to logrocket.com/podrocket. If you don't care, logrocket.com works just fine. Thanks. Brian: Today, Ben and Kate talked to David Luecke, creator of Feathers, a framework for real-time application and rest APIs. They talked about where Feathers fits in the full-stack framework space, pros and cons of building a modular framework, and what it's been like to maintain a project that needs to keep up with react for people to use it. Spoiler alert. It's difficult. Let's get started. Ben: Hey David, really excited to have you here. How are you today? David: I'm great. Thanks so much for having me. Ben: Yeah. So I'm excited to learn about Feather, which is your project or Feathers JS. Does it go by Feather or feathers or Feathers JS? David: It's Feathers- Ben: Feathers, got it. David: ... was like the original name. Often people add the JS name, but we'll talk about it a little bit that that might change too. So Feathers is like the code name and like the new name. I had an older name that was quite more difficult. Ben: Well Feathers it is. So could you tell us a bit about it? What is Feathers? How do you use it? What does it do? David: Yeah totally. Feathers is basically a framework for Java script and type script that you can build APIs and real-time applications with. And it works in Node, in the browser, with React Native, and soon Deno, hopefully too, which is like a new type script based run time similar to Node. And you can use it in all those platforms. In Node, you use it to create an API in this browser. Or in React Native, you use it to talk to that API. David: And I think one of my favorite features is that you can basically pick which way your client wants to communicate with the server. So instead of just having a rest API, or you also have WebSockets API that also gets you real-time events and people have added other mechanisms like GRPC and things like that as well. That's my favorite personal feature. What people also like is that it connects to all major databases from Mongo DB to Postgres. Couchbase is a new one. So it works with any database, but we have some prebuilt modules that you can use to stand up an API in one command and get all the basic functionality from it. Ben: Got it. So is it right to think about Feathers as an alternative to other full stack frameworks like Blitz.js or Redwood or elements as we had the creators or folks from those projects on the show on the past few weeks. Is it kind of an alternative in terms of those full stack frameworks? And if so, maybe we could talk a bit about how it compares and contrasts. David: Yeah. There is a couple of comparisons that came because it's been around for longer. The original one was Meteor. Another one was Firebase. The other one is express. Why don't you use it instead of Express? So for comparison, in that it is full stack, in that you can use the Feathers library and on the client side. But you still do the integration into your favorite framework, React Native, react, Vue.js. There's some great integrations somebody has done for that or angular yourself. You can pick whichever way you want to use it in your application. You can talk through an API directly. David: And on the server side, a lot of it tries to be very pluggable. So there's a combination of things that I say like use these things together and you get a really good experience, but each component you can use individually. So contrasting is kind of interesting. I think the biggest one is that it tries to create this concept of an application architecture overall that you use with services, which is basic you API surface for your external API surface that clients can call and hooks that are workflows around certain calls. So say we create a user, they'll use the service, we put the user in the database, but if you want to send a welcome email, you would use a hook to send the welcome email. David: So the contrast there is that that is all transport agnostic. So you as the developer never have to worry about, am I using HDP? Am I using socket.io? Am I using graphQL? And if you use it that way, people have swapped out communication protocol and were like, oh, I do need real time. So they swapped out the REST endpoint for a socket.io connection and just automatically got real-time events. And I think the big contrast there is that all of that is one architecture. Instead of having to add real-time events as a separate feature or something. It happens automatically the way you create your app. Ben: How do you balance building a framework that's opinionated? It sounds like Feathers is very modular and gives a lot of choice to the developers, but what are kind of the pros and cons to that approach versus just building a framework that's super opinionated, it has one right way to do everything from transport layer to database layer, et cetera. David: The balance is always tricky because you do want things to stay flexible and also adapt to like anything new that might come up. I've been overall very happy because the basic concepts of Feathers have been around for like 10 years. It came out of my university research that I did and the services and the hooks and all of those things still fit together really well. But there's still things that change over time. JavaScript improves, so you had to change from callbacks to promises to async await. And it's always nice to see when those things fit naturally into how it all fits together. But I think ultimately that's why it's good to stay flexible. But if you just start out, you do want some opinionated way of this is how things are being put together. Like Feathers has the generator that just generates the app and wires all of the things up for you in a way that just work. David: So it's an interesting challenge because I'm happy with how it works so far. And if you look at things like meteor, they tried to create this complete development platform, but then had a real struggle with keeping up with all the other development and web development. So react integration, or even the adoption of NPM for the front end was a big challenge if you try to do something that ambitious and big. So it's a balance. And sometimes it's been a little difficult because you can, for example, if you use Feathers Express, it's a full replacement for Express. So you can just do everything you can with express with Feathers, but you get much more functionality out of it if you use the service and whole infrastructure. David: And the challenge sometimes is that we assumed knowledge of express and some people just start out, generate the app and then end up running into questions that would be express questions that are considered Feathers questions. So I've been trying to like answer a lot of those in the docs, but that's I think is the biggest challenge that as soon as you rely on other things, integrate them into a prebuilt solution, you're responsible for any questions that come up with it. And you're also responsible for people being unhappy with it. So you can use Feathers with Sequelize and if Sequelize doesn't have great TypeScript support, but somebody uses it with Feathers, you get the unhappiness. You're the first one to know about it. You know what I mean? So it's definitely interesting. Ben: So you've spoken a bit about services and hooks. I've heard both those words. Maybe you could talk a bit more about what services and hooks mean in the context of Feathers. David: So where services come from, it's an older design pattern where you basically define your functionality of your API in your languages interface. So you say you have a user service, they use the service as a send welcome email, has a create user in database method. So it's basically this like surface of a testable surface of your API that is not related to anything about how it's being accessed. So there's no HTP handler and no request or response. It's just, this is my functionality of my API as my programming languages interface. David: It's really nice because if you do that, then you can really easily test it. You don't have to set up a server, you can just load it in and run it. You can add a controller or routes that actually just call those service end points. So it's really nice for developers too, because they see right away what your application does instead of having to dig into a request response handler and be like, oh, what happens in here? Right? David: So that is one idea for services and the advantage doing it that way is also, we can have different protocols that access the service. Or just load it into your application directly. And the hooks is basically middleware. A middleware is it's like express middleware. It's basically a flow on how you go through a certain request response cycle. So in Feathers, you have before and after hooks. So you can do things before, like validate the user object, check if the user already exists. And then after you can change the response to strip out data that the user shouldn't see like their encrypted password or send like a welcome email and say, hey, welcome to my app. And those two things combined make for a really nice way to build your app architecturally, just because you can test the hook separately, you can test the services separately. And the job of Feathers as a framework is to turn that into an API for you. So you don't really have to do much of an API test. David: I usually recommend to do an integration test, But because there's also a client that you can use, it's pretty straightforward. You just call your services transparently and don't have to worry about any of the connection. And Feathers does some of the heavy lifting. The most challenging one, technically you've mentioned that is just web sockets and authentication and making sure that the right users are getting the right real-time updates has been very interesting. And it now has a really, really nice way of putting that altogether. Ben: Yeah. I was actually going to ask a question very similar to kind of that. It sounds like Feathers is agnostic as to whether you use REST or WebSockets or some like GRPC as kind of the transport layer. But with those different transport layers, does Feathers have an opinion on how authentication works at the API layer? And if so, how does each of those different transport layers kind of have different constraints when it comes to authentication? David: The authentication layers been reworked a couple of times, especially in the current version four. It makes it a lot more flexible adding your own if you want to. And the basic mechanism is JWTs because they usually work everywhere. There's even like brand new protocols that support JWT authentication, which is just a token that contains some information and that you can verify on the client and the server that it's the right person making that request. David: So that's the core mechanism, which it's flexible enough that you don't have to use it, but it's the one that's recommended. For example, for real-time connections, not just web sockets, but other ones as well, Feathers attaches that information to the socket when it connects. So you authenticate, then that information gets added to the connection, and then every time that connection makes any request, it's automatically running through the same authentication process as you would for rest. It also supports OLS, but that only works to express HTP. Basically what you do is you go through the OAuth flow to block it with Facebook or Twitter, and then you go back and trade that information for like a Feathers JWT, and that you can use to make requests. Ben: Got it. Another thing I was curious about, you mentioned on the website, this plugin ecosystem. So I'm curious to learn a bit about what are some of the plugins that are available, what types of plugins, like what are the points of integration in the Feathers framework that plugins can hook into and what kind of functionality can it give? David: Yeah, so people have done some really, really cool projects on top of Feathers, so it's always really awesome to see. Once you probably going to notice or learn about first are the database adapters. So they're not part of the core framework, but instead are separate modules that different people maintain, but that all have the same API functionality. So there are some basic ways of retrieving data, initializing the plugin that all the official ones support. And some people actually have managed to swap out the database, mongodb for SQL or the other way around. Which is interesting. Not something that works for every app, but it's possible. David: So those are probably the highest level ones, the first ones that you interact with, if you create an app and have to choose which database you'd like to use. And then there is the client side ones, which there was some really interesting ones. One is Feathers reactive, which works well with angular, which is basically creating real-time streams out of the data. So what happens with Feathers is if you connect to it in your VJs app, for example, if you don't add any plugin, you have to listen to that events like user has been created or message has been received, yourself. And then figure out what you want to do with that in your UI. David: But because there is a way to query the database, that's pretty predictable. We can create things that say, find me all messages off the last two days, and whenever a message gets deleted, updated, or a new one created that fits that query, it just updates automatically in your UI. So you get this really seamless flow of creating a real-time app based out of all those different pieces, which was really neat. David: Somebody else has done that with react hooks, it's called fig bird. So you basically have a React hook in your app where you just say, find all messages from the last two days. Then you render each message in your component. And every time anything changes, it'll just re-render that specific message or remove it from the UI. So it's something I unfortunately never get to talk about enough because it's such an exciting thing that hasn't been done a lot and saying like, hey, look, you get automatically get a real-time app with react without having to do anything. That's really exciting. Highly recommend trying those. Ben: So I know there's a new version of Feathers on the horizon. So maybe you could talk a bit about what's exciting in that new release. David: Absolutely. So it got a little delayed due to well world situation reasons. Definitely also a part of the challenge to balance all of that out. But there's still progress being made. And the thing that I'm most excited about is adding schema definition. So how we talked about earlier saying, hey, this is how you abstract your API interface. This is how you structure your application. Something I've been wanting to do for a long, long time is to do the same thing for your data model. David: So you declare your data model once in a common format, and then it'll automatically generate Jason schema, graph QL, swagger docs, and all of those things for you without you having to implement all of those things manually, because I've seen a lot of apps where people had to duplicate the same model definitions in like three different places. So you have to create a JSON schema, then you have to create your sequelize model then you have to create... Now everybody has to have graphQL. So you have to like duplicate the same thing. And that makes it really, really painful to make changes. And I think that lessons learned from Feathers doing that on an API level could apply at the data model level here as well. So that I'm pretty excited about. Ben: Yeah, that is exciting. And I guess we didn't talk about graphQL before when we mentioned some of the API layer technologies. So it sounds like Feathers does or it will support graphQL in the new release? David: That was the idea. I'm torn to an extent about some of the decisions that have been made there. Just because what I found disappointing about just how graphQL came up in general was that it was Facebook engineers declaring REST is dead when we haven't even done things like exploring how could it work for real-time apps. Right? Ben: Right. David: And it has been unfortunate that graph QL didn't really have a good answer to that either. So now a lot of people are trying to solve a problem that Feathers has been trying to like approach in a different way for a long time. So I'd like to do that. The part of that is to the schemas because what I think graphQL is doing very well is just forcing a schema definition and the way on how you curry data. Now, it's interesting because if you are protocol agnostic, the overhead of making multiple requests is often pretty negligible. David: So I did a benchmark of comparing HDP rest versus web sockets and web sockets are about 400% faster on average. So with Feathers, you can get a 400% performance increase just by changing the protocol. So sometimes it's not the creating mechanism that's a problem, but more figuring out what is the best mechanism to connect your app to your client. Right? David: So I'd like to balance that a little and just see if there's ways to combine all those benefits. It was similar with TypeScript. I think TypeScript is really well done and making it really easy for developers to get access and know of all the things that are going on. But I've always wanted to be like, well, we already have a schema definition system in JavaScript. It's called TypeScript, right? Why not use it to define your data model and all of those things. So that's kind of what I'd like to explore with the Feather schema stuff in saying, how can we combine all of those things in a way that we don't have to duplicate a lot of the data model logic? Kate: Yeah. So David, I'm curious about the community. We've talked with a lot of open source folks on this podcast. One thing that stands out to me is that Feathers has been around for so long and it's interesting. Like how do you work with that community over such a long period of time, but then also like as technologies change. And then also as like the standard of open source, the bars kind of always being raised. How do you work in that space? David: That's a good point of raising the bar. Which, you know, is a good thing. But on the other hand, it's sometimes a challenge as well. I'm notoriously terrible at marketing. So my way of promoting things is look at us, this thing, this is why it works, check it out. But I think part of the main reason why Feathers got more recognition back in 2016 was because there was some people that were much more articulate in explaining that value to a larger community to understand that. Kate: Yeah. David: So it's always good to have different kinds of people that are interested in that. I think the challenge with marketing for community run like somewhat independent open source project is that people come and go. The longer it goes, the more common it is for different people to join and leave, which is fine. I mean, it's nobody's job. So often people move on, don't use it anymore, are busy with life events, and things like that. David: So I think that is the big challenge for us right now. I think for open source in general is how do we make things sustainable without being Facebook or Google or raising millions of dollars in funding? Because the open source business model is, unfortunately... Also I've been thinking about it a long time. It's also has a lot of challenges. And a lot of the VC funded open source companies were struggling or are not around anymore, which is always unfortunate because they managed to build some really great products. But if you run out of money, even if it's open, if your company was doing 99% of the work, then it's very likely that it's not going to continue into same enthusiasm. David: So my approach often was to just look at it and say, what is the minimal most minimum thing and how flexible do you have to be in order to allow people to join and contribute as plugins as well. Which I think is always the best way to get started. But then also looking at how can you get people to contribute multiple times? Because a lot of the open source, like you mentioned, has been gotten more, from my perspective, like more transactional. People, even if they contribute are like, hey, I had this problem at work, here's a fix. I'm like, this is great. There's sometimes great contributions that came that way. But that was it. And then sometimes if it's a big code contribution, you also have to, it's really terrible to say no. But sometimes you have to. Right? Because you're like, well, who's going to maintain it? Right? Because the problem always is with a lot of those projects is that two or three people are doing the majority of the work. David: And what's really sad when I see blog posts, there was a open Spotify alternative where you can host your own music. I'm a big music fan. So I buy a lot of music on Bandcamp. And that allowed you to host your own music and the lead maintainer. I was like, yep, can't do it anymore. I got really sick last year and I've been doing most of the work, but I have to step back now. And that's really, really common in open source, unfortunately. Kate: Right. David: So I'd love to just see a little more outside of the projects that Facebook or Google are doing or outside of the project Amazon isn't doing. Kate: Right. David: To talk about how can we make this sustainable for the people that aren't working for a company? Because I got paid for a long time to work in open source as well. Which was great, but it's difficult to find the right balance to make the project relevant in the longterm. That wasn't for Feathers, by the way, it was a different project. Kate: Yeah. David: So I honestly don't think I have an answer. Kate: Yeah. David: But I'd love to have that conversation just because I think it's important and it's becoming more and more important. And I would love to see companies like Facebook and Microsoft to participate, but also not own all the developer mind share. Because like I said with graphQL, I think there's still other good ideas out there. Kate: Right. David: We just don't have the marketing power to get heard off sometimes. So I think like having that balance and for people that are using open software to also have that curiosity to try things that might not be what everybody's talking about. Kate: Yeah. Totally. David: To try like some niche things and being like, oh, this looks weird, but I'm going to give it a try. To get to that next level. Because I mean, ultimately, we're hopefully all in it to make technology better overall. Kate: Right. So kind of another point that you just talked about in your experience in open source. Maybe we can talk about your background and kind of web development. I know you have a lot of experience. Maybe we can start your experience in open source as well as other roles. David: So I was thinking about that earlier today and I was like, yep. I made my first contribution in 2007, which was to a framework called JavaScript MVC, which was definitely ahead of its time at that point. Because six years before React or Angular, they were trying to create a Ruby on rails for JavaScript. So at that point in time, the whole idea of creating apps in the browser and rendering them and just fetching the data was not very popular. David: And they worked on creating this Ruby on rails inspired version of a JavaScript framework. It had like a model layer. It had a view engine. It had a module package manager, all those things. And that's when I made my first contribution and I got like really interested by this like, oh, just fetch the data and do everything on the client? And it's coincided exactly with the first iPhones. So that's when app development took off and I was like, well, you basically have no choice. If you want to provide the same data on your app and in the browser, you have to do it this way. Right? There's really not many other options. David: So while I was finishing university in 2010, I purposely sought out this project in saying, how would we make this good on the service side? So Feathers first incarnation was a Java project that was using remote method invocation and XMLRPC. I even tried SOAP if anybody's still... Whenever I talk to anybody about it, everybody just either shakes their head or doesn't even know what it is anymore. And then also an interface for rest, which was becoming popular at that time. So that's where Feathers originally came from. And then when Node JS came around two years later, after I moved from Germany to Canada, it was really interesting because I could apply a lot of those things that I researched during that time. And it all fit together in a way that I was really happy with. David: And then you could just ask real time to it because Node was one of the first that had really good web socket integration. So all of a sudden you had this feature that I didn't even intend to originally be like, oh, it actually fits really nicely. So that was really how all of that came together. And yeah, after that, it was around for a couple, three years in like a in a niche thing. And then, like I said, I had some people I worked with that were collaborating and presenting it in a different way. So Feathers didn't change, but all of a sudden, everybody paid attention to it. And it was really interesting. Kate: Yeah. Kind of goes from like that hipster level to common use. Yeah, maybe you can talk a little bit about how Feathers you've used it, like in other projects that you've been working on. David: So I used to do a lot of frontend consulting for larger companies that if it was a Greenfield project, I was able to use that, which was nice. And in our flagship app at my company at the Bidali, it's an app where you can buy gift cards for a hundred different countries and get discounts cash back. And we're using that to doing all the real time. Like basically all of the API layer is built in Feathers and that's how you get like real-time updates when somebody sends you money or if new offers are coming in and things like that. So it's been working nice and it's a react native app. So it was interesting to see that it also integrated really well with that. Kate: Great. Ben: So what are you most excited about in the world of front end engineering outside of what's on the roadmap for Feathers itself? David: I've been working a lot with public private key encryption and with decentralized file systems in the last little while, just because I thought that is a really interesting topic and where things are going. And I think that those two things are probably my most interesting developments, maybe not in the next year, because some of that peer to peer computing or networking is difficult. So it's getting much, much better, but there's still things to figure out there. David: But I think in the next five years, we're going to see a more accessible web where you as a developer can just put your app online. It's going to be hosted from your computer and you just send somebody a link and everybody can pull it from your computer. Your data is locally. It's just yours. And you can choose with whom you share them. So I think there's going to be, not just from an idealistic perspective in saying like giving people back ownership of the data, but also from a developer perspective, making it much more accessible. Because right now, if you want to put a Feathers app online, right, you have to learn Heroku or Kubernetes. David: And the options are very limited. You have maybe three providers that you probably end up hosting it with. And it's not accessible to people that don't have a credit card, right. You can't even get your app online. So I'd like to see a next iteration of the weapon. I can see like some almost the tech like coming to fruition now and coming together, which is really interesting and exciting. That we allow every developer to be able to share their app. David: And there's still going to be questions around how do you scale it? Or how do you do things like sending emails because nobody should be sending emails from their computer. But I think there is something there in making it really accessible for developers and getting like some really cool features. Like real-time automatically because it's peer to peer. You get offline first because that's how the databases need to be built. You get 100% uptime because the app runs locally in your browser. To deployment is basically just sending somebody a link. So there are some really, really interesting things happening there. David: And for the public private key and crucial part, it's just having your identity not having to go through Facebook or Twitter or Google log-ins anymore, but having your identity locally on your phone is also something pretty powerful. Because it gives you a lot of possibilities and ownership over the things that you own as data. And also it allows other developers without having to rely on Facebook or Google or Twitter to create apps that are secure by default because it's all encrypted. Ben: That makes sense. And I guess finally, like for anyone who wants to get started with Feathers, learn more, maybe like build their first kind of demo application, what's the best way that you recommend to do so? David: The best way to always start is at Feathersjs.com. There is a guide where you create a chat application from scratch. So it starts with the basic concepts of services without any server, then goes into this is how you create an API server, this is how you connect in the browser. And then just go through all of those steps until you finally create a plain JavaScript front end. So you have to eventually do your own research and how it works with your framework and what integrations there are. But I like that idea of having this like, this is how you use plain JavaScript to create an app that's real-time and also looks not too bad. David: And then it has the final chapter, which I personally consider very important, which is writing tests. So in how to write tests for your backend and how to add a GitHub login, because that's a pretty common thing wanting to add Google, Facebook, Twitter logins. So you basically get an entire chat application where multiple users can talk to each other in real time. It takes about two hours. A lot of the code snippers are copy and paste. So you can get the app working. And then even if you want to dive in more into details for something, there's links to the detailed docs. And then like you mentioned earlier, the awesome Feathers JS page has a lot of resources on other plugins that you can dive into. Ben: Thank you so much, David. This was great. Brian: This is Brian again. So it turns out that running a podcast is maybe harder than we thought. And so I kind of want to hear from you. I'm genuinely interested in your feedback. We have to think about new topics, new guests, we have to find them. And don't get me wrong, we can do it. But it's a lot easier if everyone else who's listening helps. So if you'd like to suggest the topic or volunteer to be on PodRocket, we'd like to hear from you. So you can do that by going to podrocket.logrocket.com/contact-us. The hyphen is next to the delete key if you're curious. If all of that is too long, you can just email me directly brian@logrocket.com. That'd be great. Also, if you're feeling magnanimous, be sure to like and subscribe to PodRocket. Thank you.