Paul:
Hi there, and welcome to PodRocket. I'm your host, Paul, and today, we have Ian Sutherland with us. He is a Node.js Core contributor, as well as an architect and developer experience lead over at Neo Financial. Welcome to the podcast, Ian.

Ian Sutherland:
Hi, thanks for having me.

Paul:
Yeah, thanks for taking the time to come on. You recently gave a talk about Node.js and zero-dependency CLIs, which is a very niche but very powerful topic. It feels to me at least, because CLIs are everywhere, but at the same time, how many people actually get in the weeds with it? Hopefully, we can learn a little bit about how you think about CLIs, and the direction that we're moving in as a developer body, and the exposure we're going to have to new tools as we develop CLIs in the future.

Ian Sutherland:
Yeah, it is a little niche. Although, as I mentioned in my talk, some of these same principles apply in other types of applications too, like lambdas or serverless functions, shell scripts, build tooling. Things like that. It applies a little bit more broadly than just CLIs.

Paul:
It's like CLIs is a good lens to look at this through, because the whole zero-dependency development ethos, it's continually expanding as we grow more and more. Unless you're developing in Deno, right? Because you got a whole ecosystem wrapped up into their standard library over there, which is a little bit of a different conversation, but-

Ian Sutherland:
Yeah, they do. I mean, Deno does have a little bit of a larger standard library than Node, for sure, and that's something that we've looked at for inspiration as well. I think it's nice to have other tools like that in the market. We push each other forward a little bit.

Paul:
I mean, now we're pushing forward in the Node. So, you are a CLI developer. You're not that now, but is that an area you stepped into very deeply? You really got into CLIs, you thought they were fun to make?

Ian Sutherland:
Yeah, I would say that's true. Yeah, I don't know. I'm not sure why. I really enjoy using a lot of CLI apps in my day-to-day workflow. Yeah, I got into building them, I don't know, maybe four or five years ago. I don't know, I find it really interesting. The terminal is a really interesting kind of constrained environment, and so you have to get creative when you're working in there.

Paul:
What's an example of something that you found to be a widely constraining quality that you wouldn't anticipate before you got into the space?

Ian Sutherland:
Well, I mean, anything that you're used to from web development. I mean, from being able to display an image to being able to do really any kind of layout. That all disappears in the terminal. At least for simple applications, you're forced into this line-by-line output. Very limited color palette, and character sets, and things like that.

Paul:
So, you're a color terminal person? You like to put colors into your apps?

Ian Sutherland:
I do, yeah. I mean, you got to, yeah. It's 2022. I mean, you got to move beyond the black background and the green text.

Paul:
I'm sure you're upsetting some purists, but we all love colors.

Ian Sutherland:
Yeah, I probably am. Yeah.

Paul:
Have you mostly been in the Node space with building CLIs? Because I know Rust is the CLI thing out there. People hail Rust for their CLIs. So, where have you been around the block on?

Ian Sutherland:
Yeah. For me, it's primarily been Node. I actually did start learning Rust this year for that very reason, just for making CLIs. Actually, there's some interesting stuff happening in Go, as well, recently. Some really interesting libraries. I think one of them is called Charm, where it's a set of libraries for building really rich interactive CLIs, so now I might have to pick up Go again as well. But yeah, the bulk of my experience has been building CLIs with Node.js. That's what I work in everyday anyways. Building microservices in Node, and type script, and front-end apps with React. JavaScript is where I feel at home, so it's a natural fit.

Paul:
Is developing a CLI in Node, does that present some special challenges that you might not find in a typical compile language like Rust?

Ian Sutherland:
Yeah, it definitely does. That's actually where the "dependencies" thing comes in. If you're building a CLI with Node and you have dependencies, you either need the users of your CLI to also install those dependencies, or you need to bundle all of your code up using something like webpack. Generally, that results in a much larger not binary, but artifact that you're releasing than if you used a compiled language like Rust or Go. So yeah, there are definitely some challenges.

Paul:
Do you generally want to use a zero-dependency project or design philosophy if you're building a CLI? Is that something we should all strive for?

Ian Sutherland:
I mean, I want to. I don't think it's realistic today. I would say the goal, at this point, is maybe just to minimize the number of dependencies you have to pull in, but it's pretty tricky to truly build something with zero-dependencies, at least at this point.

Paul:
Why is this something that you recommend and something we want to work towards, specifically relating to the CLI phenotype here?

Ian Sutherland:
Yeah. I mean, again, just for the reasons that I mentioned earlier. It just makes sharing and distributing a CLI app a lot easier if it doesn't have a set of dependencies that need to be installed, or if you can eliminate that build step as part of releasing your CLI. Another thing, too, is if your CLI has a lot of dependencies that you ship with it, your program's going to be slower to start, and that's a big deal with a CLI. If you type a quick command and you're expecting a pretty instantaneous response, but there's, say, a 500 millisecond delay every time you run a command while it's parsing all of your dependencies, that makes for not a very good developer experience.

Paul:
It's one of those things, like if you don't have your docker image cache right. Every time you rebuild it, it needs to yarn install. I just want to bury my face in being impatient.

Ian Sutherland:
Yeah, it sounds small, like 500 milliseconds. But yeah, if you're trying to hammer out a dozen commands in a CLI, that'll start to irritate you really fast.

Paul:
Those little things you don't think about. Yeah, if I'm running a command, in my mind I'm like, "Oh, it's bare metal." Even though it's never bare metal, but I'm like, "Oh, bare metal. She goes so fast." Really good call-out there.
Enjoying the podcast? Consider hitting that follow button for more great episodes.
When the new Node versions that are coming out now, 18 just came out recently, does this give us new abilities to reach this goal of, "We want as little dependencies as possible," and maybe, what are some of those for people that are interested?

Ian Sutherland:
Yeah, for sure. Node 18 is definitely a big step. Specifically, when I wrote that talk, I was looking at Node 18.3, because some new stuff shipped specifically in that version. I think they're well past that now, so if you pull down the latest version of Node 18, it'll definitely have all of this stuff. One of the big things that shipped... I guess I should start with the thing that shipped in Node 18.3, which was the built-in argument parser. I guess for people that don't know, you type your command on the command line and you probably pass it some flags, like a --silent, or -L, or whatever. Maybe a file name. Those are all the arguments to your program.
Previously in Node, you would basically just get an array. Where anytime there was a space in the command, it would just put those all into different array elements. That's all you got, and it was up to you to figure out what to do with that. As I mentioned in my talk, there's a ton of variations of arguments you can pass to a CLI. Actually making sense of that array of things can be pretty tricky, so that's where an argument parser comes in. A lot of other languages have ... Sorry, I was just going to say, a lot of other languages have built-in argument parsers, and up until now, Node didn't, and so that's what we introduced in 18.3.

Paul:
So that's in the standard library right now? You just get it out of the box?

Ian Sutherland:
Yep, that's right.

Paul:
Awesome.

Ian Sutherland:
It's called parseargs, and it's in the util library.

Paul:
We, as developers, have some sort of a mental model in our heads about how to parse an argument. It's like, "Okay, —flag. It's toggling something." If you do —— thing equals, that's like a KV arg. We have a model, but yeah, making sense of that interoperability layer in between. It changes language to language, it changes shop to shop.

Ian Sutherland:
Yeah, exactly. There are subtle variations of what features different argument parsers support. With pars args for Node, we tried to keep the API small and focus on the most common subset of those features. It's still not going to do absolutely everything that you might want from an argument parser, and so that's where you might still reach for one of those dependencies that exists on npm, but we wanted it to cover the 80% use case.

Paul:
If you're writing any old app, and you just need to be able to pass a variety of flags in one or two different formats, you got yourself covered with this, probably?

Ian Sutherland:
Exactly.

Paul:
Okay. Our parsing has always been the bane of scripting existence. As you said, you put it into an array, especially once you just start getting into doing, excuse me, sub-shell outputs as inputs into the arg V. Yeah, so this is exciting to check out. You can go download Node 18, we can go boot this up right now. Is this something that you are including, right now, in your current projects, or is there any standing work out there that you would point people to look for inspiration?

Ian Sutherland:
I'm definitely starting to use it in some scripts. A pretty common use case, too, is maybe you write a shell script, and then you go to add another feature to it at some point in time, or you want to be able to toggle something, like, "Oh, it can work in this way or that way," and you want to add just one argument, and this is a very easy way. Without having to pull in new dependency, you can just use parseargs and add that one argument, and then maybe build up from there over time. So yeah, I find myself in those situations a lot, where I've got a simple script where I just want to accept one or two arguments, and so I'm definitely using it in those kinds of situations already.

Paul:
Gotcha, Before we switch off of the new Node 18 stuff, maybe let's pull out one other exciting standard library. Something that's now in the standard library, that wasn't there before, that you're excited about that you think can empower developers.

Ian Sutherland:
I mean, I think the other really exciting feature that shipped in Node 18 is Fetch. Fetch is a web API for making a request to another URL, and that was notably absent from Node for quite a long time. The built in HTTP library is not great for just doing something like hitting an API endpoint. I talked about this a little bit in my talk. I made a shell script that does this, it hits the GitHub API. Using the built in HTTP library, I think I ended up with around 30 plus lines of code, and that didn't even, I think, cover all of the edge cases. It's an event-driven API. There were definitely some events that I was not handling at all in that implementation. Then I showed the same functionality using Fetch and it's less than 10 lines of code, and that includes all the error handling and everything. The actual Fetch part is really just two lines. That's a big improvement, I think, for developers.

Paul:
Yeah, it's been a long time coming,

Ian Sutherland:
It really has, yeah.

Paul:
Now we have it, so that's great. My go-to would be just Axios, but then, yeah, it's a dependency. It's something else I need to include.

Emily:
It's Emily again, producer for PodRocket, and I want to talk to you. Yeah, you. The person who's listening, but won't stop talking about your new favorite front-end framework to your friends, even though they don't want to hear about it anymore. Well, I do want to hear about it, because you're really important to us as a listener. So, what do you think of PodRocket? What do you like best? What do you absolutely hate? What's the one thing in the entire world that you want to hear about? Edge computing, weird little component libraries? How to become a productive developer when your WiFi's out? I don't know, and that's the point. If you get in contact with us, you can rant about how we haven't had your favorite dev advocate on or tell us we're doing great, whatever. If you do, we'll give you a $25 gift card. That's pretty sweet, right? So reach out to us. Links are in the description. $25 gift card.

Paul:
Well, moving on past Node 18. I would love to pick your brain a little bit about how you design CLIs, if that's all right? Is there an anti-pattern, or something that you see in CLIs that gets perpetrated that you think really shouldn't be? Something that hurts the way you think CLI should run? Off of that, are any of those easier problems to solve if you use Node as your CLI language?

Ian Sutherland:
I would say, I mean a couple things that come to mind would just be not following conventions. If you're used to working in the terminal and with CLIs, you probably know all the standard utilities that come with Linux or Unix. They all follow a very similar set of conventions. That makes it so that you can connect them all together very easily, and so you should definitely try to also follow those conventions as much as you can when you're building your own stuff. We talked earlier about some of the standard ways that you are maybe used to passing arguments to a CLI. Try and stick with that. Don't reinvent the wheel, don't do something different. Definitely, that's somewhere where using a library, like parseargs, or whatever it is that is included with your language, will help you stick to those conventions.
That's definitely one rule that I would try to stick to. Then, some other things that I see people doing. I mean, you should treat the interface and the output of your CLI the same way you would treat maybe a web application. You should try and have things like consistent headings and texts, whether that's font sizes, or colors, or whatever. Try to stick to consistent patterns that make the output easy to read and understand. Then another one that goes along with that is, also have a mode that doesn't do any of that fancy stuff and just output's the raw text, because that's probably what you want if you're going to be piping the output of that command into another command line utility.

Paul:
Thinking up that line of like, "Whoa. When you're designing, you should really think about any output. You should have a flag where you can output raw text." These modular objectifying design decisions are really interesting. Is there some way you wrap up a required help argument into your CLIs, like as a wrapper or subclass? Like, "Oh, this is an argument. It's a subclass of this," or do you use the arg parser to implement help somehow? Do you force that upon every command?

Ian Sutherland:
Yeah, that's a good question. Someone else asked me this on Twitter after my talk, too. The parseargs command built into Node right now doesn't do anything in terms of help. I'm not sure if we're going to add that or not. You could certainly take the "options" argument that you pass into parseargs and use that to generate a help output. That would be, probably, a great npm library, if no one's built that already. There's some more full-featured argument parsers, like yargs or commander are two of the popular ones on npm, and they definitely do that for you. If that's something that's really important to you, or some of their other features, then you may want to consider still sticking with them. I mean, I definitely recommend. You should. —H or ——help should always, always, always work in every CLI.

Paul:
Right. Yeah, implementing that standard interface. Thinking about rest, there's so many packages out there that won't even let me transpile my thing if I don't have the right endpoints, the right documentation in there. Wonder if any similar guardrails you make, even just for yourself privately, when you're developing. I mean, one of those is testing. You must test everything extensively before you release it. What are some of the strategies you use to make sure you're testing your CLI, or tools you use to do that?

Ian Sutherland:
Yeah, that's a tricky one. At least at work, with any of the CLIs that we build, we focus more on testing the functionality of each command, so a particular command might call some library methods that do something with the file system, or make web requests, or something like that. We generally focus more on testing that functionality versus passing different kinds of arguments, and things like that, to the program.

Paul:
Yeah, just making sure. It's almost like unit testing the call sub modules that the CLI, because you're making the assumption that the CLI is going to route the commands correctly.

Ian Sutherland:
Right.

Paul:
It's more like, "Let's unit test the end targets." Okay, gotcha.

Ian Sutherland:
Right, exactly.

Paul:
Do you just use a typical plugin and play, like JavaScript testing frameworks, for that type of stuff?

Ian Sutherland:
Yeah. At Neo, we mostly use Jest. We use that for testing our CLI apps, as well. I mean, this seems like a good segue. There is a new test runner also built into Node 18, so that would be a good starting point if you're building just a simple shell script or CLI.

Paul:
Okay, so we have extensive testing now part of the standard library as well, and fetch, and we got an arg parser. What do you see coming maybe next year, or the year after that, for what you're hoping to see? You mentioned briefly, you maybe help. If somebody wants to start tackling that, a required help message for your arguments. What are some other exciting areas that you're hoping to see push into the space, either through the lens we're talking about now, through CLIs, or anything really?

Ian Sutherland:
Right. Yeah, so I included a section on this at the end of my talk, which I referred to as the Wild Speculation Edition, because this is not really based on anything. I don't speak for Node. I don't know what's going to be happening in the project over the next year or two, so I included a couple things that I personally would really like to see in Node.js. I might do what I can, as part of the tooling group, to try and push those things forward. Yeah, I really can't say for sure if any of this stuff is going to happen, but... All right, I'll give you my-

Paul:
This is not financial advice.

Ian Sutherland:
Yeah. Exactly, exactly. Yeah, so I think a couple things that I mentioned that could be interesting. One would be support for glob. It's probably familiar to terminal users. That's how you can specify those star and star-star patterns that will match any number of files, or directories, or nested directories. I'd love to see support for that coming to Node. We've added a bunch of recursive file system operations recently to delete and copy files, and glob just goes really nicely with those kinds of APIs. That's a small-ish one that I'm hopeful we can get in, and then things a little bit crazier. We talked about Deno earlier, and how we're taking some inspiration from them there. Deno has a, I think they call it Deno Compile. That'll actually compile your JavaScript code and Deno itself into a binary. Because that is one issue with shipping Node CLIs, is you're relying on the user having a compatible version of Node installed.

Paul:
Yeah, the right version. Yeah.

Ian Sutherland:
Yeah, exactly. I mean, that's the other tricky thing. If you start using some of these new features, they're only available in Node 18. They happen to have Node 16 installed, or something, it's not going to work. The idea there is, you're bundling up the run time and the code into a single binary that you can distribute, and that's something we've been talking about for a long time as something we want in Node. There are a couple third-party projects that will do this for you for Node, but I would love to see that built in as a first-party thing. Then the last thing that I would really love to see, and there has actually been a little bit of discussion about this, is first-party support for TypeScript in Node. I think that would be huge. Node already supports, with ESM, custom loaders, and there is a loader called ts-node that exists already. Maybe if you try and execute a TS file, maybe we just automatically use that loader and everything just works.

Paul:
Behind the scenes, you just don't know that it's using the loader.

Ian Sutherland:
Right, exactly.

Paul:
Node magic. There's a lot of node magic.

Ian Sutherland:
Yeah, there is.

Paul:
Well, if somebody wanted to go create a CLI today, if they've never done it before, or maybe haven't stepped into the world of no-dependency development, what would you say is... Let's start with the resource. What's a really good resource that you might want to turn people to? It could be like something you wrote that you made as a tutorial off a bunch of comments, something you saw on YouTube that was really helpful.

Ian Sutherland:
That is a good question.

Paul:
I'm really putting you on the spot here, so it's okay if you don't got one. Do you have a GitHub with both public projects? Because I'm sure you have CLIs there that people can go look at.

Ian Sutherland:
Yeah, for sure. I mean, I would say, first of all, definitely watch my talk from CascadiaJS when it's available.

Paul:
We can link that below in the video.

Ian Sutherland:
I actually started working on kind of test project for some of these Node CLI features. I started making a package manager. I was trying to think of something that would take advantage of all of these different features. The argument parsing, fetch, file system manipulation. I landed on a package manager, but I wanted to really emphasize the fact that no one should actually use this package manager as a package manager, so that's called The Bad Package Manager. You can find that on my GitHub, and that's a little bit of a starting point. I haven't done as much as I had wanted to with it. It's kind of a slow project I'm slowly working on, but that could be a good starting point. Also, I can maybe include a link. Someone, after my talk, actually put together a little example of using parseargs and shared that with me on Twitter, so we can maybe link to that example as well.

Paul:
Great. I mean, that's plenty. Bad Package Manager, Ian's working on that slowly, but surely, right now. Don't use that as a package manager. [inaudible]-

Ian Sutherland:
Please don't.

Paul:
... in the name, but it's a good starting point. I'm sure if you Google that, and then the word "GitHub", you'll be able to find that on the entry web.

Ian Sutherland:
Yeah. It's on my GitHub, which is just iansu, I-A-N-S-U.

Paul:
Awesome, and then we'll try to include a link to this parseargs example that you just talked about. Then yeah, if there's any words of wisdom that you would give people getting into, just in general, no-dependency development. What's the hardest mental shift that you had to go through stepping into the space?

Ian Sutherland:
Well, I mean, it's definitely going to be difficult, like I kind of said at the beginning. I actually don't think it's realistic to build any significant projects with no dependencies at this point in time, but what I would say is: We are constantly adding new stuff to Node, so don't just assume that you need to reach for that third-party dependency. Take a quick look at the docs and see if what you're trying to do is built-in.

Paul:
Awesome. One last thing, Ian. If people want to follow you, are you on a social, or a medium, or Twitter?

Ian Sutherland:
Yeah, you can definitely find me on Twitter at iansu, I-A-N-S-U. Same as my GitHub username.

Paul:
Awesome. Well, thank you for your time, Ian. Yeah, hopefully some people are inspired to maybe step outside of the typical compiled CLI libraries that I see popping up all over the place and take advantage of these really awesome new Node features that we've been waiting for a long time.

Ian Sutherland:
I hope so, yeah.

Paul:
Yeah, we all hope. All right. Thank you, Ian.

Emily:
Hey, this is Emily, one of the producers for PodRocket. I'm so glad you're enjoying this episode. You probably hear this from lots of other podcasts, but we really do appreciate our listeners. Without you there would be no podcasts, and because of that, it would really help if you could follow us on Apple Podcasts so we can continue to bring you conversations with great devs like Evan You and Rich Harris. In return, we'll send you some awesome PodRocket stickers. Check out the show notes on this episode and follow the link to claim your stickers as a small thanks for following us on Apple Podcasts.