What’s new on Node.js Test Runner with Lucas Santos
===

[00:00:00] Hello and welcome to PodRocket, a web development podcast brought to you by LogRocket. LogRocket helps software teams improve user experience with session replay, error tracking, and product analytics. You can try it free at logrocket. com. I'm Noel, and today we are joined by Lucas Santos. He's a senior software engineer at OpenVault here to talk about what's new on the Node.

js test runner. Welcome to the show, Lucas.

Thanks a lot for calling me and super excited to be here talking to you about this.

Yeah, I'm excited about this one too. I feel like this is a thing I've like, known about for a while, but just haven't taken the time to go explore, and I suspect ~there's quite a few. Hopefully. ~Hopefully. There's quite a few other devs kinda in my shoes here as well. So what motivated you to kinda go check this out and, spread the word?

Actually, ~it was quite of a ~it was, Two things. I, before OpenVote, I used to work at Klarna. If you're in the United States, probably you know this a lot. Klarna is very big, becoming very big. But in Europe, it's super big. It's a very big fintech. And in Klarna, we [00:01:00] used Jest as a test runner and Klarna is a very big company with a bunch of people and it's like over 10 years old and as you can expect we have a lot of legacy code in there and One of the things that motivated me a lot was that I really hated to write Jest Tests ~in, in, ~in that environment.

And it was like, man, ~that should be something better. And the, ~there must be something better than this in the entire, ecosystem. And then ~we took a look~ we took a look at the most common ones, like ~v ~V test for JavaScript was the best solution. And then, node, the Node team, said that they were going to create ~a Node~ a Node test runner, like Deno does, like Deno has a test runner, Ban has a test runner, Node should have a test runner too.

And then, when they started doing this, I started ~You know, ~following them up. I have a friend that is part of the TSC. The TSC, the Technical Steering Committee for Node. js. And he was like, oh, did you see this new thing that they're doing? It might be super nice to do. And then he started ~like ~fiddling around ~he, ~I'm going to put ~it ~some code in there.

[00:02:00] So I just added this mock things in here. Maybe you can try them too. Like it's a nice thing ~to, ~to learn and to see things, how they work. And then it actually started with ~my, my, ~my whole advocacy thing started with me actually sending code to the core of the Node. js test runner. So like I implemented what is the date mocks in test runner.

And he implemented the timers. So basically when they did this, it was like, damn, this thing is super nice because in the Node. js project, we already use the test runner to run the tests so you can actually run them in using the same test runners that we were using today. And then it was like, damn, that's super good.

And then I started, people should know about this. Like it's something that. A lot of people are using like Jest VTest, even before that, like Jasmine, Mocha, things that are super, super old, and there are new things that are coming up, and things that are native to the runtime. You don't need an external library to maintain, you don't need something that it's maintained through other developers, you don't need like, [00:03:00] because Jest is, today Jest I think is still maintained by Facebook so basically, you rely on them.

Be able to make them. And the good thing is that the Node. js test runner has the hands of several developers. There are in the Jest that they are also core developers of Jest. So they also want to move ~a lot of~ a lot of things. In the Node. js native test runner, and we have a lot of other libraries.

So like, it's a joint effort of a bunch of amazing people that are just like, let's bring everything that we already do today with these thousands of different libraries into one single place where people can actually Use them without having to download and configure and run You know, documentation every time.

~So it's, ~that was my motivation. Basically. I want people to stop writing Jest configs and having problems with Jest every time, because like I've had them for long enough. And I think that's the main reason.

Yeah. So I think, ~Yeah ~[00:04:00] unification and standardization are nice goals in and of themselves. Is there anything in particular, like functionality wise or behavior or like with configuration like you just mentioned with, the existing tools, like Jest is probably a good example. It's probably the frontrunner.

That feel particularly painful to you? Are there problems that the node can't handle? Test runner solves.

Yeah, I'm going to talk about Jest because that's the one I use most. ~But there's~ Jest is a great tool. I'm not against Jest. ~I'm not~ I don't hate Jest or anything. I think it is a great tool. Has a bunch of things like it built up. The testing culture that we use today is ~like ~by far one of the easiest ones to implement and one of the Most complete ones ~to ~to use but it has two key problems for me The first thing is that it is extremely big.

It does a lot of stuff So jest is not only a test runner. It is a transformer It can actually change your code to, let's say, you want to, Jest has these transformers in their [00:05:00] configuration that you can put there. So you can write your tests in a way that you can use new features from ECMAScript.

You can use Babel transformers, you can use plugins, you can use a bunch of stuff. It's not just a test runner. So Jest has grown to be an ecosystem of stuff. And the Jest configuration file, if you're listening to this and probably you use Jest, you have to maintain a file that's called Jest.

config. js. And this file usually is the pinnacle of every problem that you're going to have with Jest in your life. Because if you want to, let's say, Set up TypeScript with Jest. You have to use a plugin called usually TS Jest is the most common one, but there is others. And from that, you have to use like transformers and other stuff that will actually transform your code from TypeScript to JavaScript.

So it can be run in Jest and sometimes it does other things. And it's a pain because when you have an error, it's not a Jest error. It's a plugin error that is going to be, Bubbled up as a jest error. [00:06:00] It's going to be a super huge error stack. It's not easy to maintain that file, especially if you like, when I'm saying this, you were relying on two different libraries, right?

Jest and TS Jest. So if some of them, one of them breaks or they're incompatible, boom, you already have a problem. So that's the big, that's the first reason, like the jest configuration file and the jest features are just They're not just a test runner, ~it's super, ~it's a super app, like of super library of tests.

The second one is performance just is widely known to be not very performant. When you have a super big test, like a bunch of tests and ~kle, ~I had ~a~ a service that had a thousand and. 500 tests. And if you had to run it using Jest, you have to stop using your computer. We had M2s Apple M2s in, in in Corona.

And ~it ~they couldn't handle it. So like sometimes you were running the tests and the computer just. Stopped working ~they ~until you finalize the [00:07:00] test and this is not only us in clarinet There is a bunch of other performance issues in this case. It's not actually a jest problem to be honest.

It's how Jest uses the watchman Which is the file watcher in there and this uses a lot of memory. So We used to say that Jest is like the Google Chrome of the test runners because it sucks in all the RAM that you have available to run your tests. And if you're running like integration tests, which are usually heavier because they have infrastructure that you need to, spin up stuff.

Then it's even worse. We had to run integration tests in band, like one after the other. It took 30 minutes to run and it consumed a lot of memory. And in the test runner, like in the Node. js test runner, these things are actually now Way better than jest because jest taught us what not to do on these things.

So in the Node. js test runner, if you run it, it [00:08:00] consumes ~very, a ~very low amount of memory, like some very low memory footprint. And it is very fast and it doesn't have any configuration files. You don't need to configure anything. So it is what is supposed to be a test runner. So if you want to use it, it's a test runner and it's what it is.

It doesn't propose to be anything else. It doesn't have transformers. It didn't have figuration files, all those things that people usually put ingest because it was like a big thing.

Yeah. So are there use cases where just has all this ecosystem built around it, all these tools within it, because it's ~is ~trying to do all this stuff. Are there cases where that is still useful that you would say like, and like this, you know, ~ this, ~these projects should probably stay on just, or is it like keep using just for that use node test runner for this 

if you want to change, let's say, oh, this probably should probably use Jes or this one should probably use no, there's ~no, ~no difference. Everything you can do with Jes, you can actually do a no test runner. The thing is to do some things with the no test runner, you would need other libraries like, let's say mocks for [00:09:00] instance.

Mocks ingest are, quoting native. They are not super native because they also use Sinon on the backend as well. But which is another library. But Jest has them out of the box, right? So you install Jest, you already have the Jest box out of the box. That's one of the key features of Jest, one of the best, in my opinion.

And one of the things that we wanted to bring in to the Node test runner, but there is a, it's a question of design principles. Because. If you want something to be a test runner, you don't need it to have mocks in it. So where do we put the mocks? Because the mocks are also part of the test, but it's not part of the test runner.

So if you want to put them on the test runner, you need to have a different class. So A lot of people in there are, like, suggesting that we put in the assertion library. So Node also has an assertion library. And Jest also uses external libraries like Chai and there is another, Chute, I think, is another one that you can use oh, it should be something, or assert is, it's more eloquent way to, to see [00:10:00] it.

But Node has the default assertion library, which it shipped from node. I don't know, since no eight or 10 already has it. So maybe we put the mocks inside the search. So you can put assert mocks or something, and you can mock the modules and stuff. So if you're using jest and if you're using node, I would say that you can use No jazz test runner, but you're going to need some external libraries like sign on to mock stuff it may be doing timers and stuff.

Now, not no. Now we don't need this anymore because they're ready in the code. But that's the thing. And just won't need just, but Everything that you can do with Jest, you will be able to do with the Node. js test runner. Some of them uh, requires a bit more configuration in the Node. js test runner because it is not meant to be created from the ground up.

Like Node. js test runner is not meant to be controlling this, but Jest does because it's like an entire framework. So I think that's the only difference.

So earlier you mentioned ~the, like ~some of the difficulties with TypeScript ingest projects. How does the node [00:11:00] native. Test runner solve these. What's the story look like there?

Yeah. So a little bit of contest. I'm from Brazil and I really do love TypeScript. So even I do even have like a TypeScript course here in Brazil that I do. And like everything that I try to do for the past five years is only TypeScript. And the biggest problem that we have in Jest is that Jest wasn't built with TypeScript in mind.

So when you create a Jest project, the first thing you're going to notice that the Jest configuration file is a JavaScript file. And it doesn't support ESM. That's one of the big problems. Like Jest has some problems when you have to use ECMAScript modules instead of CommonJS modules. So then the Good things that we're trying ~to, ~to use, like ~the, ~when you're trying to migrate everything to ECMAScript modules, you always have problems with Jest.

So that is one of the things like configuration of the TypeScript environment using Jest is a problem because usually [00:12:00] what you want to do is like, you want to have everything in one technology, right? So you want to write your tests in TypeScript. You want to have your code in TypeScript as well. You don't want to have JavaScript in the tests and TypeScript in the code.

So that's. ~Different stuff. So that is ~one of the main things I would say. But in node, that is very straightforward because since we are using ~just the TypeScript, ~just the test runner, the only thing that it does is basically it goes through your files. It's going to read those files and it's going to execute those files in an environment, and then it's going to capture the results for that from those tests.

And it's going to. Present you with a report. So TypeScript, ~nothing it's, ~it's nothing more than just extra JavaScript, right? So if you're just want to run TypeScript code, what you want to do is ~like, ~you can use this in Node. js already. You don't need Transformer or plugin or anything in jazz. You can already do this in no jazz natively by using importers used to be called loaders before, but then now it's importers.

And the idea is that an importer is basically a function that's going to take in the code that is [00:13:00] being loaded ~in the, ~in the runtime before it's being run. Right? So you're going to load the file and then you can transform that file. And then you're going to pass on ~to the, ~to the runtime as you would.

Normally. So TSX is one of the newest one. I think it's one of the best ones. And what you need to do to run TypeScript in the Node. js test runner is basically tell Node to transpile those files to JavaScript before it runs. So it's as simple as putting like just ~a. ~A flag like dash dash import equals TSX and it's going to run like there's no configuration needed And that for me that's magic because in jest you need to do a lot of stuff ~And ~you need to add ts jest you need to add transformers if you're using esm That is a whole other thing and when you're doing Node.

You can actually just use the TSX runner and it's going to take your configurations. It's going to run your configurations. ~You can write the same. ~You can write whatever you want. It's going to run normally. [00:14:00] So for me, that is the Best thing ever. And there is the whole thing like of the, if you're using integration tests, usually sometimes you have these chest environments, which I, to be honest, I never understood how it worked, but you have this chest environments that you can spin up and no, it doesn't have this as well.

So it's way less configuration that you need to do with something that you need to. Actually be more you can focus more on the code, right? You can actually focus on the tests and what you're running and the barrier between you're actually running your tests and creating your tests is way smaller, which makes people want to write tests more because it's easier to run than setting up.

And this is not only for Jess, that's other, other libraries as well, but like setting up the whole environment first, so you can run the tests. You don't need to do this. ~Like you just. ~You can just write a test and run it.

Nice. Yeah. Can I, in that same vein earlier, you'd mentioned like the performance gains, just [00:15:00] how much less lift there is here on the machine side, is there ~any, ~anything more to that story beyond just the this is, it's no native, it can, it's a little bit closer to the underlying machinery.

How is this like performance gain? 

yes, there is a few things like I wouldn't say it's because of it's no native because like just a super big has a bunch of features that no justice runner doesn't have yet. So probably that's one of the reasons why just is slower overall, but they're both fast if you're running unit tests. But the thing is when you're using the no justice runner, you already have The people that built this originally they work on the core, right?

~The, ~the test runner is built in JavaScript. It's not good in C but you can actually, we have a very amazing, like an amazing team of people that are just. Doing performance in node. So they are actually doing things to make these code performance to make it fast, to make it [00:16:00] run super easy and stuff like this.

So for now, I would say that it's not fair to compare the performance between just the node, because they do have a long. Difference of, you know, features like Jess runs a lot of stuff before it even runs your tests and node doesn't have a lot of the features that just does now. So maybe that's one of the reasons, but in general, I would say that the performance of the Node.

js test run will eventually, it will always be bigger than ~the, ~the ones from Libraries, because you're running from the native code, the one that is inside the core, instead of running from an external library or having to. So you can use the module resolution to, you know, find a library, to run the library, start up the library and stuff like this.

~it's ~it will eventually be slightly smaller or slightly faster in the future probably can be super faster than the others. It is still remains to be seen. [00:17:00] I don't want to say, oh, ~it's going to be ~it's going to be awesome. It's going to be super fast or not. ~It's, ~it might be ~faster, fast, ~slightly fast.

It depends.

~sure. Is there like, I guess ~in that vein of there being slightly fewer features and stuff going on right now in the node native runner. ~Is there like,~ despite that, have you guys seen a pretty good uptick? Are a lot of projects taking this on and dev switching over?

Yeah. Not as much as I wanted to, like, to be honest, like an open book, ~we always, ~we only use the Node. js test runner. So that's ~one~ one company that's using, we know that there is other companies that are still using, but the big problem is that the Node. js test runner is only available in Node 20 and most companies.

Are still in node 18. So the version of the node environment is the one that is blocking people to do stuff. So what happens is that When you're big, when you're in your big company like Klarna, Microsoft or any other companies, usually ~you, you, ~you're using the Node. js long term support version, which is, I think now I, I still think it's the 18 [00:18:00] for now, but yeah, until April or something, then it's going to be the 20.

So the thing is you need to have the new version of the node environment to run the test runner. And you need to have. It running your projects, right? So sometimes people are ~just ~not using it because you have to migrate from one version to the other. And sometimes people are not using it because you have to migrate your Jest code to Node.

js test runner, which is not a big migration because Node also supports the describe it and stuff like this. But if you're using like native Jest features like mocks snapshots or Even ~some, ~some of ~the, ~the assertions that he uses, you're going to have some not difficult migration, but it's some work.

It's not you know, something easy to do. So probably I think companies are ~not, ~not migrating right now because one, it's not that ~the, the, ~the support version and two, because it takes a while to do so. So eventually I think when the node test [00:19:00] runner, Pairs up in features with jest, probably that's going to be a huge migration you know, exodus from one to the other.

Sometimes even if you want to create like a script to convert one to the other, that's not that difficult as well.

You mentioned timers and dates earlier that's the piece of the code that you were working on. I feel like when people talk about tests, timers and dates are like the devs have like, traumatic experiences with time mocking, right? Like it's always the worst. Is there anything novel here ~on the, ~on the node front?

Or is it just like taking the same abstractions and trying to make them as easy to use as possible?

Yeah. Like I always say ~that ~there's ~two ~three things ~like in, in the. ~In the software development that is difficult. Everybody knows the first one is ~the ~naming things. The second one is synchronous programming, and the third one is working with dates. So the thing is that ~when I when I ~when I included the date box in the note test runner, my goal was to do something that Just does in my opinion.

It's very good. In [00:20:00] the fake timers So you basically create the fake timers and ~it ~it detaches the time from your tests From the real time and you can actually manipulate that time the way you want So that is fine super nice for now it's been a bit stalled mostly because ~we the, the, ~we've been working in the, ~this this ~core part of ~the, ~the test runner.

We've been doing a lot of other stuff together. So like we've been doing other events, we've been doing other presentations we've been doing we have to work on other stuff. So ~I couldn't have, ~I didn't have time yet to continue that. But from my side. I wanted to include more timer mocks on this.

Like the next two things that I want to include in there that's already in the roadmap is the mocks for the performance timers. So the performance dot now HR time and stuff like this, which is a bit more complex. ~It's not that. ~It's not used ~like like ~a lot in most of the places, but it's something that is very important to be mocking because some people [00:21:00] use this performance stuff now or anything to actually~ actually mock the ~tell how something ran from~ like ~seconds elapsed from one to the other and stuff like this.

So you have to be able to mock this. And after this. The next thing that I wanted to do is snapshotting. So being able to create snapshots in the Jest like the Jest way, I think it's one of the biggest features that Jest allows you to do, which is like inline snapshotting, creating snapshots ~from, ~from stuff and doing snapshot testing.

That is one of the things that I really wanted to add in the Node. js test runner, because that's one of the features that I use the most in Jest. And I think it's super, super good. And eventually there are other proposals in there. One of them is to improve in the module mocking side. So instead of mocking, just functions, we will be able to mock entire modules from the ones like justice, but just is rather confusing on this part because he has auto mocking.

It has hoisting, it has a bunch of stuff that [00:22:00] he does. It sometimes doesn't mock the module correctly. But. From my experience, I think the way that is a, that is a discussion that we can go on and maybe here today or some other time, but I don't think you should be mocking modules by injecting stuff in.

That's my personal opinion by injecting your code inside these modules, right? Because how Jess does is that if you mock the module, it. Basically captures the, the require when you do the require and replaces for your mock. What I like to do is dependence injection, right? So that's one way to mock the modules in the easier and more predictable way of then just mocking the whole thing together.

But you cannot run from these if you're using native modules, right? So FS or crypto, it's important ~to, ~to be able to do this as well.

~Yeah. ~Yeah. I feel like there's probably a lot of devs in that camp where it's like, Oh, if we can always dependency inject, that's nice. And like, everyone has this and like, you get into the weeds on some projects and you're like okay, there's like a [00:23:00] lot of overhead here. I'm just going to mock the module and this'll be fine.

Like I think it's doable. It's just, it's one of those like cost benefit analysis questions.

~I think, ~I think one of the most important things that people have to understand in this part is that if you have, let's say one of ~the, ~the biggest ones there is a module called UID and ~ no, ~no, it already has this Random UID function, right? So you don't need the UID module anymore. So a lot of people use this and in Clarinet we use this a lot and people used to mock that module, like mock directly.

So if you change the module, like, Oh, I don't want to use UID anymore. Now I want to use the NodeJet, Node uh, Crypto random UID, all the mocks just stop working. In one go, so if we're using dependence injection and the function is the same, you can actually inject another function inside and you can continue to have your tests running because you don't care about which module you're doing.

You just care about the interface that you're interacting with. That should be how we do things, [00:24:00] because if you do these, you know, proxying from one thing to the other, usually you get very reliant on the underlying libraries that you're using. So the library changes the name, boom, your, everything breaks down.

Would you extend that like almost universally? I'm thinking of just like another common use case. It's I don't know. So you have a request library, right? It's okay, I'll just go in and have my test. Just override the request library for whatever this test is. Would you argue that those should also be like dependency injected?

Like I have a project in my GitHub that's called Gothic UL. I done this a long time ago before ~I, ~there was like Apollo or anything. The idea was that this was a request maker, like Axios, but for GraphQL stuff. And you can actually. Pass on your client to this. So it used to be got, but you can pass whatever you want.

And the only thing I care about is that the interface is the same. So I do agree with this. Like if you're doing a request library, the only thing you [00:25:00] want to do is I want to make a request. If you don't pass anything you were going to use the one that already have as default, but if you do pass, I'm going to use the one who passed me.

So that is ~the, the, ~the thing that I like to do. Like I like to. Inject everything, but ~I can, ~I can relate. That's sometimes it's a pain to just move things around, like from top to bottom, ~react, ~react style. But, uh,

Let's ~like~ talk a little bit about additional tooling here, as we're talking about developer experience, is there anything just like from a developer quality of life standpoint that might, we've talked about like performance and just having fewer tools and all these things already, but like, you know, I'm thinking like cool, nice features and IDEs and like debuggers and all this stuff is this there's a lot of that stuff easier to get wired up with a native?

Test runner.

Yes, to be honest, because it not, not directly in the runner, but the test runners it outputs a top output, which is test anything protocol. So basically the idea is that you can integrate that output to basically anything that you need. So if [00:26:00] you want to move from one test runner to the other, that's easy.

If you want to move. from Node. js to whatever other coverage report you're using or whatever thing you're using. That's easy. And that's one of the big problems that test runners have right now. It's difficult to move between them. They seem to be made not to be used with other test runners. So it's difficult to move from jazz to V test is difficult to move from V test to Ava or whatever other, so.

The beauty of these is that you can output this in a TAP protocol and then you can read a TAP protocol from whatever you want to use, so it's easy to integrate.

Cool. Nice. Standards are lovely, right? ~It's ~it's so good. Nice. Is there any other like upcoming features or stuff on the roadmap that you're looking forward to or want the community to keep their eyes peeled for?

Well, since we are a very small team working on the Node. js test runner, I would actually say plead to the community to help us doing it. So, like, if you want to make the test runner a viable option, if you want to create stuff [00:27:00] you can do that. There's two things you can do. The first one is using it and sharing the feedback in the Node.

js bugs ~and ~and issues. The other one is if you really want some feature to be implemented, maybe you can just drop in some code in there, drop in some issue in there and start working on it. This is how I actually did it. Like I'm not part of the Node. js technical steering committee but I just dropped the code in there and people loved it.

And. Implemented it. So that's the one thing that you can do. So like one thing that three things that I want to see actually calling that is the original creator of the test runner, he has a wishlist for this. So one of the things that in the wishlist is like The better support for globs so you can actually search the code that one to run.

~In the, ~in the test, because today ~the, ~the filtering is not that good. Module mocking ~and is~ is another one and better assertions is another one. So that's three things that we want to implement. I will continue to work on ~the, ~the mocks of the stuff that we were talking about. Already doing, [00:28:00] but I'm going to try to make snapshot testing a thing as well.

So if I have time, because working in Node. js core is not an easy thing to do, but it's something that we want ~to, ~to implement. So if you want to help, if you're listening to this and want to help drop in, in the node repo, then say, Hey, I want to implement this thing. Let's get going.

Yeah. Very cool. Is there any, anything else you'd advise or any wisdom you'd impart on devs that are like considering switching to node test runner or, just like looking to get their feet wet.

Yeah. I think like, if you want js test runner try with smaller projects for now. Like just to get the glimpse of how you can actually run this, how you can make it run in the, the other libraries that you have and then try to migrate. Other versions of the jazz test runner or whatever test run you use to the no jazz test runner to see how this migration feels to you.

It should be easy to migrate. It's not, it was meant to be easy. So it's not something that's going to take like three months, but it's something super [00:29:00] nice that you can, you can work. So if you want to use the no jazz test runner, just. Try it, start doing small things, try to create your own stuff, writing the code and like the bit of wisdom that would say is that always try to make your code as modular as possible and try to use dependence injection as much as you can.

Try not ~to, ~to delegate stuff to the compiler too much.

Nice. Awesome. Thank you for uh, the insight, Lucas and chatting with me about tests today. It's been a pleasure.

Super nice. I always eager to test ~and to, you know, talk about it ~and talk about the Node. js and everything else. So ~if everything you want to, if I, ~if you have any other opportunities, let me know, I'm always going to be here.

Perfect. Cool. Thank you so much.

Thank you.