Vitest with Vladimir Sheremet === [00:00:00] Hello and welcome to PodRocket, a web development podcast brought to you by LogRocket. LogRocket helps software teams improve user experience with session replay, error tracking, and product analytics. You can try it for free at logrocket. com. I'm Noel, and today we have Vladmir joining us. He's here to talk about the road to VTest He's a VTest core team member and a VTeam member. Welcome to the show, Vladimir. Thank you for the invite, and I'm glad to be here. Perfect, perfect. Well, yeah, we can just jump right in. What is VTest and why should developers care? Well, why should developers care about testing their products? There you go, yeah. I guess that's a topic for a different for the whole presentation. Yeah. Well, everyone should test their code if they want, don't want them to break and Vitas. Vitas primarily helps to test the front end code using Node. js for performance related [00:01:00] tasks, , like, if you are testing your front end applications, you can go through end to end route and use Playwright or Cypress or any other end to end Selenium But it's usually quite long to write these tests. They are running quite long in the OCI. So Vitest tries to emulate this browser environment instead. So tests are usually running much, much faster. gotcha. So is it more akin to something like Jest? Or where is that performance increase coming from? Yes, it is basic, basically just like, implementation. Well, we are just running in Node. js. probably bung and yeah, basically the performance comes from the fact that we don't spin up a real browser. Real browsers are very expensive, they're creating all the contexts and we just emulate everything. . Is there any particular use cases for which VTest is particularly well suited to, I don't know, test against? Well, VueTest is a general test runner, [00:02:00] so I said before that you can use it to For front end testing. And we are mainly focusing on that, but it's running in Node. js. So people usually test also there just Node. js applications. The primary users are probably Vite users. Because Vitas simplifies the pipeline. So before with just just doesn't support asynchronous or e SMM out of the box, or it requires some flex to make it work. And it was very hard for developers who are using Vitas, their build tool chain to test something. So the team came up with a V test, which uses V under the hood. So we transform all files using the same config file. Which makes the development usually like seamless. So you have your Vite application, you run Vitest command and also just works out of the box if you have some tests. So I would say that Vitest primarily Vite Vitest. But Vitest also different from Jest, Vitest has Embraced another way of like test isolation.[00:03:00] So jest uses context API from VM module, , which creates the new global context for each test file all the time. So if, even if you write your own custom handlers, it will still use this VM module. But VTest, on the contrary, uses uh, workers for context isolation. So we just spin up a new worker, which should be faster than spinning up a worker and creating a context. So VTest has several, like, flags or configuration options that allow you to come from Jest, but still have the performance of a native test runner. Because we don't... Don't overcomplicate things like this, There's not that additional, yeah yeah, yeah, yeah. Especially for Node. js applications, it's really faster. Is that difference in kind of that global module? Is that mainly due to like differences in ESM support? Is that the main driver there? Or, are those both, derivative of the same [00:04:00] function? Well, under the hood, Vitest still uses VM module. We just use the, reuse the same context instead of creating a new one, Gotcha. simplifies also the internals because we don't need to re implement like require experts modules and so on. But , we re use VIT SSR pipeline, which kind of transforms files into ESM compatible functions, basically. So instead of imports, you have functions that require other modules. So under the hood, it's like CommonJS, but a synchronous one. So we can use VIT plugins there. but I think the reason why we used it is more historical. I think Anthony came up with the test isolation at the time. I think because it wasn't even released publicly yet. And since then it was. Just like, like a core isolation feature that we didn't really want to change because it worked well until it didn't, which there is an issue with the three times performance difference from just in a bad way, [00:05:00] which we also addressed I think in the latest version where we actually introduced the same way, the same mechanism that just uses for test isolation, Mm it turns out that there is no like common solution to test every project. Some, for example, if you're, if your project relies heavily on some global state. And you don't really want to reset it every time you write a test, or it's really impossible to reset, or some modules have side effects that cannot be reverted, then you need a isolation out of the box for when you test. So ViteSt provides you with that. But ViteSt can also be used without isolation. So by disabling isolation, we provide the same, just API, but... faster, Gotcha. like three times, three times faster. If you just disable isolation, if you don't rely on globals, if you don't rely on side effects, or if you can reset this side effects between test files, you can just disable isolation. Your [00:06:00] test will be, should be comparable to native test runner. Gotcha. Yeah, that was, you've stumbled into my next question and that I was curious if the, those changes in scope would cause difficulties for, users who are interested in trying to switch. From one to the other in that, yeah, I would assume that sometimes there is context and stuff like there's side effects to those tests running that can make it hard to convert these existing test suites, but it sounds like, like this is a configurable option now in v test. Is that right? Gotcha. Yeah. Yes. Yes, exactly. By default, we still create new workers. As a way to isolate each test file. So if you have a hundred tests, then we create a hundred workers, but depending on how many CPUs you have available it will be four workers running. Then one, one finishes another is created and so forth. But when we are using this just style isolation, we just create. workers for like, for example, if you have eight cores, we create four workers for watch mode and eight [00:07:00] workers for a CI for not non watch mode. And we just reuse the same worker, but create a new context there Was this option, was this kind of, alternate mode of dealing with context and running tests? Was this the biggest kind of roadblock that you guys were waiting to get figured out for v1? Or was there something in particular that made you feel that we were ready for? Major release stable. for me. Yes, exactly. . There was a question why is performance different that we couldn't just answer, or we didn't know why. And so we had this investigation and and tried the implementation just to give the community answers. So we can say , we have a version one because this and this. So, we know that the, why performance is like this for this test case. And. I promise like this is, was this test case and it worked out quite well, I think. Were there any other big challenges on the path to getting towards version one, like a stable release that you had to Knockout? [00:08:00] For me, this was the main blocker. Basically the idea to release the version one was to align with Vite 5. I think, I think this was like, we want to like not start a new, but kind of wanted to reset. I don't know, not the progress, but. The state of P test at this point. And then we're ready. We are happy with how config file looks. We're happy. How basically just happy with us, with the state of P test as it is. But to be honest, I do expect us to release a version two. Quite soon after the version one, because of the major changes, we will probably find something , that's a breaking change that we need to do to make, and, you know, SemVer is quite important to us To release have releases like following the SemVer. What do you foresee the relationship being between VEET version releases and VTest version releases? Will they usually be around the same time? Will VEET version updates usually Proceed of the test version update. I [00:09:00] don't think there will be any difference from how we do it right now. The team is separate. I'm part of both teams, but. When we release VTest, we don't really look at what version suite is running. , sometimes Vite exposes some nice APIs that we can reuse, but we usually have like a workaround, check the versions, so we still support everything out of the box. , I have a couple of questions , in the context of test runners and stuff, a little bit higher level. One is that bun recently claimed that they've removed the need for V test as a tool. Why, like, why is bun saying this and what is your take? This is my take on VTest. Well, let me think. I'm not sure if they claimed it like this. Did they claim that they can be removed? Okay, maybe I'm just going to say it. For now, what do I think for BAM? is now actually an effort to make VTest run in BAM. Is halted because BAN doesn't support a lot of Node. js APIs, or it [00:10:00] claims that some of them are supported, but they are have some subtle bugs or just not don't work at all. And also as our primary use case is Vite supporting Vite users and Node. js users, as long as they are using Node. js or Vite. You need Yeah. A test runner. you can't use the standard that runs on the same engine but if you're moving, switching to bond, I think my take is that if you're using the platform, use the platform, if the platform provides you with a test run, and if you're writing. If you're writing server side BAN code, of course, I think the BAN test run will be the perfect solution for you. if you don't use Vite , or don't rely on frameworks that run on Node. js, then also you can use BAN if you just have like scripts and so on. Of course you don't have to use VTest. One of the features that VTest supports, which also Jest supports, but no other test runners, as far as I know, support, is the module mocking. I know that Ban works towards [00:11:00] this. I think this is like a, quite a big a big feature that people require. I see all the time people having issues with it, because they don't understand the difference between modules. And if you run a module, it's... It actually executes your code and you can't really access it, change it after it's executed. So there is subtle differences. But yeah, I think really looking forward to Bon being really stable to, to move my projects to Bon if it's, if there is really a performance difference. But I think the main thing that Bon did or does right now. Similar to what Dino did is it pushes Node. js, like what Dino did is push Node. js into supporting web standards, right? So not started to not, maybe not re implement, but. Supported better like URL stuff and one does the same for performance. We've seen some like performance group [00:12:00] forming for Node. js the, I've seen a pull request for speeding up a file system access, file system operations. so, Yeah, yeah, Yeah, for sure. there's a lot going on in the JavaScript ecosystem and it's it'll be cool to see where these tools all land in a couple of years. For sure. , are there any like particular features on the roadmap or stuff you guys would like to address in the future you mentioned before that you think B2 or 2. 0 rather is on the horizon shortly, what are you kind of eyeing as stuff? As features you'd like to incorporate? Well, V2 will probably be. On the, it's probably on the horizon because of, it's a break, we will introduce some breaking changes to the existing APIs, but not because we will introduce new APIs. But, yeah, but when we do, there are some experimental APIs that we already provide for over a year now. Like browser support Tasting types. And we also have benchmarking support, which I actually forgot [00:13:00] about this feature, but then I saw people on Twitter discussing that, like we use this feature all the time. Just was surprised by, and there is a discussion about improving benchmarking support. So you can assert that something is became slower or faster and stuff like that. Currently type checking is already implemented, but it's not very uh, developer experience friendly. So. You have to run the separate command to your test. So you don't just run VTest and it also does your tests, your types. You'll have to run VTest and VTest type check. We want to merge them together, which is a challenge. Because of how it's implemented and of course browser support. Yeah, it was initially just a way to provide, a way to. Preview your components during your tests just to see okay. I'm clicking here. But how does it look like? Without installing third [00:14:00] party like cypress or playwright if you are not planning to use them. I All right but I think the focus will be more on browser support on component testing similar to how playwright Component testing, but the main goal in retest is similar to what we did with J so we provide the same a p I as J does or highly compatible a p i, so people can easily migrate from JS to retest. And we wanted to do the same kind of gradual migration for your tests that use GS dom or he. To use browser mode. So you can just add like, I want to this test file to run in the browser. That's just to progressively see or progressively add more tests there. And they should be also highly compatible. Like API should be highly compatible with what is already in your tests. So you don't need to rewrite everything from scratch, Like with locators [00:15:00] in playwright or tenables in Cypress, where you need to switch your whole mental model with how you write your tests. This is basically the goal for the future. What are some of the difficulties on like in implementation in the library itself For like browser support? Like what, what's challenging there? It's interesting because because actually it's not that challenging. It was not that hard because of how we implemented internally, how we run tests. So we can easily just tweak some stuff or just add a new. They're called pools. So in, in vTest internally, we use different pools for running tests in worker or Chandler process or VM. And we just basically added a new pool for browsers. And we had some providers and we also have a providers set setting in the conflict for, to, to use playwright or web driver io or not, or even not use anything just to preview or for ling for example. You can write your tests there and see a preview on the right.[00:16:00] Oh, cool. Yeah. it opens the browser, we ask browser to run tests and it runs tests tells us the results. Is the goal eventually to have, or is the vision maybe to have v test be the sole test runner that projects would need to use? Like, maybe if they're doing both, some like isolated backend and browser-based testing right now, that in the future, like V test could do or will be doing both of those. I don't think VTest can handle end to end tests. I don't think the team is willing to go that far. Component test is fine. And like the experimentation in that area are very interesting, but. The intern testing is just, I think uh, playwright team, the people who start a playwright team are browser engineers. So they understand more in that area. And I trust them more to be honest. Yeah. But experimenting uh, is really nice. I think another thing that we. Wanted to go towards is [00:17:00] better compatibility with with Node. js actually well, because with Node. js tests, currently vTest runs its tests in this like sandbox environment that emulates. It's similar to BAN. We support both CommonJS and ESM in the same file, which. Node. js doesn't support it, and we have no way to disable this. So some tests that are written for Node. js can work in Vitest, but actually fail in production, which defeats the whole purpose yeah. Yeah. That's not Yeah. Yeah. Don't use dirname, filename and so forth, because they are injected by Vitest actually by Vite node, which we use under the hood. Oh, sure. Yeah. And another point that we struggle the most. Like this whole year and basically from the start, the hardest point is how to support your packages. That like your source code is fine. We can transform it. We can run it, but your packages, we don't really want to transform every single package because this [00:18:00] will make everything slower. it's a lot. I think one time, like a year ago, we had the bag. So someone imported TypeScript file and it just stuck because it's several thousand lines. Yeah, we just couldn't process it. and Vite SSR has some solutions to that. Currently we just say not yes. , try to do this yourself. Like we, we don't care. We can't really help you that there, but it comes with issues like inter interop, like ESMCGS. So if you have a package that inputs another package. That assumes that there is a default, for example, expert, or if there is, well, the usual pitfalls of ESM, CGS interop, then we can't really do anything with it. We have to say to people, please, like, add this option to inline or stuff like that. And well, to the config file, right? And ideally we would like [00:19:00] to make this. An automatic feature, so you just run your test. We make it more seamless because currently there are some packages that will just not work correctly. The, we had a lot of issues with Svelte because they publish Svelte files as Svelte files unprocessed and it's re exported from a JavaScript file. So we assume that the package is compatible with It's Node. js and we just say, Node. js handle this package, but it just fails. So this issues pop up sometimes that we would really like to resolve. I was going to ask, when do developers typically encounter this the most often? So it sounds like, yeah, like Svelte projects. Are there other cases where devs are reaching for external packages and, stuff's falling over? there is Lodash, There is Lodash. es, which re exports every file so you can consume it as a named export, but Lodash package doesn't actually do that. You cannot use a named export there. So this is probably the most, Common.[00:20:00] yeah, common, yeah, common one. Yeah, sure. Yeah, so I guess for devs that are listening and interested and like they want to check out VTest, is there good support? We touched on this briefly at the beginning. But what does support look like for non VT projects? Like, is it a bit arduous to get running or can devs usually set it up? You can just install VTest and run it. Yeah, it works. But as I said, there are some differences from how Node. js runs it. But yeah, you basically just run VTest write your test files, run VTest it should work , but for fixing the differences, there's already pull requests. Which should address this issue and it should actually make it easier to debug. So we wanted to provide a better experience for when you encounter an issue, when you would have encountered an issue with ESM. Or cgs in your file, you usually get, you know, just, you might get a very cryptic message or a message that [00:21:00] says, like, other type module to an external package that you cannot actually do. And we would like to, like, address this issues and to maybe give some advices there. Or hints, tips, maybe to fix it. Yeah. And then I think my last big question is , when are we expecting 1. 0 to be, out and available? Huh. Currently there is already a 1. 0 beta Nice. is already here. We are aiming at VT release a few weeks after VT release, second week after VT release. But I am not sure Yeah. if I would, we'll. Would probably release it, yeah, but currently there are not a lot of beta testers, so, as I said, assume 2. 0 soon after. Gotcha. Gotcha. So I guess maybe if anyone's listening and wants to get involved in testing, it sounds like testers are needed. Yes, actually, very much. Cool.[00:22:00] If anyone listening does want to, get involved and help out, what would be the best way for them to start exploring this space? . Keep the project going or maybe just get their feet wet in V test and see. okay, there is a label in VTest issues called PR welcome, and there is also a label, good first issue. I've actually went through most of the issues and put this label on some of them. So this is a good way to help. Documentation issues are also very good. If you are familiar, if you're already familiar with VTest and you just want to help, there are some documentation issues. To improve documentation Nice. basically. And of course, if you're already using Vitas, install the better version and just see if it works Yeah. and how it works. Well, thank you so much for coming on and chatting with me, Vladimir. It's been a pleasure. Thank you. It's been a pleasure being here with you.