Alexander Jung: The idea behind building something with Unikraft is that you have your application and then you can target these different platforms or different architectures. Build once your application, but then compile many times. Eric Anderson: This is Contributor, a podcast telling the stories behind the best open source projects and the communities that make them. I'm Eric Anderson. Eric Anderson: Today, we get to talk about Unikraft, which is a unikernel technology, and I'm joined today by Alexander Jung and Simon Kuenzer, who are some of the project backers and creators. Thanks for joining, Alexander and Simon. Alexander Jung: No, thank you for having us. Simon Kuenzer: Thanks a lot, really. I appreciate this opportunity. Eric Anderson: I've been excited about this call for some time, because this has been a thing I've been curious about. I'm excited to hear about your progress. Maybe you could start us out by explaining what Unikraft is. Alexander Jung: So yeah, Unikraft is a unikernel development kit. It basically is an open source project. It allows you to build very specialized, lightweight virtual machines that are single process, and they have sort of enormous performance and security benefits that they reap, because they change the way you think about deploying something as a virtual machine. Alexander Jung: The basic premise is that there's no user kernel space separation, right? So your application sort of compiles straight into and as its own bespoke kernel, right? So it boots, it does some minimal initialization... No system D, no shell, nothing else. It boots straight to main, and Unikraft as a project allows you to build these unikernels very quickly and easily. Allows you to take your preexisting application and compile it into these lightweight brush machines. Sort of the benefit there is, again, performance. You get to build something that is enormously quick in being able to boot, enormously performant in terms of its, for example, throughput, if you were to do an Nginx application or web-based application, and has a small attack surface. There's no extra ports running that can be exposed. There's nothing additional that can be targeted from a malicious sort of standpoint. Alexander Jung: So Unikraft essentially just builds these and allows you to do that. Eric Anderson: Got it, and what you've described so far sounds like maybe the definition of unikernels. Is that right? Unikernels has been an idea, I believe, that's been around for a few years. There's other kinds of unikernel platforms. I don't know what the right word is there. Maybe you could talk a bit about what makes Unikraft unique among those? Simon Kuenzer: So compared to other unikernel projects, of course, 10 years, that there was a lot of stuff happening. So there was, for instance, MirageOS, which is actually still around as an open source project, but still focuses on the okama language, which I think... It's probably not, for everyone, the best choice, that language, although they have many concepts similar to what we have with librarization and minimizing the components to the ones that you only need for running your application. Simon Kuenzer: There was also RAM kernel, pretty popular, which we see as the... Let's say, at that time, the most closest to the POSIX compliant, so it was really easy with it to run existing applications. It was actually based on NetBSD. The problem with it was that the performance wasn't great, but I think this was more an implementation issue. This could have been fixed. Alexander Jung: That was really not very configurable as well either, right? Simon Kuenzer: Yeah. Yeah. It's not really configurable and the other thing is also, it's still based on a general purpose kernel, but we try really going for special, or purpose-built kernel, actually, to your application that looks different from application A compared to application B. Alexander Jung: You really got the whole kernel when you compiled the RAM... You got the whole NetBSD, just without the user space or kernel space separation, and you can't configure it. You look at the configuration files, one config.h file, and you kind of set a few things inside of there if you want to change the networking stack, for example. To add to that, rump is dead. Just doesn't... No one uses it. The project's not maintained. I think Antti Kantee, the guy who did his thesis on it, I think he doesn't work on it at all anymore. I think he's dropped from this landscape. Simon Kuenzer: Although it got upstreamed in NetBSD. So there's some pieces that still live. Alexander Jung: You can use it for debugging, right? The NetBSD. I think that's the main purpose. Whilst you're running NetBSD, you can debug the NetBSD kernel with it. Simon Kuenzer: Yeah, and then there's what maybe we should also mention, which also inspired us, is HermitCore and HermiTux, which have still... I would say a research focus, but they were also trying earlier, already trying to build binary compatible unikernels, what we're also trying to do with our library ecosystem right now as well. Alexander Jung: Yeah. They do basically binary rewriting, taking the binary that you have prebuilt. Imagine you don't have the source code or you don't want to compile the source code and you have the final ELF binary. You can then just say, "Okay, these instructions where you do this syscall, don't do that syscall, jump here instead." That's the basic premise. Simon Kuenzer: Maybe we can look [inaudible 00:05:27] on the history, where we come from. So first of all, our unique point from, let's say... As you said, actually it started 10 years ago. These unikernel topics. Our unique point is the specialization that we have in librarization of our unikernel system, because, let's say 10 years ago, you had quite many projects trying to build efficient unikernels, but you ended up in throwaways, in a sense that each of the projects built their own bases, ported their own application, did their own optimizations, but was never a common shared base, meaning running on different platforms or supporting a different application was problematic, and in our case, we see it as an SDK for building unikernels, and trying to avoid that issue that was happening at that point of time. Eric Anderson: Makes sense, Simon. Why don't we actually just go back to the beginning, and maybe you can help us understand how you got into this project and the story behind how Unikraft came to be. Simon Kuenzer: Yeah, sure. So also, our path started 10 years ago. We started from a research background as many of the other unikernel projects do. I personally come from NEC Labs, which is a laboratory of the NEC corporation, the Japanese IT company, and we basically started our unikernel journey with ClickOS, which is our unikernel for network function virtualization. Simon Kuenzer: So it used the click router software, which is a programmable network function in the end, so you could load a graph into it, and then you could build with it [inaudible 00:07:12] or a level two for water or a DCP server with it, and we provided that click run time based on MiniOS, which was actually a base for many of the unikernel projects at that time, to run it really efficiently on the Xen hypervisor. Simon Kuenzer: We continued doing our research with content cash nodes. We jumped a bit on the CDN high two, three years later, and again, built a unikernel with a purpose-built file system, a purpose-built web server on top, and all that work actually revealed us to the great benefits that you have with unikernels, which is basically 10 milliseconds of boot times, few megabytes of RAM usage. Simon Kuenzer: So there's this nice anecdote that we could build [inaudible 00:08:01] unikernel for Xen with just 200 kilobytes of RAM assigned, with an image size of 32 kilobytes, and funny thing was also, we had to patch the Xen toolstack to make it even possible to run a unikernel with just 200 kilobytes, because people were thinking at that time... So there was a hardcoded limit of four megabytes as the minimum for unikernel. Simon Kuenzer: Also, in terms of network performance, it was easily possible with the specialized and purpose-built network stacks to deal with 10 or 40 gigabits in network traffic with the single virtual CPU, and we speak here about 10 or eight years ago, actually. Also, some of our experiments, we also saw that it's possible to run 8,000 of those instances on a single server as well, but the problem that we became aware of is actually this development effort to build such a unikernel, because it became, as I mentioned before, really to a throwaway. Simon Kuenzer: When you get a new application or you want to get it running on a different platform than Xen, you had to do a lot of work to get that working, especially on the application side, since many of these unikernels, at that time, didn't have this POSIX compliance or Linux compatibility available, so this meant a lot of manual work to get more applications working. Simon Kuenzer: That actually led us to the SDK idea, because there's all these benefits of fast performance, working really efficiently. They really came, also, from the manual work that we built into the system. Actually, I started then internally at NEC in early 2017, building the first lines of the Unikraft project, and with two more colleagues, we launched publicly Unikraft, first time in December 2017, as an incubator project under the Xen project that actually helped us a lot scaling the project. From then on, we actually started providing kernel functionalities as libraries, like schedulers, memory allocators, and also platform support, and the biggest task here was to define APIs between those libraries so that in the future, if you have a new application or a new platform, and platform is, for us, like a hypervisor support, for example, VMware Hyper-V, is just the work of that little specifics that you need to implement to run it on Xen or run it on KVM or run it wherever you want it, but not implying the library ecosystem to adopt to your platform support. Simon Kuenzer: Also, the libraries, we wanted to keep them as minimal as possible, and also exchangeable in order to let people customize the unikernel basis as close to what people need in their unikernel project. This means, as an example, that some projects may not need ever multi-processing support, because the application logic they run will never make use of SMP, or they have some benefits by using different memory allocators, which maybe are not beneficial for general purpose cases, but for a specific case, makes sense to do, like static allocations that you can just hand over at one time, for instance, or also the same cloud for scheduling. Some unikernels may require real time schedulers, some are super happy with the cooperative one, and some really need a preemptive scheduling system. Eric Anderson: Simon, you've been working on this for quite a while, it sounds like. When did Alexander get involved? Alexander Jung: I'm a PhD student at Lancaster University. I originally heard about Unikraft through the Xen mailing list, and then saw there was a workshop paper that was published by Simon and several of the colleagues at NEC describing Unikraft was, and I found it and got my hands dirty with it, because I was researching unikernels at the time, and I was also primarily using MiniOS, which is the implementation that Xen provide to allow you... If you want to build a unikernel, I guess you can build it with that, but this primary purpose is to sort of demonstrate how to implement the interfaces that Xen offers. So if you're building a new operating system, then you can reference these. Alexander Jung: There'd been some interesting work done with it, and I was sort of looking at how you can do efficient VNFs, version network functions, with MiniOS, and Unikraft sort of came along and made it really easy, actually, to hack the code base of the unikernel and put whatever you want inside of it, and it became clear for me that this was going to be sort of the future of kernels. Alexander Jung: I actually then ended up joining NEC as an intern, so about six months later, after some heckling emails with the team, and kind of stayed on, because the community was growing, the work that was being done was very interesting. So I originally came on board to help with developing some tooling, to make it easier to use Unikraft. So we have a CLI Companion [inaudible 00:13:18] tool called kraft. kraft basically is the app-to-get of unikernels or Unikraft unikernels. Allows you to manage your libraries and allows you to quickly retrieve different components that you would like if you're sort of building them by hand. Manage versions. Sort of abstracts all the whole construction process of Unikraft itself, Unikraft sort of being written in C and using make file and K-Config similar to the [inaudible 00:13:45] kernel. It abstracts all of this. So every time you enter the command line, you can quickly build a unikernel, and you can retrieve, also, prebuilt unikernels and sort of get the ball rolling if you want to build your application this way. Alexander Jung: So I came on board to hack on this and to add some more abstract libraries, some of the stuff that Simon was saying. I think one of the core design goals in architectural decisions that is made within Unikraft is to have really abstract interfaces that allow you to plug in things that you want. For instance, one of the things that I was building, something called POSIX socket, and we know socket as the ability to open up a connection to something. Typically, it could be TCP, but actually, if you think about this API as an abstract interface, you can plug in anything. So my goal was to be able to plug in, without making any changes to the application itself, the ability to, for instance, share memory between two virtual machines that were running on the same host. So instead of doing TCP, it would just be a shared memory, and then you'd just increase the throughput between those two VMs significantly, without having to touch the TCP IT stack. Alexander Jung: So I became involved sort of in this process, but during this time and over the last two years, the community has been growing quite significantly, and we've had a lot more contributors join on, and I've been helping foster this community, I think the whole core team have. Now I help try and develop Unikraft into... It has an open source component to Unikraft, but there's a SAS and product side as well, and I aid and help and develop things like this. Alexander Jung: This is really developer-focused sort of tools and systems, right? So my interest with Unikraft is, how can we make someone use a unikernel in the easiest way possible? How can they bring their application sort of either seamlessly through... Maybe just one command line that you call within a CI/CD pipeline, for example, or how do we sew in unikernels into existing ecosystem frameworks, such as Kubernetes, for example. Alexander Jung: Kubernetes, the defacto industry standard way of scheduling services. What tooling can we make that makes it as simple as possible to make it possible to run a unikernel in that way, and sort of help develop tools that would make that possible? So we can actually do that. We created a tool called Run U, in the same vein as Run C, that sits as a sort of bit of glue in between the OCI definitions and Kubernetes orchestration framework. So this is kind of sort of my involvement within Unikraft itself. Simon Kuenzer: It's also really, for the project itself, the dream in the future to see much more adoption of unikernels, especially also in enterprise and cloud, et cetera, and the way we think how this really will work out is only if we can get the tools and also using unikernels as non disruptive as possible for end users, or DevOps, et cetera. Alexander Jung: I think that's how we believe that adoption will occur. You won't even know that your application's running as a unikernel. It just will be... Simon Kuenzer: You transparently get the performance and efficiency benefits, right? Eric Anderson: Yeah, it makes sense, and I gather that, as well, from your website, that a lot of Unikraft's advantages are more in it's kind of tooling and ability for you to easily include it and be non disruptive to the rest of your build pipeline. That's fantastic, and the other maybe aspect that's kind of interesting about Unikraft is that it has a lot of academic and kind of research university backing. That's both where you two come from, it sounds like, but it's also broader than just your institutions. There's a lot of other institutions on board, is that right? Simon Kuenzer: Yeah, of course. We have a couple of universities that are in collaboration with us, actually. We do also join publications. Simon Kuenzer: Recently, we published a work on isolating the library ecosystem that we have with different techniques, so that you could [inaudible 00:17:38] per library isolation to get more, let's say, research-driven secure operating systems built, because if you take Unikraft as an SDK for operating systems, let's say, this is quite easy to do as well. Simon Kuenzer: What we also actually work a lot on currently is for the application compatibility, which means, basically, for us, Linux compatibility, where we're also currently trying to understand how much really needs to be implemented of Linux functionality in order to run most of the existing applications. Of course, this depends per application, but you can come up with kind of histograms telling you what are the important system calls to implement, and then it's also, still, you have things like regular systems calls, like IOCTL, as an example, where you have a lot of functionality behind a single system call. Also trying to understand what's needed there in order to be sure when the next application comes, that we have the libraries available that implement part of the system call set to run that application. Alexander Jung: To help answer the question, these are all different avenues that Unikraft is exploring sort of... Yes, academically, and what they aim to do is validate all the claims against unikernels. They help prove to the wider community that a lot of thought and engineering and tests have gone in to show that this is a correct way to build something. This is a very systematic way, this is a very performant way, this is a very secure way, this is a very compatible way. So the recent security paper that we had was published in [inaudible 00:19:24] used to show that we can do per library compartmentalization. "I don't trust this library, the network stack. I want to put that in a secure enclave," or, "This POSIX compatibility," we're doing research to show, "Okay, look, these are the system calls that we care about, or that most applications care about. How can we pragmatically and how can we systematically make sure that this is the right way to make it compatible?" Eric Anderson: Actually, on the compatibility front, one question I've had as I've looked over Unikraft, you've done some impressive Linux compatibility. I believe that you also talk a lot about kind of the hypervisor platform that these unikernels will run on. Are they kind of built specific to a certain hypervisor technology, or am I misunderstanding some of the configuration? Simon Kuenzer: Actually, the idea is, for us, specializing towards the applications, or this is the, let's say, bottom up specialization, but we also want to specialize down to the execution environment where we are. Simon Kuenzer: So for instance, different hypervisors provide also different, let's say, paravirtualized interfaces, and we believe it doesn't make sense much to carry all sorts of different implementations if you anyway know your execution environment. So for hypervisor support, we have, also, libraries that implement these specifics. One is for Xen and is really optimized for the Xen environment. Another one is for a KVM guest, and the build system will, depending on your selection, build multiple images, one for each target hypervisor. Simon Kuenzer: So far in the open source part, what we support currently is Xen, this is actually where we come from, where we did a lot of work for KVM guests with chemo, but we have also worked currently in the community going on for supporting VMware, VSX, Hyper-V, Firecracker and a number of other hypervisors, actually. So that in the end, this pool of library that we have gets extended by more hypervisor support as well. Alexander Jung: The idea behind building something with Unikraft is that you have your application and then you can target these different platforms or different architectures. Build once your application, but then compile many times, so you can just, "Okay, I want to compile for Xen now," or, "Okay, I get the Xen image," or, "I get the KVM image. I compile it that way." You set your configuration statically beforehand, and then you get this different image. Simon Kuenzer: It makes, actually, sure that you are never running or actually including drivers that you will never need at your run time or your target environment. So for instance, Virtio would be included for KVM, but in a Xen part, you would have a Xen native for our virtualized network drivers, for instance, as an example. Eric Anderson: Maybe you can talk about the state of the project today. So you've been working on this for a little bit, and it seems like the project's kind of reaching an inflection point in terms of usability and adoption. Simon Kuenzer: That's true, actually. What happens today, as well, also for the core team behind is that we're trying to spin off from NEC corporation with this open source project, actually. So to build a company behind to provide some enterprise services to develop, actually, the tooling further for enterprise users as well, but also for the community and to more focus on the engineering parts that are still missing for a seamless experience as well. Alexander Jung: The community itself is definitely growing. I think one of the traits that we found is that a lot of students end up finding the open source software through hackathons that we host or projects that are open, and as a result, the number of contributors that we have are at university, and they're learning and we're guiding them and they're bringing in new features, and so over the last two years, I think we've seen such a really... Maybe even just the last year, we've had enormous growth in the community. We, for example, have a Discord server with over 150 people, and sort of always about a hundred people online actively talking about different parts of Unikraft, different libraries or different architectures or different platforms that they're aiming to introduce or modify. Eric Anderson: Fantastic. What does the year hold for you coming up? What can users get excited about or maybe work on? Alexander Jung: We have a lot of features planned for this year. Targeting new platforms, the ones that Simon said. Hopefully we'll be getting those in the next quarter or two. Also, we are always presenting our work at different conferences or open source events, and so we have planned hackathons hopefully within Europe, and hopefully within Asia as well. We're looking to try and break the Asian market. We have some contacts there, but at the moment we also have Google Summer of Code as an organization. So if you're looking to get involved over the summer as a sort of paid intern, working with Unikraft and working with us, building additional platforms or different libraries, then do get in contact, but if you're looking to get involved, do visit our website. It's just Unikraft.org, or if you're in enterprise and looking to get something a little bit more commercial, support Unikraft.IO... Yeah. I don't know. Anything else, Simon, that I might be missing? Simon Kuenzer: [inaudible 00:24:39] that we have a bunch of projects running. For instance, what we try to is bringing in other languages. So because the base system so far is written in C, but I think something like a mixed language unikernel could be something interesting too, if you can just link components together and choose the best fitting programming language for the components, like you take something type safe for an allocator, for instance, or using some other language properties for other components, like the TCP stack or the scheduler, for instance. Simon Kuenzer: There's also more work currently going on for the Linux compatibility, especially since we have different porting mechanisms. One of them is actually just compiling your source codes together with the Unikraft code, and since the application expects glibc standard Linux interface, we are currently hardly working to get muscle at work, which is licensed compatible, actually, with our project, plus is the closest to glibc, as well. Also, on the other side, what we're also going to release soon is supporting of binary compatibility layer, so that we could even run a pre-compiled application that is compiled for Linux on the Unikraft unikernel. Of course, with some performance hits, let's say, but it makes it easier to use, and then you can still decide if you want to put some more effort into a native ported version of the application to Unikraft. Eric Anderson: Sounds like a busy year for the two of you. I really appreciate you coming on the show, because this is an exciting technology. It's one we've discussed as an industry off and on for five, 10 years, and it's fun to see it coming into kind of a usable, stable, production ready thing. Shout out to your users and where they can find you. Thank you for mentioning the Discord group and your website, and appreciate all the great work you're doing. Simon Kuenzer: Thanks a lot. Alexander Jung: Thanks a lot for having us. Yeah. Eric Anderson: You can find today's show notes and past episodes at contributor.fyi. Until next time, I'm Eric Anderson, and this has been Contributor.