Anna Rose (00:00:05):
Welcome to Zero Knowledge. I'm your host, Anna Rose. In this podcast, we'll be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise to change the way we interact and transact online.

(00:00:27):
This week we have the first part of our two-part series on ZK Hardware. We do a recap of the ZPrize competition and chat with a number of different participants from an architect of the prizes to the hardware manufacturers who supported it, as well as catch up with a few of the competitors. My guest co-host for these episodes is Alex Pruden, the CEO of Aleo and the creator of ZPrize and before we kick off, I just want to let you know about the ZK Jobs Board. There you can find job opportunities from top teams working in ZK. I've added the link in the show notes. Now Tanya will share a little bit about this week's sponsor.

Tanya (00:01:02):
Ever wish you could use existing rust libraries in ZK? A friendly reminder from the team at RISC Zero that you can! Things like proof-of-checkmate, blinded proof-of-password complexity, and even proving things about protobufs are straightforward when you can use existing code. To learn more, check out the RISC Zero video tutorials from the last ZK Hack at https://www.youtube.com/@risczero. Follow them on twitter @risczero to make sure you don’t miss their upcoming 1.0 launch and the alpha launch of the Bonsai Network. So thank you Risc Zero. Now here's our episode.

Anna Rose (00:01:43):
So for this episode, we are going to be exploring hardware in the ZK context, meeting folks who've worked on hardware and will be using the recent ZPrize competition as sort of the lens through which we explore it. With me is Alex Pruden, the CEO of Aleo and creator of ZPrize. Alex is going to be the co-host and will help guide us through this. Hey Alex.

Alex Pruden (00:02:04):
Hey Anna. It's a real pleasure to be here.

Anna Rose (00:02:06):
We actually did an interview, I think over a year ago where we introduced, you came on the show and you introduced ZPrize. It was in the context at the time of like general ZK funding. So while I add a link to that in the show notes, if people want to hear kind of like what it was, but yeah, tell us a little bit about what is ZPrize today?

Alex Pruden (00:02:24):
Yeah, so ZPrize, just quick context for those who may not have heard of it. So ZPrize was a competition modeled on other examples, like the DARPA Grand Challenge where the DARPA Grand Challenge was, you know, largely credited with kicking off the self-driving car industry and as I was, you know, as a person who's in the industry, as Anna mentioned, I'm the CEO of Aleo, which is building a ZK Layer 1 being in the industry, you know, you see all these techniques that are being applied inside of different companies and you know, being someone who's very excited, of course, about what I'm building at Aleo, but also just about ZK in general. I really wanted there to be an organization or an outlet for people who had the skills and the desires to contribute to basically building up the foundation for zero knowledge that everyone could then build upon and use in the future right?

(00:03:16):
So basically an open source public good, our library of open source public goods and the key reason why we wanted to do a competition was we thought it would bring out the best in everyone, right? And so we got sponsors across the industry. A couple of them you'll hear from during this show, but you know, lots of folks from Web3, zprize.io, you can see the full list but, you know, Aleo was a sponsor, Aztec was a sponsor, Mina, lots of big Web3 companies in ZK. We also had hardware companies such as AMD provide specialized hardware for competitors. We really brought together folks from across Web3 and Web2 really to focus on promoting the development of these open source public goods for ZK.

Anna Rose (00:03:58):
I really liked the sort of network agnostic nature of it where you had even like people who kind of are competing on some level collaborating in a way. I mean, they're still competing, but at the end, once everything was revealed, it is a collaboration, right? Because they could almost share results and like find out what the other people were doing. This was meant to be kind of an open source competition, not something closed.

Alex Pruden (00:04:20):
Yeah, exactly. It was a really interesting dynamic, right? And there's a few different levels of people who participated in the competition. There were sponsors, right? And sponsors are putting up the prize money, right? And so they obviously have a vested interest in outcomes that are at least relevant to their projects, right? Potentially. You know that you have architects who are appointed by the sponsors or are the sponsors themselves that are really about specifying the prizes and that's really important in a competition, right? Because there's this, you know, you really need to make sure that everything is fair and so it needs to be relatively rigorously defined upfront. And, you know, a lot of these problems in this whole area is moving pretty fast and then the last kind of, you know, angle for participation was the competitors themselves. So these are people, many of whom, and again, you'll hear from some in this show, many of them have very little background in crypto or ZK, but they actually just saw this as an opportunity to say, hey, I can apply these skills or this knowledge that I've gained through my education or through my prior work at, you know, other companies in software or hardware and potentially win a big prize, right? And so that was like kind of the third angle

Anna Rose (00:05:24):
There, like an ultimate nerd snipe. I like it.

Alex Pruden (00:05:27):
Kind of. Yeah but I think to get back to like the motivation behind your question, I think if you look at the competition and the prizes, right? So if you just read off a title, it's like accelerating multi-scale or multiplication in a GPU. Like, you know, listeners of your show will be like, okay, I know what PLONK is, I know what Marlin is, I know what Groth16 is. Like, these are proof systems and they're very high level, but they're also very kind of, many of these things are kind of specific to different projects and multi-scale multiplication is one of these low level algorithms that kind of applies across the board, right? And that was, that was the reason why we chose to focus on the lower level stuff was because everybody found common ground as like, hey, if this gets faster on this platform, it kind of helps everyone or it's widely applicable, so.

Anna Rose (00:06:11):
Nice. Can you tell me what were the prize categories? I know we're not going to talk about necessarily all of them in this compilation of interviews, but we will talk about a few, but tell us what were all of them?

Alex Pruden (00:06:21):
So there were a total of 12 different prize categories split up into what we called two divisions. So one was this open division, which was really focused on public goods that you know, applied to more than one team and the prizes in the quote-unquote open division were accelerating multi-scale multiplication on GPUs and FPGAs. So that was kind of one prize, we later split it into two but it accelerating multi-scale multiplication on GPUs and FPGAs accelerating NTT on FPGAs specifically, and NTT stands for number theoretical transforms, which are really important specifically for STARK based systems. A Plonk-DIZK GPU acceleration, so DIZK not to be confused with the company DZK. DIZK is a technique where you can parallelize the generation of a huge proof.

(00:07:10):
And so we had that as a category accelerating elliptic curve operations and finite field arithmetic in Wazm, which most of that was focused on MSM similar to the GPU, accelerating Plonkup, which several projects in the space you're using specifically, those are as part of ZK-Garage, accelerating the Poseidon hash function and accelerating MSM on mobile and that was really cool. Just quick note on that, that was to my knowledge the first time anyone has kind of made any or developed any open source work around proving on a mobile phone, so really interesting there and then the team division category was really about, okay, you're a company, you want to just define basically like a bounty. So Aleo had three that's specific very much to Aleo. So there were two team division prizes focusing on Proof of Succinct Work, which is an algorithm that that we use as part of our protocol to distribute Coinbase rewards.

(00:07:59):
We had a fast verifier for Marlin and then the two other team division prizes. one was sponsored by Mina, one was sponsored by Anoma. The Mina sponsored one was accelerating Halo2, and the one sponsored by Amoma was creating the fastest multi-asset shielded pool circuit using Groth16.

Anna Rose (00:08:15):
Nice.

Alex Pruden (00:08:16):
And we were really thankful to have a great set of partners across Web3 and Web2 for the ZPrize. I mean, these include for sponsors and providers of folks who funded the prizes. This is Aleo, Anoma, Parity, Jump, Ocelot which is you know associated with Celo ecosystem, Polygon, Mina, Panther Protocol, Aztec, Espresso Systems, Polychain Capital, Manta Network, and the ZK Validator.

Anna Rose (00:08:43):
Oh yeah!

Alex Pruden (00:08:45):
Yeah, exactly and many more and also I want to mention the three technology providers who had AMD who provided FPGAs for the FPGA competitions Samsung who provided phones for the mobile MSM division and then CoreWeave is a specialized cloud service provisor that does a lot in the AI space, but also let us use some of their cloud resources to enable the competition.

Anna Rose (00:09:07):
Nice. So yeah, this episode as mentioned, it's kind of this look into hardware. We're using ZPrize as sort of the context. Tell us a little bit about who we're going to hear from.

Alex Pruden (00:09:16):
Yeah, so over the course of this episode we're going to talk to Luke Pearson of Polychain Capital, who worked as an architect on a couple of the prizes and really helped me kind of brainstorm the initial idea at the beginning. We're going to talk to one of the competitors, they go by the team name Hardcaml, they won the FPGA MSM Acceleration Prize specifically, and we're going to hear from them about FPGAs continuing on the FPGA theme, we're going to hear from AMD Xilinx they're a manufacturer and inventor of the FPGA devices that the competitors use and they're a leader in that category and then we're going to hear from another competitor, not in the FPGA division, but in, you know, focusing on hardware people might be a little more familiar with GPUs, so graphics processing units, and yep, that was an independent competitor who actually won two competitions.

(00:10:05):
And so we'll get a kind of a flavor what the competition was like, but I guess I just wanted to quickly say that this was really something that I came up with initially but really the result of it, which was very successful, by the way, everything across all of the prize categories, the average improvement on the existing baseline was around 10x.

Anna Rose (00:10:26):
Wow.

Alex Pruden (00:10:26):
Right? And all of this work is open source, and that would not have been possible without the many people who contributed to this. So as everyone we listed in the beginning of this, you know, as we were just discussing the sponsors and technology providers, but all of the competitors, all of the people who contributed ideas or works as part of this competition, I mean, it really is something that the community did together, and that's something that I was really proud to be a part of.

Anna Rose (00:10:53):
Cool. So we should jump into it, but before we do, if there are listeners who are interested in this, like how could people get involved?

Alex Pruden (00:11:01):
Yeah, it's a great question and, you know, we just finished the first inaugural ZPrize competition, but this is something that we hope to continue and to again, make a source of public goods going forward. And so zprize.io is the website where you can go, it has all the information about the competition. There's also links to our Discord and we have a Discord community there where would love feedback from folks around what potentially the next iteration of the competition should look like. In particular, we're really excited to meet and find folks in the community who are passionate about helping organize and lead the next iteration of this competition. I mean, again, this is a community-led effort, but it really does, the efforts and impacts of individuals really do matter. So if that's something, you know, if you're passionate about this space, if you're passionate about ZK would really encourage you to join our Discord, reach out, you know, sign up for our email newsletter and get touch with the team because this is something that, you know, we want to continue to build together.

Anna Rose (00:11:57):
Cool. So this is a four-part series, so we've decided to break it up into two distinct episodes. So this will be a two-parter and I guess it's a good time to kick it off. Shall we do it?

Alex Pruden (00:12:08):
Let's go.

Anna Rose (00:12:13):
So we're here with Luke Pearson, a partner and research cryptographer at Polychain. Welcome Luke.

Luke Pearson (00:12:19):
Yes. Thank you for the lovely introduction, and thank you for having me, Anna, Alex, and the rest of the big podcast team.

Anna Rose (00:12:26):
So Luke, you were one of the designers of the ZPrize competition. That's what we want to talk about primarily but I do want to start us off with a little backstory. What got you excited about cryptography? Zero Knowledge, the whole jam.

Luke Pearson (00:12:40):
Academically, I was studying maths and engineering at university in the Netherlands, in the University of Groningen in north of the Netherlands and then so I could work at Dusk Network and that advent came accidentally, shall we say, first caring a lot more about privacy preserving technologies. So in this case, it was taking signatures that assign in browsers can be TLS, but just something that's done like on a https on this transfer protocol system and has like a sign for an attestation every time you go to a website and then you get your little padlock and it's a big thumbs up from the internet or big thumbs up from the two hands in that handshake and to do that without leaking a single scrap of metadata. The project, the method shall I say, of looking into it was Bulletproofs.

(00:13:28):
So that was like my first kind of advent or kind of pierce into zero knowledge cryptography but cryptography itself, when I was, I used to be really into codes, code breaking design encryption and decryption languages, even just like spoken languages, but the linguistics and the structure behind them. But when I was a little younger, when I was 15, I entered a competition called the Southampton Cipher Challenge run by GCHQ a, kind of UK equivalent of NSA, except rather than Pentagon structure, the structure building they have is the donut. And basically it was all high school students, mostly those who were, who were into maths to do this after school competition and I ended up placing really well ended up being a joint finalist. So, held it in the back of my mind that I liked this particular area of mathematics.

(00:14:18):
And then when going to university and realizing and understanding and reasoning with, you know, number theory, algebra, geometry, better, I understood the broader and wider use case, which very exciting. I didn't do traditional computer science background, which had some of its advantages. It meant I got not to care about the cleanliness of my code, but also I guess that's a disadvantage in many others. But I really enjoyed bridging between, you know, a lot of the engineers that I had worked around and becoming a much more applied photographer and then fell into SNARKs when the Bulletproof wasn't viable for what is any use case really besides specialized protocol. That's not that sounds a bit too disparaging actually, but any use case that we were doing at Dusk, other than very, very short specialized range proofs. So fell into what's the next thing, what's the next stage, how are we going to scale this?

(00:15:13):
How are we going to make this stuff private? And SNARKs, that was around 2019, coincidentally fell in love with PLONK when Zach presented it, it was the <inaudible> and Deloitte some kind of school thing in Amsterdam, presentation day, I can't remember the title. Yet another very exciting ZK day and sort of implemented, begun to design around it, reasoned with some of the PLONKish stuff, made my own paper, finished my studies, and then after that decided to switch gears a little bit and change the seat I sat in from one of pure applied cryptography to research and allocation of capital to VC and investment, wherever it was necessary was kind of my hopefully succinct little story.

Anna Rose (00:16:03):
You also created a group called I'm going to probably say it wrong, ZK-Garage

Luke Pearson (00:16:12):
ZK-Garage. ZK.

Anna Rose (00:16:13):
I actually wanted to ask you about that. Like ZK, when you say it's ZK-Garage, it makes me think of UK Garage. Was that the reference?

Luke Pearson (00:16:21):
No

Anna Rose (00:16:24):
Okay. I've been wanting to ask you this for a while

Luke Pearson (00:16:28):
Okay. It was actually after a call with Yoni. Yonatan Ben Shimon that I, well, we sort of came up with this idea, the idea behind the garage. I also, by the way, spending so much time in same international communities, I just interchange and switch and swap 'Zed' and 'Zee' with prize and cash and I'm just, yeah, fine. Language is for communication. But it was basically for a similar reason that, that hopefully we'll go on in a few minutes to speak about around what the ZPrize was for. It's, I wouldn't say standardization but I guess to give the little story is that I worked right at the start of my career with Polychain or had really had the fortune of working with five or six different companies. It will help to name them. Aleo, Manta, Anoma, a couple were outside, I think one was IOHK, we spoke to a Brave and then Scroll / Privacy Scale DPSE / Ethereum, that kind of implementation and being in this more abstract capacity where I'm a bit of an investor and not necessarily tied one. I can see that everyone's working kind of on the same thing here.

Anna Rose (00:17:41):
Yeah. But separately right.

Luke Pearson (00:17:42):
Separately and that was okay. That was also very, it was about a year ago, about 12, 13, 14 months ago. Very crucially when PLONKish and PLONKish constraints were the talk of the town. It was, we have so many of these implementations. One was the one I did at Dusk, one was at Berenberg, so very Aztec, the C++ one. There was these ones I used to focus and read through and then when I saw all of these different links, and then another one PLONKish variant, we'll call it the Halo2, first one ZCash, and then from the PSE, Scroll, Ethereum Foundation, all of that is, so many people are putting so many efforts on the exact same thing that roughly to the same goals, you will never be able to create this one library.

(00:18:24):
One library serves all purposes because there are use cases, there are gadgets, there are some specifics, but there are definitely some ways we can work together to sort of standardize and make, because it was all in Rust with this nice trait system they're doing, make basically plug and play of the curves or you know, maybe the fields if you're got to do a Plonky2, the polynominal commitment scheme, because the backend was arkworks, it was originally called like PLONK arkworks, but it was just too confusing with the name. But the backend is arkworks, and they've got all these great libraries for fields and arithmetic, and they got KZGT and IPA and like both of those are supported with the Garage implementation. And I, then the final thing, or the final step was to make as abstract as possible kind of gate array or should we say, you know, width in terms of the numbers of witness wires and a little more complex than that

(00:19:22):
Selector polynomials and the columns that represent those. So we had all of these different pieces, can we work together and make this enclosure? And we did, and it integrated lookups as well, which was, which was cool but the idea behind was to standardize something that everybody could use to a certain degree and then at some point all of those protocols and then anyone in the future, of course, who wishes would fork it and change it and say how they would but there were definitely some uniform features that everybody was doing and like they completely made faster, safer and require less work from more people if we do it together.

Anna Rose (00:20:01):
I like that idea of the oversight, like seeing, you know, all of these different teams doing similar things and trying to bring them together. It actually sort of speaks to the ZPrize work as well. This idea of like a lot of teams can use some of this stuff. They might be thinking about it. Did you see some sort of parallels when you joined this project?

Luke Pearson (00:20:20):
I saw, yeah. Even they, I saw the exact same parallels, everything you just said with respect to oversight but even on another layer deeper, I think each team has themselves where we're like, at this case of ZPrize, because you know, the most of the people who've been working on this for the past few years intensely are mathematicians or maybe computer scientists, less so harder engineers and having an oversight of, and in this case we won't say the whole blockchain, we won't say an entire blockchain project, but the, you know, let's say the aim to make a ZkVM or the aim to do a fantastic state transition at great speed from a client side, super decentralized in the past 5/6 years, the proving system, the iteration of them slowed down, right until also coincidental that I spoke about in the, the introduction of up until PLONK and PLONKish, you know, there used to be Bulletproofs, Groth, Auroras, Virgos, Marlins, Libras, PLONKS, plookups, Halos.

(00:21:19):
Oh my God

Anna Rose (00:21:20):
Tell me about it

Luke Pearson (00:21:21):
My suspection, my prediction sorry, is that in the longer run, the people who will be making like quite serious leaps in terms of how approving system has moved forwards, will like probably come from academic institutes or research industry that will now pump out at a, you know, at a cadence of maybe once a year, maybe once every 2 years. Not the crazy every other month and that's because we need to get something on the ground. We need some bare bones working version and I think we sort of have that on the proving system. So what's the next thing to look into? Well, it's the hardware. A lot of these proof systems lay out these, you know, these, the bits that just group elements down onto some piece of metal, whether it be some GPU or some, some FPJ or whatever it may be and I think most cryptographers kind of go, well, I'm done here. Maybe if it's not, you know, their particular area of expertise or just because it wasn't looked into, but when looking at the whole holistic end-to-end, a lot of people's applied engineering focus moved onto the hardware and I think it speaks to the oversight is we're sort of done relatively done with these nice proving systems. We're relatively done with some of the gadgets. Let's get some optimization in another area.

Anna Rose (00:22:38):
Cool.

Alex Pruden (00:22:39):
So Luke, you were really instrumental in helping ZPrize come together, right? So you and I brainstormed a lot of ideas that ended up becoming the competition at ETHDenver in 2022. And you ultimately were an architect for one of the prizes related to Poseidon hashing and then you also helped out on a number of others as well. Can you give us your perspective on how ZPrize went from an idea to through, through the execution to the final results?

Luke Pearson (00:23:10):
Absolutely. I think it was a great success. Now looking back, I think the way in which the different parts of the competition or the different categories of prizes, let's say, without them being too specific or more specific than team and personal was really, really good. We created a method here, and I'm going back to front, I'll go from the definitely the difficulties that it was to get here and trying to suit all of these needs to kind of having a very hardware based approach for a couple of the competitions. Things that were very specifically done for, you know, number theoretical transformations on FPGAs, MSM operations that are done, you know, on GPUs and FPGAs and they were brought together by looking at all of the sponsors, the participants or was then the prospective participants, but also what the wider industry is going to need to create some kind of relatively standard as fast as possible implementation or let's say new algorithm for running all of these operations, which are crucial to so many different proving systems.

(00:24:17):
And that was very specific to hardware and working. Another thing that was really great about the ZPrize, and we got to do this because it was flexible and composable, and we did it with best intentions forwards for all of the people in the industry is people knew, or let's take the example of the Poseidon prize. I knew that Mina had some certain trick where they could fold away some particular rounds or some X boxes. I knew that the Plonky2 guys, the Mear guys, I think I say Mear because I think it was even when they were called Mear, that they created something around the lines of a full Poseidon hash, like in 1 gate I knew that some specifics with small fields had been doing and could we in fact create 1 general area or 1 repo where all of these were collated put together.

(00:25:07):
and then kind of just a well-documented open source implementation where everybody can go and take advantage of that. So that was like one sort of middle size, which was fantastic and then the third was application where something like, it was something like mobile and Wasm goes into it. So it's even a little bit further. It's the next stage to making these end-to-end proving system or these, these end-to-end designs and protocols really viable and it's putting them inside of like a browser. Lots of people don't know that or the multi-threading issues there was and we're just doing wasn't just in general, when you send off whatever code you've written on Wasm to this browser where people want to be making proofs, the Aleo trusted setup was a fantastic example. It got loads of attraction because the everyday user was able to run a Google Chrome or a Safari and not a command line interface to join a trusted setup.

(00:26:00):
Like the power of that to bring people into Web3 and to interact with some of these, I think nascent, but standardly quite complex technologies was incredible and the same with the mobile. Making I think it was Celo who did the architecting of that prize because they have the desire for people to be able to sync. They have the you know, the PLUMO with the light client straight to their chain. So three different categories speaks to some of the most fundamental needs from taking proving systems where they are to making these systems really quite viable however, creating that was very, very different.

Anna Rose (00:26:34):
Okay. Yeah. So you've, you've given the, like the glorious finish line, but what was it like to get there?

Luke Pearson (00:26:40):
It was intense. You have to try and be as abstract as possible to suit everyone's need, but not so abstract that it isn't feasible and you don't want yet another swathe of, you know, complex maybe not well documented, but very, very helpful for individual use case implementations or accelerations or algorithms. You want something that everyone who's using this, if it is the technology we so praise it to be, is able to go pull it, fork it, and use it and put it in implementation so it makes something that's fast working but also abstract such that enough people are going to work onto it. There's no point making something very specific just to your protocol because when you've got to get more people interested and if it's with some particular fields or if it's some really particular app proving system, then statistically less people work on that and I think that's a, from speaking to the other architects that was a problem they were sort of having and you've got to wrestle basically what your protocol wants and needs with making sure you get something that's helpful. So that was, I'd say that was the largest challenge was making sure we could get it to suit everybody. Yeah. But it worked out quite well.

Alex Pruden (00:27:55):
I think. Yeah, I'm glad you hit on that and I think, you know, you mentioned that challenge, but I think one of the reasons why I think this is going to be impactful moving forward is that as you pointed out, like a lot of individual teams had done their own little work on the one little thing. And you know, even though all these techniques were theoretically known, there was nowhere where like, you're like I want to start a new ZK project. Like where do I go to just like get the MSM module and it didn't exist, right?

Luke Pearson (00:28:23):
Yeah.

Alex Pruden (00:28:23):
And so this was kind of the idea behind this was that like similar to how arkworks is kind of like the repository for a lot of stuff on the proof system side, on the ZK side, on the cryptography side, it's kind of just one place that everyone can go for these kind of basic building blocks. That was the idea of the ZPrize. Now I'm curious, given in light of everything that you just said and your experience with the ZPrize and seeing it all the way through to the end what would you like to see for the competition in future iterations?

Luke Pearson (00:28:51):
For future iterations, I think we'll take very, very great use of making something far more standardized. Something where you are literally saying to people who plan on, let's say run rollers or approvers for ZkVMs or people planning to do run Aleo nodes or do some, some mining to pick parameters specific hardware, specific metal, specific FPGAs, specific GPUs, I won't say ASICs for now just because of the potential time and say that this predictably is, are the parameters, this is the, the AWS F1 instance and this is the curve that is going to be used for this blockchain for some amount of time that makes it reasonable to now create, you know, or finalize the final frontier to create the final model and say, we're going to be using a 3 77, we're going to be using Marlin, we're going to be using AWS and F1, and that's going to be the Aleo.

(00:29:49):
Now whoever can go make that as fast as possible, that's going to be the Aleo implementation. So I think well I say that as if I didn't start by saying we have three separate categories. One where that would I guess pertain to, another one which could be an exciting open research, one thing that didn't get any submissions, partly because it was difficult, has been parallel and basically same-y. So maybe they could have won the money with all of the Caulk and the Blue and things that that went on at the time just because it was so flexible making some of these prizes. Ariel Gabizon had posted some open question about reducing the time complexity of lookups with respect to the storage and the table size and that is just one of the sub-questions on the Plonkup division. No one went for it. But a place where people can work on these open problems it shouldn't be, I, I guess shouldn't say anything specifics of financials of inner workings of protocols, but from the way some of the things went in 2021, it's a very little cost for lots of these treasuries, et cetera, to have some of these problems solved outwards.

Alex Pruden (00:30:57):
Well, and also I think it was for in fact the Poseidon hashing prize was a good example is like, you know, for that ended up only getting one submission, which was quite a good submission but nonetheless it was a very high ROI activity for the individual competitor right and so this is maybe a PSA to anyone out there who thinks they have the skills to contribute to this because not only is it a great public good for the community, but also, you know, again, in light of the interest here, there's a lot of people willing to, you know, to sponsor these prizes and potentially it could be a good chance to, you know, earn some extra money for these otherwise relatively esoteric and specific skills.

Luke Pearson (00:31:36):
Both of those were given very particular hints. If people had to scroll through the document, through the PDF Hack MD it was of here's a Poseidon, but also here's a paper, but also here's an implementation by the Espresso guys where they have an X to the 5 and you picked that as your alpha to get nice inexpensive edition chains and ah, you're going to already save a third constraints and there were quite a few of those things. So it was to your point before a collation of all of these different points and good money, in my opinion.

Anna Rose (00:32:11):
I want to hear a little bit about the judging once you had those folks competing and had submitted something like, I don't know if Luke, if you were, were you part of the judging?

Luke Pearson (00:32:24):
Yeah.

Anna Rose (00:32:24):
So what was that like? Like how was it to do? Was it very hard? Was it very obvious who came first? Who came 2nd, 3rd, 4th, 5th, whatever? Was it hard to rank folks?

Luke Pearson (00:32:37):
Some of them were difficult. It also I guess helps to explain, we've specified that I architected the Poseidon and I think one more up there was is specifically under my name, but I did, you know, have the real pleasure of working with lots of the other teams. So I can speak on, on behalf of how some of the others went. Some of the competitions, especially where you're creating or you're saying run some application not on your own machine, but on a machine that will rent or will be provided or be done in some cloud instance, provide standardization, but you need to build test vectors and frameworks. Lots of these other smaller comp or this one that wasn't the one that Aleo had sponsored as the main one was, let's say the Poseidon or the PLONK, accelerating Plonkup, the competition became very, very specific in terms of how the design was or ought to be made and how these different techniques had to be exhumed and as a result, the judging had to be rather rigorous.

(00:33:39):
Okay and also if you are talking, let's say in terms of constraints, you get the same level of kind of not confusion, I would actually say like met with the same comment and and complaint from some of the wider audience. There was a definitely a level of ad hoc, well we're kind of who this person or this sponsor's paying and this is just going to be the line. It would be great to sit here around a round table of advanced study and cash out huge test vectors, but it was going to take a lot longer. So we can't talk, we can talk in term constraints if we're able to fix what we would like as a proof size, fix the array of some of the gates, let's say. So there were different challenges for judging both, but overall it went very well.

Alex Pruden (00:34:25):
And if I can just add one thing as Luke mentioned, they were both quantitative and the quantitative was very important in terms of judging. We wanted to make sure there was a very clear finish line or a bar to clear and then, you know, people's performance was very objective. but inevitably in problems like this, which are quite complex, there's always a degree of qualitative assessment as well. There's no doubt in my mind that it's true that some of the submissions that did not end up winning might actually be applicable for problems that that specific prize wasn't about. Right. So that's one of the reasons why we wanted to open source not just the winning submission, but all of the submissions is because there, you know, even though the competition was maybe set up a certain way and the specification was like, hey, this is the bar, you know, there could be another use case or another problem where you said, well, you know, the third or fourth one took an interesting technique and maybe if I take that and apply it here, then this could actually be really interesting and cool.

Anna Rose (00:35:20):
So a few episodes ago I had Justin Thaler on the show and we had talked about this kind of conflict between hardware acceleration and some of the sort of software accelerations or protocol accelerations. Luke, I just wanted to ask you a little bit about that. Like, I believe you've co-authored part of the articles that were brought up in this conversation. So yeah, maybe you can tell us a little bit about that.

Luke Pearson (00:35:46):
Yeah. I had reviewed and worked on a hacking deal with Cizik who basically took that tweet stream and I'll try and try and correctly remember the first instigating comment or tweet from Omer or the post and then the kind of Justin's follow up and then the Cizik follow up and then Benedikt jumped in try and I try and tessellate everything well but yeah, so Omer was just discussing the difference in HyperPlonk and PLONK, so as a quicker overview. A HyperPlonk is about like 40% MSMs and then 40% of these MLEs, these multilinear extensions and it's got a sum check in there. It's got this because it removes FFTs these fast Fourier transformations and the argument is there is from Omer is it's very difficult to make sum checks friendly based on hardware.

(00:36:41):
These things have a very annoying Fiat–Shamir heuristic hash function inside of them. The overview or I guess the conclusion that I were to go from, I think there's a lot more work still to be done is that I think he is correct in terms of, well he is correct in terms of the expense of running these sum checks on the hardware but Justin spoke about these very particular optimizations and one comes from if you pick Hyperplonk itself, one comes from batching these multilinear evaluations and as a result the maximum number of sum checks you do is two. So it's a constant in terms of how expensive it is or if it's fixed, if it's going to be two, but it's not going to scale forwards and going forwards with numbers, if you can do these batchings, I must say I haven't gone all the way into it.

(00:37:30):
So anybody listening, please ping and we should we should definitely talk about this. If I find something relatively groundbreaking, you can take all of the first operations of a sum check. So all of the first round of the sum check should be like heavily paralyzable. So I guess it depends on what hardware you're going to be going for. If you're doing this over FPGA, maybe you can have some host performing some operation paralyzed at the first feed it to something on the second. I don't know. But I think there's definitely something in paralyzing the first round of sum check there, and there's definitely a limit in terms of overall subject expense by doing these batched evaluations. So I don't think it's a huge killer but it definitely speaks to a new conversation of kind of backwards fitting or retroactively making these zero knowledge proving systems such as they specialize in hardware, you know, a nice, a really nice and easy way to think about it without even understanding like, you know, the pipes of of FPGAs is just field size.

(00:38:33):
You spit out in some proving sum 256 or sum 128 bit number and it's being laid down. You have to have all of these expensive carries or if you are able to do it in a 64 bit field or some some specific field minus 1, you can basically optimize the harder the already exists and that is what you just said on software versus hardware, you know, there's lots of ways to optimize hardware FPGAs with RTL and all of these very logs, but it's relatively complex. It's going to take a very bit of time and it's new or it's a new work in the blockchain industry, but software size, there's quite a lot you can do straight away. And you know that if you're doing like NTTs, you've got some something that requires like a billion NTT operations, it has to be memory efficient, you're just going to use CPU, GPU and you don't need to start rewriting all of these gate arrays and circuits and things.

Alex Pruden (00:39:28):
Well, dovetailing on that question a little bit, because I think you touched on something really important is, you know, this, this concept of hardware, software, co-design, potentially really like what's the chicken and what's the egg, right? Is the proof system written for generic hardware and therefore you're optimizing all your choices around what the proof system is doing around the existing hardware? Or are you designing the proof system around some custom hardware from the start? Right, so this was, you know, some rollup teams have taken this approach, right? And actually most recently there's a paper in pre-print called GZKP, which was you know, basically, I think it was the Scroll guys who co-authored it where it talks about how you're optimizing a proof system for a GPU specifically. So it's really interesting, right. So and this I think touches a little bit on kind of how hardware and software were designed in, in the past where it's like in the mainframe era everything was together and then in the cloud era you had hardware and things running on CPUs.

(00:40:23):
In fact, it was to the extreme like you had the cloud could give you any resource that you wanted and it kind of, you could mix and match everything. The software could just exist as a completely independent layer and now it seems like because of the overhead potentially of computing these proofs, we're coming back to an era where it's very, very important that the pipeline is very carefully designed. So the question to you then, Luke, is what do you think the future holds for ZK specifically? Will we be in a world where everything is designed end-to-end from the silicone to the application, or will it be driven by, you know, further advances and proof systems that will basically, that use existing hardware and there's no hardware, software, co-design? What are your thoughts?

Luke Pearson (00:41:07):
Well, I do not know, but I'm also confident that I think, well, I'm confident that, I think that would be most of the people's answers you asked, and that's almost a good thing. One of the difference now and one of the key things between software and hardware is one is super expensive. So I guess to the disadvantage of the argument that we'll be making specialized hardware and going backwards if you get that wrong. So and if you've spent time building an ASIC supply chain, squeezing what you can from Taiwan and trying to play into these and then the pruning system massively changed. You may have wasted a really large amount of capital from, and that's, you know, with anything from the software side, you know, changes are lovely and then they're all done on your GitHub and it's neat and it's very recyclable and very fast.

(00:41:56):
So I think for now, and everybody's doing this right, I feel everybody should do this, wait until there is a greater ossification and something far more clear in what these proving systems final forms will look like, how difficult it will, you know, I was just going to be to upgrade them, potentially change fields and how that affects some of these bits of hardware. I think the question, again, with respect to each piece of hardware is more particular, you're not going to be doing too many things on the CPU, GPU side, which is good because they're kind of in my mind like they're like the software hardware, the hardware that you would, you know, do some fancy paralyzation or you would do something in the care of a GPU for MSMs and you would, you can still probably do that from my end, from the applied cryptography end.

(00:42:48):
But once going onto the other side, the FPGA is the ASICs you are going to need to understand what level of abstraction is going to be important. No point designing, pumping out loads of these ASICs for some proving system, the whole thing changes. You will really waste a lot. One counteract to that is to make FPGAs because yeah, they're akin to basically prototyping for some of these ASICs and they're much cheaper to work with, but you still have some of the same sort of hardware difficulties. So I think no one yet knows if both sides as going forwards, software and hardware can remain as dynamic, responsive to one another and the way in which the other develops, I think will iterate forwards forwards. A personal prediction a little bit different to what I hope was objective from before is we'll have a lot of GPU accelerated algorithms for now, and in the meantime people will work with FPGAs.

(00:43:48):
The power consumption's a bit of an issue that no one really talks on, but the memory stuff is fine and there'll be a prototype basically for the ASICs. I was a little bit upset. One thing, if you wonder if this is written somewhere so I can hold myself to it, but one thing I probably said to you last year, Alex, was post merge, this is going to be great, it's going to be all of these freed up GPUs and we can just get these clusters that people are experienced in running at scale, running paralyzed, and put some these algorithms and we won't mind ease, but we'll actually make blockchains fast, safe and and private. But no, I think all of the ones I looked into, all of these farms I contacted even without the video interfaces, the GPUs had, you'd need to like replace some bits of the internal memory. You'd have to think about the dram. I couldn't really predict what would happen to the like FFT cause over time. So I think there was too much difficulty and without even speaking to many of the hardware providers and the people working on zero knowledge hardware, the fact that they're not taking as much to these farms, I think falls in line roughly with what I was thinking.

Alex Pruden (00:44:57):
That's awesome. Thank you Luke for shedding some light on kind of your thoughts, given your perspective of where you sit in the industry and also your perspective from the seat of an architect and key organizer of the ZPrize competition. Really appreciate you being here to share this with us and we really appreciate all the work that you did for ZPrize.

Anna Rose (00:45:15):
Yeah, thanks Luke.

Luke Pearson (00:45:16):
You're very welcome. It was an absolute pleasure. Thanks for having me on guys.

Anna Rose (00:45:23):
So I want to welcome to the show Ben and Rahul, both hardware engineers at Jane Street and part of the team Hardcaml. We had some debate before we started on how to exactly call the team, but yeah welcome to the show, Ben and Rahul.

Ben (00:45:37):
Thanks. Yeah, thanks for having us

Rahul (00:45:39):
Thank you, yeah.

Anna Rose (00:45:40):
I want to hear a little bit about where you're coming from, what got you interested in this topic. Ben, why don't we start with you?

Ben (00:45:47):
Yeah, so I've been interested in FPGA's for quite a while. I did my PhD in FPGA Design before I moved to the States and started working at Xilinx, which is an FPGA company and after that I moved to another company, but I had some downtime and I actually ended up doing like a one year kind of job for Zcash where we were kind of looking at accelerating zk-SNARKs and some kind of verification, improving systems around them. So kind of as a, you know, a hobby, I got really interested in this kind of space and since then I've also done some VDF competitions and just general kind of crypto stuff in my free time and in the last three years I've been working at Jane Street as a hardware engineer but also, you know, always think, you know, trying to keep up to date with what's happening in the space especially around hardware acceleration and you know, applying FPGAs to this problem.

Anna Rose (00:46:48):
Rahul, what about yourself? What got you interested in this field?

Rahul (00:46:51):
Hi. Yeah well I guess I did my undergrad in electrical engineering and computer science and I was always pretty interested in sort of the boundary between software and hardware and performance engineering. Yeah and I got pretty into FPGAs during my time in school and once I graduated I started working as a hardware engineer on FPGA acceleration. Yeah and I guess I started working with Ben and he brought up that this competition was happening and that's how I sort of got into the ZK crypto/hardware acceleration space.

Anna Rose (00:47:28):
Cool. So this prize, this competition kind of brought you into the fold then?

Rahul (00:47:33):
Yeah, for sure.

Anna Rose (00:47:34):
Like specifically the ZK fold.

Rahul (00:47:36):
Yeah definitely.

Anna Rose (00:47:36):
Nice. I have a question. Like, you're both working at Jane Street. Is there hardware at Jane Street? I don't know how that, like what's going on over there that you'd be doing hardware engineering?

Ben (00:47:49):
 So I guess we can't really talk too much about the, the detail

Anna Rose (00:47:53):
Sure.

Ben (00:47:54):
But yeah, like it, you know, FPGAs, we're using them. I think the kind of neat thing for us is that at Jane Street we use this language called Hardcaml, which is how we got our team name and it's kind of like if people are familiar with Chisel, it's like a way of improving, you know, how fast you can code or how efficient you are at writing hardware design and so we wanted to have a go this for this project. So we wrote it all in Hardcaml and that kind of overlapped a little bit of, you know, what we're doing at Jane Street.

Anna Rose (00:48:29):
Tell me a little bit about the team. There's more than just the two of you, right?

Ben (00:48:32):
Yeah, so it's me and Rahul in New York and then there's two more hardware engineers actually from Jane Street as well who are based in London and I think we're all pretty, you know, enthusiastic about hardware design and so that's why everyone was very keen to, you know, do this in our free time together.

Anna Rose (00:48:49):
Cool.

Alex Pruden (00:48:50):
Yeah, I know that a lot of firms you know, trading firms that do HFT often leverage advanced types of hardware such as FPGAs in the course of their operations. I think it's pretty common across the field.

Anna Rose (00:49:02):
Why did you decide, like, so you're working internally on this stuff you saw ZPrize, what inspired you to get involved in it? Ben, it sounds like you were the kind of one who spotted it first.

Ben (00:49:14):
Yeah, I think because of the work I'd done on Zcash for zk-SNARKs I'd kind of always had one foot, you know, in this space and was aware of things might, you know, happening and so I saw on online that this competition was coming out and I thought it would be really neat to, you know, have a go at accelerating and maybe also learning a little bit more. I think the work that I did for Zcash was a little bit like a naive implementation just to get something in hardware online because there wasn't anything open sourced really.

Anna Rose (00:49:44):
Yeah.

Ben (00:49:45):
But then for the ZPrize we really had a go at like implementing the most efficient algorithm we could think of on the FPGA and really like pushing the limits.

Alex Pruden (00:49:54):
Yeah and I actually had proactively reached out to Ben because I noticed he had done that work. It was, you know, cause again, one of the motivations for ZPrize was that to grow the body of open source work that was related to hardware acceleration and Ben's is one of the only ones that I could find for FPGAs. And so that was, that was how we initially got connected and very, was very happy that him and his team decided to contribute.

Anna Rose (00:50:18):
So I know that there, there were many categories. FPGAs was one of them, but Alex maybe you can share a little bit, like how was Hardcaml competing and how did they do?

Alex Pruden (00:50:28):
Yeah, sure. So as you mentioned, the ZPrize encompassed several different hardware platforms and the goal of ZPrize was to, you know, cover the space of different platforms where you might run a ZK prover, you know, and FPGAs. I'll let them talk a little bit about FPGAs in a minute. But, you know, FPGAs, as I said, are often used for, for various specialized applications. And the specific problems and the specific prize categories that this team competed in were relating to accelerating what's called multi-scaler multiplication and number theoretical transform, I think also sometimes called fast Fourier transformations. So these are two key underlying mathematical operations that, you know, are kind of being done in the course of creating a zk-SNARK or creating a proof you know, of a statement inside of a ZKP. So that's the significance of these algorithms. And so the FPGA specifically, maybe Ben and Rahul, do you guys want to share with us in your eyes, what is the advantage of using an FPGA to to accelerate these algorithms?

Ben (00:51:36):
Yeah, I think the FPGA kind of occupies a little bit of a niche space where you can do, you can pretty much program whatever you want on the FPGA, it's just the programmable logic cells. So you can implement any function, any kind of data path. But the problem is the clock frequency that you're working with is a lot slower. So you really need to like paralyze what you're trying to do, which is really interesting when you're competing against like a GPU, which is a much faster clock speed, right? But a little bit more fixed in the functions it can do. So I think it's interesting designing for FPGA because you have to kind of architect your design to be very parallel, make sure you're not hitting up, you know, like memory bandwidth limits and, and that kind of thing.

Anna Rose (00:52:21):
Cool. I'm trying a picture like did you really expand on the work you had done previously or was it like you're tackling a very different problem with this competition?

Ben (00:52:30):
So the work I'd done before we weren't doing proving of the zk-SNARK, it was more of the verifying. So it was like the pairing on elliptic curves. But we did try and take some of the core components of what I'd done before and just apply them to the competition and say, if we had done it this way, how fast it would it be? But we found out that it would, we would be totally smashed by the GPU when we saw like the benchmark come out. Just it was too slow what we were doing. So like all of us or four of us kind of looked for, you know, what is the latest algorithms and then that's when we changed to using Pippenger's, which is pretty much what everyone is doing and going from there,

Alex Pruden (00:53:09):
You know, we're talking about hardware, but this is a lot of deeply mathematical, there's a deeply mathematical aspect. Because you're taking, as Ben just referenced, an algorithm called Pippenger's, from an academic paper, I think in the 2010s, I can't remember. But this algorithm was described in a paper and then the challenge for a team like this is to implement that in the hardware, right? Which is of course, as efficiently as you can get, as opposed to just writing some code that implements it. Right and there's, you know, it's pretty abstract. It's pretty far from the quote unquote metal, you know, it's like very much on the metal and so that's, I think one of the interesting challenges and all of this was open source because, you know, the philosophy here was to make sure that this was available, these innovations were available for everyone to use and then leverage to build upon, so as to kind of promote a broader ecosystem for ZKPs.

(00:53:56):
But one question that I had for you guys, and you just mentioned, I think you just touched on it Ben, but you know, there's a lot of different flavors of zero knowledge proof systems, right? There's, we have SNARKs and we have STARKs, there's FRI-based, polynominal commitments, there's Halo2, one's based on sum check. In your opinion or in your eyes, like what's the competitions that you competed in the ZPrize specifically accelerating MSM and NTT, where are those most applicable and what ZK proof systems would those most help?

Ben (00:54:28):
So I think anywhere that you've got to do this kind of multiscaler multiplication regardless of kind of the curve that you're doing it on you'd be able to apply what we've done. I think as a side note, I'm not so familiar with other proving systems. Yeah.

Alex Pruden (00:54:43):
what I'm, as I understand FRI-based systems, so like STARKs almost entirely rely on NTT, whereas like Marlin is much, much more MSM heavy, right? And this is why we actually had the both competition and Polygon who's building a system that's like STARKs, it's like SNARKs and STARKs. That's why they sponsored the NTT Prize specifically is because like for them they don't really care about the MSM, it's not relevant.

Ben (00:55:05):
One thing I was going to say too, I think maybe that helped our team is that everyone kind of is has a specialty in different fields. So like Rahul has done a lot of very good like mathematical work and for me, I'm more of just a hardware engineer. So I think Rahul being able to optimize the algorithm and tweak out, you know, 5/10% performance was very key to I think us getting first place. Without that, I don't think we would've and you know, Andy and Fu Yong and London, they were also very good at like the NTT side or you know, optimizing things on the FPGA so I think it was good.

Anna Rose (00:55:39):
So how did this all turn out? How did you actually do on both of these competitions?

Ben (00:55:43):
Yeah, so we entered in the two tracks and in the MSM FPGA track we won 1st place and the NTT FPGA track, we won 2nd place.

Anna Rose (00:55:52):
Oh nice

Alex Pruden (00:55:52):
And by the way, I have to say that they won 1st place of every competitor in the ZPrize for the documentation. And we'll add a link to it in the show notes, but they're the website that they have like I say, we'll have the link, but it's very good and it's actually, if anyone's curious to learn more about what NTT or what MSM is it's described there as well as links to their solution. So thank you guys for doing that. That itself was a great contribution.

Anna Rose (00:56:19):
Cool. That's going to be a great resource.

Ben (00:56:21):
Oh yeah and Andy and Fu Yong in London, they really pushed that and put a lot of work into that. So it was great. Yeah.

Anna Rose (00:56:26):
Rahul, tell me a little bit, like Ben just mentioned that you optimized the algorithms. What are you actually doing?

Rahul (00:56:32):
Yeah, I think I approached it like from a couple of different ways. I think the core piece of the MSM sort of design is a large, like point adder just to do elliptic point additions. So based on the sort of problem space that we're in where we're streaming in points, but we know the base already we could sort of do a bit of pre computation on the host and reduce the number of actual multiplies that we do in hardware. And that was one of the more expensive operations. So reducing the total number of basic multiplies in our design helped us get this design smaller and clock it at a higher speed.

Anna Rose (00:57:12):
I want to just ask a clarification here. Cause you just mentioned pre-processing, but I feel like in SNARK-land that has usually like a very specific context. Is it the same pre-processing or is it a different kind of pre-processing that you mentioned here than maybe you, I don't know if you can jump in. Do you know what I'm talking about?

Ben (00:57:28):
Something I'm not sure about is, so during the trusted setup you come up with this set of points Alex, is that right? So the

Alex Pruden (00:57:34):
That's right.

Ben (00:57:34):
Pre-processing we kind of did for the MSM is equivalent to have if we've done it way back then.

Alex Pruden (00:57:40):
Exactly. And there's two kinds of MSM actually there's three kinds. So I've learned so much about this since being in this competition but there's, so there's fixed base MSM, which as Ben just mentioned, you know, the bases and recall like this is, you know, the whole what is MSM, you're multiplying elliptic curve points and you're really taking them to a power, right? And so that's really what, like what the whole point of this algorithm is, is you know, you're taking, because that's, that's the whole point of, you know, when you're doing the trusted setup, you basically fix the bases and then everyone's input into the setup is their power, right? And it's their secret power and you keep doing that over and over and eventually it's just tumbled together and no one knows how you got that final output string, right? But then from that output string, you then use that to construct SNARKs in the pre-processing mode, right?

(00:58:22):
So that's where the fixed base comes in. So that once the setup is done, that's the pre-processing step of the SNARK. And so now the bases are fixed. So if you're computing a proof, that is a no now that's fixed based, variable base MSM is would be in the case where you did not have a trusted reference string at the outset and then you would have to compute the bases, the points itself on the fly and you wouldn't be able to know them in advance of the computation. Like there would be some dependence effectively on like what the next point was based on the previous one. Whereas in this case they basically, you know, all the points in advance and so I think as Rahul's getting at, you can kind of get ahead of the problem a little and in certain ways and be able to kind of make it, you can chunk it up in a way that's slightly more advantageous. So the TLDR is fixed base is really applicable, most applicable to SNARK and proof systems where there's a trusted setup in advance.

Rahul (00:59:15):
Yeah, so the specific type of MSM problem that we accelerated was a fixed base MSM where all of the points are known ahead of time.

Anna Rose (00:59:26):
Cool. So this is for a pre-processing with trusted setup kind of SNARK.

Rahul (00:59:30):
Yeah.

Anna Rose (00:59:30):
So yeah which proving systems is this most built for? Like maybe some that we're familiar with?

Alex Pruden (00:59:35):
So the systems would be like Groth16, Marlin, PLONK

Anna Rose (00:59:41):
But is it like OG PLONK not new PLONK, not PLONKish or Plonky2 or?

Alex Pruden (00:59:48):
To be honest, I can't keep up with the, with the explosive growth of the PLONK ecosystem but the other one to note is there's a couple newer proof systems called, I think the latest one that I know of coming on Microsoft research is Spartan. So SNARKs that are built around KZG commitments, polynomial commitments typically do more on the MSM side and a little less on the NTT side and SNARKs that are built around FRI, like STARKs typically rely more on NTT. Now Spartan is a new proof system that's just come out, which only relies on MSM, so there's no NTT at all, which is kind of interesting because you can imagine optimizing only the MSM and not having to worry at all about this other operation. Right? So in general with hardware acceleration, the more generality, like the more the broader the set of things you have to do, the harder it is to optimize right? To make one thing really good so those are the systems in which it it's most applicable.

Anna Rose (01:00:42):
Is Groth16 also only relying on the MSM or it does it have both?

Alex Pruden (01:00:46):
Groth16 most, like the majority of the computation is the MSM, but it still does require some NTTs

Anna Rose (01:00:53):
Actually. Can we talk a bit about the NTT? What is that part of the competition? You guys came in second there. I also want to hear what you did there.

Ben (01:01:00):
Yeah, so the NTT is used as part of speeding up polynomial multiplication and so kind of what we are doing on the FPGA is just very fast Fourier transformations and our approach was just to have as many causes working in parallel as we could that would break down the problem and pretty much get it done as fast. And this competition also had an aspect of power and, you know, how much space on the FPGA you were using. So we also optimized for those kind of factors as well.

Anna Rose (01:01:34):
When you say parallelization in the hardware context, what does that actually mean? Does it mean like running more machines? Like, I'm just trying to picture what, or is it the, in the algorithm itself, there's parallelization on the same machine. Yeah, maybe you can help me with that.

Ben (01:01:50):
Yeah, so on the FPGA you, you basic, you have like a sea of logic gates and you can implement whatever you'd like and there's a lot of area, so you can basically implement a lot of circuits doing the same thing, but they're happening at the same point of time and so you feed out a huge parallel amount of data to all of these cores and they all process results at the same time and then return it back to you so you can kind of get a speed up just by increasing your area, it's like having a CPU of 10,000 cores or something.

Anna Rose (01:02:19):
Is it all about kind of space optimization then? Like are you like looking at this one space you have and just like, oh, we're going to paralyze over here, it might take up a bit more room, but it'll be faster and then we're going to not parallelize this part of it. Yeah. I'm just kind of wondering how you approach this.

Ben (01:02:35):
Yeah, definitely. When you start with this kind of problem, you look at the algorithm and you think, how can I break it down? What parts can run in parallel? And then there is definitely a part of the FPGA physically, you know, do I have enough resources? How many DSPs, which are digital signal processes do I have? How many RAM blocks do I have? You know, can I paralyze this a 100x or only 10x? And then you kind of go from there and there's another kind of strange quirk you have to look at on the FPGA is that because you're actually wiring things up between all these blocks you have to make sure that you can meet timing and there's also like power constraints and you know, so you could paralyze it too much and then your design just won't work or it might work, but it will work at one megahertz, which is way too slow, right yeah. So these are the things you all kind of have to take into account as you are working through the algorithm doing hardware design.

Anna Rose (01:03:32):
Cool. I'd love to hear about the outcomes. I feel like when looking at some of the ZPrize documentation after the fact, there was a lot of like increases, doubling, quadrupling of speeds. What happened in the case of the FPGA, the MSM and the NTT?

Alex Pruden (01:03:50):
Yeah, so the FPGA competition was a little bit unique, both and both the prize categories of MSM and NTT, which these this team competed in. It was unique because there wasn't really an existing baseline to go from and that was, this is one reason I'm really excited about the work that, that Rahul and Ben and their team did is because this now for the first time is you have a baseline that other people can bench against.

Anna Rose (01:04:14):
Okay. Then I guess the follow-up question is, what didn't you do and what could be done in order to further accelerate this, do you think? Or do you think this is the best we can do for today?

Ben (01:04:25):
No. So we definitely looked at the other submissions to see, you know, what were they doing differently and there was like a bunch of notes that we were like, oh yeah, if we had done this, it would've improved our result. And likewise, we've thought about this and we're like, oh, why did we beat them? And there's a few points that they didn't do. So I think like combination of the, you know, all the submissions would lead to a result that's maybe, you know, 10% or 20% better. Also one thing to note is like, we are putting this on the FPGAs that are in the cloud at Amazon and they're kind of old FPGAs. So if we, you know, upgraded it for a newer technology I'm sure we'd be able to get a boost as well.

Alex Pruden (01:05:01):
And that's the FPGAs that are on the Amazon cloud were specifically for the MSM, but I believe you guys had the actual hardware from Xilinx for the NTT, right. And that was the C1100s?

Ben (01:05:11):
Yep. So for, yeah, for the NTT, we had the C1100s. Even then we noticed that we were getting bandwidth limited on the PCIE bus. And the reason is not because of the hardware, but just the development environment that was provided doesn't have the, you know, latest Gen PCIE. So there's definitely like improvements for both in terms of like the memory bandwidth technology that we want to investigate.

Rahul (01:05:35):
Yeah, just touching on future work and possible directions. We thought we could go, on our documentation website we had a page where we listed several different kinds of ideas and places in the design that we thought could be improved and sped up a bit more.

Anna Rose (01:05:51):
Maybe something for next time.

Rahul (01:05:53):
Yeah, for sure.

Anna Rose (01:05:54):
Nice. All right. I want to wrap this up with a question on ZPrize and competing, like, this was the first time, at least in our space, that something like this was happening. What was it like to participate? Was it fun? Did you meet people?

Ben (01:06:07):
Yeah, personally for me it was very interesting to learn a lot of new things and also because it's part of a competition, you kind of have this fire under your feet of, you know, the deadlines coming up.

Anna Rose (01:06:17):
Yeah.

Ben (01:06:18):
We have no idea how other people are doing, so it's like we just have to push it as much as we can you know, and then the, a little bit being nervous as the deadline approached, waiting for the results to come out so yeah, it was really a lot of fun.

Anna Rose (01:06:30):
Was there any hint from like, on how other teams were doing? Did anyone brag before submitting, or was it very quiet?

Ben (01:06:38):
So we were in the Discord kind of trying to read into messages, read between the lines of what people were asking and saying. So there was definitely a bit of that. Also after the deadline, there were some papers that were released before the official results came out, so we could read those and kind of gauge how we had done, yeah. But it was a lot of case work.

Rahul (01:06:59):
It was definitely also pretty cool just to afterwards just see how everyone else approached the problems and like kind of read about their approaches and compare to ours.

Anna Rose (01:07:08):
Nice. Well thank you so much for the interview and sharing with us a little bit about what it was like, what you did, how you did. Yeah, thanks for being on.

Alex Pruden (01:07:17):
Yeah thank you guys

Ben (01:07:18):
Thank you for having us.

Rahul (01:07:20):
Yeah, thanks for having us.

Anna Rose (01:07:21):
Cool.

(01:07:25):
So that wraps up part one of our two-parter on ZK Hardware and ZPrize. Stay tuned for next week and our two next interviews. I do want to say a big thank you to the ZK podcast team, Henrik, Rachel, Tanya, and Adam. And to our listeners, thanks for listening.