Anna Rose (00:00:05):
Welcome to Zero Knowledge. I'm your host, Anna Rose. In this podcast, we will be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise to change the way we interact and transact online.

Anna Rose (00:00:27):
This week, Guillermo and I catch up with Henry de Valence from Penumbra. In this, we discuss his thoughts on the requirements for adoption of privacy systems, how these ideas led him to develop Penumbra. We then dig into how Penumbra aims to use privacy features, not as 'nice to haves' within a system, but rather as essential components that offer a new paradigm for how to think about DeFi in a multichain ecosystem. Quick disclosure, the zk validator was an early investor in Penumbra. But before we kick off, I wanna share an announcement from one of our recent zkSummit partners, Anoma. Anoma has released a new white paper that better describes their system in architecture. It also shows how all the awesome cryptographic libraries they've been developing fit together. Find this paper at anoma.net/papers. We'll add the link in the show notes and be sure to check it out. I also wanna highlight that there's a fresh batch of jobs over on the zk Jobs board. If you're looking for a new opportunity to work with the best teams in zk, be sure to check out their job postings. I'll add the link in the show notes as well. Now, Tanya will share a little bit about this week's sponsor.

Tanya (00:01:30):
Today's episode is sponsored by Aleo. Aleo is a new layer one blockchain that achieves the programability of Ethereum, the privacy of Zcash and the scalability of a rollup. If you're interested in building private applications, then check out Aleo's programming language called Leo. Leo enables non-cryptographers to harness the power of ZKPs to deploy decentralized exchanges, hidden information games, regulated stablecoin and more visit leo-lang.org to start building that's aleo-lang.Org. You can also participate in Aleo's incentivized Testnet 3 by downloading and running a snarkOS node. No signup is necessary to participate. For questions, join their discord at aleo.org/discord. So thanks again, Aleo. Now here is Anna and Guillermo's chat with Henry from Penumbra.

Anna Rose (00:02:19):
Today, we're gonna be talking about Penumbra with Henry de Valence. I'm here with Guillermo. We're gonna be covering this project the day after zkSummit8. We're all a little tired, but I do wanna welcome you both to the show.

Henry de Valence (00:02:32):
Thanks. It's great to be here.

Guillermo Angeris (00:02:34):
Thank you. It's great to be back.

Anna Rose (00:02:36):
So Henry, you gave a great presentation yesterday at the summit. I wanna talk about Penumbra, but before we do that, you've been on the show before and there's been, I assume, like quite a story since then. It's we recorded in like April 2020. I would love to hear a little bit about like, what has happened since then, what have you been up to?

Henry de Valence (00:02:58):
Yeah, so I think at the time I had been working on one of the projects to do privacy preserving contact tracing, and that eventually I think got subsumed by the Apple Google version of that protocol. I went back to working on Zebra, which is the Zcash Foundation's Zcash Full Node Implementation and then towards the end of 2020, I sort of felt like it was time to kind of move on and have a little bit of a break. And then in, in the first part of 2021, sorry, the years of the pandemic are all like mashed together, but in the first part of 2021, I had some time off, you know, there's only so many hills that you can like hike up before you go back to thinking about blockchains. And then that's kind of how I started thinking about the problem space that eventually evolved into Penumbra and kind of reflecting on the experience of spending say, you know, four years working on various kinds of ZK tooling infrastructure, trying to build all the necessary technical legos to build the kinds of private systems that I wanted to exist in the world.

Henry de Valence (00:04:19):
And yet at the same time, looking around at the ecosystem and seeing, well, there's like all of the projects that people actually use and then there are the private projects and this Venn diagram has like an uncomfortable resemblance to just two distinct circles.

Anna Rose (00:04:37):
Oof

Henry de Valence (00:04:38):
And as somebody who's right, like if you spend all of your time, like actually like, you know, you're sort of pouring your soul into these technical artifacts, the idea of, oh I'm just kind of yeeting all this stuff out into the world and it is never really getting used. It's not a great feeling. And so I had a process of thinking through sort of what would it look like to have some kind of theory of change for how people would actually start using private blockchains and what is the reason that people aren't using existing privacy tools?

Henry de Valence (00:05:19):
And is there some way that you can kind of hook these things together and find something where you get like actually good market fit for a privacy product? And what I mean by that is it's not just, oh, I'm trying to build this thing and it's private and people will use it because it's private. Like how do you break out of this mode of, oh I want to build a private X, which means like, this is the thing it does X it's like kind of worse at it, but it's private so like maybe people will want to use it, right? Like that's not really a compelling product pitch. How do you get to, you know, we're building a better X and the reason that it's better is because it has these privacy features and we don't have to spend time arguing about like, oh, is privacy a human right? You know, I believe in that, but I don't want to have a, I don't think it's workable to build technical products that are like, based on that as a pitch, you have to build something that's actually like better for what people want to do.

Anna Rose (00:06:24):
It sounds pragmatic. It's like looking at human nature. We can wish that people cared about privacy or we can make it so useful. So, so much better to actually use these private systems anyways.

Henry de Valence (00:06:38):
Yeah. I mean, I think that's, I would push back on that framing a little bit in a way that I think is hopefully interesting, which is, I think when you look around, there's actually a lot of evidence that people do really care about their privacy, but when people make choices, like this is the whole reason that I think like the idea of like revealed preference is bullshit, a lot of the time.

Anna Rose (00:07:01):
What do you mean by that?

Henry de Valence (00:07:02):
So revealed preference is the idea that like, oh, if you want to know, like what somebody's preference is then like, instead of asking them for their opinion, you just like, look at what they do, because that's sort of like revealing their like true, you know, choice or preference or whatever. But what that misses is that when people make choices, they make those in some specific contexts and the entire like technical infrastructure of our society has been carefully crafted. So that those choices cause people to choose to do things that benefit various corporate entities, right? Like the choice of, oh, do I want to have Google reading all of my emails? Right. If somebody's just asking me that, you know, abstractly of course, no, I would rather that they not do that, but the scope of that choice in practical terms is do I want to never communicate with anyone via email because every other person that I would want to talk to already has a Gmail account and so they're gonna be getting all of the email anyway. So I don't actually have a meaningful choice about my privacy, you know, based on sort of what email provider I use or something.

Guillermo Angeris (00:08:18):
I mean, there's a whole thing too, right? Like people have been trying to self host email lately, but because of like a bunch of reasons, like spam first, but also secondarily like Google, Microsoft, etc, will just like reject your emails outright if you're self hosting, unless you do like jump through all of these like crazy hoops, because otherwise they just like assume it's spam. So even like

Henry de Valence (00:08:38):
I think not to be like, you know, the Microsoft defender, but like also likely it is. Right, there's also an interesting story that I think is relevant to blockchains or other decentralized systems about email, where you have this sort of purely decentralized protocol where there's no real kind of economic model. And then you have this spam problem, which you can think of as sort of like this resource usage abuse issue. And the way that that has actually been solved is that email is a centralized system that is like dressed up in a decentralized protocol, Google just reads everybody's emails and figures out what the spam is and then bans it. And that's why email works. And I think part of the challenge of building a decentralized protocol in general is how do you account for enough of the economic logic of the system internally to the logic of the system itself. So that if it's successful, you don't have a situation where some monopolist has to come in and close that comments and then centrally manage the resources to like take account of that sort of lack of economic model.

Guillermo Angeris (00:10:00):
So I mean, I think the problem with like email and it's naive incarnation, right, is, and this is just a restatement of your points, but there's a mismatch between economic incentives of like bad actors and like economic incentives of good actors. Right? Like you are, it is very easy for someone to kind of like maybe this is a weird term, but maybe doss the comments, simply because it is like very simple, very cheap to do so. Right, while the cost burden of that kind of externality right, is very expensive for a user because like sifting through, you know, 5,000 pages of like spam bullshit is usually like particularly costly for any one individual.

Henry de Valence (00:10:43):
Yeah, I think that the phenomenon is more general than that though too. Because another example that I think about is GitHub, where you have Git, which is this decentralized system, a decentralized protocol and although it is at the protocol level, decentralized in practice, the way that Git works is that everybody has a GitHub account and GitHub hosts all the repos. And I think part of that story is that the social infrastructure of doing code collaboration requires like various kinds of technical infrastructure, like hosting like issues, having some kind of system to keep track of proposed changes that isn't just like people emailing each other on a mailing list and because those aren't part of the system, they end up being provided by Microsoft who pays to maintain all of that infrastructure so that they can sort of have ownership over this like developing community.

Anna Rose (00:11:48):
Do you feel like that to, and obviously there's been proposals to decentralize that, but it's just so enormous, like an endeavor that it's very slow to happen.

Henry de Valence (00:11:59):
I think it would be good, probably better. You know, if somebody makes like a better thing, that would be great. It is a lot of work. I know that like for instance, like Radicle is trying to build some alternative, but rebuilding like social infrastructure is a lot of work, you know, that's this whole tangent digression, that's hopefully interesting to the listeners, but where we got onto this, right, was this idea of how do you build a system that can provide private infrastructure, but with some kind of self sustaining feedback loop that actually drives people to want to use that system as a product. Privacy is actually really interesting in that it has very strong network effects, right? The larger, the anonymity set is the better the privacy.

Anna Rose (00:12:53):
Or even like the more people participating in private systems even like say it was like email or something, then you don't have that kind of like, or maybe it's the opposite, if there's more people participating in the very non-private it reduces the privacy of all, even those who are being careful about it. So I'm guessing the opposite is true. If more people are careful about privacy than generally that privacy, not even the set, but just like there aren't the level of surveillance that you would have otherwise.

Henry de Valence (00:13:21):
Yeah, right

Anna Rose (00:13:22):
By the way, just a note to the listener, we are recording this as mentioned the day after the zkSummit 8. So if we are slow aka, if I am slow, I apologize.

Guillermo Angeris (00:13:33):
So, yeah. But, but also more generally, right? Like the more people that use a system, which is anonymous by default, the harder it is to pinpoint any one individual person. Right, so this is, this is another notion of like anonymity as well.

Henry de Valence (00:13:47):
So that was one kind of line of thought. So we'll just sort of pause that and we'll see how these intersect in a minute. The second line of thought that I had at the time was thinking about whypeople weren't really using Zcash specifically. Since that was what I had been working on and I got to this perspective of kind of concluding that although payments are held up and have been held up, you know, since Bitcoin was created, right. It's like a peer-to-peer electronic cash system. The whole idea is like, we're gonna do payments and that's like the simplest possible thing. It's like the MVP for building blockchains. And if we can do payments, that means we're onto something. And if we can't do payments, that means we're failing at the most basic thing. And my conclusion was actually that that is completely backwards and that payments are actually like the hardest possible thing to build and the reason for that is that the simplicity of a payment transaction, oh, I have some ledger there's value in this account or address and it goes to this other address and I'm just like, you know, there's numbers moving around in the database is actually a totally superficial simplicity. And the reason is that the economic transaction is always bigger than the transaction that gets recorded on chain. So whenever somebody makes a payment, there's always two sides of that flow. You have some flow of some

Anna Rose (00:15:27):
Exchanged for

Henry de Valence (00:15:29):
Exactly that's right.

Anna Rose (00:15:30):
Funds and you don't see the goods.

Henry de Valence (00:15:31):
Yeah. Even in the case of, you know, for instance like a donation, like you can still think that there's some kind of like abstract flow in the opposite direction. Like somebody is, you know, getting good feelings about contributing to some project or something. And so you have a situation where the technical part of the system is recording only half of the full economic transaction. And so in order to, to use that usefully, you have to have this whole parallel social infrastructure of where that counter flow is happening off chain. And when you're trying to bootstrap that you're running into the sort of technology with the strongest possible network effects in human history, which is money. Right, and so it actually feels to me like, can you do payments on a blockchain is I think gonna be sort of like the final boss of this like cipher punk dream where like, you know, blockchains take over the world or something. And in the meantime, chasing down payments is kind of this like Moby Dick, like white whale of like, this is the thing that, you know, and all these other projects of like, you know, failed to do this, but mine will be totally different. And, you know, at every point that the ship just gets wrecked.

Anna Rose (00:16:57):
Do you sort of talk, are you saying something like you'd need to be building or bringing that value more on chain? The other side of that transaction before you could really accomplish this payments, would it work more in digital goods in a way?

Anna Rose (00:17:11):
So the thing that I was was thinking about at the time was, was actually reflecting on seeing the DeFi summer happen, you know, over the last, at the time was like over the preceding six or eight months. And the thing that I think is really interesting about all of these DeFi protocols, is that okay, although what those tokens are, is kind of limited. The types of transactions that people were making with DeFi systems were ones where the entire economic logic of that transaction was included in the blockchain transaction, right? Like when somebody makes a Uniswap trade on Ethereum, they don't need to have like some kind of counterparty that they've arranged with off chain to do, blah, blah, blah. They just say, okay, this is a thing that is gonna be useful to me. I've decided that I wanna make this trade and now I'm gonna do it.

Henry de Valence (00:18:11):
And what's really powerful about that is that if you're trying to get adoption, you've created a scenario where individual people can decide to start using the system without having to pre coordinate with anybody else. And I think this is actually the key thing that makes payments so hard. It's that, what are we building when we're building blockchains? We're building this coordination tool, but if you can't do the thing on chain directly, if you have to do off chain coordination, then it's like, oh, before I can use the coordination tool, I first have to somehow coordinate before I can coordinate.

Anna Rose (00:18:49):
Totally.

Henry de Valence (00:18:50):
And I think that that makes it much harder to get adoption. Whereas if you can start with something like, you know some kind of DeFi application where people are able to get direct value out of transactions, that they can make themselves, that's a much better way to bootstrap a user base, because once all those people are on the system, then you know, maybe actually payments do kind of work.

Henry de Valence (00:19:15):
Right. If you look at where crypto payments are really successful, like there's a lot of use of crypto payments by crypto people, but that's because they're all already in the system. And so the thought was like, okay, could you have some kind of like DeFi functionality that would let you build a private system where individual people could make a choice to start using it without having to first sort of pre coordinate with some other counterparty, you know, how would they even find that person, right. And the place where those two lines of thought kind of converged was thinking about, okay, of all the DeFi stuff, is there an application where we're having privacy is actually this kind of like killer feature where you can actually be better as an X because you have control over information disclosure. And in that case of in that sort of framing trading is a really, really interesting use case because every market is also a market in information.

Henry de Valence (00:20:23):
If you think that like the, you know, one of the purposes of a market is to determine prices, then you know, that market activity is some kind of like decentralized, computational process where people are sharing information like it's a really like fundamental part of the dynamics of that system. And so if you, if you're looking out at all this like DeFi trading where everybody has to reveal everything to everybody at all times, it's so obvious that like, that can never be the best system because it's just completely missing this whole like giant part of like what makes that such a like deep and interesting dynamic. I mean, market dynamics, I mean.

Anna Rose (00:21:08):
Are you thinkingso when you say this, it definitely makes me think of like strategies of certain groups being exposed. And then I guess people just copying those strategies over and over again, why is a private system better?

Henry de Valence (00:21:21):
I think that the idea of sort of a market as being a marketing information is such a fundamental thing that you can actually see that in a lot of different types of behavior. So one really obvious example is like all this, you know, front running, fundamentally that's about somebody sort of revealing information before they can do their action. And then that being exploited on the strategy front. Right, exactly as you mentioned, if somebody has some kind of strategy where they've, you know, done a bunch of statistics and figured out what their estimation of the prices is, if somebody else can just like clone all of that without actually having to do the work themselves, that kind of limits the incentive to do that kind of work and there's other cases like for instance, with market makers, generally, a market maker wants to stay flat with regard to price movements, right?

Henry de Valence (00:22:20):
Their goal is basically trying to make a little bit on every trade and not get rolled over by being wrong. And so if somebody can figure out, okay, I've carefully watched this market maker's activity, I can kind of reverse engineer what their strategy is. Another thing is instead of cloning it, if you can predict like what the like 2% of times they're gonna make a wrong prediction are then you, as a trader can like roll in a huge order, right when you think they're gonna do a mispricing and that can like totally wreck their whole economics, right?

Guillermo Angeris (00:22:56):
Yeah. This is a very famous term called adverse selection is the fact that if information is kind of known ahead of time and could be used, it can also be exploited, which of course is we know from MEV is a very I mean, MEV in many ways like on chain is like a market and information where not only do you have kind of the statistical notion of information, but you have a very, very concrete information about, you know, exactly what someone's gonna do and when it'll get included in a block and then you can do things to trade against it. So privacy here is a very interestingidea, right? Because essentially, if you could somehow, I mean, this is, people have been thinking about this for a very long time, not quite in this framing, but just more generally, right? It's like, if you could prevent this from happening this information leakage from happening in the first place, then congratulations, you've kind of like achieved - you've kind of cut out a huge part of this kind of, you know, people using knowledge that it's kind of silly that it gets leaked in order to trade against certain person or something like that.

Anna Rose (00:24:00):
Do you think it would become like that? The thing is that very transparent system has created a whole range of new strategies and attitudes and ways of doing it in the private system. Is it sort of a return to previous market types or is it it in itself a brand new playing field with like brand new ways that people would interact on it?

Henry de Valence (00:24:22):
I think

Anna Rose (00:24:23):
Are you thinking about this?

Henry de Valence (00:24:25):
Yeah, I think, so the thing that I think is really interesting is that, and that this is kind of whole other tangent that also comes in, you know, we'll get to, at some point when we talk about

Guillermo Angeris (00:24:36):
The later section in the paper, we'll speak about this.

Henry de Valence (00:24:38):
But another kind of interesting and underappreciated aspect of this is that if you try to build a private market, the problem, isn't just, okay, how do we make everything private? Like, how do you just like, make like every piece of this whole system private and it's all completely ZK and blah, blah, blah, blah. Well, actually nobody wants to have a market where literally every piece of information is private, because like, you want to know, like how much liquidity is there? Like what kind of trades are there like, are happening in general? What is the aggregate state of the market?

Guillermo Angeris (00:25:11):
What is the price of the market?

Henry de Valence (00:25:12):
Right. So there's a very, there's a much more kind of subtle and interesting problem of privacy, which is how do you have privacy for individual actions so that, you know, some individual trader doesn't have their specific trade front run, or some market maker can have their strategy concealed, but you still have some kind of aggregate information about what is, you know, the total trading volume, what is the aggregate set of all of the liquidity that's available? How is that moving around so that all the participants in the system are able to make choices based on that information. To go back to your question of, is this just a kind of return to tradition or is this something genuinely new? I think that there's a possibility for private DeFi to build this kind of hybrid, best of both worlds situation where you can keep some of the confidentiality of, you know, people being able to do trading without leaking their strategy.

Henry de Valence (00:26:20):
But also you're in this totally open and permissionless environment. Like if you look at the way that traditional trading works on like a centralized exchange, the entity that's running the exchange actually, you know, sees everything that everybody is doing all the time. And just like, doesn't tell that to other traders, like, hopefully, right. But also now you have like the NYC or something, which is this, you know, like centralized entity that gets to like gate keep like people's access to price feeds, right? Like you can't even like get prices without paying a bunch of money to some data provider, whereas in a blockchain system, like there is a much greater degree of kind of permissionless in terms of access to the system.

Anna Rose (00:27:11):
So I think now would be a really good time to talk about Penumbra.

Guillermo Angeris (00:27:15):
Not just in generality, but in specificity.

Anna Rose (00:27:18):
Yeah. I mean, I think we've given those some really, I mean, just the thinking and sort of the path that you were on in kind of on intellectually that leads up to this more tangible project. Tell me a little bit about the beginning of Penumbra. Like where, where does this start?

Henry de Valence (00:27:33):
Yeah. So with all that, you know, just kind of like, hold up context, you know, somewhere in your mind, the kind of, how do we actually manifest this is what kind of became Penumbra. The thought is if we want to have the genuinely private, decentralized financial infrastructure that we think is important to exist in the world, can we build a private trading system that lets people do things better because it's private and use that to like bootstrap the system sort of creation, its use its adoption, et cetera. So that's how I started on the project. At the time I was kind of in this like awkward period of visa related limbo, the US just like never issued me a work permit because they're

Anna Rose (00:28:28):
I think I remember this. Yeah.

Henry de Valence (00:28:29):
So we don't have to get into this whole thing. Like the US immigration system is just like very bad and dysfunctional.

Anna Rose (00:28:35):
Do we think Canada's better?

Henry de Valence (00:28:38):
I've never been on the receiving end of it. So who knows?

Anna Rose (00:28:41):
I just found out today that Henry and I are both Canadian. That's cool. But yeah, we don't know what our immigration

Guillermo Angeris (00:28:46):
But guys have points actually. It's not so bad

Anna Rose (00:28:49):
Actually. I heard it's actually, I heard you could do it if you don't live in Canada. Yeah.

Henry de Valence (00:28:52):
I have no idea. Anyway. And so at the time I was kind of stuck in this position where it's like, okay, well I can't technically, you know, work on anything, but I can just sort of like do research, publish ideasput them out into the world and sort of see what ripples come back. And so I had started writing up kind of this like design sketch as a kind of a way to like occupy time until, you know, someday when I would get some kind of clarity.

Guillermo Angeris (00:29:22):
Wait, I kinda remember this. This is like when we were talking about the privacy results actually

Anna Rose (00:29:28):
Differential privacy,

Guillermo Angeris (00:29:29):
No. Prior to, to that, oh, this was like,

Anna Rose (00:29:31):
Oh yeah,

Henry de Valence (00:29:31):
This was like the first half of 2021.

Guillermo Angeris (00:29:34):
That's right. Yeah, that's right. Yeah, yeah. Yeah.

Henry de Valence (00:29:35):
Okay, so that was mainly a way to kind of stay occupied and sort of actually set the course on building in public. It turns out that if you just post a bunch of ideas on the internet sometimes people will read them and sometimes those people will have a technical background that is like totally different from your own and because of that, they can tell you things like, oh, actually you should do X instead of Y or you should go read this paper or whatever. Right, and so that was like pretty cool and quite rewarding. And then eventually over the summer, you know, that kind of situation got resolved and by that point I had kind of spent enough time thinking about this, that I realized that I really wanted to actually make this real. And so then, you know, started trying to get it together to build a team and try to like incubate this project that got underway for real about last September. So we're almost exactly a year into the development of the project. And since then we've continued to iterate on the design and we've also built most of an implementation.

Anna Rose (00:31:03):
Cool. I think what we do need to hear is what is Penumbra? So we've talked like now we know it as sort of an org. Like from what we had talked about before the markets, what is it today? Because also I do wanna say like, and we'll add maybe links to this. I remember when you first started on the project, there was like this private validator sets. There's like a lot of different places where privacy was being explored, but where did you land?

Henry de Valence (00:31:27):
Yeah, so probably the, the succinct way to put that is Penumbra is a fully private proof of stake, L-1. And inside of that blockchain, there's one multi-asset shielded pool that records all of the value in the system. We have a shielded staking system that gives privacy to delegators, but you know, still accountability and transparency for the validators. And we have integration with IBC. So any IBC compatible chain, whether that's a Cosmos chain or any other place where IBC has been implemented, they can transfer assets into Penumbra as those assets come in, they are recorded in the shielded pool. So the privacy boundary of the system is actually the boundary of the chain and once you have those assets inside of Penumbra we have a shielded decks that allows people to do either sealed input batch swaps on the kind of market taker side.

Henry de Valence (00:32:43):
So when somebody submits a swap, they say, I want to trade, you know, this amount of this token for this other token, the input is encrypted and batch together with all the other swap intent in that block executed at a common clearing price and only the sort of aggregate batch total of what the, the trading flow for that entire pair is, is revealed. So you can have long term privacy, even for individual trade amounts. That's kind of on the market taker side, which is one sort of point in this trade off between how much privacy you do get versus how much control over your trade execution do you get. The other trade off point that we have is on the market maker side, where we have all concentrated liquidity and the concentrated liquidity positions themselves are public, but they're created out of this shielded pool and they're returned back into the shielded pool and the positions themselves are immutable.

Henry de Valence (00:33:49):
So it's not like, you know, on Ethereum where there's some account and it like owns certain positions and it can change them over time. It's that, you know, a position is created and then maybe the position will be closed and later another position will be created, but there's not any way to link together what the history of those positions are. And especially in a context where a market maker has some kind of more complex trading strategy, they're likely going to be saying, okay, I have this, this is my desired trading function. I'm gonna approximate that using various like smaller individual positions and which positions correspond to like which market maker or their strategy is not revealed. Maybe you could try to estimate do some kind of statistical guessing, but really what you see is just the aggregate of here are all the liquidity positions and that's kind of the like market view of what prices are, but you don't get to see any individual market makers.

Anna Rose (00:34:53):
Can I check something when you talk about the different liquidity positions having sort of this immutability, is that similar to like the NFT, like thing we see in Uniswap V3? Like, is it sort of unique? And it, and maybe it's not an NFT per se, but like where it's like, it's in itself, it's not just the collection of tokens in a setting it's like produces a new token of some kind or some sort of representation.

Henry de Valence (00:35:21):
Yeah. It's an NFT in like the technical sense and not in the like monkey picture sense.

Anna Rose (00:35:27):
Yeah. I shouldn't - different words. We need different words.

Henry de Valence (00:35:29):
But yeah. So similar to, to Uniswap V3

Anna Rose (00:35:34):
Except very private or parts of it being private.

Henry de Valence (00:35:37):
So the difference is that the actual LP data that forms that NFT that is public, because that's part of what the market state is, right. What's private is who controls that position and where the funds for that position came from. So unlike in say, Ethereum, where everything is sort of going back to some account that controls it on Penumbra, all of these positions are just, you know, they're more tokens that are in the shielded pool. So the data that's revealed when someone creates a liquidity position is okay, there's a transaction. It spent some funds from the shielded pool, it put them into a position. You know, what the details of that position are, is public. But that position is then controlled by this bear token, which is an NFT. And that NFT is also recorded in the shielded pool.

Anna Rose (00:36:38):
Interesting.

Henry de Valence (00:36:38):
Just like any other asset.

Anna Rose (00:36:39):
Wow. So like, those are mixed in sort of helping to even grow that anonymity set, like more types of tokens.

Henry de Valence (00:36:46):
Yes.

Anna Rose (00:36:46):
Okay, interesting.

Henry de Valence (00:36:47):
All you can see is like effectively, okay. There was a transfer of like, you know, 100 ATOM and like, you know, 20 USDC or something into this position. And that's it,

Guillermo Angeris (00:37:01):
Right? No, I mean, overall, it's like a very interesting architecture because evenq on the LP side, you, well, you don't get privacy of the positions themselves. You get privacy in that, like you get this notion of kind of unlink-ability. There's like a statistical notion of linkability, right. Like if I knew your strategy ahead of time, I could maybe try to unscrew whatever you've been doing, but I would've never know that it was actually you other than like, by knowing ahead of time that I knew what you were gonna do and like kind of

Henry de Valence (00:37:32):
It's a little bit worse than that from a privacy standpoint and like one way that you could have linkability between positions is like, you know, suppose somebody closes a position and then in the next block, they open a new position that has, you know, roughly the same amount of capital, but like the bounds are like slightly shifted over. Somebody could infer that actually that's probably the same person and this is why any kind of private system that doesn't have privacy of amounts is actually kind of dangerous from a from a privacy standpoint. Because if you're not careful about like what your amounts are and like what information you're revealing with the amounts, you can potentially leak a fair amount of information, but the thought is that like

Anna Rose (00:38:22):
How is that? How, why? That seems like counterintuitive no? Well, like by not having the information you leak the information.

Henry de Valence (00:38:31):
Oh no. Sorry. If the system reveals the amount that actually reveals quite a lot of information about what the activity is.

Anna Rose (00:38:41):
I see.

Guillermo Angeris (00:38:41):
So an even sillier example, which like has happened I think in some cases is people have tagged the end of like certain. So for example, you know, you have a bunch of like decimal places in a token. Right, and what you do is let's say, you know, at the 15th decimal place, you put 1, 2, 3, 4, 5. And so someone who isn't really paying a lot of attention. So like, you know, let's say I've given you, we're all doing a bunch of shielded things. I give you whatever 10.0000012345 Penumbra. And you're like, oh cool, I have like 10 Penumbra because a lot of, you know, wallets will not show you the 15th through 18th decimal place. What can happen is if you then go, so, you know, let's say everything is completely private and then you then go and you know, for some reason or another unshield, that specific value, because you let's say you've transferred it out of Penumbra into some other chain. I'm immediately gonna know that it was you because you didn't notice this like small end of this, like, you know, value tagged at the very end. Right, you like, I now know it's you because by, you know, the probability that anyone else is gonna have exactly that, you know, and like little decimal tail place is pretty low. So, and this is actually a common fingerprint,

Anna Rose (00:40:00):
But wouldn't there be gas?

Guillermo Angeris (00:40:01):
What up?

Anna Rose (00:40:02):
Wouldn't there be gas fees somewhere?

Guillermo Angeris (00:40:03):
Yeah. So, well, it depends on how it's paid. Right, but like if you were just to withdraw, for example, let's say you're paying gas fees in ATOM and you know, you're withdrawing Penumbra to a separate place. You end up with this like extra little tidbit at the end. I think some people have used as a fingerprinting technique at some point, but I don't recall who

Henry de Valence (00:40:22):
Yeah. I guess like the way that I would frame this, just like as a kind of like general like intuition on why this is a problem is if you think of like, okay, how much entropy is revealed, how much information is my transaction revealing, if that is including publishing an amount like if you have like, you know, a 32 / 64 bit amount, that's a lot of bits of data that you could potentially leak

Guillermo Angeris (00:40:43):
Henry, the differential privacist

Henry de Valence (00:40:45):
Right, but coming back to the liquidity position side of the decks, the thought processes that there's a kind of a fundamental trade off between how much control you have over the execution of your liquidity on the one hand and your privacy on the other hand and although for sort of normal users, I think it's unreasonable to expect like, oh, you should just like, you know, carefully think about your amounts and never mess up for somebody who's running a kind of actively managed market making system. They're having to think about, okay, what is the information in the market? What prices am I setting and so on? And so I think that it's like a little less unreasonable to have, you know, as in that part of the system, the idea that like, you know, maybe you decide that you should be thinking about sort of like what information your trades are revealing.

Guillermo Angeris (00:41:50):
Leaking, yeah.

Henry de Valence (00:41:51):
And so that's the reason that I phrased this sort of being these sort of two tradeoff points on privacy versus execution control, where if you just want to do a swap and have the most amount of privacy possible, you just submit it as a swap. If you're doing market making, then like you kind of need to get good.

Guillermo Angeris (00:42:11):
Right. All right. Even in the case of the swaps though, you still, you know, unless you have a lot of people swapping, you still have this notion of differential privacy, right. Like even if you just batch everyone's trades together. Right, if I also know something like, let's say there aren't that many people trading with the, you know, the market there's like you and someone else. Right. I can reasonably infer if I know something else about you, I can reasonably infer that how much you traded or something of a like, but it's much harder. Right, because there's very little public information that the only information that's made public is kind of what everyone traded in aggregate as opposed to like anyone individual person

Henry de Valence (00:42:51):
One of the ideas that we've been thinking about that on the client side actually is the idea of having some kind of UI for doing trading where a user can specify their time preference, where they can on some kind of slider between like, I want this to happen right away, versus I want this to happen over, you know, the next like five minutes or the next hour or the next eight hours, whatever. And have the client side UI be able to automatically prepare a kind of randomized subdivision of their meta swap and submit that in individual unlikable transactions at like randomized sub intervals.

Guillermo Angeris (00:43:40):
So wait, they're actually essentially implementing a time weighted average pricing mechanism.

Henry de Valence (00:43:45):
It's randomized. Yeah.

Guillermo Angeris (00:43:46):
With extra randomization. That's pretty sick.

Henry de Valence (00:43:48):
The thing that's really cool about this is that for the user, you actually get sort of two benefits at once. Number one is that you get some kind of hedge against execution risk. Like maybe you get batched in a swap with some like whale that's trading in the opposite direction or the same direction and that like wrecks your price. But since you did this, like randomized average price mechanism, it only affected a small part of your trading and you also get this privacy benefit that even if, you know, you end up, you get unlucky, nobody else was trading in the same block as you and the batch size is a batch size of one. Well, okay, you're only revealing this randomized sub amount that reveals with much less information about your total trade intent.

Guillermo Angeris (00:44:34):
Right. So, but the interesting part is this goes back to the previous conversation, right? Is that this whole notion requires like, the notion of privacy, right, requires having more people interacting with the system. So it's just like Ouroboros for like, you know, you have to have people in the system in order to have privacy in order to like, ideally also have more people in the system in order to have privacy.

Henry de Valence (00:44:55):
Yeah, the way that I would put that, which is actually like, you know, specifically intended to nerd snipe Guillermo

Guillermo Angeris (00:45:01):
Damnit

Henry de Valence (00:45:02):
Is that when you're optimizing for privacy this is not like a convex optimization problem.

Guillermo Angeris (00:45:10):
It's not? Wait something, isn't a convex optimization problem. I never would've guessed.

Henry de Valence (00:45:15):
And so, but the reason that that is like relevant is that the local minima is not the global minima or maximum, you know, depending on which way you frame the optimization problem, if you're trying to get the most privacy, maybe you decide, oh, I'm gonna build a bunch of, pieces of this system so that every action is as private as possible and it's impossible for anybody to leak any information, but then by doing so I've sort of, I've locally optimized for making all the transactions private, but I've curtailed the functionality of the system so much that now it has no users. And so it's like, oh, I have very good privacy within a very small anonymity set. Whereas another set of engineering trade offs might have slightly weaker privacy properties for each individual transaction, but then caused the system to be much more useful and have a much larger set of users.

Henry de Valence (00:46:14):
And so you maybe have like slightly weaker privacy, but like within a much larger set and so on balance, that's actually better that's what I mean by it's sort of this like very like complex optimization problem. And one way that, that plays out with the swaps is we're planning to have Penumbra do the batching at block level resolution. So when you submit a swap the trades that it will be batched together with are just the trades in that block. And if there's no other trades in that block, then you're in a batch size of one. And the reason for that is, although that's, you know, from a privacy standpoint, maybe it would be more optimal if you said, oh, you have to wait for like, you know, 5 minute intervals or hour long intervals or something. But in that case, I think it becomes a lot less useful as a trading product.

Henry de Valence (00:47:11):
And in that case, you end up with, okay, well, there's this like weirdo, like privacy project. That's only used by the like, you know, weird nerds who care about privacy and then you end up being, you know, private within a much smaller set. Whereas if you build something that is intended to be like, you know, best as a trading system, then you can do things on the client side, like that kind of like randomized average pricing mechanism that kind of mitigates the risk of the individual batch being, you know, batch of one or somethingwhile preserving the system for people who just want to do trading, because ultimately we want Penumbra to be providing just like better execution, even if you don't care about the privacy part at all, you should still be using Penumbra to do trading because the fact that the market making allows private strategies means that you're gonna be able to have market makers with like much better capital efficiency and therefore like lower spread, better prices for traders.

Anna Rose (00:48:23):
Interesting. I actually, now I'm curious to ask you, like, you're in the Cosmos ecosystem, are you competing with Osmosis?

Guillermo Angeris (00:48:32):
Ooh. That's a spicy question.

Henry de Valence (00:48:36):
I honestly, I don't think it's that spicy.

Anna Rose (00:48:40):
Okay.

Henry de Valence (00:48:40):
Like, no, not really like Osmosis is a DEX, but it is also a lot more than a DEX. Like Osmosis is this sort of whole like DeFi hub for Cosmos. And so just pausing on the Cosmos part and just looking at privacy projects, generally, there's a lot of privacy projects that are trying to build some kind of like general purpose private system, with different kind of takes on what that means. For Penumbra, we're not trying to do that. We're trying to build like one useful private application. We are an app chain, that's a DEX, and we're doing one thing which is private trading and private market making. And with that kind of in mind, I think that that probably clarifies, like why I don't really see this as being competitive with Osmosis. If anything, there's this kind of like hybridization, because we're both in this interchange ecosystem.

Henry de Valence (00:49:40):
I think there's gonna be a lot of interaction between users who use both Penumbra and Osmosis. There's all these things that Osmosis does as this like DeFi hub that Penumbra is just like not gonna do and it's actually because of IBC and because of the existence of other chains like Osmosis, or like Juno, or like the Cosmos hub or whatever that it's possible for us to have this like very narrow, differentiated product strategy.

Anna Rose (00:50:10):
Interesting.

Henry de Valence (00:50:10):
And that's another lens that I think is interesting to compare sort of the state of the ecosystem today versus say, you know, four or five years ago, right. If you wanted to build an L-1 four years ago, well, your L-1 is gonna be this like isolated little universe. So you'd better make sure that, like, you know, you have a programmability story that you have, like all of these pieces of like what it means to be an L-1, you're gonna have to go and do like all this, like biz dev to get it listed on all these exchanges, you have to like, do all of this stuff.

Henry de Valence (00:50:43):
And all of that work is just repeated over all of these different projects and the thing that's really cool about IBC and having a really solid cross chain communication mechanism is that now individual projects can be differentiated while still having the sovereignty that comes with being their own chain. Right, so we get the freedom to design a chain with like the data model that's compatible with privacy, with all of the trade offs that we think are important to make, but we get to do so as part of an ecosystem, right? Like we're not trying to build an ecosystem. We like live in a society of other blockchains and our users are gonna interact with them. And so I don't really see it as being, you know, particularly sort of this kind of like zero sum competition or anything like that.

Anna Rose (00:51:39):
I actually wanna follow up on this. So you said you're IBC enabled, but are you actually using the Cosmos SDK as the foundation, or did you have to like create a lot of this on your own and then just use the IBC modules?

Henry de Valence (00:51:51):
So yeah, maybe it's helpful to give a little bit of context for people who are not like, you know, totally Cosmos nerds. I think a useful analogy for the Cosmos SDK is that it's sort of like rails for blockchains in that if you wanna like do a blockchain project, you can like generate this kind of opinionated scaffold of, you know, reasonable implementation of various blockchain stuff. And then you can write your own extra, if you wanna do like one custom thing, you can add that in as a module. And like all the other stuff is, is written for you. The Cosmos SDK sits on top of Tendermint, which is the consensus engine. And so there's this separation between the consensus layer that's determining what was in the blocks and what the blocks are and the application level that's determining what the meaning of those transactions are.

Henry de Valence (00:52:44):
Right, so to Tendermint, a transaction is just like a blob of bites that gets passed to the application. And then it's the application that gets to decide what to do with it. And if you're, if you're trying to build a blockchain, that's like mostly the same as other chains, but does like, oh, this is our thing that we do differently. Yeah, yeah. Then I think that's actually a really reasonable approach. Right, but for us, the data model of a shielded chain is just so different from a transparent chain and the whole state model is different. We have all these different design concerns. So for us, we use Tendermint as this kind off-the-shelf consensus piece, but everything on top of that is our own custom application.

Anna Rose (00:53:30):
Got it

Henry de Valence (00:53:30):
That's written in Rust, it is built around this kind of baseline of the shielded pool as our kind of fundamental store of value which is quite different from say having accounts and we have our own IBC implementation that uses that's written in Rust that uses some of the IBC libraries from Informal, but our IBC state machine is integrated with our app.

Anna Rose (00:54:01):
So you had to redo parts of that then, like that's a lot of work, I guess.

Guillermo Angeris (00:54:07):
It's the journey I suspect to say the least.

Henry de Valence (00:54:13):
Well, yes.

Henry de Valence (00:54:15):
So the good news, the good news is

Anna Rose (00:54:17):
Henry's very good at long pauses. I gotta say he's got good dramatic pause game.

Henry de Valence (00:54:24):
So the the good news is that we were actually the first non-Cosmos SDK chain to open a connection, an IBC connection to the Cosmos hub this summer.

Anna Rose (00:54:40):
Wow.

Henry de Valence (00:54:40):
So we got one of our test nets. I mean, it required some amount of like manual poking, but it did work and so that was a pretty cool milestone. So the way that IBC works is that you have two chains and they're trying to communicate with each other and the way they do that is each chain is gonna run a light client of the other chain, so it can understand and verify its counterparties consensus state. And then also has to have a way for what the state commitment that's in the block header, a way to understand what that's actually committing to and be able to read messages that were sent by the other chain. So that kind of factors into three separate pieces, the first piece is the consensus specific light client for us, because we're using Tendermint, we didn't have to do anything on that. We just, because we're another,

Anna Rose (00:55:34):
So that one you could reuse.

Henry de Valence (00:55:35):
So, and that's good because that's the part of the counterparty chain that actually has to run code. The second part is there's a standardized way to express like a generic Merkle proof. So that regardless the idea anyway, is that regardless of what type of like Merkle tree, the counterparty chain is using, there should be some way that you can just like plug in the proof. And then the third part is that there's some kind of standardized system of like how you encode messages into your chain state so that your counterparty chain can read them. And I think actually the IBC spec is like for all of the like warts and you know, we know where the bodies are buried and whatever like is actually remarkably difficult to build a generic protocol when you only have one implementation. And although there were some like weird, you know, quirks, it was like the Bob Ross, like, you know, there's no mistakes, just happy accidents. We'll just add another layer of hashing in here to make the code happy. It was possible for us to build our own custom chain and submit proofs of what its state was to the Cosmos hub, which, you know, clearly is like not code we control. And like that code that's already running was able to, you know, understand what our state is. And I think that's a really cool accomplishment, that we were able to just kind of permissionlessly plug into that.

Anna Rose (00:57:11):
There is in IBC though, there is a message passing layer as well. Right. Like I guess the question here is like, would you be able to do something on one chain that interacts into Penumbra, like under the hood with the setup that you have?

Henry de Valence (00:57:26):
Yeah. So there's, once you have this kind of message passing layer, there's various kind of application level stuff that you can put on top of that one simple application, which is the one that we're aiming to support first is just token transfers. Another application that is very cool is interchain accounts and also interchain queries. So this is a way that you can, for instance, have on one chain, some kind of verifiable query result of data on another chain.

Guillermo Angeris (00:58:00):
Oh, okay.

Henry de Valence (00:58:01):
That's the interchain queries, interchain accounts is where you can have an account on one chain directly control an account on another chain by like passing messages back and forth. So those are like kind of more advanced features and we're not planning to implement them initially, but in the longer term, it's something that we're thinking about. And there's also some potential, like quite cool twists that we could do on those.

Henry de Valence (00:58:29):
So one idea that we've just kind of thought about is building an implementation of interchain accounts, where instead of a normal setup where you just have two chains, you know, each with their own account set. And here's my account on chain A and I'm controlling this account on chain B. On Penumbra, we don't have accounts, but what we could do potentially is have a way for the Penumbra chain itself to have a pooled account on each of its counterparty chains. So we would only have control over a single account and then track who controlled what assets within that account internally to Penumbra inside of the shielded pool.

Anna Rose (00:59:14):
Oh wow.

Henry de Valence (00:59:15):
So that you could

Anna Rose (00:59:16):
It would all come from one account on that other chain but you would actually be able to direct where it's actually going. Oh, interesting.

Henry de Valence (00:59:22):
The first way you could do like actions on other chains from Penumbra.

Guillermo Angeris (00:59:28):
Yeah.

Anna Rose (00:59:29):
Wow.

Henry de Valence (00:59:29):
Without having to sort of reveal exactly. You know, like you could have, for instance, like the Penumbra account on the Cosmos hub and there's some ATOMs in it and somebody can make a transaction on Penumbra. That's like, oh, I'm using my like bearer token that represents control of like 100 ATOMs worth of this account. And I'm staking them to like the occlusion validator and that's my action.

Anna Rose (00:59:58):
Why not? ZK Validator.

Henry de Valence (01:00:05):
It's important to have like decentralized

Guillermo Angeris (01:00:07):
Yes, of course.

Guillermo Angeris (01:00:10):
This is really some big brain stuff. I don't know. I love it.

Henry de Valence (01:00:12):
But yeah, I think there's a really cool kind of opportunity for Penumbra when we're implementing these kind of interchain standardized features where it's like, how can we implement this in a way that is both compatible with the standard, but also is like doing something that's kind of unique for the capabilities that we have from Penumbra.

Anna Rose (01:00:36):
Wow.

Guillermo Angeris (01:00:37):
Switching gears maybe a little bit from slightly, I guess. Well, we were talking about IBC recently, but from the philosophical part to kind of, how do you enable like private transactions with global state is already in itself, not an obvious thing. Right. And you talked a little bit about this in your, you know, presentation in ZK summit but could you describe exactly like how does Penumbra work? Right. And it feels like if it were so easy, I feel like everyone would've, you know, made a private DEX, but in fact as we've referenced on many previous episodes building a private DEX is really damn hard. So, and you know, I guess, how did you manage it? How did you, like, what is it?

Henry de Valence (01:01:24):
I wanna be clear, it's not done yet. We're still working through it, but,

Guillermo Angeris (01:01:30):
But even having a reasonable plan for it, I think is non-trivial.

Henry de Valence (01:01:33):
So I think probably a good way to kind of frame the problem is by thinking about the kind of basic sort of state and data models of blockchains and how we end up building private chains and this is totally independent by the way of any sort of specific details about the proving system, whatever, that's a whole awesome piece of engineering, but it's kind of orthogonal to this issue, which is like, what is the data? How do you represent state on the chain? So if you think about like a transparent chain like Ethereum, you've got this like big ball of global state, which is like, here is everything that is on Ethereum and a transaction has some kind of description of these are some state changes that we're gonna make to that state. And you have the nodes, they come to consensus on those transactions and then to execute them, each transaction basically takes a lock on the entire world and does whatever changes it's gonna make all the nodes execute those.

Henry de Valence (01:02:46):
And then they've sort of progressed from like state to state to state. And so the, the whole paradigm is around global mutable state in order to build a shielded chain. Okay, we're trying to use like zk-Snarks for something, right. How do we do that? First, you have to fragment all of the application state and kind of rewind to this more UTXO style model, like in Bitcoin, where instead of having one giant ball of global state, you have immutable composable state fragments. This is like the sort of cool way to think about UTXOs is that each transaction output is like one fragment of the chain state. And when I have a transaction that has some list of input TXOs and produces some new outputs, what I'm doing is I'm composing together those state fragments and then producing a new set of state fragments.

Henry de Valence (01:03:46):
And those are gonna get included in the chain. And although in Bitcoin, you know, that's just a simple kind of value transfer application. You could do a lot more complex types of state in principle. Okay. So why would we wanna have this kind of fragmented state? The reason is that this way we can move all the state off chain. So instead of having this like big, like Merkle tree of state fragments, we're gonna replace all of those state fragments with cryptographic commitments to the state and that way the on chain data is just this big tree of commitments that don't reveal anything about the data that they're committing to and now in my transaction, I can have, instead of, okay, I'm taking these three pieces of state combining them together and making, you know, these two new ones I can do a ZK proof of, I have honestly formed a state transition that consumes certain states and produces some new states.

Henry de Valence (01:04:57):
And now the details of that state transition are completely private. All the chain sees is like, okay, here are some new state fragments that were produced and also here are some serial numbers of some other fragments that were consumed, right and this is from a privacy standpoint. This is like, okay, great. We've like solved the problem because we've moved all the execution off chain. We're just supplying verifications that the execution was correct and so the chain doesn't like learn anything, but what's the kind of problem here. Why aren't we all using private chains then if this solves the problem and the big thing that you lose is the ability to access shared global states, right? It turns out that actually having like public shared state is like kind of the whole point of having the blockchain, going back to the kind of big sort of philosophical discussion at the beginning and the point about not wanting to have a technology where you have to coordinate off chain before you can use the coordination tool.

Henry de Valence (01:06:02):
The reason that everybody loves to have shared state on chain is that it means that they only have to coordinate with the chain, they don't have to go find some counterparty and that's why losing shared state is a problem, but why exactly did we lose the access to shared state? The issue is that when somebody is submitting a transaction on a transparent system, because the execution actually happens on chain, some pieces of that execution's inputs can have late binding. So if you think of the kind of overall state transition that's created by some transaction, there will be like various inputs to that computation and various outputs. And some of those inputs are gonna be specified by the user when the user is creating and signing the transaction, but others won't be. And so for example, with a swap, right, somebody is gonna sign over, you know, I'm sending this amount of tokens to the AMM, and I am hoping to get like this type of token in return.

Henry de Valence (01:07:15):
But another input to that computation is what is the current state of the AMM? What are the reserve quantities? And you need those inputs in order to be able to compute the final outputs. And it's because the execution is actually happening on chain that someone can create a transaction where they filled in like some of the pieces, and then they've left some of the other pieces as kind of like, okay, well, this will be filled in by like, whatever the chain state is when I actually get to be executed. Right, and you can also use this as a perspective to, on kind of the whole like MEV games and like transaction ordering and stuff. Like if you're signing a transaction where there's like, whatever the current thing is then like, obviously there's gonna be some kind of game about like, can I rearrange your transactions, that you get an unfavorable input, right.

Henry de Valence (01:08:11):
But that's really kind of the key property that lets you build useful applications is having some way to interact with the public shared state. And if we think of what's happening, when somebody makes a private transaction, because they've moved the execution off chain and they're only supplying a proof, they actually it's a like a ZK proof is a proof of knowledge. So they actually have to have like complete exact knowledge of the state transition that they're proving about. And so they have to sort of, they have to seal all of the inputs and all of the state that, that state transition touches at the point that they make the transaction. And that would appear to block people from ever interacting with the shared state in any meaningful way.

Guillermo Angeris (01:09:01):
Yeah. Fundamentally. Yeah, exactly. The big problem is when things are fully private, right, is when you have the shared state, right? Like knowledge of that shared state should not be required directly in order to like perform the action you wanna perform. And often it's not known at the time of creation. I mean, again, just a restatement of what you were saying, but that this is a huge problem with markets, right? Whenever you submit an order to a market, you often won't know the price of the order, I mean, unless you can do many things, but the point is, you won't know the time when lands or like things like that, which is not inherently possible to do with at least zero knowledge proofs. Although there might be some weird homomorphic encryption stuff you can do

Anna Rose (01:09:46):
So what did you do? What do you do?

Henry de Valence (01:09:49):
So the thought process is that really the sort of fundamental issue there is, how do you mediate concurrent access to shared state? The problem is, oh, all these people who are making their transactions, doing their execution on their own device, and then submitting that to the chain. The problem is while they, they're all kind of trying to concurrently access or modify some kind of shared state, and we don't have really a way to coordinate that. So, you know, we can't make it happen. And if you look at, you know, not blockchains, but just like systems programming in general, you can think of the approach to accessing shared state on a chain like Ethereum as like, okay, we have like one big global Mutex, and we've like put all of the data under this log. And like every transaction acquires, the log does stuff and then releases it. And if you just think about systems programming, design, if your goal is I wanna build this kind of high performance, like scalable concurrent system, the architecture that you use to do that is not like I put all of the state in like one giant lock. That's how you get the bad programs.

Guillermo Angeris (01:11:00):
The bad programs, generally speaking.

Henry de Valence (01:11:02):
Right, and so another approach would be, you could try to do some kind of more fine grained locking. You could either build that into the transaction mechanism, or you could do some kind of like, you know, very fancy Ethereum implementation. That's basically the like out of order X 86 core, but for Ethereum where we just kind of like secretly go in and figure out all the data dependencies and then reorder

Guillermo Angeris (01:11:24):
Everything, execute.

Henry de Valence (01:11:24):
And then, you know, give you this, like as-if execution through a heroic amount of engineering effort and like that's a thing that I'm sure is gonna happen just because of like the total weight in that ecosystem.

Guillermo Angeris (01:11:37):
I think it's kind of already happening. Actually, there was a paper about this somewhat recently for Microsoft research. Right. But anyways,

Henry de Valence (01:11:43):
But, but another, just from a systems design perspective, another approach to handling concurrency, which is actually like better is an actor model with message passing. So the idea is instead of like, oh, different parts of the program are all trying to like, you know, access the same data and like share state or whatever. Instead you say like, we're gonna divide the program up into these like little actors that all execute independently and they'll communicate with each other through message passing. And that way you don't need to have any kind of like locking or synchronization because everything is just like passing messages to everything else. What would that look like in the context of this specific problem of how do we share state on chain? It would look like instead of your transaction having some code that makes a call to some smart contract and gets the result, your transaction would send a message to a smart contract describing something that you wanted to do.

Henry de Valence (01:12:50):
And then eventually that would execute and you would get a response from the smart contract asynchronously. So you're moving from a synchronous execution model where every transaction gets to lock the world to an asynchronous model where a transaction gets to send messages to an application, and then some computation will eventually be performed. And in that case, you can have a really cool kind of like batched processing model where each smart contract is gonna execute once per block but on input, all of the messages that all transactions sent during that block, and then it can perform, you know, whatever kind of application, specific logic it wants to determine how to process those messages. So for some applications like swaps, right, you can actually, glomerate all of those messages into a single batch trade and execute it at once. And now you've got this like huge performance speed up because instead of having to execute all those trades individually, you're only executing one trade per block.

Anna Rose (01:13:56):
And so you can be considerably more sophisticated about how you actually do that execution, but more generally for other applications, you could also have some kind of app specific logic that like chooses, like for an auction protocol, you could sort the messages by bid or something like that. And the challenge now is like, okay, I've like sent a message to a contract. Like, what do I, you know, what is my transaction actually doing? And the thing that's kind of interesting is that what this model lets you do is it lets you retain the setup where each user has their own state for their own account balance that is theirs and they're interacting with this public state by sending messages to it. And so they can now have a kind of asynchronous programming model for their own side of that contract, where they do some part of the computation and now they have to wait, they send out some message and they have to wait for a response and you end up having a model that looks a lot like futures or some kind of async programming on the user state side, where for instance, if I'm doing a swap, I'm gonna make a transaction where I've submitted my swap inputs.

Henry de Valence (01:15:15):
I've like burned those input tokens. And now I need a way to like, you know, hang out while I wait for that swap to execute, but be able to later demonstrate to the chain, oh, I actually did participate. And the way to think about that is you need a way to take all of the kind of intermediate state of that execution and somehow freeze it so that you can pick it up later, which is exactly what happens when you do async programming with futures. Every time you have to wait for a message to come in, that's a different sort of intermediate point of this state machine. And the like really cool thing that we do in Penumbra is we have a way to Merkle-ise all of that intermediate state in a zk-Snark friendly way into the asset ID of an NFT that we record in the shielded pool.

Henry de Valence (01:16:15):
So like, if you, if you've done like Rust programming, for instance, you may know of this like idea of like, oh, you have like ownership and you can only, you can have these like move semantics for your state machines, you know, that you can't like accidentally do the same state transition twice. Well, actually, if you have like, you know, a database that's recording quantities of financial assets and you turn every intermediate state of your future into like a distinct, you know, financial asset, then like you can reuse that infrastructure to track the execution of the program and so the way that the swap works is when I make a swap, I'm gonna burn my input funds that I'm committing to the swap. I mint to myself a swap NFT whose asset ID, so that's, you know, one unit of an asset ID, which is a zk-Snark friendly hash of the trading pair.

Henry de Valence (01:17:15):
My input amounts, the address that I'm gonna admit the notes to in the end and that way I'm verifying that all those things are consistent in the proof that I make when I submit my swap and now, as soon as I see that that transaction has been included in a block, I'm also able to see what the results of the batch output are because the, the actual trade is gonna be executed at the end of that block. And now I can, in a second transaction, I can spend my swap NFT that recorded the details about my participation in the swap and now I'll make a proof for the second half of this state transition, where I now have the results of the trade execution as a public input. And I can prove that I'm privately minting the correct prorata share of that batch output and that I'm sending it to the correct address that I signed over in the first part of the transaction. And the thing that's really cool about that actually is that it means that you don't actually even need to have a separate signing step for this secondary, like claiming part of the execution, because there's no way for the funds to be, you know, diverted to some other purpose.

Guillermo Angeris (01:18:41):
Yeah.

Henry de Valence (01:18:42):
It's literally just your user agent kind of submitting a transaction that like finishes off the change to your private part of the state.

Guillermo Angeris (01:18:50):
Cool. It's very, the notion is quite interesting, right? It's like separating out intermediate steps of a state machine, right. And then enumerating the intermediate steps in a very explicit way, such that once a computation is completed, which, you know, in order for you to do a swap, you depend on data that you did not know at the time. Right, namely how much was actually traded in the output to then, you know, you can then use this like intermediate step to then complete the computation as if it would've happened, kind of like purely on chain

Henry de Valence (01:19:23):
The other thing that I think is really cool about it too, is that it actually, although this is like designed around privacy, I think this actually has really cool implications for scalability too because if you think about the sort of sum total of the execution that is gonna happen, if there's, you know, a whole bunch of trades, because we are explicitly modeling, here's the part of the computation that is only touching the per user state and here's the part of the computation that's touching the shared state as like distinct concepts, the execution of all the per user state changes is actually happening off chain on the user's device and the execution of the shared state changes is happening in one, go in a batch where you can get like hyper efficient batch processing in the case of the swaps is sort of the optimal thing.

Henry de Valence (01:20:22):
Because you can just batch all the swaps together and execute only a single trade. But in general, like batch processing lets you share work between items in the batch and the extent to which that's true, depends a little bit on your task, but it's like very rare that you have like no sharing possible, right? So you get to have like hyper efficient public computation without having to have contention on the state access and the private side, which is the per user data is like free from a scalability perspective because it's all happening on the user's device. And the other thing that's different from like a ZK roll up is that each user's proofs that they're forming are only about their own state transitions that's right. So that's a small amount of proving to do. It's not like you have to appoint some sequencer to do this like giant proof of everybody's state transitions all at once. You're distributing the computational work out to the edge, the actual end users and then you're enabling the part of it that is necessarily shared to execute much more efficiently.

Guillermo Angeris (01:21:31):
Right.

Anna Rose (01:21:32):
So you covered a lot of this in the talk at ZK summit 8 as well. So I'll add a link to that in the show notes. If anyone wants to see that

Henry de Valence (01:21:39):
It's a good one.

Anna Rose (01:21:42):
I want to say a big thank you for coming on and sharing with us all of this kind of your journey and the kind of ideas that led to Penumbra. A lot of the thinking that you have around constructing it and some of these pretty cool innovations findings, like had anyone else done that have anyone else tried to split it?

Henry de Valence (01:22:01):
I don't think, I think this is new, but the thing is that this, although I've described it in this kind of fairly general terms of like here's this kind of idea of this whole system architecture, the way that we got there actually was not by starting with, we're gonna build this like generalized programability thing and in terms of what we're actually focused on building, we're still focused on, we are going to do private swaps and we're gonna make them good. But what's interesting is that by focusing on that one specific problem, the solution that we came up with as like, oh, here's a way that we can solve it in this case, actually I think generalizes to a much more interesting state model than we would've got. If we'd started from this kind of abstract, like, oh, let's build this kind of like generalized thing but without having like a really specific concrete use case.

Guillermo Angeris (01:22:55):
Very cool. Thank you so much for coming on with us and explaining, I guess what do you call it? The state model? So I guess the Penumbra State Model, let's put in the little trademark TM or somewhere in there. Thanks again.

Henry de Valence (01:23:11):
Yeah, it's been great.

Anna Rose (01:23:11):
Great. So thanks again. I also wanna say Vic, thank you to the ZK podcast team, Tanya, Henrik and Rachel and to our listeners. Thanks for listening.