GitHub Copilot with Rizel === Rizel: ~Test, test, test.~ Noel: ~Test, test check one, two. Hello and welcome to Pod Rocket, the podcast bot brought to you by Log Rocket. Let me just take that again, just cuz we're right at the beginning.~ [00:00:00] Hello and welcome to Pod Rocket, the podcast brought to you by Log Rocket. Log Rocket helps software teams improve user experience with session replay, error tracking, and product analytics. You can try it for free@logrocket.com. My name is Noel, and with me today is Raelle Scarlet. ~Uh,~ developer advocate at GitHub. Welcome to the show, Razel. Rizel: Hey, thanks for having me. Noel: ~Yeah, of course, of course.~ I'm excited to chat today. ~Um,~ can you tell us a little bit about, ~uh,~ yourself and your role, kinda what you've been, what you've been focusing on lately? Rizel: ~Yeah, so, uh,~ my name's Roelle, like you just said, and I am a developer advocate at GitHub.~ Um,~ I started off my ca career in like help desk and then went to software engineering and then,~ um,~ I transitioned into developer advocacy, which I feel like is~ kind of ~a combination of. ~Um,~ both roles where I get to help people, but I also still get a chance to code. ~Um,~ a lot of my role is focused around ~like ~introducing people to open source,~ whether, um,~ as a contributor or helping to improve their lives as maintainers and then, Also most importantly is ~like~ just talking to people about the new GitHub products and [00:01:00] tools and figuring out how can we make it better for them, ~um,~ but also empowering them to understand how to use those products, ~um,~ better. ~And one of the things that, or~ one of the products that I've ~like, kind of been most attracted to and~ done the most advocacy for is get help co-pilot. Noel: ~Nice. Yeah. Yeah, that's, that's kinda what I, what I wanted to focus on as well.~ So can you give us an overview? I'm sure most listeners are probably~ familiar, uh,~ familiar with copilot. They've done some reading on it at least. ~Um,~ but maybe for those who haven't or those who need a refresher, ~like what,~ what is copilot? How does it kind of slot into the development workflow? Rizel: ~Yeah, and,~ and surprisingly, I've met a lot of people who don't know what it is. I'm like, oh, I need to do my job better. But GitHub co-pilot is, ~um,~ at GitHub we're saying it's an AI payer programming assistant, ~um,~ that helps you to code faster with less work. And ~like~ when I describe it to people, I'll describe it ~like~ as, ~um,~ the smart composed feature in Gmail where like you start typing and then it has ~like a,~ a bit of predictive text for you. ~Um,~ so it's like that experience, but ~like~ while you're coding. Noel: ~Gotcha. Gotcha.~ So how does, ~I mean,~ how does it work? ~Like~ what is, what has co-pilot been trained on? How does it do Its kind of predictive stuff. Rizel: ~Yeah. So, um,~ it's using, [00:02:00] so when it first came out, it was initially using an l l M called Codex. And, ~um,~ for people who don't know what an LLM is, it's a, ~um,~ large language model. And like I, I heard, ~um,~ a developer advocate at Microsoft. ~Kind of~ describe large language models in a nice way,~ but, um,~ that I didn't think of. They're just statistical probability machines, so they're like supercharged, auto complete, ~um,~ like tools, but on a larger scale. But anyway, so, um, GitHub co-pilot was using Codex in the background, and Codex was essentially trained on ~like~ natural language and~ like~ public, ~um,~ repositories so that it can better~ like,~ have~ like~ a large data set of ~like, Um,~ code, but now we've most recently moved to Sahara, ~um,~ which is also developed by, ~um,~ open ai. Noel: Gotcha.~ What's, what are the,~ what are the differences? Rizel: Yeah. ~So, um,~ both are descendants of ~like,~ G P T three, which is, ~um,~ another language, another l l M, right. ~Um,~ so basically [00:03:00] G P T three was ~like~ one of the ancestors. Then they came out with CodeDX, and CodeDX was ~like~ specifically fine tuned for programming because G P T three is used by a lot of stuff like, but more for ~like~ natural language, like dual lingo uses G P T three. But we've switched over to, ~um,~ Sahara, which is actually using G P T 3.5 turbo. And what we're doing right now is we're ~kind of~ trying to experiment with which models will be best for us. And when we're thinking of these types of things we're thinking about ~like~ cost. Because ~like ~it is really expensive for us to have these GPUs. ~Um, each,~ each suggestion and each token is very expensive of for us. So we have to think about that. We're thinking about latency and we're thinking about the response quality. ~So right now, like in this mo like~ things I think are subject to change, but right now, in this moment, we we're, ~um,~ leveraging Sahara, ~um,~ as ~our,~ our next tool or our next l. L. Yeah. Noel: Can you get into details about like why the speed there with ~like~ 3.5 and the models descendant from it are so [00:04:00] much faster and cheaper to do recommendations with? Rizel: That's a good question. And actually I'm not too sure, so I haven't seen any, ~um, any,~ what is it like I. Postings or articles from open AI specifically about Sahara. I just know from GitHub's, ~um,~ perspective and what we've announced is that Sahara, ~um,~ and how it's working with G P T 3.5 turbo. Has had a better response. Quality has been, ~um,~ like reduced latency and has been, ~um,~ a more cost effective way of doing things for us. But that is a good question and I've been waiting for them to post more details so I can read up on it. Noel: ~Yeah, yeah. No worries. No worries. I guess,~ yeah, from there ~I'm,~ I'm curious ~for,~ for devs that are ~kind of ~looking to explore this and can ~get their,~ get their feet wet, how do you recommend they go in ~and,~ and try getting started ~with,~ with co-pilot, Rizel: ~Yeah. So, um,~ first is you can sign up for a subscription or a free trial, right? But after you do that, you do have to install the GI Hook, co co-pilot extension. [00:05:00] Once you do that, ~um,~ If you're starting off with a blank file, I strongly suggest. Writing a comment, right? Because GitHub co-pilot is working based on the con, like the context that it has in the file, right? So it's looking at your file name, it's looking at your cursor, and then it's just looking at what's in the file overall and any neighboring open files within your i d E. So if it has completely nothing, right? If it's just like a new code base and there's nothing there. Write a comment ~kind of~ stating the purpose of your application if you wanna create a markdown editor. This is like something I did. I was able to just ~like~ create a markdown editor in ~like~ one go, because I gave it like a really large comment that said ~like, um,~ I wanna create a markdown editor next Js I want it to have these couple of features. Like I want it to be able to support bold, italics, ~um,~ and, ~you know,~ whatever, ~so,~ so on and so forth. I wanna be able to use these specific MPM packages or darn packages. And then once you have that like overarching comment at the top, [00:06:00] it should be able to trigger some,~ um,~ suggestions or what we call the suggestions. We call it ghost text, and you should be able to see. ~Um,~ what it'll generate from there. And you can either choose to accept the suggestion or look at~ a, a,~ a, an additional suggestion or just reject it and keep typing. Noel: Gotcha. Yeah, so I guess, yeah, ~you mentioned,~ you mentioned installing the extension is the first step. ~Are we,~ are we talking vs code specifically? Is this available? And again, any other editors Rizel: ~Yeah, good question.~ It's available in VS. Code. Yes, but it's also available in ~Visual~ Visual Studio and ~um,~ it's available in Neo Vim and Jet Breams Ides. Noel: ~Gotcha. Does it work, you know, approximately the same in all those, or are there some subtle differences between how it's functioning with different editors?~ Rizel: ~I am. Not a pers like, so I can't personally say myself, I've, I'm just hearing from other people. I'm not a person that uses the other editors besides Visual Studio Code and like them occasionally, but from what I've heard from other people, that um, the experience seems to be the same.~ Noel: ~Gotcha. Cool, cool. Um, yeah, so you kinda, you walked us through an example of what you've done. Have you, have you seen any other kind of, um, Sorry, let me re-ask this question once my dog is finished barking.~ Rizel: ~No worries.~ Noel: ~Yeah. So you, so you, you gave, you walked us through an example there of how, um, you'd used a co-pilot to kind of spin up a markdown editor.~ ~Um, can you, like,~ do you have any other ~good~ examples from the industry or like out in the wild of people using copilot to help them ~like, you know,~ write other open source code or just like large production applications? Rizel: ~Yeah. Um, I've, okay.~ I've seen people use it and maybe this is not really a production application, but I've seen people use it for a documentation, right? That's something that ~like~ people. [00:07:00] Software engineers tend to not like writing documentation. It can be a tedious task sometimes and ~like, um,~ I don't think it's co-pilots really that great with ~like~ non code, like just written pros, but if you need to add in code snippets and stuff like that really quickly, I've seen people say that's been extremely helpful for them. I've seen someone say that ~they use,~ they wrote half the static ~web CLI documentation with. I think I stuttered static~ web app cli documentation with ~um,~ co-pilot. ~Um, let me see other examples.~ I think. ~For,~ for me,~ I've just been really using it for, like,~ I've used it to build a Chrome extension. I use it a lot for like demos. ~Um,~ and I think it's been helpful for me because as a developer advocate, I gotta do demos like on the spot sometimes. ~Um,~ or ~like,~ I'm like, oh man, I don't exactly remember how to do this. ~Like,~ just writing a comment and having it generate that for me is really great. ~Um,~ And I guess ~this is kind of a,~ this is a small version of a production app, but I did create, ~um,~ like~ a,~ a web application that basically ~auto, um,~ auto invites maintainers into our private maintainer community. ~Um, And awesome.~ Yeah, that, that was really great cuz I, I haven't, I don't code every day. So that's been [00:08:00] helpful. And I also used it to create a vs code extension that like, ~um,~ deploys ~your, your,~ your web applications to GitHub pages. So off the top of my head, for some reason, I'm not remembering all the stuff that I've seen in industry, but, ~um, I,~ I have a ton of examples for myself. Yeah. Noel: ~Yeah. Nice,~ nice. I've used it, ~um,~ in generating, ~uh, like~ type files. I found myself doing a lot of projects where I'm like parsing a lot of j s o in and out of these like APIs that I don't necessarily have typings for. So ~like~ on the flag can just be like, Hey, I can just ~like,~ set that to a variable, ~like~ define a type for this and then~ it'll,~ it'll spit out a type definition for me and it saves me a ton of time instead of ~like~ having to go ~like,~ look through the object very carefully and ~like~ spec it all out. And ~it, it's,~ it's always. ~Um,~ worked pretty well for me there. ~Um,~ Rizel: ~yeah.~ I love that. ~Yeah.~ I think it's also good with like red X. ~Like, I feel like~ I think Redx is really powerful, but like I am,~ I'm not,~ I'm not gonna remember it all. Like all the, I'm sorry. All the digits and characters. ~So, um,~ that saved me a lot of time. Cause I swear I take ~like~ 30 minutes to an hour with ~like~ one line of Fred xx sometimes, but now it's ~like~ 10 seconds. Noel: ~Yeah,~ I feel like Redx [00:09:00] is one of those things, very few dev developers have actually ~like~ internalized how Red XX works. And every time they've gotta ~like,~ okay, hold on, I've gotta look at this again and figure out what this is doing. ~Um, yeah, one of my, one of my coworkers always likes to joke, and it's a joke on the internet that like,~ you fix a problem with Red X and you have two problems, the original one and ~the~ the Redx problem. Rizel: Yeah. Noel: ~but yeah.~ Yeah, I think it is, it can be handy, ~like, ~help me like. Right, right. X validates the string, ~does,~ does whatever. ~Like,~ I think those kinds of things, those kinds of ~like,~ things that we know as devs we have to do and like we reach for those tools occasionally, but we're not using them super often and like the recollection might not always be there. ~Um,~ I think that area's pretty, ~you know,~ ripe for being like these kinds of tools helping. Rizel: agreed. Noel: ~Um, yeah. Cool.~ So I guess is there, ~um,~ any other ~like,~ I don't know, pitfalls or anything to watch out for~ when,~ when getting started, when in this domain that can like ~kind of ~trip up new people or just ~like~ cause bugs in code that maybe weren't foreseen or ~like,~ have you in your experience found that there are certain patterns to look out for? Anything that may indicate that you shouldn't be using co-pilot for something? Rizel: I dunno if [00:10:00] I've picked up on specific patterns, but I will say that. Just pay attention to ~like,~ don't blindly accept the code that ~it,~ it generates. Because here's the thing, ~like~ it doesn't know what it's writing. It's just, it's ~like,~ I think some people might think, oh my God, it understands. The code base, but it doesn't, it's just following patterns. It's just following ~the,~ the trading data that it's been based on. ~And,~ and it is limited to that training data, right? So it's only as good as the data that you, ~um,~ that you've trained it on and only as good as ~the, the,~ the code that it's writing. So ~that's,~ that's where, I don't know if there's ~like~ a specific ~like~ pitfall. ~I think it's more like,~ At first, I was not really great at using GitHub colo, but I started picking up little like tips and tricks that I'm like, oh, okay, I need to be ~more. Um, like~ more specific. I need to~ like,~ give it as much context as possible so that it can understand ~like~ how to, ~um,~ write code ~or, or, or,~ or assist me. I guess a big pitfall is it can't generate an entire project. ~Like~ a lot of times people ask me ~like,~ oh [00:11:00] my gosh, can it generate a whole project? ~Like,~ no, ~it'll, it, it,~ it's best with giving short, small outputs. ~Um, for,~ for developers. Noel: ~Yeah. Yeah. I guess that I'm,~ I'm curious there, like,~ uh,~ if you have any insight into how, as ~you know,~ as co-pilot exists in the wild and people use it more and more, I have to imagine that there's some ~like~ telemetry being collected to know when a suggestion is accepted or not. I. ~To help,~ to help ~like~ co-pilot become more accurate over time? ~Do you,~ do you know ~like~ a, is that happening? B, to what extent and ~like~ where are the boundaries there? ~So, you know, um,~ Like devs can rest easy knowing that their ~like~ code isn't always being uploaded because I feel ~that~ that, ~like,~ that context would be helpful to have ~like,~ okay, I wanna know everything that was just written and what was spit out. ~Like~ I know why that might be valuable, but I'm sure that there are people that are also concerned about that. How does, ~like~ what's the current kind of line there? ~Um,~ how do you guys walk that? Like make sure the model can improve and we're getting good data back, but also not just ~like~ gobbling up stuff we shouldn't be. Rizel: Yeah, no, good question. ~And,~ and your assumption is right, if people go to ~like~ the, there's ~like~ a FAQ on ~the,~ the GitHub co page, but we do take [00:12:00] telemetry of like your user engagement, right? We wanna figure out ~like what you, um, sorry, what,~ what you accepted, what you rejected. ~Um,~ we wanna figure out ~like, What's,~ what's ~like~ the percentage of how great this thing is so we can continue to improve on it. The thing is, ~um,~ we're not taking ~like ~your actual code, right? ~Like~ we're not taking your code. We're not storing that forever. That's only. For ~that,~ that's only ephemeral. ~Right?~ We'll take it for that one moment that we need to give you back a suggestion, but then we'll throw it away. ~Um,~ and you also have the option in your settings to turn off, the ability to, ~um,~ to even take that data from you. And for ~like~ businesses who are using copilot for business, that's turned off by default. So yes, that is a thing. ~Like ~we are taking that, but. You have the ability to control. If we take that and we're not ~like~ taking any ~like,~ personal information about you, I think, I believe we have ~like, um,~ like pseudonym identifiers so that it's not ~like~ Razel Scarlet did this at this moment. Yeah. Noel: Gotcha. Yeah. So ~like~ what, ~what does that,~ what does that flag [00:13:00] control? ~The one, like the,~ the telemetry flag is that, if any, ~like~ anything's getting uploaded or like where, ~what,~ what does that actually toggle? Rizel: I believe that's toggling. ~Do you want, um,~ do you want us to know what suggestions you accepted and rejected? ~Like~ you can just completely Yeah. Yeah. ~So,~ Noel: ~So, so when a, ~when a, when a suggestion is a ~upset,~ accepted, or rejected, even if that flag. Is on enabled, right? ~Like data would, like~ telemetry data is being sent. Do ~you guys still,~ you guys still don't get the context where that suggestion was accepted. Is all you know that there was like ~this,~ this is the suggestion and it was accepted? Or is there some contextual data that helps, ~you know, like,~ might help you guys know why that was accepted? Rizel: I don't know the answer to that, but That's a good Noel: Oh, no worries. Rizel: Sorry. Noel: ~Yeah. Yeah.~ Cause I, no, that's fine. I'm just like, I would think,~ like,~ it'd be hard to know, ~like,~ like it, it'd be good to know when something is accepted and rejected, but ~I,~ I would feel without some context as to ~like,~ what is leading to that suggestion. It may be hard to know Rizel: ~Yeah. Often.~ Noel: ~like, yeah.~ Rizel: ~Off the record. I guess like right now, like at first GitHub's, like don't say anything you know about copilot. Now they're starting to like put out more information and I'm kind of learning at the same time as everybody. Like I'm just maybe seeing the articles or the drafts a little bit earlier.~ ~So we're starting to put out the information, but they haven't really said that part yet. So I don't,~ Noel: ~Yeah, yeah. No worries. Like I, I think yeah, we can, we~ Rizel: ~yeah, you could scratch that, but.~ Noel: ~Yeah, we can, we can jump right ahead. Um,~ you mentioned that there, like the flag of being one of the things that you [00:14:00] can turn off, ~uh,~ or ~being, being,~ being disabled by default and copilot for business ~or,~ or enabled by default, whichever way the flag works, whatever the telemetry is turned off in copilot for business a little bit more. And for individuals it's on. Are there other differences between the two? ~Like ~is there other, any, anything else? Rizel: ~Yeah. Um,~ there's another difference of ~like, Um,~ getting a suggestion of public code. ~Um, ~so basically ~the co~ co-pilot's not actively grabbing code just ~for,~ for listeners. It's not actively like grabbing code from repositories, but it was trained on those. Public repositories and it is synthesizing that. So there can be small coincidences where the code that's suggested matches some public code on the internet. ~Like~ if you wrote like a binary research algorithm or something like that. ~Um,~ if 150 characters or more match it, like you can have, there's this setting where you can say ~like ~150 characters more maps than public code. I don't want that suggestion for a co-pilot for business that's automatically turned off for a co-pilot for individuals. ~Um,~ it is default turned on, but ~you can,~ you can, ~um,~ [00:15:00] control that setting as well. And then also for co-pilot for business, I think the benefit there is ~like,~ you have people that join the team ~like. Um, at,~ at your company, right? You don't have to ~like~ sign each person up individually for a copilot, ~um,~ account. You just have them on the team, you add them, and then they get copilot on, on, on their, ~um,~ editor, and then if they leave, you can easily just pull them out. So it just creates ~a,~ a smoother process for whoever the engineering manager ~or~ or administrator is for that team. Noel: Gotcha. ~Yeah. Yeah. That ~having that the kind of, ~um,~ Yeah, that, that level of licensing and stuff that, that makes a lot of sense to me. I'm, I am curious about the 150 character limit thing. ~Was the,~ was the main concern there, ~like~ stumbling into some kind of ~like~ copyright or licensing issue? ~Like~ is that why like people generally would enable that 150 character setting? Rizel: ~I, I think,~ I think ~it was,~ it was due to ~like, you know,~ people, some people feel strongly about, yeah, they don't wanna copy code, ~I guess.~ But then there's some other people that have ~like~ a different, ~um,~ mindset about it. And they're like, ~well,~ it's open source code, so that's meant for [00:16:00] us to use it. So there's some people or some companies that are like, Hey, we don't want that to happen. And then there's some other people with different ideologies that are like, ~It's,~ it's public code. It's almost the same as like copying code off of Stack Overflow. Noel: Yeah. Yeah, ~I think,~ I think that is, it's just, ~uh,~ it's interesting to me ~like kind of ~conceptually, because I don't know, I assume that almost everyone's using ~like,~ Some degree of open source cord. They're using some, they're using some package that they're pulling for something, right? ~Like~ even the most regimented, ~uh,~ company I would think they're using packages for stuff. So it's odd to me that they'd be like, we never want any suggestion, even if someone's written the exact question I'm asking for and solved it perfectly, if it's an exact match to this, I don't want it. ~Uh,~ so anyway, I'm just, I'm curious what is, what's motivating these people? Rizel: ~Yeah. I mean, everybody,~ everybody has their own, ~um,~ levels of like morality ~and, and,~ and things, principles that they have. For me, I've always looked at code as something that is meant to be shared, so Noel: Yeah, Rizel: I haven't turned on. Noel: ~Yeah, yeah, yeah. I, I, I would think one would, um, is there, when, when. I guess~ I'm curious in that, ~like, is there,~ is there an option there in that setting if there's an exact [00:17:00] match found to not disable it to ~like,~ but to get a reference ~to the,~ to the match. So you as the dev could go look at ~what, like~ where this exact match is coming from. Rizel: ~Such a good question.~ We announced that this feature is gonna come out, but we announced that I believe. At Universe that we're ~like~ working on a feature that'll say, ~like,~ that'll give you information and be like, Hey, this is being used in this repo. Are you okay with leveraging it? ~Um,~ I don't think we ~like~ launched it publicly, but ~I, I, as a,~ as a staff member, sometimes I get like access to features a little bit early and I swear I did see it pop up once and I thought it was so cool. But yeah. Noel: ~Nice. Yeah,~ I could see that being very ~handle, uh, like~ reassuring I guess to it. ~Uh, like,~ and go look at what, wherever it came from, the repo or the question or whatever, and just ~kind of~ get a little bit of context as to where the code's originating. ~Um,~ is there anything else kinda like on the roadmap ~like that,~ that's kinda on the rise and coming up that. You're aware of or excited about that you can share? Rizel: ~Yeah, I mean, I think, I think we, um,~ we announced co-pilot X to most people, ~but just so people ju I can,~ I can talk about that. ~Right?~ So what our goal ~is,~ is ~like~ you are not just getting code suggestions in your editor. We're making it easier ~to,~ [00:18:00] to develop and to code in places outside of ~like,~ just overall, right? So we have, ~um,~ co-pilot for cli. ~Um,~ so ~like~ you type. Some natural language. It generates like ~a,~ a command for you and then you get an explanation of the command before you just accept it. ~Like~ it'll tell you like, oh, this is doing ~this,~ this, and this. Is this what you wanted? And then you can choose revised query, or you can choose run command. I like that because, Like you said, ~um,~ there are things that we do as developers that we don't do every single day. So we don't have that ~like, like~ rote memory right in our head. ~Um,~ and then we have copilot for docs where, ~um,~ you, you're, it'll, if you are looking through document or instead of looking through documentation, right? It gives you ~kind of like~ a summary of what you're searching for. Like you type in a question, it'll. Pop up. ~Um, ~we have, ~um,~ co-pilot for pull request. So it helps you to ~like,~ create ~like, um, Um,~ PR or commit messages and helps to ~like,~ give little suggestions on how to improve your pr. ~Um,~ we have co-pilot chat, which ~I've been,~ I've been using that one the [00:19:00] most, ~um,~ and I've been loving it. ~We,~ we, ~um,~ gave it to people who attended Microsoft Build and I'm like, they really enjoy it. It's essentially a very similar experience to chat G P T. So ~you're,~ you're chatting with an AI and you're asking it questions, but the thing is for co-pilot chat, it has. Context of your editor, ~um, or,~ or that open file that you have in there. ~Um,~ so that makes it easier where you're not~ like~ with chat G p t, copying and pasting and putting it back ~and,~ and ~you know,~ that, that's like a lot of context switching. ~Um,~ and then, ~Um,~ it also has ~like~ slash commands where you can do ~like~ slash test or slash explain. So if it gives you a suggestion and you're not really understanding why, it'll give you an explanation or it'll generate tests for you. ~Um,~ it's been really fun, ~um,~ for me~ like, um, at Microsoft Build and, sorry, I'm like going on so long about it,~ Noel: ~you're good. It's good.~ Rizel: ~it. Yeah.~ Noel: ~Yeah.~ Rizel: And Microsoft build, ~um,~ they, me and my manager did a presentation about co-pilot and code spaces. But the person before us, their pre their presentation wasn't working. So they asked us to~ like~ do a secondary, ~um,~ presentation that we did not prepare for at all. But then we were like, oh, what if we ~like~ just used copilot [00:20:00] chat and ~like~ just interacted with ~the,~ the audience and ~like,~ what do we build? So we ~like~ built like a snake game. We built the markdown editor. We built a couple of things and we were just asking the audience ~like,~ what do you wanna add? ~And it just like,~ it was really fun to see what it generated and ~um,~ it generated different things for both of us. That's all. Noel: ~Oh, cool. Cool, cool, cool.~ Yeah, I'm sure ~that was,~ that was a fun experience. ~Is it,~ is it stressful to ~like~ give those kinds of presentations or you're not totally sure what the model's gonna spit back out at you live like that? Rizel: ~Um,~ I think earlier on, ~like~ when I first started out through showing copilot demos, I was nervous, but I'm less nervous now, ~like~ I'm used to. I'll tell PE I'll let people know, ~like~ it's non-deterministic. Like ~we're,~ we're on a journey. Let's see what happens. Noel: ~Yeah. Nice, nice. Very cool. Well, I guess, yeah,~ more broadly then, ~like~ what are you excited about ~kind of~ in, in the space and in WebDev? ~Uh,~ no, like for the rest of the year. ~What, what, what are you,~ what are you looking forward to? Rizel: ~Yeah, that's a good question. I think,~ I think I'm just looking forward to, ~um,~ how our industry is making coding more accessible for people. ~Like~ I think co-pilot does that. I think things like, ~um,~ [00:21:00] Code spaces, not just code spaces, but there's other ~like, um,~ in browse or IDE or cloud-based editors that exist out there that make it much easier for people who ~like~ don't have ~like~ the greatest computer to be able to get started with coding. I think it's just nice that we're trying to make coding accessible to people. ~Cuz I feel like, like~ in the past it was like, how do I get started? I don't always have all the resources to do this. ~Um,~ and yeah, I'm excited to. To see where we go with that. Noel: Awesome. Cool. Well, thank you so much for, ~uh,~ coming online and chatting with me today, Relle. It's been a pleasure. Rizel: ~Awesome.~ Thank you for having me. Noel: ~Of course,~ of course. Rizel: ~Cool. Uh, no, I think it was good. I think I, I gave the updated stuff. I hope you all weren't like, we already talked about all this, but I think.~ Noel: ~No, it's all good. I think. Good.~ Rizel: ~Okay.~