Kelly Schuster-Paredes: Foreign. Sean Tibor: Hello and welcome to Teaching Python. This is episode 143, and today we're talking about CS education with computational thinking 2.0. And my name is Shawn Tyber. I'm a coder who teaches. Kelly Schuster-Paredes: And my name is Kelly Schuster Peredz, and I'm a teacher who codes. Sean Tibor: And this week we're welcoming Dr. John Chapin, computer science teacher, presenter at CSTA. And we're intrigued and excited to talk to you today about this Computational Thinking 2.0 concept and probably quite a few other things along the way, because it's going to be. It's. I've always found, like, as we start to discuss these, like, so many other topics come in and other things weave in. So I'm excited to speak with you today, John, and it's really great to have you on the show. John Chapin: Great. I'm excited to be here. Sean Tibor: Well, why don't we start where we always do with the wins of the week. And John, we're going to have you go first because more fun that way. So when inside or outside of the classroom from past week or, you know, there's no rules here. You could make it a month if you wanted. John Chapin: Yeah. So. All right, I'll get two wins. One. One win was. So I'm I to teach at. At the Academy Saladin, which is a public magnet school. But now I'm back at a regular homeschool and in a class, and one of the students at the end of the class goes, wow, this class goes by so fast. And I was like, yes. Yeah. Not that you do it right every class. No. Sometimes you walk out of there thinking, hey, that's dunk. You got it. We got to get a little bit better. But it's. It was really. When a kid says that and he said it like, oh, my gosh, class is over already. That was good. On a higher note, I'm a facilitator also DA which is. Does a lot of PD and is the official provider in state of UF or CS teachers. And we're working. We actually are starting to work on some grants to do professional development for teachers. Looks like, you know, we'll have to see where we go. But we're applying for some big grants to do professional development, specifically around machine learning and teaching data science and getting teachers up to speed on habitat. That's super exciting. It's not a win yet. The fact that it gets. Sean Tibor: It's a win in progress. John Chapin: Right? There you go. Kelly Schuster-Paredes: Absolutely. And if it has data science to do it, it's always A win in my book. So mine actually correlates with yours, which is great. So I'm going to steal before even Sean gets to say, I want to share mine. Because I was thinking about this. I did a lesson today. We assisted with the eighth graders, and we ran out of time. I have two weeks, Two weeks in a class left, which accounts for about five days. And we always want to do this one unit at the end. So we had a rush and we had to skip out on my data science Big Data matplotlib unit. And this is one of my favorite exercises. I could spend the whole entire nine weeks doing it. And so two and a half hours later today, I put together a pod on a podcast, PowerPoint, but it's not a PowerPoint. It was with one of the AI, where it was interactive, so sort of like pear deck. And they could draw and they could pull, and then they would get AI feedback. And I managed to do three slides somehow in the 70 minutes that I had. And everyone was still engaged. It was really cool. And we talked about these graphs and we looked at what. What made a good graph and. And what was big data and where the algorithms came from. And so I just smushed all. All of my. My five lessons without coding. Unfortunately, we didn't get to the coding part, which is the actual fun part. But, you know, if they. If we get them interested a little bit, maybe they'll take on data science later. And so it was a good win. And my teaching partner sat and did the whole. Cause she's like, why do you change things on me last minute? I'm like, I don't know. She's like, I gotta do this tomorrow. I'm like, yep. So it was a win. It was a good win. Sean Tibor: It's all jazz, improv, right? Like, it was you. When you. When it's working, just go with it and keep improvising and. And have it. Kelly Schuster-Paredes: Yeah, but it's like you two and a half hours of work and it's all gone in 70 minutes. You're just like, hey, hopefully we use this again next year or next quarter, mind you. Sean Tibor: Well, my wins this week have all been the culmination of a lot of hard work over many months and. And even the. The whole year, I just had a moment where things were starting early. Late last year, we started recruiting at Carnegie Mellon for our summer intern program, which is my alma mater. It was something I really wanted to do, and I had this belief that we would find really great candidates at CM for a lot of our Roles. We had more interns this summer, this past summer than we had available job positions. So it was fairly competitive. But we tried to make sure that everyone had a really great experience and had the opportunity to succeed. As it turned out, you know, the previous win was that the two Carnegie Mellon students that came in as interns over the summer, each of them were offered positions at the end of the summer, which was pretty exciting. And I just found out last week that both of them have accepted. So I have two new colleagues from CMU coming to, to work with me. Not directly with me, but they'll be coming into different parts of our organization, working on some pretty cool stuff. And it was just really exciting. Kind of the culmination of about a year and a half or two years worth of advocacy for my, my alma mater. But what, what really helped, though, was that it wasn't just about the school. It was about the fact that this, this whole program over the past summer has really developed into something that I think is a best in class internship experience. The thing that all of the interns kept saying was I feel like I'm doing something real that has meaning, that's like, important. And to me, that's something that I want to put my name against. Right. It's something that helps them grow and develop, that they actually have, you know, a stake in it. Right. Like they've got skin in the game and it's something that matters. That's way more interesting to me than an internship program. They, they have busy work or they're there to like, act as an, a glorified assistant. This is real stuff with real projects, and they got a taste of what it's like to be in the working world and in my company. And to me, that was pretty exciting that it's all kind of come fully around and, you know, I might be a little bit more proud of my, my CMU students, but there's some really amazing interns that are coming back full time from all over the country. It's just exciting for me. I don't get a chance to do a lot of classroom teaching, so the work that I do with our intern program is satisfying of teaching and coaching and guidance sort of itch that I have. And I'm pretty excited that it's all coming together. So now that of course, we've done that, it's all into next summer and planning and recruiting, and the cycle begins anew. Kelly Schuster-Paredes: Sounds like teaching regular time. Sean Tibor: Exactly. I'm very familiar with how this worked. Kelly Schuster-Paredes: Excellent. Well, I think Dr. Chapin is you're also a. Call me John. John. Thank you. Thank you. Feel like we're old buddies now. John, I think you were. I think you. Are you a teacher who codes or coder who teaches? John Chapin: Yeah, I coded back in the 90s, and then I started one of those Internet companies around 2000 and went crazy public. And then I did some other stuff in around 2015, 2016, or about 2013. My wife was like, so why don't you try this volunteer? So I was a Teals volunteer, and I love. Absolutely loved it. We had a company, didn't really enjoy the business we were in, so we kind of unwound that company and I started teaching nine months later. Crazy. And I've been teaching for the last 10 years. So I don't know how. I don't know what that. Now I think I'm more. I'm a teacher. Sean Tibor: Right. John Chapin: I think I've been doing it for 10 years. And I, you know, I, you know, yeah. Do a lot of stuff. I really think of my, like, people go, what do you do? I'm a teacher. That's the way I think of my. Can I teach code? Can I. Sean Tibor: Yes. John Chapin: Did I used to code a lot? Sean Tibor: Been so long ago. Kelly Schuster-Paredes: We have to have a new saying. The coder and a teacher. John Chapin: Right. And when you do, you'll get a chance to code. And, you know, I wish I did. You, you know, you're spending so much time how to. Don't answer you really like. At least for me, I don't find I get my hand series. Kelly Schuster-Paredes: I'd like me too. That's what I'm finding. I have to force myself. And I actually have it on my calendar, and it never. The calendar goes by, and I'm like, oh, I gotta go to that grading. Yep. No, I'm gonna go. And so true. Sean Tibor: I always. I felt like I always found excuses to write code. Like, oh, yeah, we need that, and I'm gonna write code for it. It was never, like, no one was asking me to. It was just like, I was. I saw something that needed to be done. I'm like, I bet I can solve that with this bit of Python code by writing this problem. You know, solving this problem with code. It was never, like, no one ever said, oh, we need you to write code or be a developer at the school. It just happened to fall out. Kelly Schuster-Paredes: Yeah. I used to yell at him all the time. I was like, did you do your grading? We got to teach. And you got to teach. He's like, yeah, yeah. Sean Tibor: Zoned in everything Got done eventually. John Chapin: That's teaching. Kelly Schuster-Paredes: So what do you, what do you code with? What do you, what is your primary length now? John Chapin: All I, all I have to now is fight all I'm doing. And I used to be a big Java guy, say class Java. But then eight years ago I was a computer science pathway meter for the academies of Loudoun and I started it up so I was like the very first CS teacher and we had to think about what our four year and we get the kids and they stay in that pathway for four years and we could design it however we wanted. And when they became juniors, like, well let's teach them machine learning. That sounds like a good idea. This is like 27, 2018 and didn't know what we were doing. Watched a bunch of Andrew Ong videos, took his online course, had had some. You have Howard Chiefs Medical Institute. They were doing some cutting edge stuff with machine learning so they helped us out. I was in Loudoun county and so we ended up so. And that's all Python, right. I remember the first meeting I had with them. They're like, they're thrown in stuff like anaconda and pandas and matplotlib and I'm like, are you talking. They were just all these words, no idea what they were. You know, I'm coming from an old Java like world and just the landscape was totally different that we had to learn and how to teach. Right. We were looking, we were taking 300 level or we have taken 300 level Stanford level courses and we had to figure out how to scaffold them for our high school. But that's a kind of a longhand question of, you know, what am I doing now? I'm like now kind of really. And we, we can talk about this a lot more. But now I'm kind of all in. Kelly Schuster-Paredes: On the pipe that like, that like just puts joy to my heart because you know I always pick on our Java teachers and they always go into. And this is why I was really excited about having you on the show. They go into the fact that, you know, we can't teach object oriented programming if we don't have Java. And I'm like, I don't know, you know, I kind of, it's hidden there. John Chapin: If you want it, do it if you want it. Kelly Schuster-Paredes: So that's always fun. John Chapin: So yeah, so I'm, I'm the reason why in Python now. And, and this is one of the kidney was last year. Well first off when I started teaching Python it was, it was probably a year or two into it. I was like, especially when you're dealing with numpy else in the machine learning world. It's like where's the for loop? There's no for loops anywhere. There's like in machine learning there's no for loops. It was crazy. And then I mean just everything was different and we can talk about the computation we know. And there's this great paper. There's this great paper that was put out in 2021 and they literally have this chart and it's in it, it's in the ACM website. I'll be downloaded for free, but it compares like regular kind of company. A 1.0 and then they call it 2.0 and they pair a whole bunch of how you think about it, how you probably every like different process. And when I saw that I was just going, yeah. Oh my gosh. Sean Tibor: Yeah. John Chapin: Wow. Holy, holy wow. And really amazing. It just like really put down in writing what I think I had in the back of my head. But they just did a job elucidating all the different. How it different and then I combined the fact different with it looking out there job and, and all of a sudden I'm realizing anybody in a STEM field or even non STEM fields that are non stem that has lots of data, you're probably going to need to learn Python and you're probably going to need to learn data science. So if you. This is my preaching here. So number one, Python the whole data science process different. And then number two is if I have high school and I have, let's say I don't know, out of, out of 400 seniors, maybe have 100 or 100 that are in STEM field that need to learn Python and data science. And they may use finance, they may econ, they may use ology, they may where F. And you may have at 100, maybe you have 10 or students that are going to be CS. Right? The CS majors. Yeah. Okay. Learn job programming. The other hundred don't need to know Java and object oriented program. There is no, there's almost no do object oriented programming. Machine learning. Sure, yeah. But you really don't. Right? And you don't need it. So here we are. I just think we're, we're at an inflection point where I forgot a surdy of STEM students or students in general. We need to start teaching them Python. They may never or even the cyber kids when they do scripting. Right. You know, the vast majority don't Java and object oriented and the whole goal should not be getting them to dsa. So I'm not saying it goes away, just saying the whole bucket is bigger now and we need each of those kids and, and we need to. I mean we need to really start now. And it's just. I think it's huge. And I, you know, just an anecdotal story. My son's up. He's in. He's worried. He's 25. He's working at British Aerospace Engineering here in Northern Virginia. And he's in Pinet and he's in a rotational program for Find It. This rotation is in finance is data science and machine learning. You know, he's creating this dashboard live updates of data that's going to be seen by 40,000. Like that's what he's working on this. And that's not uncommon. You know. Another anecdote is at Tavern's line where he's. Sean Tibor: There's. John Chapin: There's three. Three schools. One is mata. There's an Academy of engineering Technology and there's Academy of Science. Well, Archie's medical institute is this science world leading science research. They've won 30 Nobel prizes. More than Russia, right? Just absolutely cutting it and they take on a high school intern. Guess how many interns came from aos? I don't code zero. All eight interns were CF because science has become cf. So you know, and it's more the machine learning Python down that row cf. So it's. It's this movement is there. It's out there in market and, and for people in other jobs that aren't and you know, I don't feel like how big of a ship everything's had to. And I would combine that at the last tools have gotten. Gotten much easier. So you know, I think those suggestions it's super powerful now and regular kids use these. I. Kelly Schuster-Paredes: You. You're preaching to an awesome choir and I love that. I was thinking about this when you were talking, you know, the people that come in with no kind of coding degrees. When I took this data science bootcamp, we had, you know, credit people. We had some engineers. I think I was the only person in the class who knew Python and they were all like, that's unfair advantage. Well, no, because I had no kind of analytics, math, kind of side analytical side. I was more, you know, I took calc a long, long time ago. But I'm the science teacher. I'm a bio teacher. And it was really interesting to watch them when they were preparing their machine learning models or their, their graphs and everything that we were going through. And it was Just that's when, you know, I kind of, kind of looked at the Python scripts and I was like, yeah, this is such powerful, beautiful language, because it is true. And it's a different thought process, a different type of debugging and understanding that, that storytelling, it's just a whole higher level. And my other anecdote, I told my nephew, for the longest time, he graduated with a finance degree. And I said, did you take any cs, any coding? No, I don't need it. I'm going into finance. Got a job sent for a year while he's working to take Python. And I laughed at him. And I said, andy, told you so, I told you so, I told you so. But it's true. It's, it's, it's here, it's in the workplace right now. And the kids that are leaving, you know, colleges or even, you know, high school, they could go into stuff like that. Leaving that area without a strong, not necessarily knowledge, but a strong desire to learn on the fly and learn on the go with Python is probably going to be detrimental to right now for their career, definitely, in my opinion. Sean Tibor: Yeah, I'm, I'm wondering though, thinking about where, where is the frick, right? Like, where's the resistance to this? I would, I would hesitate to say that it's the students. I don't think the students are hesitating on whether they should learn how to code or how to, or how to solve problems with Python or other languages. I feel like there's still a huge entrenched, you know, belief that we, when we teach computer science, we either have to teach really traditional computer science right where it is, you know, like very much that Computational Thinking 1.0. It's the theory, the, you know, kind of the, the Java based, like we have to teach object oriented programming and we have to teach recursion and we have to teach all of these things that are traditional computer science, right? And, and to your point, that gets the 10 students that are going to go on and study that as a discipline, right? Great, we've met their needs. But then the other students that we could be teaching them so much more. It always feels like, at least when I was having those conversations with the broader community, it always felt like, yeah, yeah, that's nice, but I got to go teach statistics, right? Or I got to go teach biology and there's no time and there's no space for computer science, right? Or I mean, just the whole idea that that blew my mind was when we're talking about data fluency and Data literacy and being able to look at data and interpret it. Like we're still thinking in an educational setting about small scales of data. Right. How do we, how do we plot and measure a hundred points of data? Right. Or a thousand points. Wow. A thousand points of data. Right. And like how do you get people to see that weren't nowhere is anyone talking about a thousand points of data. A thousand points of data is Excel. I can do that. That you know, I could hand plot that if I took enough time. What about a million or a billion pieces? Right. How am I going to solve that problem? Well, you're going to have to solve that with code, right? John Chapin: Yeah. Kelly Schuster-Paredes: Can we back up one second? Because I want to. Because I was actually looking into the difference between CT1 and Computational Thinking 2.0. Let's just like define it a little bit more structurally for those listening. And I'm going to let you do it John, since you're the expert. John Chapin: I actually have the paper in front of me with this little. And you know, I don't know if you guys can provide some of the things paper. The paper definitely will. We'll definitely provide the link to the paper. It's on page six. But for example in data process so I like they list a bunch of stuff and they go and I'm not gonna, I'm not gonna read through everything but like in CT 1.0 it says formalize the problem, design a solution, implement the solution in a stepwise program. Compile and execute the program. Testing. You know, you do cross checking of output program code debug. You can track and trace the program state codes of, of every little line of code. Literally go through and see all your variables. Portability, it's really hard to take the different plot, trial and error discourage. So that's 1.0. Kind of our normal programming. CT2.0 is it's collect data and clean the data. Train a model from the available data, evaluate the model. You know how to buck experiment. You just change the knobs and the levers, you know have for it the philosophy of problem solving. CT 1.0 is very deductive and for 1.0 is inductive. It's just, it's. And then the portability. It's really portable. I can take it go wherever you need to go. It's not like you have this Java compiled program and I don't know if it'll run here and I don't know if I'm there. It's just, it's night and day different crazy. I think the big. What some of the biggest things are is bugging, right? Track, you know, tracking and trace tracing of the program states versus just you start experimenting and you hope, hope your outputs are better and like, you know, you didn't overfit and blah, blah, blah. Kelly Schuster-Paredes: I think it's also like the big though, was the difference. I was reading the rule driven. Like we're talking, we're going to sit there, we need to know what an algorithm is. We need to do our conditionals. We need to like, step out our problem. Things that, I mean, I teach the sixth graders. I think it's always. I think it's good. We're not, I'm not saying. And I'm sure you're not saying that, oh, we're going to get rid of it, right? It's just a new way of thinking for some futuristic current, current futuristic jobs that our students will have. And it's very rule driven. Was CT1 right? Think about it. I'm going to, you know, I'm going to write my procedure. I need to think about the problem. I got to plan out the problem. Where with, with machine learning and, and data science, it's all about the data. Everything's about the data. And you know, we have this data. What is it that we want to say with it? And it could be this way, it could be that way. So we need to really just figure out what our, our intentions are with the data. And that train of thought is so different. When I tell the kids, you need to tell me the story about the data. And they have no clue what I'm talking about. I'm like, literally, tell me a story. Well, the line goes up or the line goes down. No, that's not a story. That's what you see. That's. That's a description. Tell me what's going on. Why would the, why would it go down in 2021 and you know, what happened there? And they're like, oh, well, isn't that when Covid. Oh, you know, and that's a hard thing. That's a, that's a difficult, difficult skill because there's not really a right answer. It's just how you interpret all the parts and it's. John Chapin: And they get caught up in the abstract, right? They're down in the weeds of the abstraction and pulling them back up to know the abstraction's representing in a specific instance. And what does that represent? And what does it mean when you're. Kelly Schuster-Paredes: When you're talking Dog still barking. Sean, he was like, it's okay. Sean Tibor: The other, the other thing that I, I think is really different about it as well. You know, I think when we're. One of the hardest things for me to learn when I was learning computer science, going through the classes was the, you know, data structures, right? And processing algorithms and things like that, right. But what I'm finding as we shift into this world is that a lot of that is still important, right? Like understanding how data is structured and organized and everything. But so much of the work now is about, you know, where is my data coming from, how am I transforming it, right? How am I cleaning it, how am I validating my data and ensuring that it's good quality, right? All that pre work before I can put it into my model and do the training to make sure that I have decent data, right? Like, not just that I've got data, but that whatever I've applied to it is cleaning it in a rigorous way, right? So it's not just, you know, like making it look good, it's, I'm cleaning it in a way that, that I can validate and ensure is, is explainable. We start to get into all of these other ways of thinking about the code that we're writing. The way that we think about data, the way that we thinking, think about how to compute and analyze that shifts completely from I think those control structures into more like large scale data manipulation. How am I going to transform this? How am I going to analyze it? What's the appropriate analysis for this? You know, if I, if I run my model, how do I know that my model is valid, right? Or that, you know, like. And then you get into all of the ethical concerns about it, right? There's not a lot of ethics around a for loop, right? Like it's, it's a pretty structured concrete thing. But there could be bias in your model that you may not have attend intended. How do we, how do we ensure that we're training all of these other thought processes that are maybe not strictly related to coding but are more related to how we think about ML, how we think about the modeling that we're doing, how we think about the effects of the models that we're creating and what, what they're used for? There's, I think in many ways it's a richer and more holistic view of the way that we're thinking about, you know, computation. When we talk about ML, I was. Kelly Schuster-Paredes: Thinking some, you know, do research on the side. You remember that TensorFlow playground that we found a while ago when we're learning and I've always looked at and I always play around with it. But that, that is kind of one of the examples. The Raspberry PI foundation talks about computation 2.0 and that whole trying to figure out, you know, do I, do I forget what it's called by your. Whatever. There's like the formation Bayesian and then the other models and, and you have to pick which model is going to. And then how many nodes and how many, how much data set, you know, how much are we going to train it? That was the hardest thing to get my head around. Well, you know, we're going to do an 80, 20. So 80% train, 20% new data. Like it is literally playing around and having faith that you're going to sit there, you're going to try 75, you know, 75, 25, you're going to try 80, 20, you're going to try this model, you're going to tweak it a little bit. And it's really interesting because that's something that's hard to teach. That's like a mindset for sure of. I mean, you could teach why, but it's, you know, you don't know the data. It's a different cause for a lot of things. And there's general rules for each model. Right. But I don't know, I felt like when I was trying to figure out which, which ones to do for my beer, I think we were trying to find beer places around the world. And wine. We did beer and wine and I was in the night. It was a 6 to 10 class. What are you going to think about but beer. But it was just trying to. I'm like, why did you do that model? John Chapin: I don't know. Kelly Schuster-Paredes: It worked better. It came out with closer whatever to wine. And we had that, that meat. Sorry, I'm not very, I don't teach it anymore. So I have no, no idea the, the, the correct words. John Chapin: But yeah. And so, you know, and I think, I think it's really interesting you bring up that point about the knobs and the dials. Right. That you can do and you know, and that's kind of like the debugging part. But to, for my students, especially in the first year or two, I didn't do, I didn't do a great job making sure it understood what all the knobs and dials were. And so what we ended up doing was creating hand calculation or I even make them back calculate propagation by hand. But we did gradient descent. I have this little really simple thing where they can actually subtract the derivative that it actually makes sure we go in the direct correct direction. We do a really simple model, but I now have to sheet calculate it. And there's a professor out. Oh, yeah, Tom Ye at CU Boulder. He has a whole website on handcaps. He has like an RNN handcraft. Sean Tibor: Nice. John Chapin: It's crazy. It's really cool. But. But we found was that for three months, it's really important for the students to kind of build their own linear regression model from scratch. Just a regression model from scratch. See the gradient descent? See how it works? They start getting. They build the black box now. Yeah. Later on, pay to go to Keras and, you know, just shove it in. Or psychic learn and shove it in the black box, and that's fine. But then when they were doing the knobs and dials, they're like, oh, I know when I came right, that really means, you know, it's not just this. Oh, that's a word I don't really understand. But let me put it in this direction. See what happens. You know, there needs to be a little bit of intelligence around it and you can teach it. And. And it's super helpful for the students or the students at the beginning to build those linear regression models and also to hand calculate stuff. They can see that it's not just magic, that the math actually, you have the right move in the right direction in the right amount. It's. It's cool and super powerful for the student, and it sticks with the fight. I have students come back from college. You go, oh, yeah, I still remember. You know, I'm in my machine learning class. The same stuff we were doing, you know, it's just really cool. So I agree. I agree with you, Sean, that, yes, there's this. There's this piece of the. There's the data piece, which is the front end. 70% of machine learning is cleaning your data, understanding your data, picking your right data, all of that stuff. And you need to program because you're dealing with large data sets. You also, you need to program that right? Go into Excel and do it. Also the programming part. Understand at a very level what's on, rather than just, you know, moving the knobs in the dial. Kelly Schuster-Paredes: I definitely did not have you as my teacher at the night class because I know. I know machine learning is not magic, but it was so magical. Sean Tibor: I was gonna say, and I really like that point too, about, you know, hand coding it or, you know, or hand calculating it. You know, so often everything's so easy, right? Like, it's so easy to just like pull a bottle down and, and run it without really understanding. Right. Or to create a model using the libraries. The libraries are really, really good. But sometimes like slowing down and going to the fundamentals, going to the basics of what are each of these things, what are the knobs and dials, what are the things you can change and how are these being calculated lets people speed up even faster late. Right. So I wanted to really highlight that because that's something that is an excellent point. And if anyone's struggling with comprehension of their students of topic, going back to hand, drawing it, calculating it, doing it on paper always seems to, to provide a lot of good understanding. Kelly Schuster-Paredes: You know, not to switch but to touch back on something that you guys, you said before. The whole idea of the ethics part coming out of machine learning, I think that's a really big direction. I think that's a whole nother course. So we now we have these outputs and what are we going to do and what changes are we going to make based on it and how does that going to affect us? You know, we're going to, we're going to, I don't even can't think we're going to now jack up our, our housing prices because this model is showing that there is a positive trend in this, this area. And then we're not really seeing all the data because we, we skewed it. We took, we took some out. There's so many topics talk about which then lead us into the deeper AI and generative AI that schools are, are missing out on those opportunities to discuss because they don't have a background knowledge. Yeah. So what, what are you guys, are you talking about any of that stuff or getting a chance to have the fun side of the philosophical side of it? John Chapin: Yeah. You know, it's so funny. There's this, there's this like by, they call it the bible of artificial intelligence. Right. And it's big, huge thick white book. It was written, I don't know, 20 years ago and it's been updated and every, there's a section on, that's about 10 pages. I mean, I swear, I swear. And it's just so true. And we have it caught up, you know, and I don't know how like the answers are, you know, and we code because that's really what I wanted. And I'm not saying it's not. It's absolutely critical because both of you guys pointing up, you know, you have to understand what the data, you have to, well, you get into it, you have and you're making decisions on. And those decisions will affect. Strongly affect your outcome. Right. And they can be very biased. And if we don't talk about. Then, you know, things will just happen. Sean Tibor: Why? John Chapin: And prevent it. Didn't have that upfront thought process. I don't do a good enough job talking about ethics, so I'm gonna answer your question. Kelly Schuster-Paredes: Well, you thought you were gonna say you were. I thought you were gonna say, I am not gonna answer that question. John Chapin: Oh, no, no. It's one of those things I know I need to do, but I'm trying, like, I'm taking apcsp. I'm trying to see how much data science and Python I can and shove into it, because that's what I want to do. I want to make a. Because I'm trying to make a regular standard course in a high school data science and machine learning friendly and, and really get as far as possible. And I don't. I mean, I know there's an ethics part of. About it, but. Sean Tibor: No, I, I don't have always so much bandwidth. Kelly Schuster-Paredes: What. Yep, I know, right? There's. There's so much. You could actually. There should be so many more hours of cs. I. We just got done with parent teaching conferences and every, Every parent was like nine weeks. It seems so silly. Why don't you have longer amount of time? They have English, they have. There's so much to do, you know, in, in schools and we try to put everything in a little. In a little box of time, but there's a lot to cover. I was. I lost my train of thought. Sean Tibor: I have a. I have a question for you, John, about this. Like, so, you know, as we're talking about that progression, as the students are going through and they're learning those fundamentals and they're starting to apply them and they're starting to see how this all fits together. Do you have anything in. In your course or have you observed any points in the course where it all seems to, like, coalesce for students where they start to get like, oh, this is why we're doing that, or here's how it comes together. Is there anything that happens kind of in a moment, or does it happen gradually? Does it happen at all? Like, hopefully it happens, right? John Chapin: No, it definitely. Yeah, it definitely happens. It definitely happens. It's probably that. That when we start switching to Keras and then we can. Because then we start talking about all the different models, you know, supervised learning, unsupervised learning, and we start. Once we start adding those models, then they start Looking at those different use cases for AI and not everything's an LLM. In fact most things aren't an ll right. And take about and so them learning then when, then when they have their own product power and they can see wow actually project because these tools are so powerful and I've learned enough. And you know the hardest thing really is about finding data. They're tech, they can find good data. They're out there, can do it on their own. And then all of a sudden they're like. I had one student, one of my favorite ones was she created this in skin cancer detector where she picture of it with her phone and she'd run it through. Well she did it. She was tested in our own arm and it came out that, you know, it was 70% probable. It was skin kitsch. She goes what do you think? Well do you trust? She goes yeah, I don't go well go to the doc doctor they because it was pre cancer and a crazy okay. And that's a great thing about data science. Right. And the whole machine learning thing is it's super real. Like these kids can go out and they can start making stuff that solves a problem and that they're interested and they can do it in a, in a, in a year. Yeah, they could start doing things real, real things with their code. That is super power. So I think that's when the, the switch starts getting, when they and the drivers. Kelly Schuster-Paredes: I think that you, I mean that's something that we say all the time when it's theirs, when it's applicable, when it's something that interests them, that that's a game changer. And I think when you have this ability with the machine learning and, and or data science of making a change or doing something with it, it becomes like the coolest thing, the best thing next, you know, for you know, next to sliced bread or whatever. However that saying goes. And I've seen it even just with the very small scale when someone was looking at, she was looking at something that her grandfather had and she wanted to get all the data on the disease or the cure rate or whatever and she starts plugging around and this is very similar. She was just doing graphs, not doing machine learning but looking at the trends and that's very powerful. She was really connected to it. And I think that that like sums up a lot of the education. If you can make it something worth their while. Wow. You're going to come up with things that are magical. And I want, I want to ask you about your favorite activities Because I kind of read and I want you to. You had this. Does Zillow, one of your favorite ones, you were saying. So. John Chapin: Yeah, so one of my favorite ones is at the beginning I do this unplugged, totally unplugged Zillow activity where I give them. I pick 50 houses in our account and gave them, you know, number of bedrooms. This, I did this on the first day of class. And I say, what's important, do you think is important in the house, Brian? Right. It's going through that whole site the day of data. I say what's important? And they, they know enough about, you know, some of them are like, well, you know, you know, someone. But most of them go, it's big, the house, where the house is. You know, it's how many bedrooms it has, how big the lot is. It's how old it is. Okay. And then I give them a day that I gave them sell spreads with, you know, four or five param. And then I tell them up with some way to predict the house. And 80% of them basically come up with Y equals MX plus B kind of thing, which is, which is the model. And it's crazy because the whole process, see, and then they, you know, the model's not right. So then they change, they change the way weights. They don't realize their weights yet. Sean Tibor: They like. John Chapin: So then at the end of the class, I'm like, by the way, we called this. These are links. These are features. You know, you guys, the whole process is you make a guess, you see how far off you are, start figuring out how you. And by the way, I'm going to tell you mathematics, how to make a better gap. We're going to use calculus of that. And, and it's great because in this one thing, right, Something that we're basically this profit we're going to use and vocabulary we're going to use for four months, right? It's over and over and over. It's bigger, it's more complex and everything else, but the basic ending and process, it's. I love it. I refer to it constantly. Sean Tibor: It's. John Chapin: It's like really, really the bottom. And then at the end, I actually start plotting out stuff and show how I use subtract the slow of a line and how it makes it go to the cost. We taught, start talking because they know concepts like, like mean squared error. They've learned that they're. They're in calculus. So they're starting about, you know, and even if they're not. And then so you can even at the end of the class. And they're like, so that's one of my favorite unplugged ones. And then there's just a lot of hello world, Titanic, the MNIST data set. You know, there's just the classic ones that you can do and you tell them this is a hello world and you walk them through it. It's really powerful. Oh, and then by the way, the Zillow one, I actually that's one in the linear regression model. And they can see how. So it's really true. Kind of slip. And it's just, it's so much easier to just raw math. It's just. Kelly Schuster-Paredes: Yeah, but I hate to. I, you know, I know you say, yeah, machine learning is not magic, but that linear regression, when you can push out four variables and I, with one line of code and you get like the slope and, and I forget the four. The slope, the, the B Y intercept or something, you get all that in one line of code. That's magic. I was like, wow, that's cool. Who needs math and calculus and all that fun stuff? But I was. I know, I know. John Chapin: No, it's what I do like about. Sean Tibor: It also is that, that like iterative process, right. Like the, the engineering process or computer science. Iterative process is, you know, pretty well defined. Like I'm going to make something, I'm going to debug it if there's issues validated as correct. Correct and come back. But what I like about this is that it's very much tied to. Okay, I have a question that I'm trying to. Right. John Chapin: Oh, right. Sean Tibor: And, and I'm going to use some data and I'm going to try to answer that question with the data that I have. And then when I get the answer that my model came up with, right. I'm going to look at that and say, does this fit? Does it make sense? Am I fitting the model to the question? Appropriate, overfit, underfit, things like that. And that that process is really fascinating because it's. It in many ways to me it feels much more practical in terms of what students can. And relevant. Right. They can see themselves in the data. They can look at that and say that did what I did, what I create make sense? Does it answer the question? Well, and to go back to the skin cancer, I heard about a similar project like this where the, the model that they built was really good at, was really good at predicting certain types of cancers in a sample data set. But it failed on their test set. Right. And what they realized was that they had built a ruler predictor because all of the images of skin cancer had a little ruler in the corner of the image to show the relative size of it. So the model just took the shortcut. Like if I see a ruler, it's probably cancerous, right. And so but even when the model is wrong or even want to take that shortcut, that whole cycle of asking those questions and going through that process of evaluating your model and evaluating the accuracy of the out of the answers that, that it's giving is a fantastic learning opportunity. Right? Like everything is learning even when it doesn't have the outcome that you expect. Kelly Schuster-Paredes: I wanted to add in for our, our younger teachers that are teaching 6th, 7th, 8th or 9th, 10. And made me think of it when you said Titanic, the machine learning for kids website by Del Del. He has the Titanic prediction who survived with Python, but he also has a whole bunch of machine learning with scratch. So when we talk about CT2.0, you don't really need to, to be up there in the high school to start doing this because you can do Journey to School. He has one where I, I played around with these a long time ago where you can have the chameleon change colors and it's like that's necessarily machine learning, but image recognition or color recognition. But he has all kinds of really cool ones in here that I'm looking at. He, the journey to school is train the computer to be able to predict how you travel to school in the morning. And so you're making predictions and accuracy and supervised learning. So there's a lot of ways that you can get in and start doing some 2.0 at a lower level where it's not all Python, but it is that, that thought process, which is definitely a lot to learn. So I was thinking about that. Sean Tibor: Well, I'm going to do a quick time check. I know that we're getting, you know, a little bit longer than we were originally expecting, but I wanted to give John a chance to ask us some questions, anything that's on his mind or any topic that you wanted to bring up without us, you know, seizing the question. John Chapin: Yeah, I, I get, I, I, so I had two questions. One was, you know, how did you guys get started on this podcast? But maybe before you answer that, I feel like I'm, I'm, and maybe you guys, maybe I'm just been preaching the choir to you guys, but I sometimes I feel like, guys, can't you see how big of a deal this data science Python thing is for a huge portion of our students, like, that they're gonna need to know. Like, they're gonna. They need to know. They need to start learning this now. Like, it's a bit. You know, I don't think this isn't a phase. This is. This isn't the newest thing. I think because data's out there, you know, it's gonna be out there forever, and the tools are just gonna even get more powerful, and we're gonna start dealing with. Sean, I think you made a great point of even more data, right? And the only way you go with that is by program within an Excel spreadsheet that, like, just feel like, can't everybody see how. How important this is? And we need to, like, start shifting now. And I'm not feeling shift, but I do. I don't. You guys are out in different places. Kelly Schuster-Paredes: I. For me, it is. I mean, I. We teach it in sixth, seventh, and eighth. So we teach Python and. And they take principles. I always get it messed up. Principles. And then a. So they take JavaScript and then they take Java and they do the whole track. The one thing that keeps me feeling that, yes, I've done my job is. I mean, I just had a conversation with the science entrepreneurship, and she says, some of my kids want to do machine learning. They want to use photos and they want to train a model. And they said to come to you. And I'm like, yes, because, you know, I touch. Like I did today. I'm not an expert. I touch on it so that they know the words and they know the vocabulary and they know that Python can be used. And, you know, and I point them tensorflow or Sidekick or something like that, and I point them in the direction or I hand them a book. So I do think, at least with having that Python knowledge, early on, it's there. And, you know, the kids that really get it, they come back and they do. Technology competition. I had a person last year come to me and there's like, I heard you're an expert on data science. And I laughed. Laughs. And I'm like, yeah, I took a boot camp for six months and I. I thought I was an expert during those six months. But if I don't teach it for me, if I don't. If I don't. If I can't make a metaphor or. No, that's why I said, I need to sit in your class and draw it out by paper. You know, it's. It's not there. But the kids hear the words and they. John Chapin: They. Kelly Schuster-Paredes: For me, I think I. I tell them how passionate I am about Python and I pick on all the other teachers who don't teach Python. So at least I think in our school they know that Python's pretty important with generative AI. I remind them constantly. But yeah, I do feel that we are giving kids a disservice if we don't have a data science not just embedded a couple times in math class full on data science course. If not one two Post AP I think, I think they really need it before they go into College because they're one they don't like. Dr. Chuck said they could go get an internship or they can go work while they're at college and make some money on the side because they know how to code, they know how to do this data science. They can go in and get a lower level job while they get their degree in whatever they want. So yes, I, I feel your pain. I constantly preaching it but I'm in the middle school so I'm doing all I can to get them in the middle school into data science. Sorry, that was long winded. But yes. Sean Tibor: So from my side being back in the corporate world and back in business, I'm sure there's plenty of things I'm not allowed to say because of NDAs and things like that, but I mean my company, we make Oreos, right? We make chocolate, we make, we do manufacturing, we make snacks of all kind. Like when, when it comes to data science, we have no idea how much we are investing in it, how much we are doing with it, how much we are embracing it with it. Because we're, we are under a just an absolute deluge of data from every possible source, right? And that goes from everything from where I work where it's all infrastructure and systems and everything all the way out to how we interact with our customers and our consumers and the people who are actually buying and consuming our products. We're using data science in every direction. I look right now, I think there's a couple different approaches. A lot of it is things that we develop in house for data exploration or ad hoc or developing the next model that we need predict something or to track something and make sure that things are on, are moving in the right direction. There's other things that we're doing where we're trying to create ongoing processes, right? Where once we have the models or once we have the systems in place that can run the models for us, we're executing those and turning those into a core component of our business process and we make Oreos like this is not like, you know, like the Oreos themselves are not any more digital than they were before, right? Like there's no data science in the frosting thing in the middle, right? But a company like ours is operating at a scale where data science has huge benefits, huge payoffs, right? And if we're doing that right, like as a manufacturing company, as a company that makes cooks, right? Think about all of the other places, like healthcare, like media, like telecom, even oil and gas, right? Like there's data science everywhere. And what I'm seeing is that starting to become more and more obvious to people in education that like, wait a minute, we're not where industry is heading. We're still teaching in a direction that is traditional computer science only, right? Or very math based. The example, you know, when I was at, at Carnegie Mellon recruiting for those interns that I was speaking about earlier, right? When I was at CMU in the early two, well, you know, late 90s, early 2000s, the Department of statistics was like this big, right? Like it was, it was tiny, right? And there were only a few people who were, you know, stats majors. And they had a great graduate program and everything. And the people coming out there, exceptional statisticians, right? Like the professors were amazing, all of those things. But it was small when I was there. You know, 20 years later, almost 25 years later, like it is, the programs have exploded in size and now it's statistics, it's machine learning, it's data science, it's artificial intelligence. And they're not just talking about large language, they're talking about ML. They're talking about tons of predictive modeling and analysis work and everything. You know, CMU is pretty progressive when it comes to embracing technology and embracing where the industry is headed. There's probably plenty of other universities who are leaning in that same direction, but the. What they can produce in terms of graduates is probably nowhere near what the industry demanding at this stage. And you shouldn't need to go to a, you know, elite private university to get education in com. In data science and machine learning and everything. It should be way more accessible and available here. Kelly Schuster-Paredes: Here. So, yes, we just confirmed what you're feeling. Sean Tibor: It's not evenly distributed, right? Like, not everyone gets it. They're like people are starting to. But it's, it's a slow process in a lot of areas where I was hoping it would be happening faster, but it is, it is happening. It's just not as fast as I think we'd like it. Kelly Schuster-Paredes: I've seen a couple universities, I can't remember on the top of my head where they actually have the final degree is either data science, it's not just computer science education kind of degree, but it's a degree in data science or data analysis analyst, which is a whole different conversation of what is a data scientist, what is a data analyst? When that's for another show. John Chapin: But yeah, UVA has the, it's slow. Kelly Schuster-Paredes: And once I get the colleges going then it's like the trickle down. There's a lot of companies, there's data science for everyone. There's bowlers at Stanford, her, her math cubed. There's an. I was looking on the whole bunch of things. They have the graphs that, with that I use for some of my lessons of how do you understand graphs. There's a whole big movement with a lot of educators of trying to get data science in to everything, not just computer science. And I think it'll eventually trickle in. John Chapin: Code.org is coming out with their own first data science course and they're actually going to teach Pandas. And yeah, they're supposed to come out in October, November. Kelly Schuster-Paredes: Yeah, but I want to take your course where I have to do things by hand because I think that would help me a lot better. Like the Python stuff, totally easy. I can, can copy the code and make it do things, but being able to visualize it, I think that's where it's at. Unplugged exercises with data science. Those really are pretty cool. Sean Tibor: The other thing I would say, and I want to connect this back to what you said John, is that I think waiting for a data science program or a named data science degree program is a trap. I think, you know, that, that data science to your point is everywhere. I think we should be looking at it from the perspective of, of you know, kind of specific data science programs that help you learn from that, you know, starting point and then apply it to a bunch of other areas. Right. And other, other disciplines and other, you know, field. But I also think we need to go the opposite direction at the same time. Right. How do we infuse more data science into other subjects, other areas of learning in order to make sure that, you know, we're not just treating it as this, like, oh, it's this. You can only do data science if you went to a data science know degree program. It should be, well, what are we doing in biology for data science? What are we doing in finance, what are we doing in business and economics to use that into the way we teach it, the way that the skills that students are developing because the problems that they're going to solve the moment they graduate are probably going to be data science problems just in their chosen field instead of as a generalist, you know, across many. Kelly Schuster-Paredes: You know, it'd be cool. A data science. We're doing music next. That's our next unit. So I always just. Sean Tibor: If you were in that, if you want an instant job offer at Spotify, like, be a data scientist for music, you're in, right? Like, that's. That's what they do, and that's what they do really well. That's the sort of thing that you could. You can blend all of these skills together, right? And interesting. Kelly Schuster-Paredes: That'd be fun. That'd be a cool unit. I'll. I'll wait for you to do that one, John and Unplugged challenge. So we did it again. Sean Tibor: We. Kelly Schuster-Paredes: We said 45 minutes, and we are at an hour, and my kids have been quiet for a whole hour, but Sean's gonna go again. Sean Tibor: I think we should answer your. Your other question about how we get started. Started with the podcast, and I think where we. Where we came from and where this has taken us. So I. I'll. I'll take the lead on it, Kelly. And you. You fill it in for me. But was six years ago now, I was a new teacher. I was. I had never taught before. I was kind of. John, I was kind of doing the same thing you did. I. In March or April, someone said to me, hey, I know you're looking for a change. You should consider teaching. You have a lot of patience and you, like, you know, showing people how to do things right. Like, you love teaching people in a way, on one setting. You should see if there are teaching positions open, especially at the school. Kids go. And I said, okay. So I looked into it and what, five months later, I was in front of a classroom teaching seventh, sixth, and eighth graders. Right. John Chapin: Oh, my. Sean Tibor: So it was. It was definitely. It would have been being thrown into the deep end of teaching except for the fact that I had Kelly. So Kelly was my. My partner in all this. She was my designated mentor or, you know, they saddled her with me, like, hey, here's this new guy who, you know, appears to be a decent coder, but he's never taught before. Why don't you work with him? Show him what you know about and what we found that first semester that we were teaching together. You know, we're in the same classroom every day. She's teaching half the classes, I'm teaching the other half. And we're watching Each other, the way we're teaching, the way we're coding, the way we're trying to do all of this. And we were both learning Python together for the first time, right? Like, I had grown up on Java and C and PHP and all those things way back back when. But Python was new to me and Kelly. And so we're coming at it from these two very different angles. One, how do I learn? I can learn the coding really fast on my site, but I don't know how to teach it. Right? And Kelly was learning how to code for the first time. Kelly Schuster-Paredes: And every. Any language. No language. This is my first language. Like, you'd. Sean Tibor: You'd done Lego robots before. Kelly Schuster-Paredes: Lego robots and Spike prime do not count. Sean Tibor: So we were sorry. Kelly Schuster-Paredes: They do. They do. I'm sorry, everyone. They do. They do count. Sean Tibor: But she had all this phenomenal, like, teaching pedagogy of, like, how do we actually help kids not just, you know, throw stuff on a slide in front of them or ramble on? It was, how do we actually help that penetrate Zorb, Become something that they can actually turn around and use in a practical way? And so we were having all these amazing conversations. Well, how do that? Or what was this about? Or why did you say this at this time? And a lot of those conversations were happening, like on our lunch break as we're walking around the lake, talking and getting some exercise and some fresh air. And we thought. Thought, there's probably. We're really lucky that we have. We're lucky that we have someone who has complimentary skills and knowledge that we can apply, but there's probably someone sitting out there who doesn't have that partner who doesn't have the complimentary skills that they might be struggling with. What if we just start recording these conversations? We'll make a podcast out of it. We'll see what happens. It was totally Kelly's idea. I blame her. She was like, let's do a podcast. And I said, okay. And. And I think for us, the benefit has really been. It has helped us develop both. I know my speaking abilities have gotten a lot better. Listening to myself as I talk and ramble, I've gotten better about the way I speak. But I've also found that having the podcast has opened up our doors in our professional learning network and been something where I've gotten, for me personally, I've gotten new perspective from people that I never would have gotten the chance to speak to. It inspires me all the time to have conversations with people like you who are doing things that are different than what I'm doing. But when I hear what you're doing, I think, oh, I wonder if I could use that over here or how would I take that insight and apply it to this problem that I'm struggling. And we found that over the last five or six years that we've been doing this, I think it's almost six years now for both of us. It's the thing that keeps that fire. It keeps the, it's the Prometheus of knowledge for us. Right? Like how do we keep going, forging ahead. And I think the podcast, huge part of that, that for us because it's constantly renewing and refreshing our thoughts and our approaches. Kelly Schuster-Paredes: Ditto. John Chapin: Well, you guys are awesome. That's, that's really cool. Sean Tibor: Well, and I, I think the encouragement that I have for anyone is if we can do this right. It like our first podcast was we plopped my laptop down and we hit record. Like we had no microphones, we had no headphones. It was. The audio quality is terrible. We. It's so awkward and stilted and it's still our number one most downloaded episode is the first, first one. But if we can do it, everyone else can too. Kelly Schuster-Paredes: But I do have to add on for, for Sean, like on what he said, my personal takeaway now it shifted. Obviously it started off the way that Sean is, but now this is my number one go to professional development. I learned so much because when I start, when as soon as you start talking John, I'm on there Googling. It's kind of like a multi potential. Like I'm like, oh yeah. And Raspberry PI has CT too. And I, I want to find those reactions, resources and I think if it wasn't for this, this collective intelligence that we've had through the podcast, I wouldn't be the teacher that I am today and I wouldn't have the, the knowledge and the understanding. I would not. I would probably still be playing with Tinker Python and that lady who would shake her heads on the, on the corner for the if, if statement. If you don't know what I'm talking about, don't worry. You didn't miss anything. It was what we used in my first year teaching because I didn't know how to teach Python. But I learned so many things and I, I've brought in so many tools and so many ways to reach every child that I've taught because every child's different and I have a kid that wants to learn about AI and I know I can point that person in that direction or I want to know a person that wants to work with hardware and I know where to point them. So for me, that is. Even though it's a lot of, a lot of work, we stay because it's so fun learning. It's so fun learning from amazing people like yourself. And we really appreciate you for coming on the show and sharing your knowledge. John Chapin: So that's awesome. Well, Kelly and Sean, thank you very much for having me on the show and let us all, let us all be in the choir together. Kelly Schuster-Paredes: We'll be back there, we'll be screaming from the, from the bleachers. We. I have to promote early, early, early. It's April 7, April 8. We have our Innovation Institute. It's the 11th annual and talk about learning. This is where all of the ed techs and the amazing people that we work with at Pine Crest, we share our classrooms, we open the doors and it's a very small institute conference, boutique conference. I know we're talking about getting non computer science teachers coding, something like that. We're also talking about VR. They have innovation, entrepreneurship. And I think this is probably going to be the best one yet. I'm very excited about it. Sean Tibor: I think the best one was the one we keynote. Kelly Schuster-Paredes: Oh, that one we did keynote. That was pretty fun. But that was not live. And so that was not that we were not in the room. And the best thing about the institute, besides us keynoting at that one time, was being in the room with other educators. So I'm excited about it. I think they already signed me up for three presentations. Sean Tibor: One of the things I think that's really great about the Innovation Crest is it's the connections that you make. It's all small group settings. You have lunch with people that are, that are doing really cool things and you talk about it in detail, in depth. But you also get to be hands on. I mean, so many of the sessions are practical, like, hey, we're going to play with VR and you actually get to do it. You're not sitting in an audience in a big conference room or a big hall watching someone do something. You're part of it and you're doing it. And that, that to me is one of the special things the Innovation Institute is you are innovating in the institute while you're there. You're not observing, you're not passive. It's, it's an active participation. Kelly Schuster-Paredes: Cool. Sean Tibor: All right, well, John, thank you, thank you so much for joining us. We're going to post links to the resources that you mentioned and we have links. You've shared some extra bonus resources in the, in the document that we'll share with everyone and we'll, we'll put some contact information so if people want to learn more about doing and how to reach you, they'll be able to find it in the show notes. But thanks again for joining. It's. It's been a pleasure having you here. John Chapin: Thank you. It was. Sean Tibor: All right, well, we'll, we'll sign off here. So, for teaching Python, this is Sean. Kelly Schuster-Paredes: And this is Kelly signing off.