Sean Tibor: Hello, and welcome to teaching Python. This is Episode 96 and today we're going to be talking about decoupling, grading from learning, and I don't know if it's going to be decoupling so much as not conflating the two into certain extent. So my name is Sean Tibor, I'm a coder who teaches and my name. Kelly Schuster-Paredes: Is Kelly Schuster Perez and I'm a teacher. That codes, not programs though, because that's like a conversation I'm having on Twitter right now. Sean Tibor: Interesting. Okay, we'll get into that in a bit, I think, but this week it's a little bit different back to our roots. It's just the two of us having a conversation about grading and learning. We don't actually have any guests this time for the first time in a while. Kelly Schuster-Paredes: I know. And as much as I love meeting new people, honestly, I want some me time with Tyres so I can pick on him and ask him why he's drinking coffee at 530. But I was already told it was decaf. So we're okay. Sean Tibor: Yeah, it's not going to be that kind of night. Kelly Schuster-Paredes: But yeah. So I'm really excited just to be able to decompress and work through some things that I have been thinking about and something that we've been talking about prior in a lot of our conversations when we were in the classroom. But sometimes it's just nice to tease out those thoughts and get other people's opinions. Sean Tibor: It's interesting too. This is a great time to talk about it because I'm wrapping up a summer with my interns at my company and what I've had the chance to see is what happens when you have learning that has no grades with it or has no set curriculum where they're learning to be able to accomplish a goal or complete their project with the ultimate goal of getting an offer to return either as a subsequent summer intern or as a full time hire. And it's given me some time to think about how they're learning and some of the reactions that they've had as they've gone from a classroom setting or college setting to a corporate setting and what that means for them in terms of their learning. So I'm excited to talk about this also and share some of the things that I've observed as I've gone through this. Kelly Schuster-Paredes: And I think that's really important for educators to kind of educators in the K Twelve setting, I should say, to kind of have in their pocket their toolbox. We don't get this opportunity and I think this is what's been kind of nice about our little shift of who we are. It's this opportunity to understand what it's like to go from the K Twelve setting into the college setting, into the real world setting and how does that trickle down into what we're doing. In my level in the middle school, I just be working with interns who are preparing themselves like literally two or three years before entering into the workforce and they're doing what my 6th, 7th and 8th graders are doing. They're working and they're learning and they're in a classroom setting, but assume they're not going to be graded. Sean Tibor: Exactly. Kelly Schuster-Paredes: How do they know their successes? So it's going to be fun. Sean Tibor: Yeah. So, before we get into the real conversation, let's start where we always do with the wins of the week. So, Kelly, I'm going to make you go first this week and tell me what's been going oh, my God, I. Kelly Schuster-Paredes: Have such a good win. Sean Tibor: All right, go for it. Kelly Schuster-Paredes: I have such a good win. Well, by the time this airs, I'll probably be done, but I am taking the Fast API course with MongoDB and Beanie, with Michael Kennedy and about 20 other people. And this live cohort and yesterday was my no, today Tuesday. This week has dragged on. We'll get to that on another side. But yesterday I was in the cohort with a lot of developers who are checking out Fast API and seeing how they can use it for their business. And it was really cool. And actually, my win was Michael's doing his teaching and everyone's talking this stuff. I had prepped so much and I had done so much reading and I did some Pie Bites and I did some preview videos. And so as he was doing all this stuff, I was like, I know this stuff, I know this stuff. And here I am sitting with developers. So I was just like, very nervous. Mind you, towards the end, he was typing too fast for me, but we have a video to go back for that. And I did have some errors in my pie charm, but I felt super successful in the situation where I am was in. I was the only educator in that realm and it was just great. And Michael so welcoming and the people there weren't very welcoming. And it just goes to show you, the Python community is lovely wherever you are, whether you're in a cohort on Twitter at Python. It was such a high and it was 2 hours of talking about Python and type hence. And I'm texting Sean the whole time, but did you know about this? And I learned how to write my requirements in file and use my terminal to make a requirements TXT file, which was super cool. And then I learned how to use the was it like JSON to Pidantix or something so that I can get like a full right out of type hence. I just learned so much and it was incredible. And that was only day one and I have five more days of it and I just can't wait. Sean Tibor: I'm really glad to hear that because I know one of the things that you and I have talked about over the years is that journey that you've gone on, from not writing any code before this to being a fullfledged Python teacher and being a developer in your own right and to have those affirming moments where it's like, yeah, I really am getting this. I know my stuff is super satisfying. Kelly Schuster-Paredes: It is. But this is probably the first live course and I think it is literally the first live course I've ever taken in my four year journey. And I feel like it's a big difference. And I've watched tons of Michael Kennedy's courses and I learned a lot from them. But having a teacher in front of you just goes to show having that teacher in front of you who's a good educator, who's there to ask questions, be like, Wait, how did you get that? Oh, you have to import pie tools. Oh, okay, I didn't see that. And you have that live question ability is such a game changer. And I want to do more live courses versus recording. Sean Tibor: I think that's good. The variety of it is what matters. Right. There's only so much you can do that's entirely self paced. There's so much that you can do that is text based or book based. Like, you have to have that variety in those different channels that you use to be able to do your learning so that you're stretching your mind in different directions, right. Stretching your learning and testing your knowledge. So I'm glad that that's working well for you to be able to have that experience and have that affirmation that you're figuring it out and you know. Kelly Schuster-Paredes: Your stuff well, I know day one, so let's not get overconfident but yeah, day one, I'm ready to go. Sean Tibor: Well, you know, then you go do a day two and day three and just like one thing at a time, right? Kelly Schuster-Paredes: Absolutely. And I already have an idea that I'm going to use hopefully with this fast API, and it has to do with our topic. Sean Tibor: That's really cool. Kelly Schuster-Paredes: Do you have to give your win? Sean Tibor: I actually have two wins, and one was kind of a teaching win, and the other one was a non teaching win. So I'll start with the non teaching one first because it reminds me a lot of your win. I had what I'm calling my Pinocchio moment. Not that my nose grew, but that I felt like a real cloud engineer. I'm like a real boy. And it was one of those things where there are a couple of my teammates that are very experienced and very knowledgeable, and I've been learning a ton from them and reading their code, watching how they describe things. They explain things to me in a really good way. So I've grown a lot working with them. And we were trying to solve a bug in one of our applications we're running. And for those people who understand the words that are about to come out of my mouth, I didn't know most of these words six months ago, but we have this container that was running on one of our servers that was extracting metrics, which is my job. So we have it running in this Kubernetes cluster of machines. It can float from various computer to computer doing its job. And one of the things that's important about it is that it maintains its data in a virtual disk or virtual storage that persists with it. So as it gets shut down or started up, it has the same storage and the same object, so it can kind of read and write to this disk. Well, we have made a change with that, where the storage was going to be encrypted so that only certain people would have access to that storage. Just that's our standard practice. And we had forgotten to give it permission to be able to encrypt and decrypt that storage. But it just wasn't starting up for us, and it wasn't passing tests, and there was no good error logs. It was telling us exactly what the problem is. So I ended up going through three or four layers of debugging and trace backs and figuring out what was going on and why this thing wouldn't work, and setting up a test environment and creating the whole thing and getting it to work. And then I realized what the problem was. I fixed it, I tested it, it worked, I deployed it. And all of this happened from, say, about 08:00 at night until about 130 in the morning with no help from anybody, like nobody was available. And I figured out the whole thing by myself. And I was like, I feel like a real cloud engineer right now. Because this was a hard problem to solve. It wasn't obvious. I had to keep digging and keep digging and be persistent with it until I figured it out. And then it worked and everything was okay. And I felt really good about that. And I got that nice kind of affirmation and I got the nice affirmation the next day, like, hey, nice job solving that. That was not easy. Well done. It felt really good to have that. Kelly Schuster-Paredes: Moment, and it justified the 01:00 A.m.. You know what, all in teaching and everything, but 01:00 A.m. Is way past my bedtime. Sean Tibor: I fell victim to that. Hours later, I fell victim to that one more thing, one more thing trap, where you just I want to get that one more thing working, and then before you know it, it's way past your bedtime. Kelly Schuster-Paredes: Oh my gosh, not me. I don't know how you people that work at home do it, because I get home and I'm like, oh, that couch and that bed looks really nice. Sean Tibor: I still have those moments, too. That night was not one of them, but I had that kind of non teaching moment, which was kind of a personal growth journey, and felt really good to kind of mark that milestone as being a real good moment. And then the other thing that happened over the last couple of weeks is that as my interns projects are wrapping up, I'm seeing more and more of her growth and development along the way. And I've played more of a teacher and mentoring role over the course of the summer, trying to coach her and guide her and help her clear roadblocks and everything and just watching her progression from the beginning of the summer. She's a very capable student, very capable learner, but didn't necessarily have a lot of experience with the kind of work that we had asked her to do. And she has been just going and figuring it out and working her way through it and being persistent, being determined, and just being able to go see her kind of get through the process of learning and figuring things out and applying. It has been really satisfying to watch, and it's made me feel really good about kind of creating those opportunities for people to learn and grow and develop. And so I've got a book for her, actually, it's a good book recommendation for you, too. I got a book for her as her end of summer kind of thank you gift. And it's called The Missing README, and it's a guide for new software engineers about all the things that are not about writing code, some of it's about how to write code, how to make it maintainable all those skills that they want you to have. But it's not specific to Python or JavaScript or Java or anything else. It's really about how to do your job well as a software engineer. And I've been reading through it before I give it to her, and it's got a lot of really good advice in there, so I'll post a link to that in the show notes. Kelly Schuster-Paredes: That's awesome. That reminds me of the book that you gave me about Think Like a Programmer. I mean, that's still kind of my one that I didn't read the whole book straight front to cover, but going through certain chapters and just reading the thought processes of it and things that you don't Kelly, learn in school, which is, I mean, yes, you learn it in context, but you never learn it in application in the real world setting, maybe you do. I haven't done many CMU courses, but as in any CMU robotics school. You did robotics school. I can claim that I did. I did do robotics there. But yeah, that's the thing, right? Teach that content. You can teach the opportunities to solve that content, but until you're thrown into that fire, you don't really get the opportunity. So that's pretty cool for her. Sean Tibor: Yeah. So I'm really proud of her, and I'm also really sad to see her go because this is her last week with us, and I'm planning to stay in touch and really want to talk her into coming back and doing another internship next summer. I think she'll be pretty effective no matter where she goes. So I'd like that where she goes to be with us. Kelly Schuster-Paredes: That's awesome. Sean Tibor: Selfishly, of course. Kelly Schuster-Paredes: The good news is you get some more interns, hopefully in the next summer. Sean Tibor: That's right. So maybe I'll post a link to those job postings for anybody listing and we'll see if we can get some good candidates. But it is definitely a lot of my job lately has been around thinking about recruiting and team building and how we're going to grow our team over time. And a lot of the work we've been doing is really globally focused. So how do we recruit team members in other countries and other regions and bring them in? And it's been really fascinating to talk to and learn from people that are interested in working with us from all over the world and got to have some really interesting interview conversations and learn more about the people that are out there and what they want to do. Kelly Schuster-Paredes: That's super cool. Did you have any fails this week? Sean Tibor: Well, except for all the fails along the way to that wind. It took a while to get to that one. Kelly Schuster-Paredes: How many hours was that? Like 1516? Sean Tibor: Yeah, it was a while. I did also have a moment where one of my servers got out of sync somehow and started firing off an alarm like the world was ending every five minutes for the better part of 12 hours. So I woke up to just dozens of messages in my inbox about how everything was not working and it turned out it was just a metrics error. I guess that was the biggest fail that I had this week so far. Kelly Schuster-Paredes: I had a classic fail last week, although it got saved and it didn't turn into a terrible loss. For those that don't know this, I do all the educational software and ebooks for the five schools that we have, and I work with Class Link of Importing, all the books and et cetera, et cetera. And we shut down Class Link Access so that the students can't see the rosters. Because it's a very important day when the teachers get to call their students and say. Hey. You're in my class and welcoming you. And it's kind of a nice experience. But always everyone wants to know whose class their child is in. What teacher they have. Because they have their favorites. Siblings have had the teacher before and they want the same teacher. Et cetera. Et cetera. So anyways, we lock down the rosters, but in order to do books and software, we need to have the rosters in there. So anyways, last, I don't know, like 630 in the morning when I was working, the house was quiet. I had to set up a resource CVS sorry, I was going to say CSV for the books that were associated with each course. And I'm going through the steps and I'm like click import and all of a sudden I see this whole stuff like importing and it was all the rosters and everything, and I was like, oh, God. Texting our tech person. You've got to shut it down. You've got to shut it down. Thankfully, he was on his email in his chat at 07:00 in the morning, and we managed to shut it down, and I was just like, okay, maybe I should have some more coffee next time before I work. It could have been tragic. And this was a good five days before the release. Sean Tibor: We actually had a moment this summer where we shared each of the senior engineers, the people that were supposedly the most experienced, most knowledgeable, most on top of our stuff, shared with our interns each of our stories of the catastrophic mistake that we made as a junior engineer, right? That oh, no moment where you realize, like, oh, man, I totally messed that up, and I deleted something I shouldn't have, or I destroyed access for everyone or whatever it is. And it was kind of humbling, I think, to go around the room and realize that all of us, no matter what we've done in the past, have had that moment where you're like, oh, no, I've really messed this up. Right? And the point was that it happens to everybody, right? And as much as we want the computers to be infallible and as much as we want them to be always up and always on and always reliable, we're still humans operating the machine, and humans make mistakes, and that it's okay. That there's reasons why we have controls in place. We have safeguards. We do backups. We do all these things because we make mistakes, and it's okay. Right. And one of the things that you start to realize the longer that you're in this business is that there are mistakes that are made because you did something stupid, and at some point it becomes someone else's mistake because they didn't account for your stupidity, right? And there's also a level where it really goes beyond that. So we've talked a lot about kind of how to avoid those things, how to be thoughtful and how to kind of be kind to yourself and keep moving, to own the mistakes when they come up, but then to be able to move past them when you realize that there's nothing more you can do. Kelly Schuster-Paredes: And that's the thing. There's nothing more you do. You got to figure out how to move past that and be okay. Well, I think I would have been in big trouble if the rosters had gone out, but I'm glad they didn't. Sean Tibor: Yeah, they didn't. So you're okay, right? Kelly Schuster-Paredes: I got saved. Sean Tibor: All right, let's talk about grading and learning, because I think it's an interesting topic with a lot of discussions that you and I have had over the years, and I don't even know where to start exactly, but I think the best place to start is the recognition that grades don't always affect learning. And the learning is not always reflected in the grades. That often what we can measure and what we grade on are not the things that really matter when it comes to learning. And in fact, traditionally a lot of the grades that are given are given because they're convenient or easy for the teachers to create and assess and don't necessarily reflect the actual growth of a student in terms of their knowledge and understanding. Would you say that's fair? Kelly Schuster-Paredes: That was really well done? I think that was like an expert teacher. Sean Tibor: Well, you taught me everything you know, so that's where I'm getting it from. Kelly Schuster-Paredes: Yeah. I think going through a criterion based school to an A through S kind of school system I've sean it all and it's always been something challenging. As much as we I like standards, even those are not often all inclusive of the actual learning growth that you see within a child. And sometimes we just don't necessarily get it right, even though we know subjectively where that student might lie. Because we've done so many assessments throughout the year and we get to know these children and their abilities, but their grades don't match what we think they should have. So it's always been a hard spot for me and it's something that I really and I know a lot of teachers struggle with the process of giving a grade because we have to. Sean Tibor: So the question is, why do we have to? What is really the purpose there? Because when we talk about grades and we talk about what we have to give them, what's the purpose of a grade? What is it supposed to indicate? Kelly Schuster-Paredes: So in the educational system, people want to know where they are in comparison to others. We compare schools, we take standardized tests to say this school has a percentage of students that are greater than another school and we like to rate things. We like to have children with different GPAs and we like to say this percentage of students will have a 10% of 3.99%, or now it's like inflated 5.25% GPA and they like to have that as a comparison. But what does that mean for a child or a student that's going into the workforce? They're great in memorizing content. Yes, they were great in saying what was in the book or what was on the test. But how do they really apply that knowledge outside of a school setting? So I think there was this philosophy where we had to do something, but we didn't necessarily ever get it right. Makes sense. Sean Tibor: Yeah, I think yeah. I mean, it's a really complex thing because grades are lossy, right. They're taking something that's very complex and very nuanced and it's reductionist by nature. We're reducing all of these things that a child has learned or an adult learner has learned knowledge that they've acquired. And we're trying to combine or reduce that down to an indicator of that. How well did they learn it right? And there's a lot of different ways you can assess this. You could say, here's what they should have learned, here's what they actually learned. And the grade is a reflection of how much of that they accomplished, right? Like their acquisition of knowledge in its worst form. A grade is a reflection of how well organized the student is and how well they turned in their assignments on time. Right. It doesn't actually reflect what they learned. Right. And so you can have different grades mean different things depending on the school, the teacher, the subject, the course content, the grading system. All of these things are factors in that assignment of the grade to the outcomes of the course. Right. And then you also start to get into the challenging the assumption that does the grade accurately reflect the learning outcomes that we wanted the students to achieve by the end of the unit of learning? Right. And if you think about it as a course level, but then there's also, like the individual assignment level or the individual task level that requires that teacher. Kelly Schuster-Paredes: To say, these are the individual tasks and abilities of each child. And sometimes that doesn't always come through. Right? Here's the content that was required to know. Great. That's your knowing level. But where's your application level and where's your skill level and where are you on that whole spectrum? So then the grades kind of become a mixture, like you said, of everything. And I keep thinking in my head, can you imagine being at your work and your boss says, okay, here's my bell curve. I have 100 employees. Where are you on this bell curve? I think you're over here in the I'm going to give you credit. A 96 percentile of workers at this job. It's kind of how people do the grades. Not everybody can get a 99. Why not? Don't you want 100% of your workers to be in the 90th percentile? Don't you want that productivity to be at positive mark? Do you want to have to fail somebody or push them into not always getting an A? And we kind of do that in an educational system, and that's kind of where teachers are going. There's a lot of trending going on with this hashtag ungraded. How can I get rid of that mandatory zero to 100% system in my classroom but still reported in a school setting and the school report card? And it's a lot to think about because let's think about computer science and let's just only focus on what we teach. You either can make the code or you can't make the code. There's no 92%, right? You either can do a print statement or you can't do a print statement. You can either write a conditional statement, think through logically of a problem solve, you know, a little challenge, or you can't there's either fail or pass. But yet somehow in between our students are getting zero to 100. Sean Tibor: Well, there's definitely still a gradient, though. I mean, in the corporate setting, there's a stack ranking of employees, right, where they look at who are my best performers, who are my worst performers, and that gradient can be very narrow. Like, even your best your worst performers at your company might be stellar employees at a different company depending on their level of maturity and the skills that are needed. But I think where you're going and where I want to poke into this a little bit and learn more about it and talk more about it is the whole premise of is knowledge is learning. Is it something that is binary? Is it something that you can just do it or you can? Right. And what are we really measuring? Because I think the grade is a heuristic, right? It's a way to simplify something complicated in a way that we can grasp it and communicate it. So in one way, the greatest way to communicate to the student, to the administration, to parents or other stakeholders that are involved, how well that student performed in that course over that given period of time or on that assignment over that given period of time. But there's, I think, a difference, and we would kind of want to discuss this and understand it in a deeper level. What grade should someone get who walks into a course knowing everything that needs to be known, all the content, they know how to code it's, effortless for them from day one, and they make no growth. Right. Versus someone who starts with no knowledge of the subject matter and grows to that kind of proficient level where maybe they don't know everything, but they're adept, they're capable, they're able to achieve the outcomes that they want. What great does that person get right? Are we measuring growth or are we measuring aptitude? Kelly Schuster-Paredes: And nine times out of ten, we're measuring aptitude. Right. That child that was good at math, always. And they come in, and the concepts seem easy to them, so they're always in that top grade level. And you're right. It's like, at what point do we condone or what type? Not condone, I guess, but as we appreciate, we celebrate growth level and achievement level and the ability to move past where you were and get better at something. Yeah. There's so many thoughts going in my head right now because this is something like I've really been trying to tease out in my classroom, and they've taken the point that your concept of someone knowing something at a better level. So we have children that come in that have been coding since they're in 3rd, 4th grade. They know Python. They're trying to outsmart the teacher. But the whole time in the classroom, they really haven't challenged themselves to become better in something else. Right. They're so stuck on the point of saying that, oh, yeah, I can answer all your. Questions. And it's kind of like, hey, I'm there, I can answer everything. You tell me where you have another child who absolutely has no level of code in their background and they are able to produce everything. And I feel like the person that works harder, the person that has more growth, is that person that's always going to have growth, that's going to always challenge themselves that they almost deserve a better grade. But you can't do that, right? Sean Tibor: It's hard to do everything right? Kelly Schuster-Paredes: Yeah. Sean Tibor: Here's an interesting thought that came to mind as you were describing this. I'm not sure exactly where it came from, but I once worked at a company that had an unwritten policy of not hiring anyone with a 4.0 GPA. Kelly Schuster-Paredes: I would love that. Sean Tibor: And the philosophy was, and the reason for that is that they would much rather hire someone who has a 3.8 or 3.7 and has extra curriculum, good projects, they're more well rounded, they have all these other activities. The thought was that someone who's always chased perfection in their grades may not be putting the effort on the right things. They may be so focused on perfection that they lose sight of good enough and being able to move on. They can't deal with failure, they can't handle issues. And obviously that's a gross oversimplification. I'm sure there's plenty of 4.0 students who can handle that without any issue and get that 4.0. But the idea was that the grade was something that they were chasing so hard that they lost sight of the bigger picture of why they were chasing it. Kelly Schuster-Paredes: So one of the other points that comes out with this whole ungrading movement is this idea that students aren't really appreciating learning. So that goes into the point of this 3.8 child probably either struggled, came across other challenges, learned other skills during this process of learning, and I wish you could see my air quotes, but this learning that's happening because that 40 students and not every 40 students. Some people are just amazing learners. They've got great aptitude, they've got a great brain. Like you can hold more short term memory stuff. I'm like at a three, four on good days, five items short term. Some people are just more skilled in certain things. But there was this movement that if a child is pushing always for the grade, then they're not really pushing for that learner. And do you really want an employee who's just doing X in order to get that promotion? Or I'm doing X to say, hey boss, give me that extra money or whatever, get that self gratification are you doing? Or do you want that employee, that person that's trying to solve problems at work or in the classroom because they appreciate the point of solving the challenge or that point of learning or that point of growth. And I almost would like nine times out of ten want that person who is doing it for a greater reason than just that external gratification. And that's kind of how I see with those grades constantly, because what am I going to get on that? Like, really? Does it matter? Didn't you just see? You wrote 200 lines of code and you didn't know how to code five weeks ago. Isn't that amazing? Yeah, but what grade am I going to get? And I feel like sometimes we have to just say that these grades are pushing down the learning or stopping the learning. Sean Tibor: If the grade is affecting the learning, why are we giving it? Kelly Schuster-Paredes: Exactly. Sean Tibor: Right? So if the grade is affecting someone's ability to truly learn, it's not helping anything. Right. And what we've lost sight of and I'm not saying we as teachers, I'm saying we collectively, everyone involved, is that the grade is an outcome. Right. It's a post learning result that occurs, but so many times that gets brought forward into the learning process as the goal. Right. Getting the grade is the goal that we lose sight of the actual learning. And so I think to turn this into something maybe more practical as we start to hopefully wrap this up a little bit, because I want to bring it together. There are some things that I've watched you do that I think we're really effective at decoupling that grade from the process and putting it at the end where it belongs. Right. And removing it from the equation for students as they're doing this mental calculus of why am I doing this? Right. Kelly Schuster-Paredes: Yes. Sean Tibor: So point number one that I've seen you do a bunch of times, and I love it, and you should do this all the time for students is give the grade before the learning. Right. So they say, well, what grade am I going to get for this? Well, it's out of ten points. You get a ten, you get an A. Here you go. You got your grade. Now what are you going to learn? Or how does this apply? So you've guaranteed the grade. Here's your grade. Don't do this for things that are really maybe important in terms of assessing understanding and knowledge, but really around things that are completions or things that are smaller portions of the grade. Take the grade and give it to them right away. Here's your A. You've got the A. Now what? Right? You've gotten the grade. That's no longer a concern. What is your actual concern? And I love that approach that you take with it. So what has been your experience doing that, though? Does that work every time? Are there places where you use it or places where you don't? Kelly Schuster-Paredes: Yeah, so some students really do well with that, so that's not a stressor to them, and then they continue to learn. I do agree that on those, it's kind of like holding the carrot in front of the horse and saying, let's walk to the finish line. Here's your carrot, your ten points are there. You see it, it's in your site. You just got to keep walking through. And I personally would just do that for everything. But it doesn't necessarily work that well. For example, for a project or for a product where a lot of times I'll share out a project that a student does and a lot of people on Twitter will say, yeah, but why did they do it that way? Or you notice that they had this mistake or they could have written it in four lines instead of 20. And I'm like, but that wasn't our goal. Our goal is to get a working product out of these specific learner goals and that child or that student did it. So in those cases, I would not just give them a ten and say now the learning is over. I think in those situations, that peer reflection and you get your ten, it will happen, but it might take you 20 more attempts than the other child who might do it a little bit quicker, or I might push them a little bit in a different direction. So this is my baseline and the person that already knows how to code might have a higher baseline. So in those situations that ten right away in malwut doesn't really work make sense. Sean Tibor: So then the other end of this, the other thing that I've seen you do and I learned a lot from, was also if you can't just give the grade, remove it from the equation, make sure that the grade is the important part of the outcome that you want to achieve, right? So this is where the rubrics that you write are not necessarily focused on the functionality of the program. Or one piece of it may be the functionality of the program, but the other pieces are the real outcomes that you want them to demonstrate, right? So demonstrate that you know how to break down this larger problem into smaller pieces or demonstrate that you work well with others, demonstrate that you solve the problems and you work through the process of debugging this and understanding it. So the demonstration piece of it is aligned to the skills, the competencies, the behaviors that we want our learners to be able to exhibit and prove. Right. So that if you are going to the other end where you have something that is deliverable, make sure that the grades aren't just well, you answered a bunch of questions on a quiz and you were able to guess your way through it. It's the points that are being assigned. If they're going to be a reward and they're going to be something that those students are going to chase, make sure they're chasing the right things in pursuit of those goals. Kelly Schuster-Paredes: Yeah. And those are like a single point rubric. There is no did it at an exceptional level and you got a four that's either yes or no. And what becomes great and this is something that I'm thinking about adding to that process this year. I'm worried about time constraint, but I want it to be self graded. So yes, I've sat with another peer, or I've sat with me, your educator teacher, and I said, yes. Have you met that? Show me where yeah, you did great. And we have a conversation about meeting the goal or not meeting the goal. And when they don't meet the goal, okay, well, you want to go back? Do you want to make it better? Do you want to achieve that goal? There lies the point, though, that everyone should, in theory, get 100 at the end of the course and so be it. Sean Tibor: Yeah, those are two that I've seen you do and I've used myself that I think are very effective. Either removing the grades from the equation when it comes to learning or making really sure that the grades are hyper aligned to the outcomes, the skills and behaviors that you want to see, recognizing that if they're going to chase the points, make sure they're chasing the right things. Kelly Schuster-Paredes: So I've had a couple of different options in my head, haven't decided exactly what I'm going to do. So if you're listening out there and you start groaning at them, please forgive me. One of the options, though, just to change things up is I was thinking of doing little challenges and having everything else like nonexistent in a great book, but the challenges being so low and so frequent that the points in essence, would flatten. And so, for example, do a print statement today, or here's your project, you need to make this happen. And here's your five point kind of thing. You can work with a partner. You can get debugging, and the idea is to get a product or something and make it just about those steps and not necessarily like, here's how you practice, or here's how your learning happens. And not just grading on the learning part, but grading on the outcome. So that was one scenario. Something else I was thinking about doing, which is kind of hard in a nine week course. If I had a year course, it probably would be nice, but having one on one conversations with students throughout the nine weeks, if it was a year course, I think you can have more frequent conversations of where the learning is, more reflections, more dialogue. But in such a short nine weeks course, it's almost difficult. So maybe talk about one project or here's your final project of that nine weeks, and let's talk it through and meet like sort of what you did with an intern. Where are you in on your standing? What do you need to learn? And kind of work through it that way. So these are all my wishful thinking for moving away from the carrot stick and more into a marathon. I guess if I had to make an analogy of learning to code so that the passion comes in the actual process of learning how to code and not necessarily that's a on a piece of paper. Sean Tibor: Yes. And one of the things that I found fascinating about working with interns this summer and having conversations with them, with them, I really wanted to make sure that they had the freedom and the autonomy and the latitude to be able to tackle their project in a way that made sense to them. And almost every single one of them has described to me this really kind of jarring transition for them. Not in a bad way, but just kind of a surprising revelation, I guess, is that they realize that in school everything is laid out and planned out. Here is the start of your assignment, here's the end of your assignment. And there's a sequence of steps that if I do them, I will get to the end and I'll have a good grade. And what we do intentionally is, here's a project description, here's what we want you to accomplish, here's what we think that should do, and here's why it's important. But then how they get there is up to them. And we can guide them, we can coach them, but there is no series of steps that we figured out, because if we had the series of steps already figured out, we would have just done the project. Right? So they've said that it's really different to go from this really prescribed. Here's the assignment. Here's exactly what the outcome is going to be to kind of charting their own course and finding their own way to that desired outcome and being able to go back to any one of us engineers and say. I know you had me put it together this way in terms of the outcome. But that's actually not what we need. We need to go in this different direction instead and do it this other way based on what I've learned. And our response to that is, okay, let's do that. That's your project. And I believe what you're telling me is the right way to do it. As we think about grading, one of the things to keep in mind is that most jobs out there, most things that we do, don't have a defined beginning, middle, and an end and a sequence of steps in the way that we're going to teach it. I didn't approach teaching that way. Right. I had things I wanted to cover and things I wanted to accomplish. But charting that course each day was something that you and I worked together to figure out, right? And I think where we need to look at grading is how do we really communicate and assess and reward that process of figuring things out. Charting the course. Finding the way. Whether that's something that's very highly structured or very abstract. There's got to be a way to reward that behavior that either uses grades to be able to make that process better or gets them out of the way so they stop causing problems. Kelly Schuster-Paredes: Yeah. For me, it's always I wouldn't say always because I don't like that word. It's a lot of the problem with education. The grades get in the way of the learning. Sean Tibor: And a lot of that is shame on us. Kelly Schuster-Paredes: Yes. Sean Tibor: Because a lot of that, at least what I found was it was something that was a crutch. It was something we could fall back on. It was often for our convenience as teachers and not for the students learning. Right. And thankfully, you and I are blessed to work with administrators who give us a lot of freedom and latitude to figure out our path within those boundaries. Right. Here's the thing that we need to do. We do need to have grades to be able to communicate effectively, but they give us a lot of room to be able to figure out what those grades should mean in context of our course, as long as that would be backed up with real information, a solid thought process, and some demonstrated evidence from the students. Kelly Schuster-Paredes: Yeah. And we're not saying we have all the answers because we're still obviously picking through all the things that are successful or not successful in our systems and our teaching and our jobs. But it is something to think about. And I think this is something that never really goes away for a teacher. Getting rid of that subjectivity, getting rid of that carrot stick, getting rid of that need to express stuff and just really kelly having fun with that learning process. So I'm interested to see what other people think and say about undergraduate and where they're going to go with that. Sean Tibor: And that's one of the best things about being part of the teaching community and the people that we've gotten connected with, is that there's a whole variety of opinions and they have a lot of everyone's coming from a different place. Everyone has good reasons for why they think the way that they do, the experiences that they've had. And it's all worth considering, it's all worth discussing and understanding. And I don't think that this is a problem that will ever truly be solved. I don't think it will ever be solved. Kelly Schuster-Paredes: We don't say never ever. It's not done yet. Sean. Sean Tibor: It's the pursuit of the goal that makes us better. It is not the achievement of it. If we spend our time working and striving and struggling to figure out how to better assess our students and how to better motivate their learning, to reward their learning outcomes, then we're doing the right things. If we stop that's when we have problems. Kelly Schuster-Paredes: Absolutely. Well, I am looking forward to trying a lot of things. That's the benefit of having quarter classes and seeing where I go with that. I've tweaked my choice. Boards are no longer in squares and I've done. This is opportunity to learn. This is opportunity to show your learning and this is where you're going to be assessed. I'm going to only grade the final checks to see that they understand them and however they get to them will be their choice if they want to extend their knowledge. We'll see how it works. I have my son as a guinea pig this year. Sean Tibor: I forgot about that. I'm actually a little sad I won't be able to watch that firsthand. Kelly Schuster-Paredes: For those of you who do not know this, my son is entering 6th grade this year. It is very stressful. My baby is no longer going to be a baby. He's going to be a middle schooler, and I'm going to get to see what that's going to be for him. And I've known all his friends since PK, and I'm really nervous for them because they're still babies in my eyes. So I've taken a new look. Sean Tibor: Don't worry, they'll grow up the second half of the 6th grade year, not the first half. They'll still be babies the first half. Kelly Schuster-Paredes: Oh, I know. And I'm just like, I can't believe they're out playing nerf guns in my backyard. And here I'm going to make them code. So all of you parents that have middle schoolers, I'm feeling your pain right now. Sean, you're in a couple of years. Sean Tibor: Yeah, I've got a little ways to go, but the interesting thing that sort of developed over the course of the last four or five months is that I've become more and more involved with the learning and development team at my company. So I'm working with them on a few different projects, really around it, skill development and creating opportunities and pathways. And a lot of the conversations we've been having are really about engagement and enrollment and getting people to buy in on this concept of professional development and motivation. And it's been a really interesting thing to try to marry up all the things that I learned as a teacher, working with students who frankly had to be there, right, they had to be there, versus working with adult learners who they have different motivations, they have different reasons for continuous learning and development. And as I've tried to share and tried to exhibit, I don't think there's a profession on Earth where you can just stop learning, where you will know everything. You always have to be learning something new and creating something new and making something different out of yourself. And so I'm looking forward to applying some of these ideas because I'm in a situation where there are no direct grades for the work. Kelly Schuster-Paredes: Right. Sean Tibor: There's no assessment. I myself am studying for an exam, for certification exam. And yes, there's a grade on the certification exam, but once I have it, my certificate looks just the same as anybody else's, whether they got a barely passing grade or a perfect score. Kelly Schuster-Paredes: Isn't that great? Crazy. And yet, yeah, I'm not going to go back into the conversation. Well, speaking of learning and development at a corporate level, there's a super conference that is in November, and I went to it as an educator, and I loved it, the Learning 2022. And I'm trying to go back to it. It's a little bit of a hard time because it is in November. But Sophia, he's going to be there as a keynote speaker, and it's going to be Kelly. Interesting. And that goes back into learning development. How do you keep engagement in the workplace? How do you diversify the workplace? And that's pretty cool. That just made me think when you said Learning Development and there you go. Maybe we should meet up at that conference, go see Sophia Speak keynote speaker. It's in Orlando. Sean Tibor: I think for a moment when you said the robot woman, I thought you meant Simone Gertz. Who's, the queen? Kelly Schuster-Paredes: No, the actual robot. The actual robot. Sophia. She is giving one of the keynote. Sean Tibor: Okay. That could be really good. Kelly Schuster-Paredes: Yeah, I really want to go. So it might not be Python for me this year. I might be going to Learning 2022 just to meet Sophia. Sean Tibor: Yeah, we'll see. I think I'm trying to figure out what my professional development schedule looks like for the next year or so, but that could be fun. I think that's in Orlando, right? Is that correct? Kelly Schuster-Paredes: Yes, it's in Orlando. Yeah. And I don't remember exactly when, but we'll put it in the show notes. Sean Tibor: All right. I think that is pretty much it for our conversation today. It's been really enjoyable to just have a one on one conversation with you again. Kelly Schuster-Paredes: Yeah, we could ramble on for a couple of hours. I feel like we don't get to ramble as much, so anytime we'll just. Sean Tibor: Have to do a live stream or a zoom meeting with our Patreon supporters or something like that, where we can ramble for a while. Kelly Schuster-Paredes: Yeah, that new mayor website is pretty cool, and people can just talk and cohort and maybe one day. Sean Tibor: Yeah, I think we'll get that. We got to get through the start of school first. Kelly Schuster-Paredes: Oh, yes. Don't even remind me. It's only day two of going back, and I'm already tired. Sean Tibor: I get it. All right, well, I think that does it for this week, if you'd like to and please do email us, chat us up on Twitter. The best way to get a hold of us is through our website at Teaching Python FM. We are on Twitter at teaching Python. Kelly is at Kelly Perez. I am at Smtber. And we are both eager and waiting to hear your thoughts on the idea of grades and learning and how they relate, or don't, as the case may. Kelly Schuster-Paredes: Be, and some of your thoughts of just what you might do to try to break that carrot stick of getting the A in the class. Sean Tibor: Exactly. So maybe we'll have to do some sort of, like, hashtag chat and get that trending and talk about maybe we can take over an ed tech chat or something. Kelly Schuster-Paredes: That'd be fun. Sean Tibor: Yes. We'll work on it. Okay, so for teaching Python, this is. Kelly Schuster-Paredes: Sean, and this is Kelly signing off.