Episode 64: Rubrics and Assessments
Sean Tibor: [00:00:00] well, you know, the points don't really matter,
Kelly Paredes: [00:00:01] yeah. Points don't really matter.
Sean Tibor: [00:00:03] points.
Kelly Paredes: [00:00:04] 1 million points.
Sean Tibor: [00:00:05] like whose line it is anyways.  Welcome to middle school where the points are made up and the learning is real. 

Hello and welcome to teaching Python. This is episode 64, rubrics and assessments. My name is Sean Tibor. I'm a coder that teaches.
Kelly Paredes: [00:00:35] and my name's Kelly Schuster-Paredes, and I am a teacher that codes
Sean Tibor: [00:00:38] so Kelly, it's  a rare weekend recording for us because we are just starting spring break.  We actually have some time to catch up on a few things we've been working on. So  what's your big project this weekend?
Kelly Paredes: [00:00:51] Wow. So I'm still working on  my bookstand trying to,  diminish the amount of books on my nightstand. I should say, not my bookstand move them off my neck, stand onto my bookstand. I did make it through to last week.  My big time goal is just to get through the rest of my books that are sitting in there.
Tons of them have a couple of, uh, AI, AI books, and some scrum books left by Jessica. So.
Sean Tibor: [00:01:19] Very cool. Very cool. Yeah. I know  what that's like, I have a couple of new books coming in for data analytics and machine learning that I'm ready to dig into and learn a little bit more. And I think David Amos's new Python basics books , through real Python is sitting on my doorstep right now.
So after the podcast, I'm going to go pick it up and dig into it a little bit and see what he's put together for that. I know I have the ebook version of it, but there's something just more satisfying about the physical copy.
Kelly Paredes: [00:01:44] Yeah, so exciting for them too. Right?  
Sean Tibor: [00:01:47] Yeah. I think they launched and they hit number one on the Python programming books, bestseller list. And I think  if I saw correctly, they're at number 83 on the top 100 books on Amazon overall. So it's pretty exciting to see a nice big launch for them. The folks over at real Python and David in particular have been really good friends over the last couple of years.
And Roy helped us learn a lot. So was pretty exciting to see.
Kelly Paredes: [00:02:11] Yeah. And we also have, uh, Al Swaggart book. We're looking at.
Sean Tibor: [00:02:14] Yeah, we'll get to that a little bit later in the podcast. I think we're going to do a little bit of a special book review session, or maybe a little mini book review and talk about it, but there's a new book coming out soon called the big book of small Python projects. So it's kind of fun to dig into and have some ideas for how we can use it in the classroom.
Kelly Paredes: [00:02:32] Very cool. Cool.
Sean Tibor: [00:02:33] Right. Cool. So before we get any further, , wins of the week, Kelly, , same place, we start every week. Something good that's happened inside or outside of the classroom.
Kelly Paredes: [00:02:41] Yeah, well, we were kind of talking about this, but I out nerded you. So I guess that has to be my winner of the week, right? The fact that I knew something completely dorky, well, I researched it, but I still knew it before you knew it, which is a first and I feel like that's a huge win.
Sean Tibor: [00:02:58] Yeah, it was, it was funny. Cause you, you were making a new t-shirt for the online store and you sent it over to me and I was like, I don't, I don't get it
Kelly Paredes: [00:03:07] And I was just like, I can't believe you don't get it. And then I sent back, it was it's about the ghost. So I was doing some research about, cause I loved Pac-Man. That's probably the only game that I used to play. , back in the days I would go crazy when I would see them at a restaurant or a little.
Bowling alley or something. And I, or like my parents would always take us to KOAs camping all over the world and all over the United States. And we would always see these Pac-Man games. So I would play it all the time. I didn't really appreciate coding as much, obviously back then. But out of the four ghost only Blinky was set to,  follow Pac-Man.
All the other ones had grid-like tracking, I guess, or I don't know, they were like set up in a grid pattern. So Blinky, actually the algorithms that were written for Blinky were to go to the place where Pac-Man had last been. And so as you increase in the level,  Blinky starts going faster, right? And then he reaches a state called Cruise Elroy state.
And that's when , Blinky is now running at the same pace as Pac-Man thus making it harder to not get eaten by or stopped by the ghost. So I thought it was pretty cool.
Sean Tibor: [00:04:28] There's no real blocks or anything like that. There's no way to slow him down. He moves at the same pace you do. So you have to think about ways that you can potentially outsmart them. So really good find I had no idea that was the case. I mean, I remember playing a lot of Pac-Man when I was a kid.
And I remember that there were even strategy guides for Pac-Man, right? Like here's all the tips and tricks for how to get really good at Pac-Man. And the only one that ever stuck with me was to move in the direction that you want to turn before you get to the corner so that you don't stop, you know, that you kind of move  .
Into the corners quickly and smoothly. And I just, uh, I never really quite got the hang of it, but it's definitely a classic game. Yeah.
Kelly Paredes: [00:05:03] I think like back in the time, I can't remember the, the guy who made it. Um, I was looking at Tor Ru I would Tawny, but he was, he was a legend for the way that they set it up with this  pixel game. So it's pretty cool.
Sean Tibor: [00:05:17] Yeah, there's a lot of stuff, stuff in there that have, I mean, just really clever genius ways of making the most out of the space that you have and the capacity that you have. And so when you tear apart, some of those consoles, there's almost nothing to the hardware by comparison to what we have today.
But now it's like just amazing to see what they could make happen with just the basic microcontrollers and circuits and everything.
Kelly Paredes: [00:05:42] yeah, pretty cool. Plus you can kind of make a Pac-Man ish game on, on a microbit. So now that I know that I can add that little 2 cents in for the sixth graders.
Sean Tibor: [00:05:51] Yeah. And I'm sure it would be hard to come up with something like arcade or something like that  for Pac-Man. So there's a lot of really cool things you can do with retro gaming , in Python  and helping to learn how to program. It's. One of the reasons why we play so much Oregon trail is because it's a great way to learn about loops and all the things that you can do with just an ongoing game loop and see how it works from day to day.
Kelly Paredes: [00:06:09] Yep. Very cool. And so you're one of the  week. What was that?
Sean Tibor: [00:06:13] I have to say it was really just all of the wonderful feedback and everything that we've gotten from the broader Python community around just being on PyBites last week and recording an episode live with Michael and Brian. And it's always just nice to hang out with them and catch up. And they're just really great fun guys to talk to.
 I think the last time we saw them, both in person was at PyCon US in 2019. So hopefully. PyCon 2022, we'll be able to get back together and hang out with them some more. But it's just really nice to do that in the live stream aspect of recording. The episode was pretty cool. So hopefully we'll get a chance to, to make that happen for the teaching Python podcast too.
Kelly Paredes: [00:06:52] Yeah. Like I always have such a great time. I'm talking to. A lot of the Python developers, but laughing a lot with Michael Kennedy. My son, my youngest still thinks it's so cool that I had a recording with Michael Kennedy again.  I don't know why that name stuck, I guess. Cause I listened to his podcasts so much when I was first learning.
So it's always like that's Michael Kennedy? You saw him again. Was he wearing his glasses?
Sean Tibor: [00:07:19] they're basically celebrities,
Kelly Paredes: [00:07:21] They are. So it's pretty cool.
Sean Tibor: [00:07:24] Yeah,  it was a lot of fun.  Hopefully we'll get to do that again soon.  And it was a lot of fun to guest on someone else's podcast, mainly because we don't have to do as much editing after the fact, which is nice, , but uh really a nice time.
And it was fun to do it live. So hopefully we'll be able to do more of that.
Kelly Paredes: [00:07:38] Cool.
Sean Tibor: [00:07:39] So we will get ready into our main topic and then we'll save our mini book review of the new Al swagger book for after the main topic. But this week we wanted to talk about rubrics and assessments and Kelly. I thought this was a really apt topic at this time of year because we're getting towards the end of the year.
 Most students thoughts turn to finals and tests and all these big cumulative assessments. At the end of the year.  I think it comes as no surprise. We view this a little bit differently in our classroom for a number of reasons. But one of the major reasons is because we have a nine week long course.
We don't have this big end of year sort of exam schedule.  We have to find ways to do assessments of student knowledge , in a lot of different points in time. Over the course of the quarter. So it's a really good time for us to talk about how we do that and the ways that we do that. And then some ways that these rubrics and assessments could be used outside of the classroom as well.
Kelly Paredes: [00:08:34] Yeah. I mean,  we know, or students well, and  formatively, we can rate all of our students coding level on a one, two and three, but using the rubric. That the kids can understand and use as a point of reference, I think has been a really cool thing that we've introduced. So yeah, I'm excited about talking about this topic.
You know, me, the nerdiness, my nerd appeal is always with the teaching stuff. So I'm excited to share what we, when we found out about these.
Sean Tibor: [00:09:11] Yeah, it's definitely pretty cool. And what's just to set the stage for it. What is the purpose of assessments from a pedagogy standpoint?  What is  the reason for having assessments in the first place?
Kelly Paredes: [00:09:23] if used correctly, the rubric should guide the student as to what it is that. we as teachers or as a curriculum wants students to achieve it's a place to, , set out right away. The objectives of learning it's gives both of us teachers and students a goal. Saying,  maybe our objective is student must be able to write , a basic function with two key words, that's in a rubric.
So the kids can look at an object and head towards that level, another thing that you can do is it provides an opportunity for some constructive feedback. When you have a specific. Goal or specific criteria, you then have a way to measure above or beyond that criteria. So you are able to give more meaningful feedback.
Oh, well here you did not. And I'm very, I'm simplifying this,  to basic terms, but here you only put one argument, but you didn't set a default.  Keyword argument or something, you're able to give them specific quality feedback in a timely way because it's in a checklist sort of. , and one of the other important things about rubrics is it allows students to actually analyze and grade their own work.
I know when we give the rubrics , we like the kids to go through and highlight. Where or if they have completed that objective, so students are able to pretty much just grade themselves right away , and see where they've met the qualifications. And if they didn't meet the qualifications, they can go back and make sure they do.
So it's like this benefit where they are almost in a no-fail situation.
Sean Tibor: [00:11:11] Yeah. And I think the right way though, too, it's it doesn't make it easier for them to  get a grade. It doesn't like lower the standards, but it makes the standards more transparent and clear to the student to understand what the expectations are for them in order to achieve certain proficiencies.
. And this is something that's very different than when I was a student. So coming back into teaching and seeing how rubrics and assessments are being employed. Now you have a whole range of very traditional.  Things that are very familiar to me.    The teacher may have a rubric, but then you never see it as the student.
Right? So you just get this grade at the end. And the teacher's using it simply to establish like a standard of fairness in their grading, our consistency, even though there are plenty of studies that show that there's still plenty of variation, that's there just to provide consistency in their grading and give them a framework.
But what was. Surprising to me in a very pleasant way was how you could use rubrics to increase that transparency for the students set clear expectations for them in terms of the standards that proficiencies that they're supposed to achieve and then make it so that it's a more of a partnership and learning, 
here's what I expect you to learn. Here's what I expect you to be able to demonstrate for that learning. And then the student can turn it around and say, Okay. Here's what I've done. And then when you come back with the grade for it, if they have any questions about how their work was evaluated, you have specific points within the rubric that you can look at and say, here's where it measured up.
Here's where it exceeded. Here's where it fell short. And the student has a really clear understanding of. Did I get it? And I think that's really what we're trying to get to with us is that student has a sense of, am I getting it? Am I understanding it? Am I achieving what is expected of me? Do I have realistic expectations in terms of the kinds of work that I'm producing as well as the kind of work that's expected?
And it just helps with that whole alignment. Between expectations for the student, the teacher, and then the broader community,  whether that's administration or other  courses , to be able to show here's the fair and transparent way that this has been assessed and evaluated
Kelly Paredes: [00:13:13] Yeah, I think that's so true.  The transparency's where it's at. And I can't remember when I started giving out the rubrics. I know it was early on always when I had large projects, because most of the time we would use rubrics for a presentation or some sort of scientific lab report or such,  Big assessment that we would have  in a science classroom.
So as early on, I used the rubrics, but I think when I really started getting into understanding their transparency is when I was teaching MYP and design technology, because we had almost, it was almost subjective, I think because we had rubrics that said by the end of year five, Of the MYP, the student will be able to do this.
So if you're teaching the first year of MYP, you had to extrapolate, what would it look like for that sixth grader? Who's starting out in design technology and you're aiming for, , where they are going in five years, which was nice, but  a lot of teachers and a lot of it administration , Would have issues with that, but for us in computer science, that just makes sense.
 We know where we want the kids to be before they go into high school. So we are able to already automatically extrapolate,  , first year, second year and third year students in the middle school. So it's a nice thing to see.
Sean Tibor: [00:14:39] Before we go too much deeper, what's establish a baseline of what is a rubric. So Kelly, can you explain it to me? Like, I don't know anything and I've only been a teacher for a few years. What is a rubric, what is a rubric? What makes it a rubric, . Versus other alternative assessments that can be employed.
Kelly Paredes: [00:14:58] Okay. So it's funny because a lot of people, a lot of teachers have a love hate. Relationship with rubrics more or less,  rubrics have, and depending on what type of rubric you're using,  we use a different kind and we'll talk about that later, but most of the time a rubric is a set of objectives, either bullet point objectives or single objectives and, , a level of completion or a level of accomplishment.
For each one of those objectives. So some people would have , one, two, three, four, four able to write PEP8 or follow PEP8guidelines and a four would be amazing. You're a developer and a one might be a newbie. So there's a huge gap. And a rubric just sets out, , almost  like a Likert scale, maybe.
Level does that sound about right. A Likert scale level of , where that child or coder has met or that student has met in there in their learning paths. So that is , the basic form of a rubric. It's a table Scott criteria down the side, and a level of achievement going across. Does that sum it up?
Sean Tibor: [00:16:16] yeah, I mean, that sounds familiar and what's interesting is, My first experience using rubrics was not in an education setting at all, but it was actually working in a corporate environment using them as scorecards for evaluating different vendors. So whether it was a technology vendor or I was actually working in the marketing space for a bit, we would use.
 Rubrics to evaluate different proposals from vendors, to be able to see how they matched up and to be able to clearly and logically evaluate each of them on the merits of their proposal, both the tangible and intangible things. So you could really get to a better score for each of them.  So the grade that they got was how close do we believe that they're going to fit the. Outcomes or the goals of the RFP or the evaluation that we're doing. And the critical part for that was to have the rubric before you started interviewing suppliers.  So before you started looking at vendors, you needed to have your success criteria established to be able to decide who got the work and who got awarded the contract.
But this gave us a really clear understanding of what they could do against our original goals. And of course it wasn't set in stone.  We could still, if we needed to , change the rubric later based on new information, but it was once I came into teaching that I saw the value of having that same sort of system in an education context and how we were able to use that for evaluating students in a fair and consistent way, and also giving them a clear set of guidelines in terms of what their, what was expected of them in order to get the grade that they wanted.
Kelly Paredes: [00:17:51] So using it to evaluate,  other marketing or other businesses is exactly the same way that we would use to evaluate students, except for the fact that we give the rubrics to the students. I think it's really important that the students have the rubric. Now, this is where a lot of problems come with teachers wanting to use rubrics because they are quite labor intensive. You have to know specifically what you are trying to get out of the assignment, just like when you were looking for, , Or evaluating a client or , a business of such you knew exactly what you wanted , from that business, you knew that you needed to have , certain costs, a certain timeframe or whatever.
And so you were able to build out that rubric prior that takes a lot of work. And sometimes if these rubrics aren't built right, then either the students get all A's are not really judged. Critically, I should say, , not,  punitively, but critically enough to give proper feedback or they're too hard and the kids can achieve the goals.
So I think that's,  one of the hard things about writing a quality rubric, and that's why we've done our rubric a little differently.
Sean Tibor: [00:19:05] Yeah, you really do have to have a vision of what a good project looks like or a good piece of work looks like you can't just after the fact, look at it and say, well, I've got 20 different submissions here and I'm going to take the best five and you're right back into being subjective again.
So the rubric is there to give structure at least to that subjectivity and make it more objective in the analysis and evaluation.
Kelly Paredes: [00:19:31] yeah, while not. Like hindering the creativity. So there's like that fine line between being , too prescriptive. , this is how you're going to do it. You're gonna put this number of lines. You're gonna do this many variables or whatever. When you do that, then you've actually just had regurgitation of what you said.
So it's a really fine line between creativity and,  objectivity, I guess that's the word I was looking for.
Sean Tibor: [00:19:57] Yeah, I think that's right. I think that's right. So, , let's talk a little bit about how we use it for Python programming. So how we use  rubrics to evaluate code and to make sure that. , we're able to take something that can be wildly different. Two different students code submissions are often as unique as their fingerprints.
 So , how do we use these rubrics to evaluate their work?
Kelly Paredes: [00:20:19] Yes. So we actually have used, what's called a single point rubric. Now  I was just trying to Google her name. I can't remember her name from the cult of pedagogy. She's really brought these back into mainframe. She's written a couple of articles, Jennifer Gonzales. She's a cool person to follow.
So if you are an educator and you want to follow someone, cool, Jennifer Gonzales is great, but she brought them back into light. Now she didn't. Make up the single point rubric. , this was actually done in back in 2000 by a person named Mary Dietz.  There was a huge study on the effectiveness of a single point rubric.
And what I mean by a single point rubric is we set these goals in the middle. But we haven't done the task of saying, this is what a below level, and this is what above level looks like. We've set the standard. This is what I want you to know to achieve. And anything above and beyond, you are going to be able to identify that.
And so a single point, rubrics, easy to write, easier to write, I should say. , and I think. A lot less information that will overwhelm a student.  
Sean Tibor: [00:21:33] Let's give a concrete Python example for this. So let's say we have.  An assignment where we're really evaluating, or at least one aspect of what we're evaluating is their ability to use lists, for example. the middle of the road example might be, they can create the list. They can append things to the West. They can pop from the West and maybe they can access an element by its index. And the West as well.  So that would be like your single point. Here are the criteria for what we expect someone to do in this area to get the meets expectations or the middle of the road score for this , and that would be the only criteria.
So the above that and the below that would still be blank and you can look at it and say, well, what more have they done with it? Or what less have they done with it? . Because we omit the above and below, it gives the students the opportunity to extend in a variety of different directions here.
Whereas if we were very prescriptive about it, we might say, wow, you know, someone who exceeds expectations would also be able to use lists slicing, and they might be able to use some other list methods. They might use the itertools module to iterate over things. But then when we put all those things into that exceeds category, so that upper box like  that shows that what we've really done instead of giving them.
Opportunities to grow. We've given them another list of things to accomplish. We've boxed them in to doing what we think should be exceeds expectations instead of going way beyond that and finding something really exciting for them that actually does exceed expectations.
Kelly Paredes: [00:23:01] absolutely.  And we actually, we put in for data types, we made it even more generic. And we're still always tweaking this. So we're not saying we have the,  ready to sell copies. Although, , we might have that on the teaching Python store later. No, I'm just kidding.  But we're still working on editing this and I'm trying to make a generic one for sixth grade.
We've made the one for seventh grade. Cause I think seventh grade at least is easier. I don't know why, but it's easier to start in the middle. We know what a middle year child can do. And then the sixth graders, eighth graders are in the other side. So for example, Data types, the student uses correct data types and understands the reason why they've used  that type.
So why would they use a list over a dictionary or,  a string over an integer and they can justify it. So maybe they've chosen to use a list instead of a dictionary. And we're like, dictionary is probably a better choice. Will they say they can say yes, but with this list, I want it to do a, B and C so they can explain why that data type is chosen.
And then that allows for the flexibility of that code to happen because of student choice.  So it's pretty cool.
Sean Tibor: [00:24:22] No, I really like it. What I see is,  the students that want to just do enough because maybe it's not their best class, or maybe it's not the one that they're most excited about, but they still want to get a good grade this way. They at least know what that entails. They know what they need to do in order to get the grade that they want.
My students that love this. Stuff. They just love the course. They'd love Python. They love programming. They see this and they say, Oh, that's all you want me to do. Let me go do  four other things, because I'm really excited about this and everything. And then the kids who are falling behind or who are struggling to know what is expected of them maybe they don't get everything, but at least they got a few more areas improved because they knew what they were shooting for.
They knew what to strive for instead of just missing. Okay.
Kelly Paredes: [00:25:08] Yeah, and I think it was really cool. So we were actually starting to build in  , and I was giving some thought about this for the sixth grade, the pet pet. Roles at documentations, huge,  , and going through the documentation for a sixth grader, seventh grader, eighth grader, what is it that we want them to know in order to have nice code?
Well, in Mo it always will tell them up, talk about the white space , or the, the use not use of the white space. And you can use the tidy button. So for maybe for PEP8, the student  uses basic PEP8 rules. We're talking,  the four spaces, the formatting of the code blocks, the line length that doesn't exceed 79 characters, clear and meaningful variables, right?
That's maybe like a sixth grader doing that, but then the sixth grader might say, Oh, I've gone above and beyond because I started to learn basic functions. And I now know that I have to do two lines between each function and the documentation has to, or, , Whatever doc, doc strings have to be inside that function underneath whatever, you know, I'm pushing, trying to look for stuff, but that's what we're playing with.
Sean Tibor: [00:26:17] Right. And there's, there's plenty of things there. And I liked this idea of having a generic rubric as a starting point, because if you think about your assignments, you're giving a lot of these rules for the rubric are going to be the same from assignment to assignment. So if you have 80% of your rubric has already completed in a generic fashion, it means that you, as the teacher can extend the rubric to add an additional 20% that may be assignment specific or project specific or something like that.
And this means that you get that still, that great level of customization and adaptability to the assignment at hand, but you're also not reinventing the wheel all the time. You've got something that was designed to work and cover the 80% of the assignment and the right 80%. It's not just the like, Oh, here's all the boring stuff that Mr.
Tibor asks for every time. It's no, these are the basic expectations. We expect you to have good style. We expect you to , have a program that actually works and functions and is bug free.  Those are things that we can put in there that should be the base expectations for every assignment that's turned in.
Kelly Paredes: [00:27:18] Yeah. And , that's why we wrote for the seventh grade. And that's why we Guinea pigged them for this year. And we're still, like I said, still tweaking it, but the cool thing about this , and. And why? I think like it works for us is that, that evidence there's actually, in our single point rubric, we have a four columns.
We have N not yet , here's our expectations. Then we have an evidence. And so not only are the kids saying, Oh yeah, I've done this, but now they need to go in and say, how did I meet that standard? Oh, I did that on online. Give me an example. I did this online line for, I did this online 100 and if they go above and beyond say again, using generic PEP8, maybe they did something extra and they went to the documentation and they actually read the documentation.
Maybe they had been doing it  , all along through the coding, but they actually went to documentation and found that specific. , they went to go research something on their own and learn about it. That's that area where they've gone beyond the basics. And we've added in that.
 I joke around 0.5 to one point extra for the evidence because we can't give them over a hundred. That's just silly, but , we could, in theory, if we wanted to
Sean Tibor: [00:28:35] Well, you know, the points don't really matter,
Kelly Paredes: [00:28:36] yeah. Points don't really matter.
Sean Tibor: [00:28:38] points.
Kelly Paredes: [00:28:39] 1 million points.
Sean Tibor: [00:28:40] like whose line it is anyways.  Welcome to middle school where the points are made up and the learning is real.  That's the whole goal is like  the points at the end of the day are there to act as a signal it's just there to signal to the student.
How much of the. Expectations did they achieve? And we want to give them some room to see that , they,  went past what was expected of them. And in our school, we do have this cap on the final grade, where once they hit a hundred percent, they can't go any higher. So there is some limits to what they can do.
Although there are plenty of students who secretly know that they got 107, even if the report card says a hundred, .
Kelly Paredes: [00:29:18] Yeah, it's  quite funny. I think eventually we're going to have to raise our expectations. We already have pretty  high standards, but the kids keep meeting them. It's just amazing what they can do and having these rubrics to show them at the end of the day, we use this mostly at the end of the course this year.
 We're thinking about using it for other projects to start teaching, but I don't want to teach to the rubric. I kind of have fun , making it up as we go now,  
Other Applications for Rubrics [00:29:42]
Sean Tibor: [00:29:42] the other thing that's really important here, and this is the part , where I believe it goes beyond kind of the teaching pedagogy and the nerdiness of like education and learning and into this, just being a really useful tool. It does not have to be limited to education.
You can use this for things like. Vendor evaluations or suppliers or whatever you want in a business world. You can also use it with junior developers. So if you're a senior developer mentoring, a junior developer, if you go through a rubric with them  of their code and do a code review with them and talk about the different things that they're doing in their code, it helps them see where they have.
Are doing very well, where they have opportunities to improve this sort of scorecard rubric evaluation. If it's done in the right way with a level of trust and understanding that this is there to give structure and context to the growth that they are undergoing a rubric could be very great for that mentoring relationship as well.
But you can use it outside of work,  it doesn't have to be work or teaching or anything like that. And you can use it when you're buying a car,   if you want to buy a car, you can make a rubric for here are all the things that I want in my car. And it helps you structure it. Any sort of complex decision with multiple dimensions or variables into a set of.
Clearly defined success criteria so that , what each factor means to you. You can even go as far as putting weights on it, that sort of thing. So that you get to a really clear understanding of this is the right decision to make. So it's interesting when you step even further back from rubrics, not just as.
A teaching tool or not just as a form of assessment, there are decision support system. They're there to help you make better decisions. And in this case for teaching, we use them to make better decisions about what grade a student should get. And the student can use it to make better decisions about what they're going to learn and where they're going to put their effort into the coding.
But there are decision support tool that you can use for just about anything where there's a fair amount of complexity and ambiguity that you need to be able to clearly.  Work through and come to , a well-supported decision.
Kelly Paredes: [00:31:42] Yeah. Can you imagine,  when, instead of those job criteria where they, , tell you, no, you have to have this many years of work or whatever, it may give you a rubric. And instead of just showing your project on, GitHub or whatever you've made in the past, that could have been helped by. Whoever they give you a rubric and you say, okay, here's your rubric.
You have to complete this to be a junior, whatever, a junior developer or front end designer. I am sure they have something like that out there in the workforce. I would hope so. I know they have that  when new people that come in to teach , I add a rubric when I graded you.
Did he crawl on the floor during the lockdown? Yes. Check higher.
Sean Tibor: [00:32:22] Five bonus points for that.
Kelly Paredes: [00:32:24] I want to say one last thing. , about types of rubrics that we F we forgot to mention. I found this really cool, , rubric, very convoluted must have been a math teacher  , I put the link up on their show notes, but it must have been from a professor of math or something who, who dealt with this, but it was the a 2% amazing category.
And they had this percentage is you got . 40% for this, something else for this. And then the amazing category was the 2%. And so you still had that ability to , wow. Somebody actually, I think it was a, programming rubric from did you pull it up?
Sean Tibor: [00:33:04] yeah, I pulled it up on the screen here, so it's actually a, a weighted rubric. So they have a weighting with it. They put in a written component with correctness. So there was just one single point there, but the programming  component of it was program correctness was 55%. Readability was 23%.
Documentation was 20% and  amazing was 2%. So if you did everything they expected from you and did it perfectly, you could get a 98 on the assignment, but to get a hundred, you have to find something amazing to implement. And I liked this challenge. Because it's not just for the students that always have to get a hundred,  the perfectionist out there it's for everyone.
Who's like, wait, hold on. Yeah, red bull. I'm going to code this because I know I can do something amazing here and they really get excited and into it and they get rewarded for doing so. So maybe they're not the ones who go from the 98 to a hundred, but they might go from an 88 to a 900 because they found something amazing that they could implement.
Kelly Paredes: [00:34:05] I think it's from Northeastern , university, Cory college of computer sciences. So we'll put that in the show notes. I thought that was interesting. I don't think I can do all. I have time to do all the math that they did, but Hey, there you go. There's another example for you
Sean Tibor: [00:34:20] there's this really cool programming language that might be able to help you with that.
Kelly Paredes: [00:34:23] Automating it
Sean Tibor: [00:34:26] Yep. Yup. You can do the calculations type in the scores and have it, have it all calculated. So, I mean, that's maybe the other thing to do too, is give a student an assignment where the assignment is to create a rubric program that they can put in the different weights and the different criteria, and they have to code it with Python.
So maybe there's some ways that you can make the rubric, the assignment and not just the evaluation tool.
Kelly Paredes: [00:34:47] that would be  , that's another cool thing that you can even have the kids, if you want, you know, maybe you have some middle schoolers or some high schoolers and you can say, what do you need in order to make a program that does X and then they need to build out a rubric of criteria. So you can have a whole bunch of kids pulled together and write a rubric, which is always a nice thing to do.
We used to do that with MYP. What did you need to make an app? What would make an app be the number one app? What would you do for , architectural design? What would make it high level building or et cetera. And you have the kids build out the criteria. And as you're doing that, you're having a conversation about the objectives and you combine all the kids.
It's a lot of work, but it's a , really effective rubric building. So another fun thing.
Sean Tibor: [00:35:37] Well, I have to say this is probably. A deeper dive into rubrics than I thought I would be doing five years ago before I became a teacher. But you've been a good sport about how, like going into Docker containers with me and talking about all kinds of really geeky tech stuff. So I love going into the geeky teacher stuff with you.
Kelly Paredes: [00:35:57] How you said, yeah, this is going to be a short recording, like 20, 30 minutes. And I, you know, you put me in talking about rubrics and I'm like, Oh yes. What else can we talk about? But I'll stop there. We won't go any further.
Sean Tibor: [00:36:06] Well, speaking of what else we can talk about , what's do a little mini book review on the new Al Sweigert book. That's coming out. We happen to get an advanced copy of it. It's called the big book of small Python projects and it looks like it's coming out. I believe in April, although I wouldn't quote me on that.
It's going to be coming out soon and you can pre-order it now on no starch presses website. I think they also have a pre-order available on Amazon as well. So you can get started on it, but the, uh, Kelly, why don't you tell us about the premise of this book? Like, what is that this book and who is it geared towards?
Kelly Paredes: [00:36:40] Oh, who's it geared towards now. You got me stumped. Um, I think it's all beginners. It's pretty, not beginners, but maybe intermediate it's definitely. I would not have picked it up at first because it doesn't really walk through the code. , for me it seems like a really good. Teaching book for teachers because of the, , the questions that are offered in the book.
I'm just trying to pull up the book as I go. But  the whole concept for me is that after you've gone through these projects, Al puts in a little bit of thought questions there and Guide you in ways on how you can explore the program further. So I just think it's a really neat concept of how do you get those questions and those conversations going with maybe students you're working with.
Sean Tibor: [00:37:35] Yeah. And what I like about this? the book itself has  81 projects in there and they, he may sneak a few more in, before it actually gets published. We got. This advanced copy of it. But it's coming up to like 394, 400 pages of Python projects. And the goal of this is to give you a lot of different project examples.
So you can see different ways of implementing things. , I agree with you. It's not really designed for beginners. It's not necessarily a book  that teaches you through code the way that he does with automate the boring stuff or invent your own computer games with Python has two previous books.
But in this case, it's more like here's a program. Here's what it does. Here's the source code. And then here's some questions to help you understand what it's doing. So It doesn't break programs down into chunks, as much as the beginner books, but it definitely gives you a lot of code that if you've completed those beginner books and you want to look through it in a deeper way or in a different way to learn ideas, it works pretty well.
So yeah.  For me, when I was looking through this, there were a couple that stuck out to me as projects that we've done in class. for example, there's a blackjack game in here, so we do a lot with making blackjack in the classroom. But if you look at the way that he's got it set up, there's a lot of great ideas in here that students could pull from.
So he has things like how we do the move. Calculations are how we determine the players moves. How do we play the cards? How do we represent the cards on the screen? All of these different things that have a student made their own basic version of blackjack, they have an example problem that they can look at and see the solution in a different way than that.
They came up with to be able to see some of those new ideas and see ways they can make it work. , 
Kelly Paredes: [00:39:17] Al has the Mondrian art generator. And I love coding the, giving the kids, the challenge of doing this and,  the turtle library. And it, we teach only basic functions on how to make a square and how to make a rectangle. And we use a couple of parameters with color and position, but he has it coded with, and I'm just trying to flip through this,  With, , system import try.
I'm looking for this old things. Yeah. Just in basic kind of basic kind of code. I'm assuming where would this come out? Oh, it's a pretty lengthy one. It was one of the longer ones. It's 184 lines of code. We'll see. But , yeah, it's just driving out on the canvas. It's not using turtle. It's using canvas points in length is kind of how it, how he has these questions at the end.
What error happens if you do this? What happens if you change it? So again like the, Oh, it's in pillow. There it is. , Use the pillow module, which I never used before. So Hey, learn something, try something new. But I think I like it. I like how it has just these little snippets. I know some of these  are not new games are not new projects, but it's the co the questions at the end of the projects that make you think  
Sean Tibor: [00:40:29] so I've been using this with my eighth grade students. We went through and just grabbed the bagels program at the beginning. So it's a game where you're trying to guess a three digit number and you get clues based on what you guess, whether you have the numbers right or wrong or in the wrong place or whatever.
And it, what I like about is the way that he introduces it, he sets it up and then just. Like what's the program speak for itself. There's good comments in there.  So you can follow along with what's happening, but there's also the level. We're not, everything is explained. So you still have to look at it and try to understand what is this line do.
 One of the things that we're doing as we go through it with my students, Is to go through it line by line and keep asking ourselves that question. Why is this line here? What does it do? Is there a way that we could do this better?  And so that idea of questioning every line as you go through, it is really critical for the learning.
And I remind them that like what the source code for this is all out there. You could copy this right from the book or from the get hub page or whatever, and paste it into a, an editor. But then you don't actually get that process of going through and trying to understand what the code is doing. Now. One of the things that's, you know, I think maybe.
Good and bad is that, you know, I noticed that there's a lot of things that are different in terms of the approach than what I've been teaching in the classroom. And I'll give you an example, you know, in the code here, a lot of these examples use a string dot format method. To insert values into the string.
And I've been really trying to get most of my students to use F strings for their programs. Now I know that the dot format method is more compatible. There's a lot of good reasons for that. So a little bit longer lived, but the F strings have been something that we've been teaching in the classroom. So when we encounter it, it's a little bit foreign to the students, but that we also get the opportunity to be able to say, you know, Hey, whenever you see a dot format, Method, you can replace that with an F string approach.
And here's how we would do that. So they are engaging with the code more as a result of this, because they're then modifying the code to fit something that they're more familiar with and getting it to output the same thing.
Kelly Paredes: [00:42:29] Yeah. And then my, my, I guess it's not an issue, but it's a more of a, a sixth grade level issue of trying to get some of these ASCII art. Programs up in Mu is a little bit difficult. Sometimes they just won't run or it won't take the library. I am not sure on the technical, I'm sure you can explain that.
 They just won't run out the cool app or the cool artwork, for example, there's like the bitmap one, which is really cool that I w I put in there and I was, I was like, Oh, I'm going to use this one. And it just comes out in one single column and move editor and then trying to get the kids and vs code or pie charm is not my primary goal.
And  , it doesn't run in replica.  , those are just a little bit of the sad things, but I am going to do it myself because I thought that was cool with the bitmap. And then the hang man has a couple, so lots of visuals, which is. A hundred percent awesome for sixth graders, but it doesn't run that well in mu yet.
Sean Tibor: [00:43:31] Yeah. And, and the visual learners, this could definitely be something that you could grab before doing a hundred days of code learning process. Right. So if you're doing a hundred days of code and you've got a, a system or a methodology that you're following for that. That's that's great. But if you also had this, if you were struggling for, what am I going to code today, this would be something great to, you know, grab one of these programs that looks like it's interesting.
Follow along with it, make your modifications, use the extension questions that he creates at the end of those, so that you get a really solid like hour or two long project out of each of these. Um, they're not that long by themselves, but it's that process of going through line by line, understanding it.
And then doing the modification where you get the most valuable learning experience.
Kelly Paredes: [00:44:17] Yeah.  Just one last thing I promise. , what I really like about Al's code is that. I can understand it as I read it. It it's very clearly written. There's not a lot of hidden , Lambda or  list comprehensions. It's very clear code  for the beginner. So that's my 2 cents.
I can read it. I can read it and I don't get scared. Okay.
Sean Tibor: [00:44:43] Yeah, there, there's definitely a lot of choices that are intentionally made to make the code more accessible for beginners. And for people who may be only learned a few things like, yes, you could use a list comprehension and one of this, one of the places, and it would be a single line bit of code, but maybe doing it, it is a more traditional loop is a better way to do it for someone who's only been coding for a few weeks or has needs that ability to quickly get in and understand the code and make it accessible to them.  there's very clearly a lot of choices that have been made in that direction. That's where you can really see that come through in the writing, both of  the code and the nod code sections of it is that this is really  almost like a love letter to those beginner programmers. Like here's a bunch of really fun programs that you can write and they're not all games and they're not all art, but there's always something interesting about each one of those that you can take away from it and using your own programs.
Kelly Paredes: [00:45:35] Yep. And they also have this calendar maker. So I think we're going to look at that when we do our schedule app, that'd be a cool little add in here. Throw that in and see what if we can map them together. That
Sean Tibor: [00:45:45] that one's really fun. That one's really fun. So I started coding that one, I think, most of the way through it today. And it's been really nice because yes, there are calendar programs are lots of different ways that you can do this, but he's doing it all with the date time module. So you get to think a lot about those.
How do I use the dates the days of the weeks? How do I get all of these things that we need to be able to make it work?  Then can I create a visual calendar out of that? And most of it is all ASCII art, right? So the calendar that gets created, there's a text output of it and it's all lines and boxes using ASCII art.
And it's pretty cool.
Kelly Paredes: [00:46:19] Yeah, and I just, that was a nice little code to bring into tuples and teach them why it's a tool. It's kind of nice.
Sean Tibor: [00:46:26] Yup.
Kelly Paredes: [00:46:27] Cool.
Sean Tibor: [00:46:28] So lots of little nuggets and snippets in here. I think it's a great companion book to automate the boring stuff or invent your own computer games with Python. If you want just some, almost like a little smorgasbord of ideas to play with. . That's what it's there for.
Kelly Paredes: [00:46:40] Yeah, it's gonna be a nice compliment because I am still back into finishing my, automate the boring stuff video course, because  , the book wasn't enough for me.
Sean Tibor: [00:46:50] You can always learn it more. Always learn more. , so that's the new book coming out from Al swagger. Hopefully within the next few weeks you can pre-order it now on no starch press. And I believe on Amazon, the book is called the big book of small Python projects and we look forward to seeing it hit the shelves soon.
Okay. Kelly, I think that kind of wraps it up for us. We've ah, We've gone a little bit longer to topics this week instead of one, which is  a fun thing.  We're going to try to bring in more of these book reviews. Cause we do have more books heading our shelves. Hopefully we'll get a copy of Python basics reviewed for everyone as well, so that you can see that.
Anything else you want to share with everyone, any other upcoming events to think about any new products in the teaching Python store?
Kelly Paredes: [00:47:31] um, Oh, well, new products, I think that's my new hobby. I am so addicted to building t-shirts and we have a, we maybe it's not. As clever as everyone thinks, but we, I think we're clever. We come up with stuff on it in the morning at seven in the morning, seven 30. But, yes, my, my favorite cup is the power of yet.
We have a coffee cup. It's so nice to drink out. You get the reminder to say this. I think for me, it's the fact of always adding in that word yet to the kit for the students when they say can't code. And I'm just like yet, and with the power of yet , my coffee cup sitting on the desk, they don't have a choice and they  roll their eyes.
Sean Tibor: [00:48:14] It's a really good one. And we're going to have more of these coming out. I think we're doing the same sort of thing that we do with like content creation or with , my approach to photography is we're going to create a lot of things and see what people like and the things that people like.
We'll keep things that they don't like we're going to get rid of. And we'll just keep making more stuff and having fun with it. So kind of this podcast brought to you by us, it's  supported by the teaching Python store. And we'll put a link to that in the show notes.
Kelly Paredes: [00:48:40] Yeah, we don't need John to lose his hair before with all the editing.
Sean Tibor: [00:48:46] That's true.
Kelly Paredes: [00:48:47] So we need to pay for that and maybe get a beer.
Sean Tibor: [00:48:53] possibly, possibly. So we're also looking forward. We're going to try to be at the PI con U S education summit virtually in may and if you're able to join in on that'd be great. We do have a lot of new, uh, social media platforms to connect with us on, we have a Facebook page.
Now. I think we have an Instagram page all set up so that we could connect our store to it, but we're going to start publishing more of our content to those mediums as well. So if you'd like to reach  out to us the best way is still through the teaching Python website. That's teaching python.fm or through Twitter at teaching Python, but we're now on Facebook as teaching Python.
We're on Instagram as teaching Python. Kelly, you
Kelly Paredes: [00:49:32] Yes, and yes, and we are going to be presenting at the innovation Institute. There are still a couple spaces available. Last minute spaces you have to get on quick. If you're interested, we will be using our microbits to do a wild about robots presentation, but the boxes that are going out are worth the cost of the Institute.
So even if you can only attend our session, the box of stuff, you get. Bach is just great. And then all this stuff will be on recording. If you join the Institute,
Sean Tibor: [00:50:06] Yeah, I don't know if I'm supposed to say this out loud, but I can't see how we're not losing money on this because the box has so many really cool things and it like you in our presentation alone, you get a microbit, you get servers, you get, Oh, electronics, you get 3d printed parts. You get really cool stickers for your laptop.
Um,
Kelly Paredes: [00:50:23] a breadboard.
Sean Tibor: [00:50:24] of the sessions. A breadboard, right? So there's just all this stuff that's coming your way and all kinds of really cool things that you're going to be able to play with and learn about coming in the box. So if you want one of those boxes and you want to attend the innovation Institute, we'll put the link to that in the show notes as well.
Tell them, when you sign up, the teaching Python sent you because our boss  comes gleefully, running across campus to tell us about it. Every time someone signs up through teaching Python and we're excited to have more of our listeners attend as well. I think the cost is fairly reasonable since it is virtual this year.
I think it's a little over a hundred dollars,  for  the three-day session. So it's definitely worth looking into, and you probably will get as much
Kelly Paredes: [00:51:04] think you get about $150 worth of stuff for $150. We're working for free again. That's why we have a shop to support us.
Sean Tibor: [00:51:14] Exactly. Exactly. So that's the next big conference that we'll be at? It's the pine crest innovation Institute look forward in the show notes. I think that pretty much does it, there was something else, some other social media site that I signed up for that I was like, I got to tell Kelly about this on the air, because she'll laugh at me for it, but, , Oh, I'm on clubhouse
Kelly Paredes: [00:51:32] Oh, co clubhouse. Yes.
Sean Tibor: [00:51:34] Yes. The new social media network. I'm @smTibor on clubhouse.  I was on the,  Python weekly hangout on Friday night for a little bit of time. And it was
Kelly Paredes: [00:51:43] At 9:00 PM. I was asleep.
Sean Tibor: [00:51:45] Yeah. That was the only downside I would have asked you to join too, but I knew you were already unconscious,
Kelly Paredes: [00:51:50] I did get an invite, but I did not join. I am also on it and I am @kellypared.
Sean Tibor: [00:51:56] Well, so that's our show for the week. Lots of pedagogy. Lots of new books, lots of social media, lots of good things happening in the Python space. We look forward to hearing from you and thank you all for your support for teaching Python. This is Sean.
Kelly Paredes: [00:52:10] and this is Kelly signing off.