0:00  
Hello, and thank you for listening to the mathematics teacher educator journal podcast. The mathematics teacher educator journal is co sponsored by the Association of mathematics teacher educators, and the National Council of Teachers of Mathematics. My name is Eva Anheuser and I'm talking with Kate melhuish, who is an assistant professor of mathematics education in the Department of Mathematics at Texas State University. Today is a special day because we're going to discuss Kate's and my article, the student discourse, observation tool, supporting teachers and noticing, justifying and generalizing published in the march 2019 issue of the mathematics teacher educator journal. We will begin by summarizing the main points of the article and discuss more in depth lessons we shared in the article our successes and challenges, and how these lessons relate to our other work. Kate, thank you for joining us.

0:54  
Thank you for having me.

0:55  
All right. So let's get started by just giving a brief summary of the article including the results.

1:01  
Yeah, so a lot of the work that we do, and this is actually in conjunction with Development Group, which is a professional development provider based out of Portland really focuses on this idea of promoting student centered classrooms where students make sense of mathematics through a continuous focus on justifying and generalizing. And of course, this work, we really found that promoting, justifying and generalizing is completely a non trivial task. And in order for teachers to burn the set of activities, they need to be able to understand them and also notice them as students are actually engaging and justifying and generalizing. So this article really focuses on one particular tool that's was developed to kind of aid in the understanding and noticing of justifying and generalizing in the classroom. And that's the student discourse observation tool. analysis that we did was really focused on looking at a group of teachers within a larger professional development, advocacy study, and analyzing how they were looking at justifying and generalizing in their classrooms, and particularly how the student discourse observation tool could be leveraged to help support teachers in developing their noticing and understanding of these constructs. And the paper in particular, we document some of the aspects of really rich conversations that teachers had around these, these ideas of justifying and generalizing after classroom observations and provide some empirical results that teachers who work with this tool extensively have provided more robust descriptions of justifying and generalizing and generally we're able to characterize student discourse, written student discourse with a greater degree of accuracy in these two categories.

2:33  
So I'm imagining being a listener on this podcast and not having seen the innovation. Could you briefly describe what this tool looks like?

2:42  
Yeah, absolutely. So the tool itself is a paper tool. And the idea is that it provides a place for teachers to scribe down some of the things that they hear students say and do when they're observing classrooms. And it's within a couple of categories. So you can take notes on what you're actually hearing, and then categorize it in terms of Am I hearing things that are using procedures and facts? Are they justifying? Or are the students engaged in generalizing, and addition to having the area to kind of scribe these down, there's also some rich descriptions of what these activities are at the top of the tool was really the focus on the nature of the activity itself. So it's not a matter of trying to decide whether something is a complete or correct justification or a complete or correct generalization? But really focusing on the nature of what students are doing our students really exploring that mathematical, why are they connecting to meaning? Or are they leveraging procedures and facts? Or are they thinking about moving beyond specific cases to build conjectures and arguments for general cases?

3:45  
And so just specific definitions are on the tool and also described in the article? Correct?

3:50  
Correct? Yeah. So it serves not just as a place to write down notes. But there's also this kind of reference to what we mean by these greater argumentation activities.

3:58  
So before we jump into the locker part, could you really briefly summarize the main results that we found?

4:05  
Yeah, so the, the main results here actually come from just a small survey study, we had in, I guess, a large urban school district that was participating in a larger professional development, or one of the big things that was focused on was this idea of understanding, justifying and generalizing and being able to notice that in students discourse, so we developed a survey that contained I think, seven pieces of sample writing of students in response to a mathematical prompt. And some of them were students engaged with mathematical structure and providing justification, some generalization and some were just these different iterations of using procedures and facts. And we asked teachers and this was I guess, there was a stratified random sample of the schools. So the teachers who have participated in the extended professional development characterized each of these pieces of student discourse, and then the group of teachers who didn't spend extended time with the student. discourse observation tool also categorized these pieces of discourse. So we compare it across to see if teachers who have worked with a tool, were more likely to characterize them in ways that was consistent with how we characterize them in terms of justifying and generalizing. And then we also looked at responses to an open ended prompt about what they think justifying and generalizing looks like in their classrooms. And we're able to document that the teachers who spent more time with the student discourse observation tool, were able to provide richer and more accurate descriptions of justifying and generalizing in their classrooms.

5:32  
Okay, I'm gonna get to why we care about this later. But let's kind of move on to who is the intended audience for this paper, obviously, everybody.

5:45  
I mean, I think folks who work in service, professional development are probably likely to gain the most particularly if they're working within schools and have a means for observing classrooms, because this is really very much a observation tool, a tool that teachers can use when they're going into each other's classrooms. We also in the article provide a lot of some protocols and questions that could be used to really orchestrate debrief discussions afterwards. So in that sense, it's a very sort of practical tool for people doing some of that work in classrooms. I also think researchers who are interested in how teachers think about generalizing justifying would find this article interesting. We do spend some time kind of categorizing the ways that the teachers were talking about these things in their classrooms. And then there's some new results, for example, about how teachers are thinking about what generalizing is in the elementary level.

6:32  
So would you say that this tool could be useful in elementary school or possibly all the way through college

6:39  
there? Well, we studied it in particular, the it is third to fifth grade teachers in classrooms that this was occurring in the tool itself, which was initially developed by to just develop in group has been used it definitely at the K 12 level to a lot of success in classrooms. As far as the undergraduate level, I'd say that's more of an open question. Okay.

6:58  
So let's talk a little bit about what is the important problem that is addressed with this tool and with the paper.

7:07  
So I think the main, the main problem that we were thinking about is that in order for teachers to vote justifying and generalizing in their classrooms, they need to be able to notice it. And understanding what we mean mathematically by generalizing and justifying can be really difficult with mathematical argumentation comes with a whole different set of norms, there may be other ways we use this language. And being able to actually then identify students engaged in it is a pretty complex task. So this the ESA tool, and this paper was really focused on what is something that we can use to really support teachers in developing their their noticing of justifying and generalizing

7:43  
Yeah, I think that you and I had noticed when we talked with teachers that sometimes when we use the word justification, it becomes very clear that what we mean by it isn't shared by the teachers and vice versa. So would you say that this tool helps kind of get a common ground on what we mean by justification? Yeah,

8:04  
it definitely provides a means to having sort of the shared understanding around justifying and generalizing, and particularly when it's partnered with debrief discussions afterwards, where you can really dig into pieces of actual civil discourse and make sense of whether what the nature of the reasoning is involved.

8:20  
And there is some examples in the paper, right?

8:23  
Yes, yeah. So there are a lot of examples of, and we we tried to share examples that weren't just standard complete justifications or generalizations, but activity that might be classified as justifying that are incomplete, or even maybe something that would be considered mathematically invalid, but to really illustrate the different ways that students can still be engaged in rich activities as they're in the process of learning.

8:46  
How would you say that this article builds on existing work in the field, in particular theories or prior work you did, or other grants you're working on?

8:56  
I think this work could really be situated in a number of areas of literature. First, there's a lot of work that's been done about how teachers and pre service teachers conceptualize ideas of proof and justifying and I do think that we expand and build upon that, in particular, some of our work kind of lines of like arose out of proof schemes and what people find convincing. And the fact that often the often the we come in with ideas around justifying and generalizing that aren't, aren't normative. So that literature kind of framed our problem. And then we expand it out by really tried to categorize some of the ways that teachers also thought about generalizing, which we see as kind of the partner type of reasoning for building mathematical understanding. And there's really not been a lot out there about conceptions around generalization prior to some of the things that we've been looking at. We also have a Jmt article which digs into some of these things from a little bit of a different view and really unpacking what it means to notice justifying and generalizing and introducing that framework which but pretty explicitly on the Jacob's ladder and Philip noticing framework about describing, interpreting and then deciding how to respond on the basis of what you what you've noticed in those first two steps. So thinking about that through the lens of justifying and generalizing. So in that way, we're pretty situated in the noticing world, but moving into this world of thinking about mathematical reasoning and practices. And if I remember correctly, that paper you just mentioned is cited in this article. So if anybody's interested in they can find that yeah, that's correct. Okay, so let's move on a little bit. Maybe we already talked about this. But then let's just briefly summarize, let's talk a little bit more about the innovation. How does the innovation the student discourse, observation tool address the problem of practice? Yeah, I'm just gonna start calling it s dot because it's easier.

10:48  
And s dot stands for what

10:50  
we're doing discourse observation tool. Okay. Yeah. So this is definitely at its heart. It's an observation tool, and with the intent for observers, whether those are teachers or school leaders, that principals, coaches, whatever work with this, and it gives them a place to really note down student discourse. And when we're saying discourse, we mean it pretty broadly. That could be what students write with, do you say gestures?

11:13  
So let me pause you for a second if I'm imagining. So you said earlier, this is a paper tool. So I'm imagining this on a clipboard, and I'm standing in a classroom and what is happening, and what am I doing, you have this

11:27  
in front of you, hanging on the back of the classroom, or maybe you're walking around as students are working on some sort of mathematical task. And then the idea is, while you're walking around, you want to be noting down the thing that you're hearing students write, or the things that you're hearing students say, with the purpose of spending some time interpreting those then as the type of reasoning later on. So that would be kind of a column, one of the tool would be jotting down just the different things that you're hearing to the best of your ability to.

11:57  
So I would be transcribing things I hear as I walk Exactly.

12:01  
Okay. So it gives us space to really kind of focus in on the students thinking and what the students are actually saying, is kind of step one in the same process. So that's kind of would be column one, then there's a place that you can actually characterize the discourse is justifying generalizing or using procedures and facts. And that's something that could happen as you're walking around, noted, it's also something that that you have described down that you could reflect on after the fact,

12:28  
could you give us an example maybe of like a piece of discourse and how we might characterize it. So one

12:34  
example might be that a student is summing up some multi digit numbers, they use the algorithm, and then they check their response by doing by subtracting and verifying that they are correct. Okay, you might jot down what you see them, right. And maybe the student explains, well, I know that my answer is correct, because I checked it using subtraction. So you might actually write down some of that language as well. And then, actually picking this example because it's one that came up a lot with our teachers have, is this a piece of discourse that is justifying generalizing or using procedures and facts. And kind of, according to how we're conceptualizing this, which involves really leveraging mathematical meaning and structure to make arguments in order to justify this would be a classification of this is using procedures and facts. So it's using the procedure for addition, it's verifying that the procedure of subtraction for there's no real connection to the mathematical meaning behind their argument, so that would be the type of classification for that piece of piece of discourse. Okay, so that ends up a lot of opportunities to talk about, well, there is an argument for why this answer is correct. Why wouldn't this fit into the classification of justifying that? Sounds great. Thank

13:44  
you so much. Sorry, I didn't interrupt you. Did you want to talk a little bit more about this innovation? Yeah, I

13:50  
mean, I think most of the pieces we had mentioned earlier, when you asked me to kind of unpack what's on there that I'd mentioned, they have those definitions, which can be helpful for just kind of keeping a shared understanding going on from your example that you just shared. One of the things to highlight is that procedural is not equal to negative or bad. Right, right. Right. Absolutely. Right. There's a lot of impact. You have to reason with procedures, in fact, when we're dealing with mathematics,

14:16  
so it's really more a classification of different kinds of reasoning, and not necessarily, you only want to see justification,

14:25  
right, right. Yeah, yeah. So the focus is really on making sense of these activities that students are engaged in, and ultimately do want to see justifying and generalizing So thinking about how we can that can emerge complimentary to using procedures and facts and the other types of reasoning that go on in the classroom. Okay,

14:43  
so let's move on to the research questions. What research questions did you study to document the effectiveness of the innovation?

14:52  
jot it down to the two research questions to what extent to teachers who routinely use the s dot have more robust conceptions of justifying and generalizing then teachers exposed to the stat only during a Summer Institute, and to what extent the teachers who routinely use the s dot have greater ability to characterize student discourse as justifying generalizing or using procedures and facts when compared to teachers exposed to the s dot only during a summer institute. Okay.

15:17  
And usually we asked what were the research questions that you studied? And then what are some challenges

15:23  
or challenges in relation to studying these research questions or to the innovation itself?

15:29  
Sure.

15:32  
You knew that was coming.

15:35  
I mean, every one of the challenges when, when studying something in particular, that something so one tool does so that tool is that the usage of the tool is in such a bigger system of things that are going on, right? There's this was part of a very large professional development. And it was certainly one of the focal tools in this picture, justifying and generalizing was one of the main themes, but it's really impossible to detangle what exactly was the the impact of the tool without all the surrounding supports, and the particular PD facilitators that we're using in our project and some of those things. So I think trying to, to make the argument that we want to make while also attending to the fact that this is part of a larger setting, with one of the one of the challenges to answer these research questions authentically.

16:23  
So I'm going to try to rephrase the research questions in a little bit less complex of a way. So this was part of a larger research study, you said, right, and some teachers did a lot of professional development and use this tool a lot. And some teachers had like summer professional development and use the tool there. So some people, all people use the tool, some use did a lot more and professional development. And so those are the two groups we're comparing. And then the question was, what did the tool or the questions were like? Did the tool help teachers, in some sense? And can you quickly just say, what were those two things that we looked at? The the teachers who use the tool more of? Are they better able to categorize? Was that one of the questions? Yeah, so

17:14  
one of it was they had a bunch of pieces of students written work. So we're teachers who work with the estimate, better able to classify them as justifying generalizing or using procedures and facts. So that was one piece of it. And then the other piece was, all the teachers responded to a prompt about, you know, what does it look like when students are justifying? What does it look like? And students are generalizing in my classroom? And trying to see if these descriptions were richer, more accurate for teachers who spent more time with the

17:42  
staff? And I think you said this at the beginning. But just to summarize, again, I think for both research questions, the answer was yes, teachers who used the tool more could categorize better, and yes, they had richer descriptions. Yes. Okay. All right. So what evidence do you present in the article that shows the effectiveness of the tool or what we just summarized,

18:06  
so we looked at these, we had seven pieces of student discourse. And what we found is that on average, the teachers who work with the staff more extensively classified 75% of the discourse pieces correctly, versus 60%. For the teachers who were only introduced the tool over summer sessions. Okay, so that pretty much amounts to being on average slightly over an extra piece of discourse being classified correctly for the the teachers, okay, in terms of how they were describing, justifying and generalizing that took a little bit more, more work to make sense of and what we ended up doing was developing a classification scheme of how detailed the responses were, and how accurate they were. So for an example, a teacher who says that a student is generalizing when they use a known rule to do something would get classified as an inaccurate portrayal of generalization, or is it a teacher who says something like, well is teepees making a connection? That would be something that would get a partial term for for understanding or the description of generalizing because making connections is an important piece of it, but there wasn't enough evidence to say, Oh, this teacher has a robust understanding of generalization versus a teacher who talks about, you know, students make conjectures about what happens in the general case, would have an understanding that we would have said, Okay, this is pretty complete and pretty accurate to how we're talking about generalizing. And so we looked at all those responses for generalizing and justifying across the two groups. And what we ended up finding is that the proportion of s dot teachers who provide a complete description of generalizing was significantly higher than the summer only teachers and the proportion of providing incorrect descriptions was significantly lower. Similarly, the proportion of s teachers providing partially correct descriptions of justifying was significantly higher and the purpose Providing incorrect descriptions of justifying was significantly lower than the teachers who only had minimal exposure to the tool.

20:06  
Oh, yeah. So wrap up question, this deals with the new contribution. What's the new contribution to our field that this article makes? Or the innovation makes? And within this question, part of the goal of this podcast is to distribute the knowledge and like, maybe hear from the authors and think about like, how would you like other people to use this tool? What might you want to hear back if somebody used it? Those kinds of things. So let's start by doing a contribution. And then from there, the contributions

20:41  
from this article, a little bit of no record, in general bounds around justifying and generalizing is one establishing that really noticing, justifying and generalizing is a skill that can be developed, and that our teachers in our study really grappled with these notions. And then we had this evidence that when they leverage something like the stat that really focuses the work on observing, justifying and generalizing that really has the potential for teachers to develop some of this noticing skill. So then the the tool itself is also one of the contributions of this paper up here is a tool that seems to have been pretty successful and supportive for the teachers that we've been working with, around this idea of justifying and generalizing and noticing when your students are doing these things.

21:24  
And it's a pretty straightforward tool that you can just print out and use. Right. So that's a nice way. So let's say somebody else downloads this tool and uses it, what might you like to hear back from them?

21:37  
Okay, good question, I made a number of things that I'd be, I'd be curious about the, the definitions themselves, which is the bulk of the part of the tool that you'd be putting out, or very much formed in the context of teacher development, group work and some of our work, I would love to know if these definitions are useful and ring true to people who are maybe less situated in this particular work, if they are, provide a helpful lens for thinking about student discourse, or if there's any parts that could be clarified on that front, I'd also be just generally curious of the different ways that this could be used. So one of the things that we thought about and that we haven't tried out is that this is a tool that could be used if you are watching video recordings of classrooms and not necessarily going in and out of like classrooms. And I would imagine that this would be very adaptable, that type of analysis and discussion, but I'm curious about what that how that would play out with other people.

22:32  
So to close out, can you talk a little bit about how the work that you described in this article fits with your work in general?

22:40  
Yeah. So I think that most of the work that I do, and the research I focus on, has a lot to do with developing classrooms where students are really building conceptual understanding. And I see justifying it generalizing is one of the big, big tools, big ways that students can explore mathematics meaningfully. And this is an area that I this conceptual understanding area that I've looked at, across the K 12, all the way up to abstract algebra, where I do a lot of my my method research. And so moving with word forward, we really been focused on looking at what's going on in classrooms in particular, and what are the teachers supports that can allow and promote students engagement and argumentation and in particular, justifying and generalizing. I mentioned earlier that this is part of a larger PD advocacy study, that was a Dr. k 12. Grant within itself. Since then, we've gotten a follow up grant, where we are working on developing a much more complex observation tool, but it does include some of the justifying and generalizing but this version of is actually an iPad app where we are looking at not just our sins justifying and generalizing, but some of the other thing that occur in mathematical reasoning, as well as all those teacher moves and supports that can promote justifying and generalizing and sort of the big goal is to identify the things in the classroom that get us towards justifying and generalizing identifying the move that provide opportunity, learning opportunities and access for students. And we are working on refining and conducting a validation study of this iPad app, with the hopes of being able to make some really nice claims about what are the particular supports? What are the things that are happening in classrooms where we see students justifying and generalizing a lot, and how can we explain what's going on there.

24:31  
So we're very excited and looking forward to the podcast on that article that you're gonna be publishing. All right, well, thank you, Kate, so much for joining us. And this is the end of the podcast.