0:00 Hello and thank you for listening to the mathematics teacher educator journal podcast. The mathematics teacher educator journal is co sponsored by the Association of mathematics teacher educators, and the National Council of Teachers of Mathematics. My name is Eva Sennheiser, and I'm talking with Randy growth. And Jennifer Bergner, who with their co authors, all from Salisbury University in Salisbury, Maryland, authored the article undergraduate research in mathematics education, using qualitative data about children's learning to make decisions about teaching, published in the US, June 2020, issue of the mathematics teacher educator journal, we will begin by summarizing the main points of the article and discuss in more depth the lessons they shared in the article, their successes and challenges, and how these lessons relate to their work. Randy and Jennifer, thank you for joining us. 0:59 Thank you. 0:59 Thank you. 1:00 Can you get us started by just briefly describing the innovation? Sure. So 1:05 this article is based on research experiences for undergraduates project from the NSF that we started in the summer of 2014. And we run a cohort each summer, and we have eight undergraduates in each one, bringing undergraduates in teacher preparation programs from the local area, some from our own country, and include some community college students as well. And the overall goal is to design and test mathematics learning units, mathematics learning sequences, so we think of it as kind of being like an engineering research project where undergraduates come to campus and design and test these instructional structures while collaborating with each other and with a faculty mentor. 1:50 How long are these experiences, 1:52 so each summer, it's a 10 week experience. First week, we spend prepping the undergraduates to do interviews, we give them a sense of what the instructional look like over the summer, we actually start that process even before they arrive on campus, we have a one month online module that they complete, so that they can really get the most out of that first week, it's pretty intensive first week of preparation. By the end of that week, they're ready to do pre interviews with the children that they'll teach over the summer. They do the pre interviews, and then each week they teach a lesson for the next seven weeks, they teach a lesson to their group of students. And then they collaborate with their mentor to analyze the video of the lesson, analyze student work, and decide on the basis of student thinking what they should do next very end of the summer, they do post interviews as a summary to see what students have learned. And then the final week, the 10th week is dedicated totally to just finishing up research projects like abstracts, PowerPoint slides, and a poster to summarize everything. 2:57 Can you give us a brief summary of the article, including the results? Sure. So 3:03 in this article, we start out in the introduction, by just making an argument that undergraduate research, even though it's traditionally been the domain of just the STEM disciplines, not necessarily STEM education, we make an argument that teachers have a lot to gain by having undergraduate research experiences as well. And then we go ahead and give our model as an example of what it might look like to do undergraduate research in mathematics education. We talked about how the model is built on just continuously refining instruction and kind of a design based research type of a paradigm but give some of the specifics talk about how the parents of undergraduates work with math education mentor, get into some of the work that they've done, give some examples talk about how undergraduates learn to link their instructional decisions to what they see of students thinking. And then we kind of close it out by giving an example of a smaller scale implementation. So I've been talking just about the full blown project. But we've also had people one of our co authors took this model and used it in a mathematics methods course, and tried to keep some of the essential elements of it while still making it manageable within those constraints. So our article closes out with a smaller scale model that people can implement if the larger scale models a little bit too time intensive for the setting that they have at this point. 4:29 So this leads really nicely in our next question, like who should read this article, 4:34 as Randy had just mentioned, since we highlight our two audiences, those who want to do it on a smaller scale? So methods instructors who have field based assignments that involve interviewing children or working with children, or those that summarize qualitative data and want to make decisions about their teaching, and want to make decisions about their students thinking or on a larger scale people who are interested in doing And are you in mathematics education? 5:02 So it's really kind of two different goals. One is this like larger idea of the research project over the summer or over a longer time. And the other one is taking this experience and being able to use it in methods courses. Exactly. Okay. Let's talk a little bit about what is the problem of practice that you're tackling? Well, I 5:29 would say in the bigger picture, what we're looking at is trying to make sense of qualitative data, teaching, you know, prospective teachers how to make sense of it, they are going to encounter one day, a host of qualitative data. And so we are hoping that this experience will help them learn how to make sense of it, and how to learn from it and how to make instructional decisions based on it. 5:51 As I'm listening, I'm trying to imagine that you're looking forward to winter teachers in the classroom. And this experience would prepare them to be able to listen to their students and reflect on that, 6:08 I would say yes, it also, I think it shows them, it gives them a model of what that could look like at the that if they take the time to sit there and reflect upon what their students are doing and what their students are learning that they could make better instructional decisions and meet their students in a better way. meet them where they're at. So we try to model that in pathways and the longer 10 week are you. And then in the in the methods class, when this other instructor did this, she is trying to engage you know, these prospective teachers in the art of listening, 6:44 listening to your students. So is then the goal or the idea. There's many goals and many ideas with this project, I imagine, but is one of the goals to get towards a more student centered instruction. Yes, 6:59 yeah, that's definitely a good way of describing it. And just the idea that teachers really, it's amazing when you stop and think about it, how much qualitative data teachers encounter in the classroom, just on a minute by minute basis with all of the student work, the student comments, just an amazing amount of data. And so one of the things we do hope they take from this is the ability to make sense of the data to kind of develop the habit of mind to focus in on Okay, what is this piece of student work telling me about the child's thinking? And what can I do with that information, as I you know, either transition to the next activity during that lesson, or I design the next day's lesson. So we really aim to foster that habit of mind of making sense of the data, and then using inferences from that to really decide what to do next. 7:52 So there's some formative assessment in there, there's some belief in how students learn. And there, there's all kinds of things that you're mixing in there, which actually leads me really nicely into the next question that we have, which is how does your work build on existing work in the field? So we take a lot of inspiration from other professional development models that emphasize learning from practice. So I mean, one, you know, in the earlier literature, one of the examples as just, you know, this idea of action research, which people have conceptualized different ways, I wouldn't describe this just strictly as action research. But 8:32 the thing it does have in common with that strand of literature is the focus on solving problems of practice, like, okay, for example, students don't understand measures of center, they don't understand, or they're working on understanding by various data conceptually. So what can we do? What can we try out? What can we test out to solve that problem? So we have that I think we have that in common with action research. But there's a very strong similarity to I would say, the lesson study, because we have kind of a reflective cycle that they go through and plan each lesson. they implement the lesson, they video, record it, they sit down together and analyze it so that the briefing piece that's also in Lesson Study is present in this project. And then those observations are used to improve the next lesson. So there's, I think there's a very strong similarity to that model. But there's a little bit more there too. So we characterize it as being closer to design based research. They're really building an entire sequence of linked lessons like related lessons. So the other piece, I think that makes it a little bit different from the other models I've mentioned is the focus on qualitative data analysis and a more enough, I want to say rigorous but at a more intense level, then you might find in some of the other models, we've done Lesson Study before and you certainly attend to the data and you certainly attend to your observations of student thinking, I think and this project being an undergraduate research project, it takes it to a little bit of a different level, because we're really sitting down with them and teaching them how to code, student reasoning patterns, how to take those codes and group them to get a sense of not only get a sense of what to do next and instruction, but also to report out to an audience like they would encounter it in undergraduate research conference. So because we have those dual purposes of teaching, and having them also share this work more broadly, we found it useful to have that increased kind of emphasis on qualitative data analysis in this project. And that serves a purpose, I think of both teaching and reporting the research out to a larger audience outside of the project. 10:49 So in the article, you do describe your innovation in a little bit more detail. So could you give us a sense, maybe with some examples of what exactly this innovation looked like, and maybe some examples of this qualitative data analysis. For me, this 11:09 has been completely enjoyable because I do teach in the math department, but it's wonderful to get to mentor future mathematics teachers and work with young children. So my groups usually work on the concept of multiplication. And depending on the grade level, they are, sometimes I have third graders, fourth graders, fifth graders, we would pick the topic of multiplication. And with my undergrads, I would talk about the ideas of like, what mathematically is multiplication? What does that mean? We use illustrative mathematics a lot to look at task. And we also access, we look at learning trajectories for where multiplication should be situated in the curriculum, and what kind of multiplicative reasoning would be going on at the different grade levels. And so we do a pre interview that first week, and then they get to literally see what the kid they get to see themselves, ask the kids the questions and see what the kids answer. And after that, they design their first lesson. They know where the kids are at. Sometimes they're surprised. They think that the questions are too easy. The children will whip through them, they're not going to, you know, struggle, and why are we asking them these questions. So for the undergraduates, it's often interesting for them to be like, Oh, I thought the students would be at a different place. And I always tell them, well, you don't know where they're going to be until you ask them. And that's a big theme throughout the summer is, you know, ask the students where they are ask the students what they think try to engage the students. So we develop them the first lesson, they go implement it, they videoed themselves, they go back, they transcribe the interview, they come in the weekend after and then we go through the interview line by line. And we try to develop a set of codes very organically for like, what are you seeing with respect to student thinking? What patterns are you noticing, and then we develop that codebook. And we use it throughout the entire summer, to help reflect upon what's going on. So basically, that middle seven weeks is always the same. Design The lesson, implement the lesson, look at the video, make decisions about what you're going to do, what they learn what we can do next, go design another lesson, and then just reflect and so it's like that over and over and over again. And then the message to the undergraduates the whole time is think about what your students are learning and use evidence to back that up. So look at the video, look at their performance on tasks, think about their thinking and ask about their thinking. And then at the very end, like Randy had mentioned, we have them try to synthesize all this in a big under you know, poster that they could present somewhere. You know, at national research we've had some of them go to Randy, how many of them have been to incur quite 13:55 a few we've had some I would say almost each year. And actually one thing that we do have all of these presentations archived on our project websites. So if people are interested in seeing the posters that have been assembled, and the different presentations and some of the articles that have been written as a result of this, all of that is posted online at www dot saulsbury. spelled Sal is bu r y.edu slash pathways. My group just to give a couple of additional examples as to what Jen was talking about with her group, my group, you know, has gone through that same research process. And we focused on grades six, seven and eight Common Core State Standards for statistics and probability. Because that's a lot of my other research work lies. Now just mentioned that that's kind of a nice side benefit for mentors in a project like this as it gives you an opportunity, especially if you like to do really fine grained analyses of Students thinking, it's really interesting to, you know, try to bring undergraduates into this research process. I mean, a lot of studies are based on studying what undergraduates are thinking. And we do that. But we also bring them in as colleagues on this, which I think is again, kind of a unique aspect that I hadn't considered before we started this work. 15:20 Yeah, and I would say, for me, as well. And I think for some of the other faculty mentors, it really is almost like math Ed research summer camp, you really get to know the group of students you're working with, you get to know the little children as well and their parents. And it's just kind of really neat, because you're part of this community that's going to, you're really focused on these young children and getting to understand what they understand about math and encouraging them to love math, they usually just love the undergraduates, it encourages the undergraduates and lets them see, like, I can do this, I can teach, I can also be a reflective practitioner, I can work with this, when we say qualitative data, it sounds so scary, but then they realize, Oh, this is just stuff I'm bombarded with constantly, that, you know, when I'm in my classroom, one day, I'm gonna have to be making decisions almost on the fly. And so I know for me and others, it's just, it's a really fun way to spend our summer. And it also helps the children out, we have a variety of, we always try to get a variety of children within each group so that we can represent, you know, on a very small scale, what a normal classroom would be, you're already making me jealous, I 16:36 want to be part of this. Now, I just pulled up that website, Randy, that you mentioned, and it's really cool. And you can see all the cohorts and all the posters. And so if anybody's interested, I was also wondering if one of you could talk a little bit more about this matrix that you developed with the students? 16:57 So that the the data analysis matrix? Yes. Okay. So I believe that one, I believe the one you're referring to is the one that we use to have them track the codes that we design during our, during our data analysis process. And that's something that yeah, I mean, it actually took a while to really land on the best way to do this qualitative data analysis with undergraduates, because early on for the first few cohorts, we had them look at the strands of mathematical proficiency, and tag their lesson transcripts using those five strands, which was okay. They used words, they got a sense of like, what it meant to have conceptual understanding procedural fluency, which was a positive outcome, but we wanted them to dig a little bit deeper into more specifics like okay, what does it mean? Or what kind of reasoning patterns? exactly would you expect a student to have in order to say they have conceptual understanding of number and operations, fractions, whatever the case might be. So over time, we shifted more toward this open coding kind of a paradigm not starting with the predetermine categories right away. So that's really what the table is about. We break the lessons up into segments. So each row of the table is a segment of the lesson. And a segment we define actually, in the article, we talked about, like the technicalities of what we consider a segment, it's really just In short, it's just when a new task or your question is posed, but we break the lesson into segments. And then we have some in the far right column of the table, we track the reasoning patterns that each of the children in the lesson exhibited. So it's almost like a scorecard in some ways of what happened during segment one, what did we learn about student one's thinking during segment one, what you know, based on what that students said, What can we assign as a code that would really capture what that student is thinking? And it's really interesting sometimes, if a segment is blank if it's totally empty, and we can say nothing about what students were thinking, because all of a sudden, then our undergraduates are like, Well, wait a minute, I guess we didn't learn. We were doing all the talking and didn't learn much about students thinking in that segment. And so then it becomes a goal. Well, we got we have to figure out what's going on here. So even the empty cells serve us well, sometimes in the table. And so we go through each segment like that, and keep score at the end of all of that, when we're done with the entire lesson. We go back through and look for patterns, we actually have them. Do, you know, group the codes into families? Like what are the main codes that you saw in this lesson? And what does that mean for the next one? So that's kind of the next level of it after the coding matrix is complete. Let's look across the whole thing and see what the global patterns are as well. 19:56 Yeah, I really liked this matrix. No, thank you. This could be really useful. So thanks for sharing a little bit more about that. I think we understand the innovation a little bit better. Now, could you talk a little bit about what your research question was? And what evidence you use to answer those research questions, 20:16 probably the implicit research question that we had that guided this particular article and probably, you know, guides our our work in general with this project is how do undergraduates engage and understanding qualitative data under the pathways model? And again, the pathways model can be implemented, you know, on a large scale, like, are you on a smaller scale in a methods classroom? 20:40 Could you also share how you answered the question and what evidence you used in this particular paper, 20:48 we have some examples of how undergraduates engaged in the process. So we talked a little bit about how they grew as both teachers and researchers, those are kind of the two main goals that we hope or two main objectives we hope to achieve with this project is that they would grow as teachers, and also that they would grow as researchers. So some of the data that we drawn and empty paper, we administered some of the surveys that are usually given to undergraduates and other NSF ru projects. And those were interesting. They focus mainly on aspirations for graduate study. There are some questions about teaching certification. So some of our participants said that they, you know, the experience made them more likely or very more likely to pursue teaching certification. Some of them started thinking about possibility of doing graduate work in method research. So I would say that's one broad category. Under this model, one of the things we hope to achieve on that we've gathered data on as far as growth as teachers give you an example of one of one of the instruments that I've enjoyed looking at the results for is, before the undergraduates give the interviews before they do the pre and post interviews, let me back up a step before they do the pre interviews with their students. We asked them to predict how their children how their students will answer some of the key questions on the interview protocol, it's always interesting to see the difference, what they think at the beginning versus what they think at the end, we asked them to predict before the pre interviews, we also asked the undergraduates to predict how the children will respond before the post interviews. And so that's as far as growth as teachers goes, we've seen some interesting results there. In the paper, we give an example of a couple of examples. One of them is with the statistics group, and how they better anticipated that students would focus on some of the visual features of crafts at the beginning of the summer, they thought students would have mostly a procedural view didn't really take a lot of conceptual things into account. And then as the summer went on, they were surprised at some of the features of graphs that students did focus on. And by the end of the summer, they were much in much better position to predict students tendencies to focus on those features. So those are some of the things in that. So again, they're really two broad categories, the growth as researchers growth as teachers, and so being able to anticipate students thinking and then use that as something that we've seen growth and with the kind of the second broad goal there to help them grow as mathematics teachers. 23:36 Alright, let's wrap up our conversation by reflecting a little bit about the contribution his article makes to the field, which we kind of have discussed already. And a little bit of how this work fits into both of your larger works. 23:54 Yeah, I think the one thing that I would just like to add that we've touched on a little bit, but I think it's just valuable, repeating or emphasizing a little bit more is that, you know, traditionally, these research experiences for undergraduates have been pretty much dominated by just STEM fields, not necessarily STEM education. If you go into a school of science, it's really common to see professors, you know, having funded projects for research experiences for undergraduates, but really, like groups like NSF have really encouraged STEM education reuse. So I think it would be great to see more of these. And when undergraduates think about well, I want to pursue graduate study eventually. Well, you know, why not also think about pursuing graduate study in mathematics education, it's not just the STEM fields. I mean, there's actually argue, a bigger need for graduates in mathematics education field. So I think that's one of the things in addition to really helping teachers think about their practice. I think it would be great if we could See more opportunities, you know this nature to get undergraduates maybe thinking of themselves as future math education researchers. Yes. And 25:09 I would say also this project, I get given me a chance to help model and to work with, you know, future teachers and show them that mathematics teaching is not just about getting up at the board and writing beautifully in chalk from end to end, that it's more about the fact that mathematics is community based and, and you should think about your audience, and you should think about your students and who you have in front of you and how they're understanding the words that are coming out of your mouth. This past summer, the group we worked with one of the ideas that they got stuck on that they always thought they had to ask the students about with some the constant of proportionality. And it was funny, because one day in the chat feature, because we had to do this on zoom this summer. So in the chat feature, we're like, ask the student what they think constant proportionality means, and they asked the student, the student was like, uh, and it was really eye opening for them that it was like, oh, like, I can throw this language around. But it doesn't mean to my student what it means to me. And so to have the opportunity, you know, to work, you know, over this 10 week time period very intensely, and then engage in good practice with mathematics, instructor instruction, and also to show them like that you're part of the community, we want you to be part of the community, you are the community, and to produce future mathematics education researchers, which, you know, actually all teachers in a way that teach mathematics are researchers, if they're doing it right. 26:38 Well, I have to say, You sold me I'm already thinking about how I could potentially do this at Ace, the larger project seems a little bit more daunting than the smaller project in terms of how to make it happen. So I'm sold. 26:54 And we welcome you, you could come and stay on the eastern shore and enjoy the summer. 27:01 I was wondering if you want to just say a few words about how this summer was different from other summers, 27:09 I will start this because Randy has had quite the summer on zoom, I've had less I did teach course through zoom myself was an advanced calculus course. So I did it through zoom. But he's had quite a lot of zooming this summer, not as much as me, I actually enjoyed it a little bit, because we could sit there and chat with the students as they were engaging with our undergraduates, while they were engaging with the students and make suggestions in a way that wasn't maybe so intrusive. Like if you're in the classroom with them, sometimes that's a hard thing to I call it butting in. And I always joke with mine, I'm like, I'm sorry, if I bought in, you know, and try to get maybe the conversation steered another way. And, you know, they're just learning, they're undergraduates, they're just learning about teaching and engaging the students with the content. And so sometimes they might freeze, and they don't know what to say, which is completely fine. I mean, it's a learning process. But I found it very nice to be able to kind of sit back and you could really see what was going on. And you could see the dynamics, I also thought it was really nice to engage the schoolchildren, in a meaningful set of zoom sessions, the kids loved it. So I would have the moms, you know, texting me or emailing me and telling me how much their children appreciated it. And then I think it also was of benefit to the undergrads because we are in a different world now. They could be looking at doing more zoom sessions in the future, you know, in the near future, and in the far future. So I see it as a way of like teaching them how to monitor, you know, the zoom session and engage their students, although sometimes that felt so unnatural, but like, if that's what you have to do, that's what you have to do. 28:51 Yeah, I mean, I think I would just add from so you know, again, with the kind of two goals of the project, so we've got the goal of producing better teachers, we have the goal of giving them a taste of mathematics, education, research, some of the research logistics are harder, and some are easier. I mean, on zoom, just some of the nuts and bolts of it are are somewhat easier. As far as gathering data, zoom will actually produce a transcript, you have to go back and clean up the transcript, but you're not necessarily starting from scratch. So yeah, I think that will really be keeping an eye on how things play out in the coming months. Because we do have in the current funding cycle for this project, we have one more year for sure, on the books that will be funded. And so at this point, you know, it's hard to say if that entire summer will have to be online again, in anticipation that it might be or that we might do this work again, I think one of the next things that we'll be taking a look at is just really analyzing the data that we gathered this summer from all of these zoom sessions. We have planning sessions that we facilitated with We have the analysis sessions. And so I think we still have a lot to learn from the data. I mean, we have a wealth of qualitative data that we're going to be in process of trying to make sense of as we plan for this project next year. 30:14 Exactly. Yep. 30:17 There's lots of opportunities for additional research questions with these changing times. Right? Yes. Well, thank you both so much for joining us today. It was really enjoyable to talk to you and learn more about this experience. 30:33 Well, thank you for chatting with us and for giving us this opportunity. 30:36 Yeah, thank you for having us. 30:38 For further information on this topic. You can find the article on the mathematics teacher educator website. This has been your host, Eva Sennheiser. Thank you for listening and goodbye.