Sean Tibor: Hello, and welcome to teaching Python. This is episode 116 and today we're going to be talking about natural language processing with ines Montana. My name is Sean Tibor. I'm a coder who teaches and my. Kelly Schuster-Paredes: Name is Kelly Schuster Perez and I'm a teacher that codes with a horse voice. Sean Tibor: Yeah, it's been a little bit of a rough start to the school year with getting a little bit sick before it all begins. Ines Montani: Yeah. Kelly Schuster-Paredes: Tomorrow is the first day with the 6th graders. I'm so excited about installing move for the I don't know how many 55th time. Ines Montani: Yeah, I can't wait. Sean Tibor: It's all downhill after the first 52. And Nez, welcome to the call. It's so good to have you here this week. Ines Montani: Yeah, thanks for having me. Very excited. Yeah, I'm kind of new to the whole teaching side of things. I don't have kids, but yeah, we had a Ukrainian family staying with us and the twelve year old was like really taking to programming. So that was kind of what really got me interested in okay. How to teach him more. He's really smart. How to do any of that, I have no idea. Sean Tibor: Well, we're happy to share what we've learned so far. And I love the idea of talking about natural language processing with students because I think it really brings coding and algorithms to life when they can interact with the program that they create. Ines Montani: Yeah, no, it's real. That's also what really attracted me to the field. It's not just abstract programming. You're really working with something that's like super familiar, which is language and text. Sean Tibor: Nice. Well, we'll get into that in a second. Before we begin, we're going to start where we always do with the wins of the week and that's something good that's happened inside or outside of the classroom and ines. We're going to have you go first this week and share a win that's happened for you. Ines Montani: Yeah. So this one is not specifically about programming itself, but about the community and the Python community because we as Explosion, we are sponsoring PyData Amsterdam, which is happening next month. So we're right in the middle of preparations, actually. I don't want to reveal too much, but we're planning things for our booth. It's our first time having a booth. We're going to do cool activities for the community and just getting ready to meet people. I feel like that's one of the best parts of Python is the community around. Yeah. So there's been a lot of excitement in our team. Kelly Schuster-Paredes: That's probably the best thing for me too. I think that's what kept me into Python is all the people that we've met. The community is so welcoming and when you're giving some talk or explaining stuff and everyone's just so supportive, everybody wants to learn all the time. I think that's why I like Pythonistas. They just want to learn all the time for no reason. Ines Montani: It's a very welcoming community to anyone. And yeah, that's what I love as well. Sean Tibor: I love that everyone's willing to share and help. I mean, no one's like, oh, I can't talk to you about that. They're like, oh, no, let me teach you about what I'm doing, or show it to you, or check out this cool thing that I'm doing. It's a very sharing, open community that I really enjoy. Kelly Schuster-Paredes: Yeah, 100%. Sean Tibor: Kelly, how about you? Kelly Schuster-Paredes: Oh, me. So I'm going to take my win from last week and it kind of goes with what we talk about or sort of talk about, but I did an AI for beginners presentation for teachers, teaching teachers last week and it was the only one that was offered and I did it with a colleague. It was fantastic. The doors had to be opened up, people were sitting in the hall. It just goes to show you how many educators want to learn about what's going on with AI. And I kind of talked to them a little bit about LLMs and explained to them, this is what a large language model is and this is kind of what's going on. Very basic, didn't go into any coding and I kind of explained to them, like, it's not just chat GPT, that's not everything in the entire world, so just stop saying that AI is chat GPT. Let's just talk about everything that's been happening for the last 60 years. And it's not brand new, it just hit mainstream news channel and change your channel, there's other things. So it was really cool and a lot of people were interested and it was such a good win and people wanted the presentation and they wanted all the links and they were just really it's a little bit less scary when you introduce something as it is for real and not just the hyped up version. Right? So it was a really good one and I wish I could offer that again as another talk for educators, but yeah, I was on a high last week. I think that's where I sucked out all my energy. Sean Tibor: That's really great. I know we've been having a lot of conversations about that at work, as well as what's the right role to use for AI, not just for writing code, but for all of the other things that we're building and doing and where can we use it. Just even things like we generate a huge amount of security log data and being able to use that trained models to spot for anomalies and potential intrusions and things like that is hugely helpful because it's such a small positive effect, right? You could have billions of lines of log data and you need to find the right two, right? And it's challenging and it's a hard problem and it's a really important one. So a lot of conversations in education and enterprise, everybody is talking about how we can leverage this. So I'm definitely not surprised for a variety of reasons. There's a huge amount of interest, and you're also a really good speaker, so I think that helps. Kelly Schuster-Paredes: Together I had a good co host. It wasn't you, but It was my counterpart in Boca and she did a great job. So we did it. I would have much rather had a presentation with you, Sean. Ines Montani: Sorry. Sean Tibor: We'll do that another time. For me this week I'm going to pick a small one. I rebuilt my Circuit Python Lightsaber this week and it was really cool. The real Win was not the lightsaber. The real Win was just building it again and going through the code and remembering what it felt like to create it for the first time and see all the paredes that the Ada Fruit team has done with some of the hardware that's being used. So it's using a new Raspberry Pi 2040 microprocessor now, so it's smaller. And they took all of the prop maker controls for like, servo controls, which I'm not using, but the NeoPixel, the speaker amplifier, the accelerometer, all the stuff that was on a separate board is now all fit onto a tiny little feather board. So it's even smaller and lighter. And it's just really cool to see that and have that feeling of like, wow, I can make things and I can build stuff. Ines Montani: Yeah, I actually relate because I'm totally not a hardware person. But recently I was like, okay, I really need to play with this now and I can program for those who want to check it out. I wrote a blog post about it. I had this really stupid idea to build a machine to pet my cat remotely because I figured, well, it is this robot arm and then I attached this brush and then anyone from our team remotely can then use an app and pet because everyone loves our cat. Kelly it didn't quite work like that. Like, I've included the video. It's not cat's quite scared of it, but it was very satisfying to build. Sean Tibor: And I also used yeah, I'm not sure that cats are the best. They might be like, looking at it. Ines Montani: Side eye, but then cat's like, what is this? Kelly Schuster-Paredes: It can now just be like, just tell ines pet ines to transfer the pet to the cat. There you go. Everyone will just go, pet, pet, pet. Ines Montani: Yeah, the Web app is great. It had like this hand that follows your cursor and that you can move. I was very proud of that, too. Sean Tibor: That reminds me, I think one of my first hardware projects I did was a Servo arm that had I don't remember. Kelly you maybe remember this, and as I don't know if you've seen it, but there was like a board game called Don't Break the Ice where you had a character on top of some ice blocks and you had to tap out the ice block. Anyways, it came with this little green hammer and it kind of was like a Jenga like game where eventually someone taps out the wrong block and the whole thing comes crashing down. Ines Montani: Right? Sean Tibor: So I took this little green hammer and I put it on a servo and it would ring one of those service bells, like the little tap tap bells you put at a front desk or something. And I made it so that you could ring the bell from Twitter. And so every time it got a hashtag, it would ring the bell. And so every now and then I'd be sitting at my desk. But I love those ideas because they're fun and they're silly. And I was talking with my team about this. This week we learn more from building something by having this stupid idea that we think will be fun to make and we don't know how we're going to get there. And the process of getting there is what makes it a learning experience. Kelly Schuster-Paredes: 100%. Sean Tibor: Yeah. If we went into my list of fails this week, it would be a much longer show. I think this has been pretty rough between trying to parent and be productive and everything. My list of fails this week is pretty long, so we won't go into that, but just want everyone who's listening. If you're also failing your way through the week, you're not alone. I'm right there with you. Kelly Schuster-Paredes: Me too. Mine's the software and ebooks and trying to get everything up and running. My goal was to get everything up and running by tomorrow. And I still have one book that's not running in ClassLink. But luckily the teacher is amazing and she's super supportive. She's like, don't worry, we never open the book until like the second week of school. So fails happen. And I'm at the point right now where I'm like, okay, it's Friday. Let's get this kids in the door. Ines Montani: Yeah. Even in our company yeah. We build fancy AI and people are like, wow, you do so much, but a lot of the stuff you do doesn't work. And they're like, all kinds of I actually have this document where I collect the funniest bugs, all kinds of weird things that happen that I don't know, drove us crazy trying to debug. And then it was like some really interesting reason. Like, I don't know, a Unicode character that looked exactly like an e but was not an e in the file name and then type test doesn't pick it up. Or you get some weird failures when you install a library and it's like, why is this happening? And then hours later we're like, that wasn't an e. Yeah. Sean Tibor: One of my fails this week was from a bug I introduced nine months ago and caused a twelve hour outage. Kelly Schuster-Paredes: We're going to have to have a fail of the life, the fail of the daily thing kind of show. Let's talk NLP. Let's talk. Sean Tibor: Let's get into it. Let's start with just a little bit of background about yourself, the work that you're doing, how you got into this space. What is NLP for those people who don't know what it means or what it stands for, because every three letter acronym has like, 25 different possible explanations, right? Ines Montani: Actually, there is the other NLP, and they had the NLP hashtag on Twitter for a long time, which is why the Natural Language Processing Community, which is my NLP, used Nlproc as the hashtag to find each other, because NLP was used by Neurolinguistic programming, which is a whole different thing, and also doesn't have to do anything with the type of programming we're talking about here. But yeah, it's a thing. Kelly Schuster-Paredes: Yes. Ines Montani: Well, to go back so I don't actually have the classic tech programming background. I was like, kind of an indoor child. Indoor teenager, grew up on the Internet and was making websites. So when I think we got a computer when I was like, maybe 910. And then at some point I discovered that Microsoft Word lets you export pages as websites and then you could upload them to the Internet. And maybe nowadays that's really normal to people, but back then I was like, Whoa, mind blown. So I was making websites and that was even, I don't know, as a younger kid, I would make, like, little magazines and that was always like a thing I enjoyed. I'm like, great, I can do that on the Internet. And then I kind of went from there and really went all in on making websites. Then it was PHP WordPress. So that's kelly how I started programming. And it's always followed me ever since. I studied Communication science and linguistics at the time. It's interesting, looking back, it all worked out, but I just didn't see myself as a programmer back then. I was quite young when I finished school. I was like a girl making websites, and I just thought of programmers and computer scientists as these boys I knew from school. And I was like, I don't fit in here. And so I didn't go for computer science, but it just always still followed me around. But I did linguistics as part of my degree, so there was always the language thing. And then I worked in media for a while, and then by coincidence, I met my co founder Matt, who had the much more classic academic background. He was a linguist, then turned computer scientist, did his PhD, was really into this kind of idea of like, oh, if we want to analyze language, we need to do that empirically and properly. And that means we need to analyze a lot of it. And that means we need to write good programs, because if our programs are slow, there's no point. And even the Internet and the tech stays is growing faster than computers are getting faster. And so that's how he got into really writing C code and writing fast software. And he had just left academia because he saw that companies wanted to use his research code because it was the fastest. It was like on the top of leaderboards, and companies were like, oh, can we use this in production? And he's like, yeah, now that's like research code. It's supposed to exit, print a number and that's it. But that's kind of what inspired spacey writing, which is an open source library to analyze large volumes of text and work with text in Python. But that was really designed from the ground up to be used in real projects and not just academic research. And that's kind of what got me started in that area. I started working on the open source library and then shortly after, we decided to form a company around it and really focus on developer tools around working with machine learning, but especially around text. So training your own models, building pipelines, doing every, finding out more about text, which is, yeah, nowadays is clear. It's a use case in pretty much any field. We all have text. We've amassed text over decades and now we're like, wow, we have way too much text for anyone to read and we want to find out more about it. Sean Tibor: And I think it's kind of a subtle thing too, because as humans, we're trained our whole lives to process text and handle text and think about the meaning behind it. Ines Montani: Right? Sean Tibor: And I think anybody who's taught English before, taught language, really understands the struggle and the development, the growth that happens. But by the time you become an adult, you read and you write and you input text, you output text, you're able to find nuance and meaning in text. But to look at that through the lens of analysis right, of extracting or quantifying meaning from a given text and being able to classify it is something that I think is a really hard problem. Right? It's not something that's like, oh, yeah, it's so straightforward, we could all do it. Ines Montani: Yeah, I mean, even for humans, ask humans. Or one experiment I often do, or like when I give talks, I ask, okay, take these two sentences, I love cats and I hate cats. Are they similar or not? And then you can take hands and often, depending on the audience, you might get a 50 50. Or people can argue about that because often about similarity, people think, hey, that's like, easy, I should be able to tell that. But it's like, well, it depends if out of all the things you could possibly say in the English language, those two sentences are extremely similar. They express, like, sentiment about cats. But if you're building, say, a dating app and you want to match people up based on their profile text, then that should be really dissimilar because that's like opposite sentiment about text. And that's just the tip of the iceberg. There are tons of these things where even humans disagree and where there's like, a lot of knowledge and a lot of it is very specific. And yeah, it's hard for humans, it's hard for machines, but there are also things machines can just do better than humans. Like machines can remember stuff. A lot of the things you can now do with computers and with NLP are very difficult to perform for humans because you can't keep thousands or millions of documents in memory and then form connections between them. That's simply not feasible. Kelly Schuster-Paredes: So let's back up a little bit because it reminds me of this funny, I don't know if it was instagram or something, but you're flipping through the words and it's like hair, hair. And we put a th, it's not there. And it's all these languages and different pronunciations. I was just thinking about this joke, but when it talks about NLP, natural Language Processing, what does that entail? If you can break it down a little bit. Ines Montani: Traditionally, I would have initially said it's usually about taking text as input and outputting something about the text. And often it is used when talking about structured information. So finding things in text that go beyond just keyword search and that can be linguistic stuff as well, like what's the verb, what's the noun, who does what to whom? These things are actually really important for understanding text, but also things like what's the text about? What persons are mentioned, what companies are mentioned, and then doing stuff like, oh, is this company, that same instance of the company Microsoft, or is there someone else? And I think a lot of people, especially nowadays, would also include more generative things in that same field, which includes even something like machine translation, where a text comes in, a text comes out on, of course, things like chat, GPT, anything, dialogue. But basically, I think what everyone we all agree on is, well, text comes in and something related to that text goes out. Kelly Schuster-Paredes: So depending on what you put in and I'm thinking sort of because it's the NLTK library, which is similar to the Spacey, is that correct? Ines Montani: Back in the day, like when Spacey, when Matt started developing Spacey, NLTK was kind of the only main library around and that had a slightly different focus because there's a book that goes with it and it was really designed for teaching as well as using it for practical things. So it implements all the important algorithms back at the time. So there was a different focus. Whereas for Spacey, we always said like, hey, we implement one algorithm and that's like the best. And we try to have one model implementation and focus on because there's less of a focus of, hey, we show you different things you can do, and more like, here is a way to do a thing. But yes, that's definitely the same direction. I think I just kind of cut you off. Sean Tibor: I'm trying to think about how to make this real for people too. Examples of things you can do. Right. Because we're giving it basically just strings of characters, right. Like, here's a bunch of symbols that go into these tools. Some things that we can do that. Right. You mentioned sentiment analysis, right. So what is the sentiment of this text? Ines Montani: What are things your spam filter does? Something like it like an email comes in and then it predicts, is this spam or not? And there are like a lot of different factors that go into it. I'm sure sophisticated spam filter also combines that. It's not just one model. It's like a lot of different signals, but ultimately that's like a very classic text classification problem. Sean Tibor: Nice. So there's like, sentiment classification, there's summarization. Right. So I've seen some really cool examples where you can take a large document and summarize it down to an executive summary yeah. Ines Montani: That keep the meaning of a generative task. And then other things that are less visible, but also used a lot are things like span prediction or named entity recognition, which is, say you take a text, is that a person, is that an organization, is that a country? Stuff like that. That's often very useful on the way to other things, like if you want to fill a database or from more boring things like processing insurance claims or anonymization. That's actually super important. In a lot of areas, there's strict rules about personal data, especially, for example, in the medical field. There's like one company we recently talked to where, hey, if doctors are talking, for example, there's certain personal information that cannot be shared. And you can use a machine learning model to predict that and alert, hey, that looks like a person. Kelly Schuster-Paredes: And that, I think, is the key. So I'm thinking the kid in me, the teacher talking to a student. Well, I could just put everything in and chat GPT, or I can just put it into some other generative AI. Yes. And I think that's the difference between running your own or making your own kind of NLP project because it's more specific to your specific goal. And I was reading a little bit and I was watching a couple of your talks. I think it was a EuroPython talk. It was about the fact that you start with a problem. Yes. I can say, oh, I have all these recipes from home, I don't know, from my, whatever, grandmother, and I want to get the most common ingredients for all of it. That's something I don't want to dump all my grandmother's recipes in there because I don't want the whole world seeing all my grandmother's recipes. But it would be something that you could make for yourself. Is that kind of correct? Ines Montani: Yeah, I think exactly. Especially because to take a step back, it really comes down to this really fundamental thing we've always had, which is we want to be telling computers what to do. I mean, that's why we program and that's the fundamental problem. We know what we want and we need to find a way to make the computer do that. And there are different ways to do that. Like before machine learning was viable, you could write rules or you can write regular expressions and stuff like that. That's an approximation. Then with machine learning, that was actually really cool and new, instead of just writing logic, you give it examples and you can really give it these examples, show it to the model, gets a point if it's right, gets a minus point if it's wrong, like very simplified. And then it builds up these statistical weights. And then hopefully, if it sees other stuff that's similar, it can make the right decision. And that's kind of a big plus because if we can express something in examples and congenitalize, that's great. And now there's a whole other way, which is the prompting. So we can come up with a prompt and with a question that hopefully gets us what we want. And there are different there are pros and cons to all of that. Like the prompt, obviously, it's like fast, especially for everyday life stuff. It's really easy to just ask it a question that's general and get a good answer back. And then it gets a bit more difficult if you have something really specific in mind and especially something that's really domain specific because you have this very general, broad model that's, like, kind of very wide but not very deep in that kind of sense. If you know exactly what you want, you can create examples. It can be a lot more reliable, faster, easier, more private, to just say, hey, I create a bunch of examples and then I can train a system to do exactly that and understand exactly what it does. I can analyze where it's wrong. And there are a lot of use cases if you really care about the outcome and really care about the problem, where that's absolutely worth it, because if you hit a ceiling, then you want to build something better. And I think previously yeah, Sean, you mentioned the fraud detection or like, logs, what was it? Security stuff that's always kind of a race against each other. And that's one of these cases where I think I called it better is better. You don't want a model that does surprisingly well, given it's not trained for this. You want a model that's as good as it can get, and that's worth the effort. Companies spent millions on this stuff. And the fact that we have to create examples to train a model, it's like a feature, it's not a bug. It's like an opportunity. Sean Tibor: Well, I think that makes a lot of sense too, because I think Kelly and I have talked about this before. It's a lot easier for us to learn and understand through examples that are given as humans. And so it becomes easier for us to create that. So I'm thinking about, as you're describing this, one of the things just like the lightsaber that we talked about earlier. Finding a project and building towards it, right? Or finding some crazy idea or the bell ringing thing or whatever it is. Having some crazy idea and finding a path to making that happen is where real learning happens. And so where I'm seeing this opportunity with natural language processing is that there's a whole section or a whole domain of project ideas that students and learners can have, where they have their interests and they have things that they want to be able to do and being able to provide examples to the system that they create, to say, this is the type of thing I want you to find. This is the type of thing I want you to go look for. Ines Montani: Right, yeah, that's actually the hard part. And I think, again, that's also something that's not, in my opinion, talked about enough and that's often because it's not what the research is about. But the hard part is not the algorithms. Like, sure, if you want to learn about it, it's good to understand how does that work? But that's pretty solved and a lot of the optimizations there, it matters a lot less. What usually it always comes down to is the examples. And if you analyze those, actually, I think even for people who are learning, there's a lot of interesting stuff you can find out there. Like you look at even if you let several people annotate and create examples on the same data and then compare that and see where do people disagree? And then you can talk about it and then you very quickly have very deep conversations about language or like, I don't know, stuff like is the forehead part of your face? People can debate that for hours. There are a lot of these things where you could show, hey, this is like inconsistent data that you created there. Like here you included the doctor in the person name and here you didn't. No wonder your model is confused by that because now trying to make sense of why is it this way here and why is it this way there? And there is no sense. And that's what a lot of problems really come down to. And I think that is something you kind of don't see and don't get to experience if you just interact with a black box over an API. Sean Tibor: And I know a lot of our oh, go ahead, Kelly. Kelly Schuster-Paredes: Well, I was just going to say, I just got one of those ideas again as you were talking. I can see a lot of educators going, oh, what? You know, we're talking about AI detection or plagiarism or everything. And I'm thinking, oh, can you imagine? You can add in a whole bunch of the kids data or their writing over the years and then instead of having this massive AI. Detector you have actually a single student because that's what it is. It's about the data that you put in. So I think when you control your own project, you can control your own data. You can talk about the biasness of the data or whatever might be skewing your outcome. And I think that's what's neat about having made your own product, your own project yourself with that kind of feature. Ines Montani: And also actually bringing up the question. Even something like bias is much easier to look at in that context because you're really doing something with a specific purpose rather than having this model that's supposed to do everything and then can produce really bad things under different scenarios. Like if you train model on ancient texts yes. Even the weights you initialize it with. You sort of want a lot of this to be biased because the text is biased and you don't care about there are a lot of scenarios where it's like, now, that's not where you want a model that's optimized for an equal representation of men and women, because they're literally like, no women in this. And that's, like, a problem, but not the model's problem. Or if you're analyzing, I don't know, online comments with hate speech, you really don't want a sanitized model because otherwise it's never ever seen hate speech before. And then you can misclassify that. Or like your nice email filter will hallucinate a label. If it's never sean bad language before, it will hallucinate a label. And then, I don't know, your accounting department gets forwarded all the hate speech because your model has hallucinated that this must be accounting because it's kind of weird. So it's like there are a lot of these things where like, hey, really controlling what data goes in your model is useful and also really helps understand how these things work. Sean Tibor: I can see where you can make this really fun too, right? So a lot of the things we talk about is solving a problem, but you can also make it very entertaining. Also, for example, like an English class where you train a model on all of the Percy Jackson novels. Like, here's all the text from Percy Jackson and the Lightning Thief and all of the other sequels. And now everyone has to write a short story and we're going to see if we can make it. Trick the model into thinking that it's actually a Percy Jackson story, right? The same style, the same content, right? Ines Montani: That's a cool idea. There's also actually the spacey community has some really awesome projects. Like there's one model pipeline called Hobbit Spacey. People have trained pipelines for Latin. There's a lot you can do. And actually, yeah, I've never really seen it from that perspective, but even really doing some of these annotations yourself even with a group of students to really ah, okay, if we're training from labeled examples, here's how that process works. And then you go through like, oh, here I don't know, dave, why did you disagree with Alex here? It's interesting to have these conversations about the neurons of language. And actually, yeah, these are as I said, I think these are skills that are much more important than a lot of programming skills. Like, we had a team from The Guardian use our tools, Spacey and Prodigy, to build models, and they wrote, like, really detailed annotation guidelines. And you could tell that, hey, they're like journalists who really used to thinking about nuances of language and yeah, of course, they're really good at that part of it. And that's something that a lot of teams are missing and a lot of skills that don't get enough focus. Sean Tibor: Yeah, I love that. Just the idea to tie it into other domains and other areas and make it something that brings the language to life. Ines Montani: Yeah, actually, now that we're talking about it, I just had an idea because we could maybe talk about this a bit more later. Our first product, or the first developer tool we launched, is an annotation tool called Prodigy that really brings together the scriptability of Python and Eui for creating these annotations. So you can run that locally, load in your data, create annotations, and then it even has workflows in it to compare for disagreements. You can train a spacey model straight from that. So if someone's listening to that, then thinks, like, hey, this could be awesome to do in my group with some students to really do a bit more hands on annotation work. We have a research license program. I can give you some info. You can put in your show notes after this. And we can definitely we're always happy to support these sort of projects. And of course, if someone tries it out, let me know how you go. You're muted. Kelly Schuster-Paredes: That happens when you're coughing and you're covering up. I want to switch gears a little. Ines Montani: Bit. Kelly Schuster-Paredes: Because I think you have a talk coming up about maybe the future of NLPS and the LLMs, or maybe you already had the talk. I can't remember. I was looking and just as a person that's learning about all this stuff, go, I don't even know what the question is, but go. Future of NLPS, ll, machine learning, it. Ines Montani: Kind of makes futurists out of all of us. Like, there are a lot of takes. There are also a lot of takes I don't necessarily agree with. You have people who will tell you with a lot of conviction and also motivated by everything you see when you scroll through LinkedIn every day, that large language models will replace everything. And we don't need programmers anymore, we don't need Python anymore, we don't need anything else anymore. But from our practical view, that's not where I see things going. And actually, it's a bit of a short sighted view of things because it doesn't really distinguish between just capabilities that are generative, which yes, now work so much better. Like, you could generate text at a really high quality. That's great. And that's an effect of the models just being bigger. But all these other things people are doing with NLP and have been doing for the past decades in companies, I would say the majority is much more around information extraction, predicting things. And yes, large language models can do that, but currently they are a lot worse at a lot of these things, especially specific tasks, than what is out there. If you just fine tune a model with just more examples, like if you have low examples, it's still nowhere near the performance. And it all makes sense, as we talked about, and you still have this problem of, like, oh, parsing this output, which is text back into something structured, which also has a margin for error. And some people might then go and be like, well, it's just a matter of time. Models will get bigger and it will all get better. But even if model gets bigger and you can get much better accuracy than now, there's still kind of no logical way in which it beats a model that's very specifically designed and trained. There's a whole next step that goes, okay, now we have this massive model that's pretty good. How can we make it smaller and even better? What can we feed it to make it do even more? And that's totally possible. And I think that's where we see the industry heading. The sort of very futuristic idea basically assumes that companies and everyone will drastically change everything they've been doing and use completely different technologies for everything. And that's not something we've ever, Sean, really seen in the past. It's more of a very gradual development. And the research backs this up. Yes, they're like huge changes and huge things that we couldn't do before. But there's also a much more nuanced take I have on the future. Sean Tibor: It's interesting, you brought up a good point about kind of the evolutionary incremental change that a lot of organizations take, right? They aren't necessarily looking to transform their entire business. They want to just make parts of it better, right, or solve specific problems. And I think one of the things that struck me coming from that world before I started teaching around Enterprise and around the way the approaches everyone was taking, that became very familiar. And then when I came to teaching, especially in middle school and with these kind of teenage, early teen students or preteens, was amazing to me, was that they did not think that way at all, right? Because everything was new to them. Right. There's no incremental change when you just learned about this six months ago. So why not do something completely radical, right? Especially since they also have an underdeveloped sense of risk aversion. But they tend to, because everything is new and everything is equal in their minds. There's no history, there's no pattern of this is the way we've done it before, or these are the things we want to do. We saw a lot more experimentation with Kelly and I in terms of how students would tackle problems and how they would look at things, and they were willing to go try the completely different way of tackling a problem. And I wonder if, as we bring these kind of machine learning NLP ideas and models into those younger learners, if we'll see more of that kind of big risk taking strides, like, let's go combine two things together that nobody thought to do. What interesting thing could happen? Ines Montani: That's an interesting question, but I definitely think there's kind of a new generation of people who are much more like, I don't know, how do people call it? Like AI native and have a very different approach to things. But I do think, of course, there's also risk taking is good, but there are also aspects, especially if you have this sort of essentially black box model, there are a lot of things that can happen that are much more difficult to predict than if you just wrote some if else code, right? But I do think that, again, that's just a different learning process. Like, when you wrote your if else code, you realize at some point, okay, the whole thing crashes, or depending on the programming language, you just get like a SEG fault, and then you have no idea what went on there. And it's sort of the failure modes are all different. And that's something we have to adjust to more. But yeah, to the original point, I think transformative technology does not mean that we just throw everything away and do everything different. And especially if different means more expensive and worse. That's not how transformative technology works. Sean Tibor: That does not go over well in the annual performance review right. Ines Montani: And improves things, that there's no shortage of things we want to do as humans and companies want to do and have always wanted to do. And even before, maybe a team could do like two projects now, the prototyping, that's where LMS are great. You can get started more quickly without having to annotate examples that's like even I think also actually for teaching before, it was actually kind of difficult to explain these things and really walk through the process. Because you need to do so much work to actually get to a system that produces some output. Because you need to label some examples to even have a model that learns. And then you need your evaluation data, at least with large language models. And with these plugins, like in spacey, we have a whole add on where you can basically have the same experience and the same data structures, but just but powered by, say, OpenAI GPT four or even open source models that you can download if you want to run it locally. And then you have a thing that's working immediately out of the box and then you can work on improving that and see like, okay, let's create some evaluation data. Again, we will always need that. You always need to empirically be able to tell is the thing doing what I want. And for that you need examples where you know the answer. Actually, I guess from a teaching perspective, that's like the most fundamental thing. It's like a lot of exams. There's always a component about like, oh, you need stuff where you know the answer. And then based on that, you can decide whether someone is right or wrong and how they perform. Sean Tibor: How do we know they're learning, right? Ines Montani: Yeah, exactly. It's kind of similar. I do hope that nowadays it's not all multiple choice exams, but there is a component of that and it's similar for a model, but we can get to a working state much quicker. And I think that is a big transformative part. Not that we just throw everything out. Kelly Schuster-Paredes: I was just thinking it kind of reminds me of large language model. Give me feedback for a student, it's going to give me, yeah, you're doing well, or keep focus or sit down. Very generalized. But if we take in the other side where we're getting more specific for the job, we're getting more specific for a situation, it's going to give more specific feedback, stuff that can actually be done to improve. So that's kind of where I'm getting my metaphor on it, of the difference of the two summarizing what was just said. Ines Montani: Yeah, you go. Sean Tibor: I think it's fascinating that we're taking all of this amazing capability, all of this capacity that we have in terms of, to your point, millions of documents together and being able to relate them and make connections and hold all this stuff in memory. And we're having a very old conversation about a very new topic, which is the role of generalization versus specialization. Right. Where is it more valuable? Is it more valuable to have generalists and people who know a lot about everything but maybe not as much detail? Or is it better to have specialization where we have models or people that can really focus their capabilities in one area and create more value in that space? Ines Montani: It's kind of both, but I think both as in the world and also from a technical perspective, I think the answer or the highest value lies in both. A very generalist knowledge and capability lets you get started quicker, but then you can kind of take it to the next level with specialist knowledge. Kelly Schuster-Paredes: Quick question. Is there generative and predictive NLP? Like, does it depend on the way you do code or is it just generative for more of the large language machine learning AI? Ines Montani: I don't know if it's something we've coined that way or if it's something everyone uses, but I do think it's a super important distinction, which is, yeah, well, generative, you're really producing new output. And that's kind of also really sort of the underlying idea of how a lot of these models work or what. Now it's really unclear. People call everything large language models now it's really confusing. Like there are a lot of these foundation models that have been around for years and the architecture is kind of the same, the algorithms, it's all the same. It's just that there's now a new term for it and people are using that even for older stuff. But that's basically this idea of, hey, let's try and teach the system to predict the next word in a simplified way, the underlying experiment. And it turned out that not only do you train, they effectively train the system to predict the next word based on a huge corpus of raw text that actually works well. But it also the weight you're building up under the hood that make this possible kind of naturally and logically encode a lot about the language and the world, because you have to encode that in order to be able to predict the next word in context. And that was this whole revelation basically or that was the finding that really changed things years ago. Because not only can these models produce text, but they can also be used to initialize other models. Like for tasks where you really have a component or machine learning implementation that has the job of taking in vectors representing a text and outputting some label which under the hood is also like an integer. That's the job. And that works much, much better if every word gets a weight from those statistical weights that were trained by predicting the next word. Because then every word in context has all of this knowledge encoded within it and then these models work better. So that's how it all started and that's the whole idea. And then people went and said, oh, what happens if we make these things bigger? And then OpenAI was like, okay, hold my beer, what if we make this even bigger? And it was just like, oh it's big. And then wow, it actually works better and better. But I do think when talking about these things, I think it's incredibly important to differentiate between generative tasks where really the output is a text that includes translation, summarization, and then tasks where the output is much more machine facing, where it's like structure about the text. The text goes in and then it predicts, like, for every word. Is this part of, say, a person or from which word, to which word are words about a company name? Or even much more linguistic things like base forms. Sounds kind of boring these days, but it's actually super important, especially for languages that are not English. Like even if you want to make a word cloud, you don't want to include all forms of a verb. You want to include it once and that means you have to resolve it back to the base form in English, that's quite easy. Gets a lot harder in other languages, like German, Dutch. Yeah, English is quite easy. Is also the language that has the most research done in it. So the closer your language is to English, the easier it is for NLP, because that's what everything was designed for. Sean Tibor: It's giving me a ton to think about. And I think that's the fun part about talking about NLP and thinking about it is that bringing us back to where we started the conversation. It gets you to think about something that is very ingrained and intuitive, especially by the time you're an adult, the way we communicate, the way we express ourselves. What also occurred to me as we were talking about this is that the other thing that's interesting about this is that we may have in our mind this idea of like, well, there's the perfect text that is correct and grammatically solid and expressive, and it's good text. Right, but we don't speak that way. Like, my last three sentences were complete gibberish. Right? But the idea of being able to look at real text and real language, the way people communicate, the way they speak, the way they write, the way that systems generate text and create output generative programs, things like that, it means that we're really looking and working with real, tangible things. And I think also bringing it back to your point at the very beginning for learners and for teachers, what makes that real and makes it compelling is that you're not dealing strictly with theoretical or the abstract or that perfect idea of text. You're dealing with real things, real text, real words. And that's messy, right? Ines Montani: Yeah. And I think one main point I would make is that there's much more than just generating text or chat GPT. There's actually so much interesting stuff that's kind of at the intersection of human and machine and especially extracting structure from text. There's so many fun projects, even beyond, like, all the boring enterprise company stuff. I don't know. There's so much even as a hobby project, like if you're into Pokemon or I don't know what young people into these days. But even something like that, you can imagine a task of you want to build the database? Of everything that's ever said about every I don't know, Pokemon or someone from your favorite band or some other something or sports that's done a ton of. Kelly Schuster-Paredes: It'S got to be fortnite. Fortnite. So we analyze what that person's saying. We know that they're going to come up and set an attack or something. I can see it now. Ines Montani: Yeah, like some things people talk about Fortnite all the time and then they're different characters and they're different seasons, and I don't yeah, I know nothing about Fortnite. And then you want to from everything, from news articles, from Twitter, from Reddit, and you want to pass that and you want to really find out what is here. My categories are person, I don't know, fortnite season name. I don't know if it has different countries or different places, whatever. And you want to extract that and you want to do that live over everything on the Internet, which is going to be load. And then over time you want to populate your database so you can query that and say like, oh, what were the most talked about things? Or what do people say about this? And that sounds like, yeah, it's kind of fun, but this is actually incredibly similar to what a lot of large enterprise companies are doing and what's driving incredible value. And there you would like a good way is as if that you train a model specifically for that. You label a lot of examples, you highlight, go through them, highlight all the different entities you're interested in. Maybe your first idea is wrong again, that's also designing these label schemes, deciding what you want to predict, that's much harder than like, training a model. That's the hard part. Even as you start doing this, you realize this is ambiguous. I can't decide if it's this or that category. And then you're like, okay, let's scrap this, let's start again from scratch. The world doesn't divide neatly into categories, and language doesn't either. So it's like, that's the work. And I think learning that will also teach you a lot about the models and the technology that comes out of it and the limitations failure modes. Sean Tibor: I think I'm looking at the clock and I'm looking at the time, and I think we could probably talk for forever about this because there's a lot. Ines Montani: Of really I can talk about this. Sean Tibor: I think maybe the best place to wrap up is around maybe Prodigy a little bit and where teachers can get started. So if you have an idea about what you could do with NLP or you have an idea for a project, you mentioned earlier that you have Prodigy. There's research licenses, there's a community of people that are doing things with spacey where's a good place for teachers and educators to get started playing around with this and getting a feel for what's possible and what they can do with it. Where should they start looking. Ines Montani: So I think we actually put a lot of work into the spacey documentation. It also really goes through like, spacey one on one, I'm sure, yeah, I can send you the link to that, where it really starts at the very beginning, like what different things you can analyze about language with some code examples. Like, it does assume a basic knowledge of python, but also luckily that's something that's like, there's so many resources for that, but then there's really a lot of code. You can even run that in your browser and start experimenting with that and learn as you go. And then once you get to a point where you're like, oh, I have a great idea for maybe a text classification or an entity recognition model that I want to train on some raw text I have like our Fortnite idea or maybe I'm sure people have better ideas for what to do then usually. Well, what you want to do is you need some examples you can train from where you know the answer and of course also some examples you can evaluate on so you can actually check if your model is learning something and that also has to be good. And that's where Prodigy comes in because it's also a scriptable tool. You can write Python functions and integrate pretty much everything into it. And it also comes with a lot of things out of the box so you can start annotating named entities, it opens up the web browser streams in your text and you can just start highlighting that's like the most basic thing. You can also load a model in and improve an existing model or just look at what does my model predict in an interactive way. And it has interfaces for all kinds of different things and then at the end of it you can run train and then it will train you a spacey model from these annotations you've collected and evaluated. So there's very interactive aspects to it as well. We always actually like doing kind of workshops. I know a lot of lecturers in university, they like to use it with their students because it's very easy to get people to really look at the data and have that connection and not just have this abstract corpus that you calculate accuracies on. And also university, they need them to create data sets and the students are like good workers. They can also produce data as part of their courses. Sean Tibor: Nice. Yeah, it's definitely prompting some ideas for projects and hobby things that I want. Ines Montani: Nowadays you don't need that many annotations anymore. It's like absolutely not. Even if you're training a model from scratch with transfer learning, like if you start out with good word embeddings, something like Bird, a foundation model, you could get by with like a few hundred examples. And if you have an efficient tool, which is, again, why we always built Prodigy and the UI to be efficient, like, as little clicking as possible, just like, really just the knowledge you really need. And then you can easily do that even in a few hours, you can create quite a lot of data. And that's, like, not that much if you get your own machine learning model at the end. And that's like a model that runs on your local machine that can even run on a modern MacBook these days. That's, like, no problem. And it's 100% yours. I don't know, people have run it on a Raspberry Pi. I haven't really played much with it, but I know that's possible. Sean Tibor: You could run anything on a Raspberry Pi. Ines Montani: These? Yeah, you could actually copy your spacing model and then run it from your Raspberry Pi and I don't know, people can enter some text and then it can predict something about the text. I'm sure they're like even cooler things. You can, again, going back to the video games, I don't know. You can stream your one where people type in and chat. You can kind of stream that to your model and then predict things in real time about what people are talking about in the chat or how is the mood today? I don't know. So many ideas. Sean Tibor: I would love to have it predict how good they are as a player based on how much they type or what they're typing. Right. I think that'd be fun. Right? There's all kinds of great ideas. Ines Montani: Yeah. There's so much you can do. Kelly Schuster-Paredes: Don't get them started. Don't get them started. We'll be here all night. Ines Montani: Nobody sends me hate mail about, I don't know, my lack of Fortnite knowledge. Your listeners are mostly teachers and not the students, so I hope I'm safe. Kelly Schuster-Paredes: You're safe, absolutely. I don't even know what goes on in Fortnite and my kids play it. I just know they run around screaming at each other on the phone and they're like, come here, come here. Ines Montani: Cover me. Kelly Schuster-Paredes: I'm like, turn it down. Sean Tibor: But anyways, well, I think we can probably wrap up here. We'll grab a bunch of the links and put them in the show notes and share links to spacey, the documentation prodigy explosion so that people can get a sense of the work that you're doing ines. Thank you for joining us. This was really a lot of fun and super interesting. Ines Montani: Yeah, keep me updated if you end up trying things like annotating your own data, some of your listeners definitely keep me updated. I find it super. Yeah. Sean Tibor: Excellent. Ines Montani: Cool. Sean Tibor: Kelly, any announcements for our listeners this week? Kelly Schuster-Paredes: No, but keep an ear out for my next project. You'll like this, sean. Actually, I bought one of those magic mirrors. When you said Raspberry Pi, it reminded me it's a magic mirror, smart mirror. It's a project for my exploratory. We'll see if we can put it together. It doesn't come with the Raspberry Pi, so I had to buy the Raspberry Pi, but apparently, you know, we're going to have temperature and I actually have a mirror because we have no mirrors in our bathrooms. For some reason, past two years, they took them out. They just never put them back. I guess they were like, it's easier to clean. There's no mirrors in there. So now I had a mirror. That's going to be a smart mirror run on Raspberry Pi. We'll see. I might be calling you and getting some help. Sean Tibor: Sneak me on campus. I'll help you set it up. Kelly Schuster-Paredes: Sounds good, but nothing else happening. Good luck to all the teachers that are starting for their new year. Sean Tibor: Yeah, we'll be here for you if you want to. Get in touch with the show. Teachingpython FM is our website. You can always connect with us at teachingpython on Twitter. I know we've said this the last few weeks, but if you're enjoying the show and you think it's valuable, share it with someone. Send it to someone directly and say, hey, give this a listen. Or find an episode that you really like and share it with someone. Make that connection. Talk about it. Have the conversation with another person. I think that's really where in this crazy new world of social media, it's the human connections that matter. So make a human connection. Share an episode. Have a conversation. Talk about it. Talk with us about it. We're happy to listen and hear about it. If you liked our show, if you disliked our show, leave us a review. Hopefully just the ones that you liked. Leave us a review on your favorite podcast player. It helps just give visibility so other people can find the show as well. And if you want to support the show financially, of course, there's our Patreon link. Thank you to all of our Patreon supporters who keep the show going. I know it may not seem like much, but it helps cover the costs of hosting the show and makes it just nice for Kelly and I to see all the different friends of the show who are helping us out. It's a really nice thing and it makes our and as thank you for joining us. Ines Montani: Yeah, thanks again. Sean Tibor: Looking forward to having you back in the future and we'll talk more soon. So for teaching Python. Kelly Schuster-Paredes: This is Sean and this is Kelly signing off.