JOHN: Welcome to Greater Than Code Episode 159. I'm John Sawers and I'm here with Chante.

CHANTE: Hi, everyone. This is Chante Thurmond here and I'm with my co-host, Jacob.

JACOB: Hello and it's my pleasure to introduce today's guest. Lauren Maffeo has reported on and worked within the global technology sector. She started her career as a freelance journalist covering tech trends for The Guardian and The Next Web From London. Today, Lauren works as an associate principal analyst at GetApp where she covers the impact of emerging tech like AI and blockchain on small and midsize business owners.

Lauren has been cited by sources such as Information Management, Tech Target, CIO Online, DevOps Digest, The Atlantic, Entrepreneur, and Inc.com. Her writing on technology has also been cited by researchers at Cornell Law School, Northwestern University, and the University of Cambridge. She has spoken at global events including Gardner's Symposium in Florida, The World Web Forum in Zurich, Open Source Summit North America in Vancouver, and DrupalCon in Seattle.

In 2017, Lauren was named The Drum's 50 Under 30 list of women worth watching in digital. That same year, she helped organize Women Startup Challenge Europe, which was the continent's largest venture capital competition for women-led startups. She has served as a mentor for Girls in Technology Maryland chapter, and DCA Live included her in its 2018 list of "The NEW Power Women of Tech". Lauren was also shortlisted for the Future Stars of Tech Award in AI and Machine Learning by Information Age in 2019. Lauren holds an MSc from The London School of Economics and a certificate in Artificial Intelligence: Implications for Business Strategy from MIT's Sloan School of Management. Welcome to the show.

LAUREN: Hi. Thanks for having me.

JACOB: We like to start the show off by asking all of our guests the same question, which is what is your superpower and how did you acquire it?

LAUREN: My superpower is that I remember what I call useless yet sentimental facts about people. I'm actually really bad at remembering people's names when I first meet them because I'm too focused on making eye contact with them. But if I meet you once and you tell me your cat's name is Penelope, I'm never going to forget that you have a cat named Penelope. And I'll ask you about how Penelope is doing if I see you three months later at another random event. And I think I acquired that through my mom who actually has sometimes not the best memory but will remember people's birthdays or anniversaries or things of that nature. And it's really not something that I put a lot of effort or energy into. So people sometimes act surprised that I can remember these details about people and I just do that. I don't know how else to describe it.

CHANTE: That's awesome. I actually think that there's a study about that, that if you remember like a couple of factoids about somebody and their name and their face, that you have a better chance of remembering them later. And also it's great when you're doing sales because people want to know that you've picked up on something personal, right?

LAUREN: Yeah, I've often been told that I should have considered a career in sales. It doesn't sound very fun to me, to be totally honest, but I've heard that before.

CHANTE: Speaking of that, I think it'd be really interesting to kind of hear how you got into the work you're doing over at Gartner.

LAUREN: Sure. So I started my career in tech in London. I am from Boston, went to college in Washington DC and then immediately after that moved to London to do a master's program at the London School of Economics. And I was pretty convinced from high school onwards that I wanted to work as a journalist. And so, I oriented everything from my academic majors in college and graduate school to my internships around that very specific goal. I was especially keen to work in broadcast journalism. So I did internships at various news stations in high school and college. I was a broadcast reporter for a DC area radio station when I was in college. And so I focused on getting as much hands on work experience in that sector as I could. Unfortunately, I was doing that not only during the last recession but at a time when ad spend was really moving from news over to online advertising and specifically to tech companies like Google and Facebook who now own an enormous percentage of total ad spend all together.

And so that collapse of the business model for news coupled with the recession made finding a job extremely precarious. And so I did want to try making it work. And after graduating with my master's, I spent a year working as a freelance journalist reporting on tech news from London, which at the time had a small but rapidly growing tech sector of its own. And in the years since then, the tech sector in London has received very large amounts of investment and they're really a leader in the AI and machine learning space globally, which is great. And so getting to make a lot of connections in that sector at a time when it was growing but still relatively intimate, was a huge asset for my career. I realized quickly as a reporter that I was going to need a beat. And so I chose tech for no other reason other than that's what I was networked into. And I quickly developed the Rolodex that I needed to find stories and cover them for outlets like The Guardian and The Next Web. And I really enjoyed the work a lot. But I realized quickly that being a freelance reporter in your early 20's in one of the most expensive cities in the world is not an easy task. And so because it wasn't sustainable in the long term, I moved back to the States five and a half years ago. I briefly left tech for about eight months to do marketing for a larger healthcare firm and decided that it wasn't for me. And so then I went back to tech. I joined the marketing team of a Silicon Valley based SaaS startup, which was fully remote. So everyone worked from home. And for the past three years, I've been working as an analyst at GetApp, which is a Gartner company focused on helping small and midsize business owners find the best technologies to grow their businesses. And that's where I focused a lot of my research on bias in AI and giving presentations to business leaders about how they can mitigate it.

CHANTE: Wow. So cool. Thank you for that backstory and telling us just a great segue into what you're doing now.

JOHN: It seems like a lot of reporters who write about tech and especially about AI are boosters for it and very excited about AI and all the cool things it's going to do with our self-driving cars and all that. And you're focusing on bias and some of the downsides of AI. I'm curious as to what drew you to that aspect of it rather than the sort of glossy, shiny future aspect.

LAUREN: I was actually attracted to researching AI as a side project within my work at GetApp and Gartner because at the time, I was focusing on covering the project management and accounting software markets for small and midsize businesses. But this was in 2016 and we were starting to hear the rhetoric about how AI will "take jobs from humans" and I wasn't really sure what that meant. And so I started on my own time going through Gartner's repository of research on the subject to learn more about the nuances of that. And that was how I realized that a lot of the issues surrounding AI have a lot of confusion surrounding them, especially when they're reported by the mainstream media. So for example, Gartner writes a lot about the differences between automation versus augmentation. Automation is when you completely outsource a task to a machine, which is a lot of what any SaaS product does. But then there's also augmentation, which is when a tool helps a human perform a task more efficiently than they would be able to do it on their own. And so that's a big distinction that I think doesn't get enough attention in the mainstream media. And it's something that I started to research more as an analyst so that I could help readers and clients understand the distinctions between the two.

What really struck me and specifically struck my interest about bias in AI was when we had a guest speaker come talk to Gartner's AI research community about the concept of bias in AI. We had Meredith Broussard, who is affiliated with NYU and a former engineer affiliated with MIT, talk about the implications of bias in the data sets that are used to specifically train machine learning algorithms. And that was when I wanted to focus on AI and specifically bias in the data sets as a technical problem to solve because I don't think it's enough for people to say that the algorithms are biased. And I actually dislike when they're referred to as such because they produce biased results or they produce sexist results. But there are reasons why the algorithm produces those results. And if you can break those reasons down into various components, then you give the technical teams building those algorithms and opportunity to solve those challenges.

And so really, I would say my research on bias and the data sets is about approaching a year and a half old. So it's not something that I've been doing for a very long, but it is an opportunity where I've been able to focus a lot, especially because now I cover business intelligence for GetApp, which encompasses things like data mining, predictive analytics, big data. And so now, I've been able to turn what was initially a side project for me into work that I do full time.

JACOB:	I sort of had a dovetail question about, because John has like, it does seem like there's a lot of media about how just sort of cheerleading for AI. And I think the flip side to that, and it's more of in minority is, one is sort of the, like you alluded to like, that AI is scary. We don't know what it could do or what the implications could be that we should all just be very scared and highly suspicious of it. And I feel like people who don't consider themselves technical might be sort of wondering, "What do I do?" Because on one hand people are saying this will make everything great, this will fix all of our problems. And on the other hand it's going to destroy the world. And I feel like people are just saying like, "I don't even understand what this is. This sounds like something from science fiction and who do I believe?"

LAUREN:	Well I think it's interesting that it seems like the general consensus from this panel is that the media is overly bullish on AI. And I actually have slightly a different take on it, which is that the media is very alarmist about AI in a way that's unnecessary and unproductive. So it is true that any rules-based repetitive task is realistically going to be automated within the next five to 10 years. And so anyone who has a job that is based on those tasks is at risk of having the role automated. Although I think it's much more likely that those jobs will change rather than being eliminated altogether. But the reality is that the AI is not -- I think there's a lot of fearmongering in the media that is not always productive rather than having the media champion every new AI project. I think there's now a healthy dose of skepticism in the press, particularly around issues of data mining, which is tangentially related to AI. So now that the public is more aware of how much data large companies have on consumers, we're seeing a relatively minor but growing backlash where people want to know how their data is being used. And so one example of that is the Census Bureau in the States is going to hold its 2020 census next year, which is a big deal. And they are in the process of revamping the security for that project because they found a loophole where anybody using advanced data tools would be able to get data on approximately one in six respondents to the survey. And so the Bureau is currently not only revamping the security of the survey, but also putting in place strongholds that would make it illegal for anybody from a politician to a marketing team to target people based on their responses to the survey. And so you can say that this response is coming too little too late because big companies already know everything about us and that power is already concentrated. But I think we are seeing in the press now a move past the initial fearmongering into more realistic accountability reporting. Another example is the news that Google is now partnering with a lot of very large healthcare providers to upload patient data into the Cloud without being fully transparent about how that data is going to be used. And again, the fact that this is being reported at the outset rather than years after the fact I think is a positive sign.

CHANTE: Yeah. All right, so this is Chante. One of the questions I have is just in terms of your daily work and working with the small medium businesses, what are some concerns that are coming up when you're talking with folks, particularly from the business [inaudible] and business standpoint, from small to medium businesses as it pertains to bias in AI?

LAUREN: That's a great question. I think there are several challenges. One is that there's a lack of understanding about the implications of bias in AI. One thing that I always stress when I'm talking about this subject to technical teams is that whenever an algorithm is found to have bias, your only recourse is to scrap the model and retrain it from scratch back to a point before you saw bias, and that creates an enormous amount of work for your technical teams. And in the case of a small or midsize business where the product is based on that algorithm, you could in theory have to remove an entire product from the market, which could in effect close your business. If you're small or midsized, these larger companies can afford to take one product in their portfolio out of the marketplace. But a small or midsize business building their entire business on one cannot afford to do that. They also can't afford potential lawsuits that could result from someone who is adversely affected by the bias that those algorithms provides. And that's becoming more and more of a reality because the overarching reach that these algorithms have on people is huge.

If you look at the stock market and the fact that the top five most valuable stocks are all big five tech companies, they're using AI and machine learning to produce products that affect basically everyone. And so as a result, these tools affect much larger amounts of people. And as mentioned earlier, if one of their products is found to have bias within it, it's not as much of a financial implication for them to take it out of the marketplace. But a up and coming SaaS business, which is using algorithms to reach consumers could be put out of business. But if in the worst case scenario where someone brings a lawsuit against them for having produced a product that adversely affected them. One example of this that comes to mind is using an algorithm to decide who gets approved for a home loan, which is something that is a big problem largely due to the fact that these algorithms are often trained on data that was outdated and/or based on practices that are now illegal. And so that's just one of many examples.

I think also there is a misnomer that bias in AI is often intentional when the reality is that that's not the case. It's not that technical teams set out to create products that overtly discriminate against people. The reason for bias in AI is much more complicated than that. And so understanding how and where the bias creeps into the datasets is really the step that you want to take before you try to solve it. So I think in terms of bias in AI and how that affects small and midsize businesses, all of this can seem too high level. But the reality is that if that smaller midsize business owner is developing a product based on machine learning, all of this was hugely relevant because the implications could affect their business in a much more negative way than it could affect a big five tech firm.

CHANTE: Yeah, that's great. And actually the reason why I was wondering is I wanted to hear what people are concerned about, but then more importantly, I would love to know, and maybe you don't have all the answers, but I'm guessing you might, like if you do have a client that's considered a small business or medium business and they have an AI-based solution that does have bias, what can they do for the recourse to reset and to pivot? And is it Gartner that's providing or your organization that's providing best practices or do you have partners that help you do that?

LAUREN:	We do. Gartner is a research and advisory firm, and so we publish research for clients about how they can mitigate bias in the data sets used to train AI. And then GetApp where I work is a subset of the Gartner organization where we help small and midsize businesses find the best software and technology tools to grow their businesses. And so that small or midsize business owner can work in the computer software industry. Most of our users do work in computer software, but they can also be managing any type of small or midsize business, whether it's a bakery or healthcare provider. Really any sector that you can think of. And so in terms of what we advise people to do, the first step really is to define and document your priorities for the algorithm you're going to use upfront. And that really involves answering two questions in your text back. You want to address which methods of fairness you're going to use to measure success in the algorithm. And you want to know how you're going to prioritize them. And you need to also, at this stage, declare what's known as sensitive attributes out of bounds within the data set. So the best way to think of a sensitive attribute is to think of it as any class of people which is protected by law. So things like race, religion, sexual orientation, it's illegal to discriminate against people based on any of these sensitive attributes. And so in your algorithm, you should declare them out of bounds unless you have an explicit reason for including them. And again, you're going to want to document that, because this is really the first step in what's known as explainable AI, which is where you have documented rationale for how you're using the algorithm, why it makes the decisions that it does. And that way if anybody, like a client does come to you afterwards to want to know how the algorithm made certain decisions, you have that written down on paper. And this is equally important for people who are not just developing algorithms but looking for software that is purportedly powered by AI. Because the reality is that if you are a small or midsize business owner and you're searching for vendors that say that they are powered by AI, you are therefore using that vendor's algorithm to make decisions about your business and then you that power and responsibility transfers to your business.

And so it's really important that people who are shopping for software that uses artificial intelligence to make more informed decisions about when they're buying these tools. And frankly, if the account management or customer success teams can't answer those questions about methods of fairness and prioritization, then that's a sign that you should maybe be looking elsewhere at other vendors. And they might need to get that information from the technical teams, but they should be prepared to answer questions about how they're using AI within their products. And if they can't answer those questions, which many of them can because they overinflate their products' capabilities, that's the sign that you should maybe not be integrating that tech into your own tech stack.

JOHN: We've been talking about generally ethical people making decisions about how to use AI or how to market their own AI product. But I feel like there is a bigger problem where the ethics are breaking down. For example, just the other day I saw the headline that the court had allowed the maker of a biased sentencing algorithm. There was controversy about whether they should be able to use that or not. And the court said, "Yeah, it's okay if you use a bias sentencing algorithm, as long as you acknowledge that it's biased," and [inaudible] for that, whichever one is really good at doing, of course. So, what things can we do to try and push back against those sort of deeply entrenched things like an algorithm used by a police department where we don't have a lot of influence over what their purchasing decisions are?

LAUREN:	Yeah, that's a great question. And I think that's an important point to make because I do think the legal Titus turning and that in five to 10 years, there will be much more accountability expected of businesses and the algorithms they produce. And that's why we're, for lack of a better phrase, trying to future proof these businesses today so that they make the right decisions five years from now. But the reality is that the law at this moment is very behind where the tech is. And the example I always give is that there's an algorithm called COMPAS, which is used to predict the likelihood of a person to recommit crimes. ProPublica published a lot of research explaining how that algorithm made biased decisions based on race. But when one man tried to take his case against COMPAS to the Supreme Court, the judges refused to hear his case. It's called Loomis versus Wisconsin. So listeners can look it up if they want to. And I always cite that as a hugely problematic example of where the law is on bias in AI because that choice not to even hear his case signals that the most influential judges in the country condoned law enforcement using a biased algorithm to make decisions about people's court cases even when that algorithm was found to have been incorrect. And so, I think the key from a consumer perspective and a citizen perspective is to get informed on the issue and then really start influencing your legislators by making it clear that you don't want these tools in your community. So specifically you talked about police using facial recognition software. That's one area of the law where I am encouraged because we are seeing bands against that technology in cities like San Francisco and in states like New York. And I think a lot of that is due to having enough education amongst those populations about the adverse effects of these tools and having enough foresight to say that this isn't something we want in our communities until it can prove itself to be much more beneficial to [inaudible] these types of tools would hurt more than it would help.

JACOB: And how in a legal sense do you define bias? It sounds a little more obvious to, I think, us on this show, but how do you do that? In terms of like an AI?

LAUREN:	It's tough to do and I'm also not a lawyer, so I'm not the best person to ask on that. Georgetown University Law Center does do a lot of research into bias in AI and so they're probably a better resource for people wanting to learn more about that. But the advice that we generally give when you're thinking about this from a technical perspective is that when all else fails, you want to look at what the law says in terms of what is illegal to do in terms of discrimination. So again, it's now illegal to discriminate against someone based on their race, their sexual orientation, their gender, et cetera. And so you want to start with the minimum legal requirements when you're building AI. And that sounds very basic, but you'd be surprised how often that does not occur.

And there are also many ways throughout the data life cycle that bias can creep into an algorithm. It's definitely not enough to say at the outset, we're not going to discriminate against people based on race, gender, et cetera, because that's not really how it works. That's an example of direct bias in an algorithm. But most bias in AI is indirect. It's where a byproduct of sensitive attributes like race correlate with nonsensitive attributes like zip codes, and that's where the bias can creep in. And so I think it's actually a misnomer to say that you can just declare at the front, "Oh, we're not going to target people based on race," because that's great, but that's not enough. And so I think that's another distinction in the work I do is trying to get people to understand the difference between direct versus indirect bias in the data sets.

CHANTE:	This is Chante here again. I actually teach this, so I'm a diversity inclusion consultant, I have a business around this. And we call it unconscious bias, right? That's, I think, probably the correct term for people who are listening, just to know that there's lots of ways in which you can check your unconscious bias. And there's lots of organizations who provide training whether it's on demand or in person for that. And just for those who are listening who maybe have never done anything on unconscious bias, I would recommend that you check out Harvard's Implicit Bias Test. It's completely free and start there because we all have them. Everyone has an implicit and unconscious bias.

LAUREN:	Yup. No, I totally agree. And so that's it. That's an example of where knowing about it and acknowledging that it exists sounds very basic. But if you don't do that, then really all of the advice that follows is secondary to it. Because understanding really how these data points within an algorithm interact with each other is the key to mitigating it throughout the life cycle of an algorithm.

JOHN: And it sounds like a lot of what you're doing is trying to help surface some of the research that's going on around early detection and prevention of bias before it gets into the system so that the people who are trying to take care of these things before they happen can be up to speed and have their systems corrected for, so that we don't get into that state.

LAUREN:	That's exactly right. And I think that's a byproduct of me having been a journalist because when I learned that bias in AI is a huge problem, I was very interested in the problem, but I also wanted to know why it was occurring. And that led me to research the technical [inaudible] how and why bias can creep into a data set. Because the other thing is that artificial intelligence isn't new by any stretch. It's been a technology that has been in play for more than half a century at this point. And it has had more far reaching implications on our society than I think we even realize. So AI isn't new. What is new is the informist volume of data that is used to run these algorithms and the production of that [inaudible]. And that is where we're seeing a lot of the issues with bias in AI come into play. It's really [inaudible] managing the volume of data that's available. And the fact that volume of data is very high, but it's unevenly distributed largely amongst large tech companies that have broad portfolios of internet connected products that are collecting data at the consumer and also the business level.

JOHN:Yeah, I think you're right to point out that like in the 70's, they did a lot of AI research and things like that, but it just wasn't possible to collect a dossier on two thirds of Americans and start running that through a model. So we're sort of in a new space with that.

LAUREN:	Right. And also the level of computing power now is so much higher than it was decades ago. And so, bias in AI has existed for our googly as long as AI has existed. But the implications of that bias have never been higher than they are today because that volume of data is so high and particularly that volume of very sensitive data is so high.

JOHN:One thing that just occurred to me, I don't know if this is the case because I'm by no means an AI expert, but I know a lot of the initial, the early on AI like research was focused on top down. Like let's build a model of the world and then the machine will know this thing and then it can go do what it needs to do based on that understanding of the world. Whereas most of the algorithm, most of the machine learning and stuff that we do these days is very bottom up which is say, let's throw all the data at it. It builds the inference and then it can make all of its decisions based on just what it saw in the world. And I think that again, like you were talking, we need to be very careful where your source data is and make sure that there's not bias in the data itself. But it also leads us to the point of the fact that if the AI is trying to replicate the world as it is and the world as it is, is biased, then you have to do extra work to try and counteract that.

LAUREN:	Yes, that's another really important point and I think that addresses the issue of fairness versus accuracy in an algorithm. The thing that I always bring up when I'm talking to technical teams about bias in AI is that you do often have to make tradeoffs and algorithm's "fairness" versus its accuracy. And so that obviously leads to questions about why that's a trade off in the first place. And the example that I typically give is, let's say you have an algorithm that is found to make biased predictions against women when recommending who should move forward in the hiring process. And this is a very real scenario because a lot of the first looks given to resumes are not given by humans. They're outsourced to machines. And so then, those machines determine which resumes get moved over to people for interviews and such.

So one example of an algorithm doing this was Amazon. They had to scrap an HR hiring tool that was found to be making bias predictions against women in the hiring process. And the challenge there is that algorithms like that are trained on historical data whereby it is more likely that a man had a higher ranking title in an organization like CEO or VP. And so the data in that case isn't wrong because it is correct that a man is more likely to have had these higher ranking business roles because it's statistically more likely for a man to be senior in an organization than a woman. So it's not wrong. But you would also, I hope, want to correct course and not implement that in the future, which means that it is something that you have to be watching for. And I use that as a cautionary tale for why you shouldn't be outsourcing the data in these algorithms to machines entirely. That's another misnomer about AI is that you can train a machine and then let that machine take care of particular tasks without any human oversight. And that's a prime example of why that doesn't work if you don't have a dedicated employee on your technical team who's monitoring the data that your model receives in the production and the deployment environments, that's how you run into trouble.

JACOB: It seems like, and I definitely agree that you got to have diverse teams on your technical and organization wide. But this point is making me think about maybe the real problem is that we are expecting too much out of algorithms that we're expecting them to be able to produce a world that's more equitable than the one we live in currently. And gosh, just having better diversity on a team that produces an algorithm isn't going to help if we then turn around and say, "Oh, this algorithm is going to fix everything."

LAUREN:	Yes, that's certainly part of it. And I think that goes back to why I don't love when people say that an algorithm is biased or sexist, et cetera, because it's applying a human attribute to a thing and you're attributing those attributes to a thing that is created by humans. And so even though machine learning models figure out rules enough to train themselves, they only know from the data that they're trained on and that data is fed to them by humans. And so, that's another example of where we both in some ways, underestimate AI's power, but in many ways we overestimate it and I think that's equally as problematic.

CHANTE:	Yeah, this is such a great conversation. It makes me also want to just say that I think in addition to what Jacob said and what you said, Lauren, that we are living in a world right now where race, relations, and our identities and equities are at the forefront of everything. I mean, every day I pick up something and it's so blatant and so right there. And so, with the work that I'm doing right now, I've always kind of started off the conversation by saying like, to your point, how can we expect this AI, the input to be any different than the ways in which we think and view and have socially constructed our current world. Until we've made end roads to basically commit to being truly more equitable and more accessible and inclusive as a society, we can't really expect any of our technology or the data in which we produce would be better.

LAUREN:	That's correct. I will say that I'm heartened by the fact that there seems to be pretty universal consensus in the computer science community today that bias in AI is a problem. And so, I think we're past the point of having to convince people it's a problem, which I think is huge. I would say we're not there yet with diversity and inclusion. And so in that sense, I think we've actually oddly made more progress with bias in AI because there's again, pretty widespread consensus that it's a problem. The next frontier is solving for that problem. And the hunger for knowledge around this topic also makes me cautiously optimistic that it is a problem we can solve for. Because again, if we have those conversations now about what companies do with our data and how they build algorithms, I'm hoping that we can future proof five to 10 years from now when these algorithms have even more hiring, reaching decisions when computing power is astronomically higher than it is today. And so I'm hoping that if we again, can start at the beginning before many of these algorithms are created, they can hopefully be created more mindfully. But you'll always have challenges with bias in the data sets used to train AI. So I always caution that it's not a totally solvable problem in the sense that it's always something that you have to watch out for. But it is a problem that you can manage. And having the knowledge about how to manage it can really go a long way towards preventing a lot of the problems we've seen with algorithms thus far.

JOHN: It sounds like you've collected a lot of knowledge about this preventative work that's going on. Are there specific resources that you can point people to so that they can start getting educated?

LAUREN:	Sure. If someone's a subscriber to Gartner or their organization is, we publish lot of research on this subject for clients. And so, I would encourage them to start there. If someone is interested in data science, specifically open source data sets that you can use to train machine learning algorithms, the Towards Data Science Blog on Medium is a great resource. I use it a lot for references that I can use in my own articles. And so, those are two resources on the web that I would point people towards if they want to learn more about how to prevent bias in AI because the Towards Data Science Blog, in particular, host content from a lot of leaders in this space. And so, that's a free resource that I would guide listeners to if they want to learn more about these subjects.

And also the other thing that I'll mention is that if someone is interested in developing an algorithm but they are not sure how they can get the data that they need to train it. I know that GitHub has a list of open source data sets by industry. And so, let's say you're wanting to open a restaurant business and you need historical data to train an algorithm to make predictive decisions about when your restaurant's most likely to be dead, GitHub has that list, that repository of open source data sets by industry that you can use to train your models. The caveat I'll give is that you always want to look for larger data sets. More is more when it comes to training an algorithm because you need to give that algorithm both positive and negative examples of specific classes so that it learns what any particular variable is. So, a data set with 4 million data points is always going to be more valuable than an algorithm or a data set with 4,000 data points because the volume matters a lot in this case.

CHANTE:	Thank you for that, Lauren. That's very helpful. I would love to kind of transition a little bit and pivot to another topic that I think is of interest to you of these days is, I think we talked about this at the very beginning of the call before we started recording, but it sounded like you had been doing research or working towards understanding teams that are distributed and maybe virtually working together and not at the same location. So, can we talk a little bit about that?

LAUREN:	Sure. I'd be happy to. It's not something that I researched specifically, but it is something that I experienced in my last five years working in tech. I've worked on very globally distributed teams. And so, that has been the reality of my working life, and so it's something that is very different from the standard nine to five which you are presented with college and even in popular media like office space in the office. The reality of my working life is actually much more similar to what we're doing right now. It's getting on video calls with my colleagues and managers to connect with them because oftentimes doing so in an office on a day to day basis isn't even possible.

CHANTE:	How has that experience been good or bad for you and your team?

LAUREN:	I have actually over the last five years been very surprised by how much I like it. I did not expect when I first started working remotely to enjoy it nearly as much as I did. And there were two things that really stuck out at me. One was the amount of time it takes to commute to your job and back. When I didn't have to do that in my previous role, I got really used to waking up at 8:45 and getting online to work at 9:00. And not only not skipping a beat, but because a lot of my team was based in Central Simon on the West Coast. I was still early in getting online before them. And so not needing to spend up to eight hours a week or more on a commute was huge.

It also had the effect of me getting more involved in my local community because at the time when I was working for this fully distributed team, I was the only employee in my area. I live in the Washington DC area. And so, I didn't have that camaraderie in person of a team to fall back on. And so, I knew that I needed to go out of my way to get involved in the DC community. And so I started volunteering with my grad school's alumni organization. There is a way of doing that and hosting events for people and making connections that I couldn't get from an office. And so I think that was hugely beneficial because if I had not been so alone in the sense of working from home, I wouldn't have been as motivated to get involved in my local community. And so I think that was a surprising aspect of working from home that turned out to be positive.

JOHN: Yeah, I really like that. I've heard people talk at various times about how working at home can be very lonely. And I think that's a really good antidote to that. Not only does it increase your community engagement, but it also substitutes from that community that you normally get from a work environment. So I think that's a really good solution.

LAUREN:	Yeah, I agree. And it definitely can be lonely. The thing that I will say about it is that working from home and working with distributed teams, I would say you have to be very responsive and you have to be self directed. So if you're someone who tends to walk away from Slack for three hours without in a way message or an explanation, working remotely is going to be much more challenging because really, you can only be micromanaged to a certain extent because your boss isn't in the room with you. And so, the antidote to that is that if you're not being proactive about explaining where you are, putting away messages on Slack or in your office calendar, that trust can get broken down pretty quickly if you don't know how to use the digital tools your team is on in an effective way. But the counter to that is that if you're not in a client-focused role and you have a lot more of an opportunity to set your schedule and work when you're most productive, which for me actually tends to be later in the afternoon and in the early evenings. And so that's a huge benefit not only to quality of work but also being able to manage your energy on a day to day basis more effectively.

JOHN: Yeah, I think finding the schedule that works for you where you know the ebbs and flows of your own energy and being able to customize your working hours around that a little bit. Obviously, that's not available to everyone, even every remote worker. It depends on the team you're in and what your responsibilities are. But when you can, I think that can be really beneficial.

LAUREN:	For sure. I would say the big differentiator I've noticed among my friends in this regard is that anyone who does client facing work has a schedule that's much more beholden to the clients. And so, you don't have as much autonomy in your day to day. But in many roles in tech, as long as you have internet and a laptop and deadlines, you're really free to do the job how you see fit. And I think especially for anybody in a technical position, again, if your job isn't to client facing, you do have a lot of autonomy over your day to day, which is really the key, I think, to working remotely in a successful way is to own your schedule and to be proactive about communicating. And those two things I have found very helpful in terms of working on a remote team. And it's also really important because I often hear people say they worry about that lack of in-person visibility and they worry about that having an adverse effect on their careers. And that's all the more reason to be more proactive about communication because the reality is that if you're not in a physical office, you do need people to know you're there and you exist and you're contributing to the company. And there are ways to do that effectively over digital communication. It certainly doesn't have to be a trade-off of going down the company ladder versus having more autonomy in your day. You can have both if you find the right team.

CHANTE:	Speaking of that, do you have any best practices or resources that you can share with people who are listening in terms of maybe what your team already does because you are distributed or what you do personally to control and to make yourself more productive in a day?

LAUREN:	Yeah, sure I do. We use Slack like almost any remote team, and we're very active collectively on it. And so that I don't personally have my work email on my phone. However, I do have Slack on my phone because I want people to know that if they need to reach me, they can do that and my goal is always to respond as proactively as I can. We are working across many time zones on my team, and so people are very proactive about using the snooze feature in Slack, which prevents messages from going through to them after a certain hour. And that's a really important but subtle way to set boundaries with colleagues, like is saying that you'll get to their message in the morning if it's after a certain time period. I often hear from people and read, like I just read in the New York times an article about how to prevent Slack from taking over your life with the implication being that it's tempting to be at its beck and call every day. But the reality is that a lot of these SaaS tools have controls that allow you to mute or snooze notifications. And so learning which tools and features your software offers to do is really important because it's often a case of people not using the tools they have to the best of their ability. One example of a boundary I've also tried to set is with social media on my phone. The iPhone has a control which will cut off all of the social media apps on your phone if you've used them beyond your daily allotment. So, I have my daily allotment of all social media apps on my iPhone set to one hour. And after I've been online for a collected hour on the apps through my iPhone, I get a message saying I've reached my limit. I can override that limit and use the apps more during the day. But that's another control that I have found helpful and illuminating because prior to using it, I didn't think that I was spending as much time on these apps throughout the day as I actually am.

JOHN: Yeah. Slack just added a new feature where if your time zone is set and all your team mates are, it'll tell you what time it is in their time zone when you're about to message them. So you can realize that it's 11:45 PM in Dubai or wherever, and whether they've snoozed it or not, you can make a decision about whether to interrupt them.

LAUREN:	That's the bigger issue is that again, I think like with bias in AI, there's a temptation to blame the tool. And I would say that's an issue with the manager, not the tool. If the colleague or manager doesn't respect the boundaries of that person in Dubai, that's a team problem. That's not Slack's problem. And so, you can't abdicate responsibility as a manager to a tool. You still have to be using human best practices in order to use the tool effectively. And that goes for Slack as well as something more complicated, like a machine learning algorithm.

JOHN: That's a good parallel. I like that.

CHANTE:	Touche. I can't let the call go without asking this. Have your team or have you all gone under or done anything in terms of diversity and inclusion and culture as a team to kind of help you understand the values and the things that you all are trying to strive towards and to help you set boundaries for the work that you're doing?

LAUREN:	We have definitely, as an organization over the past few years, had conversations about diversity and inclusion. And I think those conversations are reflected in the people we've hired during really rapid stages of growth. Our business unit in particular has increased exponentially over the last two years and it has gotten notably more diverse. And so, I can't speak to particular programs that we've implemented, but I do know that is something that's top of mind for the leadership team. And I know that we also have a VP of D&I at Gartner who is very committed to these types of issues. Another SaaS company that I think is doing that really well is Slack. The Atlantic wrote a pretty in depth piece about their attitudes towards diversity and inclusion, and specifically how they think it's every team lead's responsibility rather than delegating it to one specific person. And then they talk about the effect that that attitude has had on their organization. And I think they're a really positive model there. Another person who I really look up to in the D&I world in SaaS is Aubrey Blanche who works for Atlassian. They produce a portfolio of technical tools that are widely used by almost any tech team today, whether it's JIRA or Trello or anything of that nature. And their commitment to D&I is also one that I think is really admirable.

CHANTE:	Thank you. Those are great resources. I've seen all of the ones that you mentioned and those are great. We can include those in the show notes.

LAUREN:	Awesome. Yeah, they're great resources if anybody is looking to learn more about the specifics of implementing D&I.

CHANTE:	Yes. Thank you.

JOHN: So now we come to the time on the show where we do reflections where each of us talks about the takeaway, the surprising thought, or the interesting idea that each of us is going to take with us after the show. I think for me, I really liked Lauren talking about the work that she's doing to pre-educate people so that they can prevent themselves from getting in trouble even before they build their models or learn how to correct these things early on in the process rather than years later when someone finally points out that there's been a bias problem for years. Because I think that's the only way that we can really address these issues is by getting ahead of it, learning all the different ways that you need to work with this technology such that you can keep that bias out of it. Like you were saying, you need operational responsibility, you need people reviewing the data inputs on a daily basis. You need, I think, awareness in the organization that this is what you're trying to prevent. And knowing that those resources exist and that that body of knowledge is being built out, I think it's heartening to me. So, it's nice to know more about that.

CHANTE:	Yeah, I would say one of the things that I enjoy about this conversation is just that you mentioned, Lauren, that it's not enough to just be doing this internally and thinking about your own product. It's actually really important to be thinking about the folks in which you're using it such as vendors and other areas that might be a source of where you're getting your data for instance, is key. So, just a great reminder to anybody who's listening that it's not just internal. Bias happens in all shapes, sizes, and forms. So the more that we can have awareness of it and trying to think about where we're sourcing information and the source of the issue perhaps is not just on us, it's on everyone. I think. So, thanks for bringing that to the conversation today.

JOHN: I just thought of a metaphor in relation to that. It's like, you probably want to know if your hardware manufacturers are employing child labor. And in the same manner, you want to know whether your info-product vendors are importing bias into your organization that you don't even know is there or don't want it there.

CHANTE:	Yes, totally. That's a good analogy.

LAUREN:	Yeah, that's a great analogy.

JACOB: Just to put things on a grim note [laughs], it's making me think about how in a biased society, we can't expect completely unbiased data and therefore we can't train an algorithm on this sort of theoretical, perfectly equitable world that we might want to create. And I guess it's just making me think about how, I think where that leaves us is you have to just recognize that even the best product that you create is always going to have a certain trace of the bias in the world that we have now. And I think what it's making me realize is that the real problem can arise when you believe that that isn't so.

LAUREN:	Exactly. I would say that's pretty [inaudible]. We need to [inaudible] bias exist in the first place and I know a lot of people in, for example, accessibility in computer science who have decided not to spend their time convincing people that accessibility matters. They want to spend their time working with tech teams and developers to implement accessibility and build accessible websites because that's not only an easier solution, meaning a lower energy solution, but it's really acknowledging that you have limited time and you need to spend it with the people who are committed to caring about the issue rather than trying to convince someone that it's a problem in the first place.

CHANTE:	Yes. I would say from personal experience, that is absolutely the truth. And I'm so glad to hear that there's other people who feel that way because, why? Why waste anymore energy on people who don't care and who want to argue with you? And there's a lot of people who do and there's so many other ways to make an impact in the world and the lower or the lighter the width, the better.

LAUREN:	Yep. I agree.

JOHN: Great. This has been a fantastic conversation. Thank you so much for joining us, Lauren.

LAUREN:	Oh, thanks for having me. I had a great time and I'm really looking forward to hearing the episode.

CHANTE:	Me too. Thanks, Lauren. Thanks everyone, Jacob and John.

JOHN: Goodbye.

CHANTE:	Goodbye.