Happy Hour 99_mixdown.mp3

Harpreet: [00:00:06] What's up, everybody? Welcome. Welcome to the Artist Data Science podcast. It is happy hour number 99. Friday, October 7th, 2022. Happy hour number 99. Next week is happy hour number 100, man. Hope you all could join it. Look, if you are one of the people that are tuning in, you know, later, like listen to the podcast episode, please come to the happy hour. Come to live happy hour. This chat you know you guys know how to register write bit.ly for ads. So come to the happy hour session. I'll be more than happy to have you guys here. Or if you just want to send me a, you know, a question that you want to ask. You guys know my email address. The answer Data science at gmail.com. Hit me up. If you're watching on LinkedIn, if you're watching on YouTube and you got a question, please do let me know. Just drop your question right there in the chat wherever you are watching, and I'll be happy to get to you. I just got done with recording a podcast with Richmond, Awlaki. I was on I was on his podcast. So I feel hyped up and like, you know, there's a reason why I stay on this side of the microphone and stay the question asker.

Harpreet: [00:01:18] Because when I go on other people's podcast, I don't know if what I'm saying is just rambling or if it makes sense to anyone or from providing any type of value. So, you know, that's why I tend to stay on the side of the podcast. But I had a great conversation with Richmond, excellent host asking great questions. So shout out to Richmond if you guys don't know Richmond, check out his podcast. There's some great work if you're joining in on LinkedIn and you got questions, let me know. Shout out to everybody in the building. So far we got Russell Willis and the building the coast of Krishnamoorthy crusted was at the Brisbane airport waiting for a flight flying out to Sydney, hopefully having a good safe flight out there. Shout out to free dot bellow in the room. Happy to have you guys [00:02:00] there as well. If you've got questions, let me know. Madam, I'm happy to take your your questions and kick off the stream. Then what's on your mind, man? Let's start there, man. Let's see what's on Vin's mind to just riff off of that.

Speaker2: [00:02:15] Wow. That's a Let's put me on the spot. Worried about the economy, worried about jobs reports. And right now, just finishing up like budget season and strategy planning season, wrapping that up in the next couple of weeks. And it's it's weird out there that's that's what's on my mind right now is just it looks good for data science looks good for data professionals in general. But just tech as a whole looks looks like there's some really bad times ahead because there's so many companies that aren't profitable and all of them are just getting beaten. It's ugly out there for them.

Harpreet: [00:02:53] Is there anything like in particular that's top of mind that you think is is causing some of these companies to not be profitable or, you know, not have the returns that you think they would?

Speaker2: [00:03:06] Yeah, money's been so cheap for so long. That's really what it's been. You've been able to borrow and fundraise and everything else. So valuations were, I don't know, auto wax, not serious enough, but crazy's too serious. I think that was the problem is companies I know companies that have been around for eight years, nine years, ten years, never been profitable, never had to be profitable. And now over the last year, their stock price is taking like a 90% hit, somewhere between 60 and 90%. Founders are they're looking at their cash piles dwindling. You know, if they didn't reraise at the right time earlier this year or late last year, they're they're trying to figure out what to do. There are more established companies that they can't service their debt. And so if they didn't [00:04:00] refinance their debt over the last six months when everyone was yelling at them to, but some companies actually didn't refinance and they're in trouble, too, because they can't you know, they can't push their debt off right now. And so they have to service some of those big payments coming up soon. And like I said, if you're not profitable right now, all you have is cutting staff. And if you look at companies like Peloton, Peloton is just I mean, they're cutting people they can't afford not to cut. You're starting to lose critical service levels. And I think you're going to see that with a lot of other companies where service levels are going to drop to the point where they can't keep their customers anymore.

Harpreet: [00:04:39] Facebook's actually announced something similar as well, writing like hiring freeze and restructuring it and letting folks. Oh, but let's go to Kosta real quick. Go for it.

Speaker3: [00:04:50] I'm not sure how all you guys can actually hear me. Pretty much. All right.

Harpreet: [00:04:54] But see. Yeah. Yes.

Speaker3: [00:04:55] Good. Okay. Awesome. How much is that? If that is like I want to call it COVID inflation, right? Like there's a bunch I mean, Peloton, obviously they they took an absolute skyrocketing during COVID because they provided a service that was naturally valuable at that time. Right. And how much of that is just generally US tech companies looking at, you know, VC funding, where a lot of it comes down to how much can I set a valuation now that I know I can increase the valuation to sell off again, as opposed to, hey, we're actually providing genuine value, right, as a as a tech company because in Australia we're seeing kind of a mixed bag. We're seeing some early stage tech companies that are super speculative, that are getting hit pretty hard. But then there are other tech companies that are doing pretty well and they're profitable and it's fine and they're not really getting hit nearly as hard. So you've got some companies that are able to, you know, essentially consolidate and grab a lot of the talent. How much of it's like the COVID thing, [00:06:00] how much of it is the general overvaluation? How much of it is something else? Is there something else?

Speaker2: [00:06:08] Know, there's definitely that macro trend. I mean, a lot of it is demand got pulled forward. So if you look at what's happening with AMD right now and Intel, you know, the PC market went crazy because there was everybody was at home, they needed new machines they needed, so they were buying and that pulled demand forward. So there's definitely a lot of that Peloton's I think a different case because they had demand pulled forward and then they bungled it. I mean, they acted like it was going to be like that forever. And so from a supply chain standpoint, they had massive overproduction, you know, And you see that Intel did a little bit of that. But Intel also has some issues with, I don't know, issues with supply chain and decision making, just kind of across the board. They have issues executing. And so you're kind of hearing like this is the mixed bag that we're experiencing right now. There's demand pull forward. There's still supply chain issues that companies that don't know how to use data, which kind of baffles me at this point, how do you not have a handle on your supply chain? Apple is one of the most complex supply chains on earth, and they figured it out. I mean, our companies that are supposedly more advanced than Apple not figuring out their supply chain by now. So that's where.

Speaker3: [00:07:28] How much how much of something like that comes down to an experience as well. Right. Like you're seeing the profile of tech company leadership. A lot of people are young people like, you know, I'm in my late twenties, right? I don't know how efficient supply chain is managed, but if I put together the right, you know, package, I'm sure like maybe four or five years ago, I could have got a pretty decent VC funding round or, you know, early stage angel investor to fund me a fair amount without actually knowing how to solve some of those problems. How much does if it comes down to [00:08:00] are we too myopic in terms of what a good tech leader looks like is, you know, if you're young under 35 hotshot CEOs that are doing that, how much are we undervaluing that in our hiring process is like, I'm finding that I want to look at companies where there is a really good balance of youth and really good balance of people with solid experience and especially operational aspects of the company. Right? How much of it comes down to are we too myopic as an industry?

Speaker2: [00:08:31] You know, a lot of it, a lot of these companies, you can see a good step forward. And this is actually something investors love seeing is when they bring in an experienced CFO, when they bring in an experienced CEO. And those are kind of the mid stage when you're trying to scale. If you're smart, you bring in somebody who can handle operations, somebody who's focused on revenue and sales, somebody who's focused on on the financials. And those are all good hires you're starting to see now those people are showing up. In some cases, it's too late. You got companies promoting CFOs to CEO roles. That's a bad sign If you're a tech company, that is the beginning of the end for you. That's basically your investor saying, Yeah, we don't believe growth anymore. We want to see you guys cut a lot. And that's what the CFO comes in and does. But you're right. I mean, when you talk about leadership profile, the CEOs and the founders, you've got a president, co CEOs and those types of co founder type joint leadership. They're doing what they're supposed to be doing and they know they're part of the business.

Speaker2: [00:09:38] They understand they're part of the customer base. It's the smart ones that bring other people in as they grow and start augmenting the leadership team. I mean, you see that at the very beginning when they bring people onto the board and they'll go to VCs not just for cash, but to get a couple of advisors and you'll start seeing them go to companies like Bessemer or Sequoia. They don't need the cash. They need some smart [00:10:00] people. They need a couple of people from their from their analyst group or a couple of their parachute team, you know, the people that they go to and they'll put them on the board and they become advisors. And then those relationships lead to a CFO or a CEO or CRO or something like that. And those are I mean, I every once in a while get brought in as part of something like that where a new investor comes in, they bring in a new C-suite, and then I'll come in to take a look under the covers to see what's AI and what's BS. So I mean, that's the that's the maturing. And if that doesn't happen.

Harpreet: [00:10:40] Missiles. If that does not happen. Shout out to Richmond like I was just talking about Richmond at the beginning of the hour that I was just recording with you. And now you're here. Man. Good to have you here. Hi. Right, guys. So a question about valuations. And one thing that I thought was interesting and I don't know how much like, you know, I'm probably misunderstanding this, but it seems like valuations are very subjective, like a startup could value itself out whatever it wants and get, you know, get investment at that valuation. Is that how it works then?

Speaker2: [00:11:19] For a while it wasn't just startups. I mean, major companies are just throwing numbers out. That didn't make any sense. Just across the board companies with no profit where multi billion dollar companies. And that's that's something that almost every every economist just kind of said what how is that even possible? You aren't making any profit. You're losing money for several years in a row and you're worth many billions. Really. There's there are some backwards valuations where companies that never made any money were worth more than Exxon and these really massive companies. So yeah, for a while it was just all witch [00:12:00] SPAC. All of them, all of the above. Spacs were kind of like peak peak insanity. I think we're going to look back at SPACs in the same way we looked at at some of the the mortgage backed securities and the junk mortgages back in 2008. I think SPACs going to be said in the same in the same light here pretty soon. But yeah, the valuations for a while they were if you could get a VC to believe it. Yeah.

Speaker3: [00:12:29] Sure.

Harpreet: [00:12:32] What is a spec? I don't think I'm familiar with that.

Speaker2: [00:12:36] Long story short, it's a whole bunch of very wealthy people that put money together in such a way that they can acquire companies that can invest in companies. They can get around a whole lot of regulations. They can do some some interesting accounting. There were some fake it till you make it. That's it. There were some benefits to a SPAC that I don't 100% understand, but everybody put one together because it was beneficial pooling of cash that you could get away with some things. From what I understand, there were some some loopholes that you could get through, and I don't want to call out a client, so I'm not saying particularly what loopholes.

Speaker3: [00:13:16] Yeah.

Speaker2: [00:13:17] I'm getting mad.

Harpreet: [00:13:19] Eric says a special purpose acquisition company, that's a shell company that is traded publicly, that acquires other companies to take them public. That is interesting. That is the interesting.

Speaker2: [00:13:30] I don't know why you needed a SPAC for that. You could do that without a SPAC, but. Sure. No, I'm glad. I'm very happy for the people that made money on those SPACs. Right now, there's so much money in them and no one's no one's writing checks. And the people who invest in those SPACs right now are going, Oh, come on, guys.

Speaker3: [00:13:51] I think there's like some accounting benefit, like you mentioned two SPACs as opposed to going through the normal [00:14:00] tried and tested rule, which is IPOs and as in the SPAC. King Chamath I can't say his name. I do listen to his podcast and he's put, I think, maybe two of his SPACs as in a solid return of money to investors, because I couldn't find the right vehicles to put the money into IPOs. Spacs, they sound like a blank check for start ups to essentially get into and just do what they do relatively as not a lot of SPACs as being very successful with their valuation that's just dropped. It's been it's been ridiculous lately.

Harpreet: [00:14:42] This is something I've never heard of until today. I'm going to have to check this out a little bit more. Thanks so much for bringing it up. Shout out to Eric Sims in the building. What's going on, Eric? Jacob in the building as well. Good to have you guys here. If anybody got questions or comments or anything they want to bring up, please do let me know. I'm monitoring LinkedIn and YouTube. I see you all. I'll see you all there if you're watching on LinkedIn. Go ahead, Smash. Like Eric. What's going on, man? How's your how's your week been this this week?

Speaker4: [00:15:09] I think it's been a good week. I'm trying to remember the last couple of days have been kind of crazy. Yesterday I just was like working on something and the product manager is like, Hey, I need these two numbers to these two big things. We've done this last quarter. And I mean, could you happen to me by the end of the day? I'm like, Yes, yes I can. But by the end of the day, I was like, as long as it's by the end of the day, like midnight, we got it. No problem. So got that, got that taken care of. And then we're buying a house. And so we had our home inspection today, which was and it was fine, but there's just always so much craziness associated with the home buying process, which I don't know. You think we would have it simplified after having done it for like a couple hundred years or something as a as a country. But now it's just complexity, just breeding more complexity.

Harpreet: [00:15:59] Well, congratulations [00:16:00] again in the house, man. That's awesome. I'm super happy for you and excited to to to maybe see some pictures. Send some pictures. Doesn't that and that group chat we got going on.

Speaker3: [00:16:10] Yeah.

Harpreet: [00:16:11] So talks about if you can kind of just like at a high level this this interesting request that came in what was kind of like the nature of it. How did you kind of navigate that that challenge?

Speaker4: [00:16:23] Sure. So the request is just to get some numbers to kind of summarize the impact of some improvements that we've made over the last quarter. And there have been two important things that we've done, both related to how we route the lending tree where lead generation company, right? And so it's to how we route leads to experience a experience B or experience B, one or B two. Right. And this this one in particular was a little bit tricky because I had kind of a couple of different layers. And one of the ways that has really helped me organize my thoughts for getting numbers to somebody is one I got to think about like, what does the product manager care to see? Right? Because I could just send them a deluge of numbers and a spreadsheet and he would like probably just he would find a smart guy, but that's not really his job. His job is to do product these stuff. And so what I've started doing when I'm dealing with projects that are a little hairier or have more layers to them than I can like store in RAM is I get out a word document and I'll just like start writing questions, hypotheses, whatever. As I go to be like, what was the percentage composition of this? What about the absolute something? Blah, blah, blah, blah blah.

Speaker4: [00:17:44] I kind of write a few things out and I'll just go one bullet point deep on each on that first one and just go, go, go, go, go. Until I've kind of answered all my questions and I back up and then go, go, go, go, go down. Another cascade of bullet points to the next question that I had. And that way, you know, [00:18:00] one or two pages later when I'm like, Oh, crap, did I validate such and such a thing? I just scroll up. Yes, I did. Okay, we're good. Freakout averted. You know, go back and continue on with my work and then when I'm done, I've got this nice little sheet of work that I've done with numbers that I can look at and then pick out the ones that are the most important, put them into bullet points, and then just really try to distill it down to the very simplest thing that I can share with somebody. And I've really gotten into it. I didn't really do this kind of word document type, stream of consciousness, whatever you want to call it, interstitial journaling type thing until, I don't know, maybe like six months ago. And I found it really helpful for keeping keeping myself organized as I go through a nonlinear process.

Harpreet: [00:18:48] Eric, thanks so much for sharing about your workflow, man. That's I like that depth that's that that you shared there. If anybody's got questions on LinkedIn or on YouTube, do let me know. Monitoring all the channels. Happy to take any questions. Otherwise, we just we just go with the flow, man. Richmond, it's good to have you here, man. Good to get to have you here on the first time. I know we've been in contact through your podcast. You've been pushing out some amazing episodes of your show with I know Vince been on an episode there and a couple other friends of mine. Man, How have you been?

Speaker3: [00:19:21] I'm been great. I've been great. Today's a good day because I've got to speak with you. Yeah, Minutes ago. Yeah, today's a good day. Today's a good day. Couple technical things going my way. So it's always a good day if that happens. Yeah.

Harpreet: [00:19:35] So you're working as a I guess you. The title is like an architect, right? Solutions. Architect. If I.

Speaker3: [00:19:43] Have I. Machine learning architect.

Harpreet: [00:19:46] Yeah. So this is a title I don't really come across very often and I think it's one that is probably not familiar to a lot of people. I know Vint talks a lot about machine learning architect as well, but how is the role of an architect [00:20:00] different than, let's say, a traditional data scientist or maybe even, you know, a machine learning engineer? What are some of the differences?

Speaker3: [00:20:11] Yeah, as in my background, into the field was working in startups, so I used to work as a computer vision engineer or a male engineer before becoming a machine learning architect. And the key difference between the engineering level and the architecture level is mindset, right? So at the architecture level, you think about things that are very high level. So I find myself thinking about what the data engineer and what the data engineering team is doing, also what my ops team is doing and what the engineering team is doing, and being able to take this different perspective of building intelligent products or machine learning systems and be able to communicate that in a client in several different ways. So it can be verbally through diagrams or through proof of concept is is a unique skill. Whereas within the engineering side of things, I just had to deliver a feature, right? I just have to build a computer vision model that could classify, I don't know, cats and dogs or whatever on an app. And, and that was it. So it's mindset shift and awareness of the overall aspects and roles within machine learning essentially. And it's an interesting field. So I started off, I started off exploring the data engineering side of things and now I'm exploring more ops and man ops is a crazy world. It's a crazy world filled with loads of tools.

Harpreet: [00:21:35] Yeah, filled with a lot of that. Vc money has been pumping through ML Ops tools for sure. So it was being like an architect. That's something like somebody can jump right into after, you know, completing some training or just require like years of experience.

Speaker3: [00:21:52] So I'm not going to I'm not going to front and like I've been I've been in the field for decades. I've been a machine learning practitioner for [00:22:00] three and a half years now. Two years? Yeah, three and a half years. Two years as a computer vision engineer. But here's what's happening actually now. Two and a half years, two years as a computer vision engineer, seven months as a as an architect. And I was a computer vision, computer vision engineer startup. And the reason why I'm mentioning that is because I have to be able to think about the entire system because I was almost responsible for the entire system in a startup, right? So in a startup you wear different hats. So I was talking to the CTO and I was talking to the to, to the customer, to the sales team, talking to the designers and just explaining what I'm doing, very high level. So in a startup you have to wear different apps. And I felt like that prepared me for this machine learning architect role from from the mindset perspective. Whereas if you're working in a large corporate company or big tech company, you'd be very focused on your role because they got people to do the other things. So you might not get that exposure, but it's machine learning, architecture, something you can learn, and then jobs jump right into the role. I would say no, you need the experience of actually building system and understanding this system to be able to execute on this job function. Although there are system architecture lessons you can take and there are loads of books, there's practical mock ups, machine learning system designs, and there's so many books that you can take or books that cover things at a high level. You can consume that we prepare you for this role, but experiences can.

Harpreet: [00:23:44] So talking about that might mind. Shift. So when you're working as just a say, gestate, but when you're working as a engineer at the engineer level, where you're kind of just doing one feature, I imagine that you kind of working in terms of just writing [00:24:00] a lot of code and just kind of narrow, focused on small bits of it. And it seems like for the architect, maybe you're not coding as much directly onto one particular feature, but you're kind of taking the bird's eye view and seeing how different pieces fit together, I guess. I don't want to put words in your mouth. Why don't you tell us this mindset shift and specifically the mindset shift you had to make?

Speaker3: [00:24:24] Yeah. One thing is, I would like to say being a machine learning architect is not the same as being as, I guess a building, an actual physical building architect, because they can draw the the plan and the the scheme of the building, but they can't actually build the building. But when you're a machine, an architect, you need to know how to think about things from the architecture level, but you also have to be able to implement these things as well. So you need to have you need to be basically have a balanced skill set and that is, I guess the title is a bit misleading because you think it's just dealing with things, the architectural level, but you are writing a bit of code as well and you have to have some not in depth knowledge, but enough to be able to deliver either proof of concepts or be able to communicate the advantages or disadvantages of sets and decisions or to or tools essentially. So I find myself doing a lot of knowledge building, right? I've got a bunch over the next coming months, I've got a bunch of articles that are going to be coming out on state engineering, data ops, and there's a lot of learning to do, and especially with a fit of machine learning. It never stops.

Harpreet: [00:25:40] And you mentioned MLPs, like is that synonymous to like a machine learning architect? Is that like an ML ops engineer, machine architect the same thing, or is it just the overlap is really great?

Speaker3: [00:25:52] Yeah, I think it's more like the overlap, like the machine and architect is thinking about things from a high level, so you need to know what machine and ops [00:26:00] engineer is doing and also how to do what they're doing as well, which is the key. The key thing. I also need to know what data engineering is doing and also how to do what they're doing, but I don't need to know it at the depth that they know it. But I also need to know how how they do it to be able to either come in to the picture or to be able to actually drive some technical requirements as well. So that is my ops is just my area of focus. Now before it was data engineering. Like I said, I'm just building that knowledge base and I'll be moving more into data ops. It's time to look more interested. So maybe in a three months time I'll be going into data offices. Mark Mark Friedman puts a lot of great content on data, so I'm consuming a lot of his LinkedIn post.

Harpreet: [00:26:49] Yeah, Mark's got the pod, not the podcast, but the newsletter launching about data ops and a whole bunch of content around that. That's something that he's really big into just talking about. Real quick, you mentioned you're creating articles as you're learning and like it. Talk to us about that because you're quite prolific on on Medium. You got a lot of articles, huge following on on Medium. Is writing a way that you kind of teach yourself or how's writing fit into your learning scheme?

Speaker3: [00:27:23] Yeah, Is it Writing is essentially part of my of my learning. It's I can't learn about not writing and putting an article because if I can't communicate what I've learned, I don't feel like I've actually learned it. And Kavcic is on the call. So me and council went to the same university and I used to write articles of most of the conversations we were having about computer vision and deep learning. And I'll go back to my room and just try to think about it and write it like I'm teaching it. And I'm still doing that [00:28:00] three or four years down the line. So nothing has changed. And, and, and it's it's part of my process. It's part of my process.

Harpreet: [00:28:08] That's one thing that I've been working on a lot lately, is kind of like building a second brain, building a better relationship with information and just finding a better way to consume it, distill and then create. So I'd love to you know, there's there's a lot of prolific writers on the call. For example, Venn puts out a newsletter that comes out multiple times a week, and it's just filled with so much knowledge. I want to drill down into like kind of your guys's processes for, for how you navigate. Okay, here's the world of my experience in the world of content. Here's how I go through and then create this new piece of of work. So you start with Van, then let's go to Richmond, then I'll talk a little bit about my own process. How do you go from ideation to output? What? What's the kind of process like?

Speaker2: [00:28:56] That's an interesting one for me because I'll plant out about a week in advance, sometimes two weeks, and I always get derailed by something that's way more interesting. That happens during the week or at the beginning of the week. So I'll typically get one planned post out and two, sometimes three of them are just, Well, I got this great question from somebody. I think with me it's probably different than a lot of other creators because I'm older and so there's a lot of stuff in my head that I could write and I have to kind of stop myself sometimes because I shouldn't be writing about things like ML Ops or data ops or ML engineering. So I think part of my process and it's probably to be what's unique from from yours is I have to back away and not write about some stuff. I also want to look for an angle that's different because everything that's out there is going to be covered eight ways, nine ways, ten ways, sometimes 15 different, same ways. So what I want to look for [00:30:00] is something that I can bring to it from a perspective standpoint that's unique, where I can reveal something. So if you're reading one of my posts, you're going to discover something new. You're going to get a different perspective on it. And for me, a lot of times that's just bringing leadership into it, bringing the business into it, bringing a larger picture view into it, talking about how like on Monday I was talking about how macro factors can break your model. And there's a significant risk right now of having macro factors just absolutely decimate your model if you're not monitoring it correctly and not monitoring your data correctly. So not getting too far into the stack, but bringing insights to people that you maybe won't get elsewhere. I think that's the biggest part of my process.

Harpreet: [00:30:44] Then. Thank you so much. We're going to come back to what a macro factor is and how a distribution model after we hear from Richmond in his process. Richmond, go for it.

Speaker3: [00:30:52] Well, I like Ben's answer. I also like your right. And I just got one of your emails because I'm subscribed to you, Substack. And the way Vin writes is so it really encapsulates the concept of leadership because he writes from such a macro level where it lets you understand and relates it to the to data professionals and you understand you do come away with something new. I take a different approach, or because I'm a bit more selfish, I'm writing for myself because I'm trying to learn. And I feel when I hit publish in an article that is a knowledge solidified in my brain. So it's there now. So. So whenever I'm writing articles, I have the idea. The idea is usually birthed out of the necessity if a skill that I need to sort of improve in, in my role or a new skill that I need to sort of explore. So I begin exploring these skills. And out of that, let's say my ops is not a skill essentially as a field, but let's say MLPs. I can start exploring MLPs, I can create ten articles from there and each of those articles [00:32:00] are focused on different aspects. And I dive deep and I make sure I'm able to explain things in my own words and in my own code as well. So that's how I approach, that's how I think about articles. And right now I'm exploring feature store data versioning. So I've got about four or five articles just on that topic alone. And one problem I think I seem to have is I don't know when to stop or when to because there's so much knowledge, right? So you're like, Is this enough? Is this enough? Then eventually I have to hit publish. And that's that's when that's my process, essentially. Yeah.

Harpreet: [00:32:37] Do you have like a system for like, you know, you're doing your research then from research you got notes and then from notes you end up with like a final product. Do you have like a system that you use to help facilitate that creation?

Speaker3: [00:32:53] Yeah. So I use notion for my for writing. So like the first draft of my article and in terms of a system. Right. So I could give like an example let's say my feature stores is a topic I need to learn about. What I would do is I'll create, I'll look for several good resources, which could be research papers, which could be articles, which could be books as well. I usually have about 10 to 20 tabs of this open that I start to select a few which ones are important, which ones make sense. Then? Then I start to break down, break it down into the topics. So I look at the high level topics first. So what are the topics I need to understand? And usually what drives this are the the content page of really good books on a topic. So I start to look at, okay, these are the areas I need to explore within feature stores or databases, and then I dive into it. I dive into it very deep. I'm not reading just one book at the same time. I'm reading maybe four different books because they're all cover the same topics. So I just [00:34:00] look for what this what for a set of feature stores or offer be a set of features. So now I have the same knowledge over and over be and I create my knowledge, my understanding of this, of this particular topic. And that's my process. And I just iterate and I try to make my article readable as well. So I write it like I'm teaching like I'm speaking, which is helped a lot because it helps my teaching as well. I do a lot of teaching on the side of my Imperial College business school. But yeah, yeah, that's, that's, that's my process. That's my process.

Harpreet: [00:34:33] That's awesome, man. Thanks for sharing that, by the way, for watching LinkedIn, YouTube, You got questions? Do let me know if you're watching to smash that. Like, yeah, my process has I've adopted based on this book called Building a Second Brain, Tiago Forte took his master class as well, and I realized I just had such an unhealthy relationship with information, like I was just consuming stuff, but I wasn't. I had no record of it, had no way to, you know, get the ideas out of my head. So what I do now is like his whole process capture, organize, distill, express. So capturing is, okay, you're coming across a ton of information on the Web. You know, how do you how do you figure out what to read? So now what I do is I come across an article and I put a filter on it, right? And the filter is, you know, there's about three or four things right now that I'm most interested in or that are most important to me, my career or what have you. And if the article I'm coming across doesn't immediately address those three or four things, I'll just reject it. I'll just, you know, whatever. It's good. Maybe I'll come back to it, you know, whatever. But if it is something that touches on my important questions, then I'll do it. You know, give it a once over. I'll, you know, kind of screen it and see. Okay, Does this have good quality? Is it does it look like it's giving [00:36:00] enough details then if it does, then I save that to like a read it later app and the read it later app I use.

Harpreet: [00:36:07] I know some people use like Instapaper, that's one and you know a couple other ones but I use matter just get matter dot app and save everything to that read it later app and once everything is not really later app I'll go through and I'll highlight in matter and the highlights from that article get synced automatically to this second brain of mine called Obsidian. Then in Obsidian all my notes are there. I'll organize them to, you know, what category they fall into and then I'll distill it. So I take the highlights that I've got. I'll distill it either even further, just so I get the main essence of what that article is about. And then on top of the note, I divide it on top of the note I'll express. So, okay, given this body of work that just read, if I was to write a LinkedIn post, what would that post look like? Then I put that right there on the top. So now that's my expression. That's how I understand this. And it's it's been so helpful, man. It's been super helpful. And just managing this deluge of information that is that that is out there. So let's let's circle back real quick. Macro factors and machine learning models. Right. Let's talk about this, Vin. Break this down for us. First of all, what is the macro factor?

Speaker2: [00:37:23] So macroeconomics, just the highest level fundamentals of the economy that affect everything, everyone. That's the simplest way to explain it. But when you look at them from a data science and machine learning standpoint, those are the things that indicate there is some change in the marketplace coming. And so anything that touches a customer or anything that touches a complex network and this is what I was bringing up on Monday's post was anytime you have a complex network, it's going to touch one of those macroeconomic factors, those macro factors, things like inflation. That's a macro factor, [00:38:00] your issues with supply chain, and that's the one I brought up. Because Nike was struggling with inventory that they all of a sudden have and Target as the same problem. Walmart at the same problem with Walmart, though, was a little different. They didn't have the right mix and so their models weren't sensitive enough or their monitoring excuse me, wasn't sensitive enough to changing customer behaviors. And so they only found out after they made a bad decision, ordered a ton of inventory, and then realized, whoops, customer behaviors have changed. We didn't see that coming. Why not? So there's different types of impacts when you deal with these complex systems, and your models always touch complex systems, Sometimes your models are supposed to be complex systems. We say they learn functions, but now models are more complex than that, especially when you have multiple models interacting with each other, trying to simulate your supply chain, trying to understand how inventory should be planned.

Speaker2: [00:38:56] And one of the things that Nike didn't do was look at it from a. What could impact negatively our inventory? What could cause a shortage? What could make this worse? Or what could cause us to be in the situation that we're in right now, which is to have this glut? They weren't looking at how the supply chain kind of evening out and suppliers coming back online after COVID lockdowns could be this horrible, perfect storm that they're in right now. So instead of monitoring and really changing the feature, set a little bit more. To include a different definition of inventory where it's what's ordered and late, plus what's in transit, plus what's in your warehouse, you know, doing something like that and watching what good impact each one of those. And so from a macro factor standpoint, you could have watched the price of containers because those have been dropping. And that's an indicator that demand is dropping, Supply chains are normalizing. [00:40:00] And so if you're looking at that, you can say, okay, that's a leading indicator that I need to be paying attention to. One of your suppliers coming back online after a COVID lockdown, you say, okay, all of a sudden these orders that have been back ordered are going to all of a sudden start flowing in. So that's something I need to take into account. I need to slow down my ordering.

Speaker2: [00:40:21] And if you look at what Macy's did, Federated did a pretty good job of this, where they have less excess inventory. I think it was only 7% where most companies were double digits. I mean, high up their double digits. And those are the types of macro factors that you can monitor. I mean, you can't predict something like that because a lot of macro factors change right now, especially change unexpectedly. And so you can monitor the data to say, okay, all of a sudden shipping container prices are coming down. That's something I need to pay attention to across ordering pricing supply chain. And those are the kinds of things you can monitor and say, okay, if that changes, my model is probably going to become unreliable and you can advertise that to users, you may not be able to retrain it immediately because you don't have enough data or you don't really understand how the problem space has changed. So you may not be able to immediately deploy a new solution, but you can at least tell your your demand planning team or the people that are buying your buyers and purchasers. You can tell, Hey, I've detected something. This model's not so hot. Don't rely on it as much as you used to. Here's what I'm seeing and here's what you need to know now. I need smart people. We need to go back to you guys being smart, you girls being smart.

Speaker2: [00:41:39] Help us out. And you can give me a hand here because my models are unpredictable. And that's one of the problems with sort of these descriptive models that are being passed off as something more reliable than we just learned the data set because you're getting used to it being right because the data set is representative [00:42:00] of the times that we live in and then all of a sudden times change. And if you don't understand where the holes are in your model and where it's going to start behaving unpredictably, you don't tell your users and they're like, Right now there's a whole bunch of people at Nike going, but it wasn't our fault. The model set. And that's, you know, that's where I say look at macro factors, but understand how they impact your model because that's going to make a big difference, especially to front line users just being able to advertise it. Because I mean, think about it. If demand planners and buyers were making decisions based on a model from a data science team at Nike, how much are they ever going to trust that again? Being that the stock just got humbled, I mean, ten, ten or 15%, I can't remember. I can't remember what it was taken down. But it was it was a number that your CEO will fire you for. And so you got to ask yourself, is anyone in that group ever going to trust your model again?

Harpreet: [00:43:00] So these macro factors like I guess who would be who should be, I guess, responsible for monitoring this because I imagine, like, you know, a front line data scientist doing their thing, they might get caught up in like the nitty gritty of the day to day and might not have the bandwidth to monitor to track this stuff. Like, is it are you supposed to catch this through maybe a model observability type of platform, or is it just something where you just kind of take your head out of the sand real quick and just look around and see what's happening? How do you, I guess, monitor.

Speaker2: [00:43:35] Just by third party data? I mean, financial companies both consume and a lot of them produce and sell these types of data sets where, I mean, some of them are a real time where you get it day to day or hour to hour, but I don't think you need that much. That rigor is usually having something that's watching. A data set that gets updated weekly is enough to give you the insights into [00:44:00] and then just having an understanding of your model to know that these are some of the problems that you could encounter. And a lot of times you can do that just with experts. And I love graphs for this. Basic structural causal models are kind of going a little further than most companies do, but just creating a basic graph of what features are important to your particular model, or at least what are the most important features, And then letting really smart people, anybody in supply chain that's been there for 15, 20 years, you want that person staring at that chart and saying, you know what you're probably not thinking of? And they'll tell you this I've watched in the past or I've seen this happen before, or I remember this one time back in 1990 when, I mean, they are a gold mine. They will tell you where your bad assumptions are. And if you give them a little bit of training so that they can look at it and you basically present this graph to them. Here's the features. Here is the thing that I think these features impact whatever metric that may be, and they will come back at you and say, okay, so that one right there sounds good.

Speaker2: [00:45:10] Not doesn't work. And they will give you a whole different view of the assumptions that get baked into your model. And they're going to also then call out new features. You can realize, why have I not been at weight? So I could have just been asking experts and they would have been doing half of my feature engineering for me. Really. That's all I had to do was get smart people, and that's kind of a surprise for a lot of data scientists is the monitoring and what you should be monitoring and what data you can buy. Those are all a lot of times you can talk to experts in the field, you can talk to analysts at investment houses, especially if you're in a company that's covered by analysts at large investment houses. You can you have people in your company who have access to them and you can talk to them and get some answers about what are you guys watching, What are you worried about? What do you think's important when it comes to our company? What are the fundamentals? And [00:46:00] you get just these rich answers from investment analysts because they want they want some info from you, too. They want to get a little bit of what you're thinking and how just there are so many different relationships with experts that you can tap to understand the types of macro relationships that are out there.

Speaker3: [00:46:17] Can I speak to Vince? Answer as I love the fact that Vince always takes the business, the macro level. Very good, because it ties in with what I've been exploring in my space. And the question was how do you detect and monitor this these problems? Right. So what what we're mentioning here is data drift, right? So you have a change in the data and the data set and that leads to things like concept drifts and and feature drifts. So there are tools out there within the ops ecosystem that are built to detect this change, at least in the in your in your data. So in your feature distribution, they can actually pick up any sort of significant change in feature distribution or maybe you set the threshold. So I think the industry is starting to realize that there needs to be some sort of infrastructure to be able to enable the monitoring and the detection and essentially to be able to allow people to act. The question is how quickly can you detect a problem before it becomes an issue for the customer or propagates further down the pipeline and it becomes irreversible. So it's something that a lot of companies are trying to solve, and that's why there's a lot of VC money going into the OP space, because now we have what is machine learning models in production doing stuff. But the human world is very unpredictable, especially in this time of COVID that we have this massive economic downturn, supply chain issues, geopolitical tensions. So there's really uncertain times, which means our models that were built on datasets from relatively what we'll call certain certain times [00:48:00] are not really performing well in in today's market or in a future market. So there's a there's a place for this ecosystem of monitoring tools. And that's what this is essentially one of the principles. Anyway, my answer is very, very great. I love it. I love listening to them.

Speaker2: [00:48:19] And also what you're saying is you're actually calling out one of the gaps that we don't have enough tool coverage in. So we're monitoring our models really well. We're monitoring the data that feeds our models really well. But the indicators of change happen one step upstream, and we don't have tools for that unless you create. I mean, I've seen cron jobs doing this. It's it's terrible. It's just basic ugly logging of a database. And you get a report that will tell you whether some of those data points that feed into and have well correlated, if not believed to be causal connections to the input data for your model. And when those change, you can see those happening before you see the pain happening in your model. And so which one is calling out is this gap that we don't have for from a toolset standpoint is we've got a lot that's focused on features and on the model itself and on data drift and concept drift and all of the things that are fairly well known. But we don't have something that validates our assumptions. We don't have anything that's doing continuous assumption validation. And so when he's talking about like geopolitical instability, that's that's we don't have anything that monitors that. You have to set this stuff up manually. It's it's ugly and it's underserved.

Harpreet: [00:49:53] Potential startup idea for a I mean, would you get a what would a startup look like [00:50:00] that did this? Like, would you be like a data vendor? Like, I feel like you just I don't know. I'm not sure.

Speaker2: [00:50:09] Yeah. I mean, if you could basically all of your pricing models have similar macro factor impacts. When you have inflation. Obviously your pricing model needs to change. When customers become more price sensitive, your model needs to change. When the behavior shift like they are now, they're going a lot of different segments of customers are going from products to experiences. And so that's going to cause drift. And each one of those I mean, if you just had this is like I said, this is stuff that finance companies already do. They're already tracking these. A lot of hedge funds have unbelievable visibility into these very small changes. And they have these models that they've spent forever building and validating and rebuilding and re validating. And the people who do it are very, very wealthy data scientists. And so, I mean, it's not like it's impossible or it hasn't been done, it's knowledge that hasn't gotten out into the rest of the data science field.

Harpreet: [00:51:12] How do we get that knowledge, man? Like, what do we got to do to get that knowledge? Like, that's. That sounds like a gold mine opportunity for a people who are in the field and maybe thinking about what's coming up next in their career.

Speaker2: [00:51:24] It's weird that a lot of people know this, but a lot more people don't. It's just it's not like it's a it's not like it's unknown, you know what I mean? As you see Richmond nodding his head, and I'll bet you Russell's kind of nodding his head, too. We all kind of you know, most of us know this stuff, but you'd be surprised how many people don't. And it's not for lack of all of us talking about it. It's really I don't know if maybe they don't have enough time. Maybe that's it. They've got too much else to do. Their focus is a little bit too granular. [00:52:00] They're not being asked to build at a higher level of reliability. Maybe the use cases that they're that they're going after right now are still pilot project level and could be a lot of things. I don't know. I can't tell you why we have so many huge companies that make the same mistake as someone else made a month and a half ago. And then and I'm serious. In another 45 days, you'll have somebody else make the same planning mistake where I mean, you saw Peloton do it in December last year. They over ordered they had to drop their inventory back. They basically warned everyone in December, watch your supply chain, watch your inventory, watch for overstocking. And no one I mean, we're we're ten months later and everyone's still making the same mistake.

Speaker3: [00:52:52] How?

Harpreet: [00:52:55] So I guess our our tech companies that don't have any like physical products or they immune from these type of issues or what considerations should should they make that or lessons that they can learn from some of these physical goods type of companies?

Speaker2: [00:53:10] I think anything digital is going through the same thing. You just don't have inventory. But what we're seeing is a lot of the inventory was talent because the bottleneck for most R&D is we can't get enough people, we can't scale because we can't get enough people. And so Google scaled, Facebook scaled all of these companies over scaled. And now Google is looking around going what these people do. That's literally what Google is doing. The CEO of Google right now is walking the halls going, what are you doing? What are you working on? Hey, so what are you working on? That's he's taking a poll right now and they're going to reorganize and probably slim down a little bit. And Zuckerberg did that, I think, in March or April of this year, where he started walking around going, wait a minute, do we need all these people? What what [00:54:00] are they working on? So that's where the Overstock has come in, is they over hired? Amazon did it with their warehouse workers, but they know demand eventually will catch up for them. So they're going to keep a lot of their workers as long as it's feasible because they understand right now the bottleneck of talent for them is a bigger issue than spending a few tens of millions.

Speaker2: [00:54:22] That's nothing to them on not having more talent than they really need. And so that's the equivalent is the talent, because the cloud made it easy to scale up, scale down. And so there's really no overstock of hardware or resources or compute data doesn't get overstock, you know, I mean, there's no such thing as too much. Well, can I take that back? If it's terrible, there's too much there's no such thing as too much good quality data. But for them it's really talent and you're going to see a lot of it also spending. If you look at moonshot projects, a lot of companies right now are having to make some really tough admissions to their boards that are going to be public in January, February next year, where they spent billion plus on something that will it's just not going to work. And those are kind of you know, it's the same idea where when you're demand forecasting and you're looking at an innovation project and you're saying, you know, it'll get there, demand will materialize, like especially a lot of these headsets that no one really wants. The company said demand will materialize and.

Speaker3: [00:55:28] Nope, it won't.

Speaker2: [00:55:29] So those are kind of the same lessons is if you expect the consumer to spend forever, they won't. If you expect the labor market to be tight forever, it won't. If you expect demand for luxury goods to go up forever, it won't. It's just every one of these trends. If you think it's going to go on forever, then you're going to miss the you're going to miss the bigger problem. You know, it's the crypto learning.

Speaker3: [00:55:58] One final is going to add on to that is [00:56:00] the awareness piece, right? Personally, for me, the other reason why I'm aware of the things that are happening, especially the tools that could be used, is because I'm very close mates with someone who's who's works in the risk management side of things within a financial institution. So it's their job to be able to sort of think about the worst case scenarios of things that could happen. And this is what I was talking about with financial institution, right? They have all these risk models that are able to take into account several different variables and be able to then decide if maybe an investment is worth is worth pursuing or so on is worth investing in. And really within the tech space, we don't have that sort of infrastructure because we've never really needed it right at this point in time. It is moment in time in history. We've had a domino effect of several different global, well, I would say unfortunate events, right? So we had the pandemic and from the pandemic we had inflation, global economic downturn and that now we have the VC market drying up. We had a crypto market drying up, We have geo geopolitical issues. We have famine because of wars and grains are not getting into other places. So there's a bunch of domino effects of things that are happening one after the other that they haven't happened in pretty much, I would say, history in terms of the sequence at which that happened. And the gap between them as well is very close. So financial institutions, they're basically.

Speaker2: [00:57:36] Built.

Speaker3: [00:57:36] To be aware of unpredictability, whereas I guess modern day companies might not be because we don't have to take these things into consideration or the sequential events of unfortunate events happen frequently in. And that's, that's my $0.02, for that matter.

Harpreet: [00:57:59] Russell, thank [00:58:00] you all so much. Don't see any of the questions coming in on LinkedIn or on YouTube or right here in the chat. Small little crew for the happy hour today. Hopefully next week for happy hour number 100. We get it poppin. Get a lot of people in here. I'm going to actually set up a Facebook event. I'm going to sit down like I set up an event on Facebook, not Facebook. Linkedin. Jesus Christ. Setting up an event on LinkedIn. It's just it drains my energy. I don't know why. It just feels like there's so much effort required in doing that. But I'm going to do that and hopefully get get this thing popping and hopefully get some old faces back. I remember like two years ago doing this thing and it would be like 50 people in here going crazy, man. It was it was a good time. But nevertheless, I'm thankful for you all for coming here. This happy hour not happened without your guys participation, without you guys watching and listening. So thank you so much for being here. Take care, my friends. Have a good rest of the weekend. Remember next week, Happy hour 100. Join in bit.ly for ads. So register come hang out. If you got questions that you want to send to me, feel free to send them over at the artists of data science at gmail.com or at my friends. Take care. Have a good rest of the week. And remember, you got one life on this planet. Why not try to do something big? Cheers, everyone.