Matt Drooker 2
===

Noel: ~Test, test, check. One, two, test.~

Matthew: ~Test, test, test. One, two. Test, test.~

Noel: ~You guys sound fine. Do I have I, is my like audio breaking at all? If I sit here and like talk for a couple seconds? I don't know. I don't know.~

Matthew: ~fine.~

Noel: ~don't know, man. Yeah, yeah. Is, uh, before I give us our intro, is it Druer? Am I saying that correctly? ~

Matthew: ~It's Drucker. Yeah. Dr. Drucker. Yeah. I've, I've heard all kinds, you know, hooker, but, we'll, we'll stay p we'll stay PG 13.~

Noel: ~with a D. Sure. Yeah. It's okay. Like everybody gets all of my name wrong as well, so it's like,~

Matthew: ~Well, I almost called you Noelle, so.~

Noel: ~I'm used to that one. That one's like nine times outta 10. That's how it goes. So cool. All right, I'll kick this off. ~[00:00:00] Hello and welcome to Pod Rocket, the podcast brought to you by Log Rocket.

Log Rocket helps software teams improve user experience with session replay, error tracking and product analytics. Try it free at logrocket.com. My name is Noel, and with us today is Matthew Drucker ~Dru. I just, I'm sorry. I know I just asked and I messed it up. Anyway,~ Matthew's the CTO of SoundCloud and former CTO O of CNN Digital.

Welcome to the show, Matthew.

Matthew: Thank you. ~Uh,~ longtime listener. First time

caller. Um.

Noel: ~Yeah, I'm excited.~ I'm excited to chat.~ Um, yeah, so usually I just kinda like turn it over and have a, uh, people, you know, kind of tell me their background and their journey. Um, but I kind of did a little bit.~ So maybe let's ~just~ talk about c n n briefly, what you were doing there and then like how that ~kind of~ led to SoundCloud.

~Does that sound sound good?~

Matthew: ~Yeah. ~Yeah. Great.~ Um, you, I've lived here in Atlanta, Georgia for, you know, 40, 45 years. Uh,~ went to school at Georgia Tech, ~uh,~ ended up starting working at Turner Broadcasting, which is a fairly large media company here in the United States in my journey through Turner Broadcasting. Had about ~10 different,~ 11 different jobs, and some of the last two were really, ~uh,~ focused in the digital space.

And so I was,~ uh,~ the chief architect for many of our backend systems, ~but you know,~ the last two ~were really, um, it~ took me to where my current career is now. And so one of 'em was becoming [00:01:00] the c t O over at CNN Digital. And so CNN Digital is pretty well known as a, ~you know, a.~ Digital property providing news to the web.

Most people know the three red letters on air. ~Um, and so, you know, we're very proud of the three red letters.~ And, ~you know,~ taking over the reins of that brand on the digital side was, ~you know,~ something I hadn't really done before. ~Uh,~ obviously. And so ~when you,~ when you~ kind of~ step into that, ~You,~ you don't know what to expect.

And ~you know,~ all of the years leading up to that, whether it be again, ~at,~ at Turner, ~um,~ running our ad sales systems got me to a place where I was, ~um,~ still developing. ~So, you know,~ even today, even at SoundCloud, I still developed, but I started at the, at CNN in 20. 13. And ~so, you know,~ from a, from where the web is now to where it was in 2013 is a much different world.

So, you know, got got to CNN digital. ~Um,~ we were in a, ~you know, right~ in the middle of starting to investigate how to redo our backend stack, our origin stack, serving stack,~ uh,~ using node. And so this [00:02:00] is early node, this is like 0.6. ~Um,~ Lot of, ~you know,~ lot of issues with serving at the scale that we were serving on.

We were also changing our front end render stack that used to be, ~um,~ very h like XML based templates into a very d or more dynamic, ~um,~ page layouts. So ~it was, it was, there was a,~ there was a lot there, but node was ~kind of~ key. ~Um,~ and this is really where, ~you know, the,~ the education started on how to start.

Sites and brands that scale worldwide at some of the numbers that we were talking about for c n but it was, ~it,~ it was ~quite a, quite a, um,~ quite a journey to make sure we could get, ~you know,~ no again, back in the day to do its thing. ~Uh, which we, you know,~ we're ~kind of ~used to it now, but back then it was not a, it wasn't a sure bet.

Noel: that is very like surprising to me. ~And I'm,~ I'm sure ~most,~ most listeners that were like, ~kind of~ in the development space at the time, I bet if ~they had, you know,~ you'd asked them the over underrun, the backend that like CNN Digital was running, no. Being, no. That would've been a pretty long bet for most of them.

~What was like.~ Were there other ~kind of, I don't know,~ big players at the time, at the scale that CNN [00:03:00] Digital, was that like traffic wise that you knew of, that kind of served, I dunno,~ as~ as something that to give you some confidence or was it ~kind of~ a leap of faith?

Matthew: ~Um, you know,~ the New York Times was always somebody we were taking a p at Financial Times over in London were folks that we were always looking at one. One of the benefits that we have and still had then ~and,~ and continue to have today, is our partners in the CDN space. So we were a huge fan of early Fastly.

~Um,~ and so Fastly, ~uh,~ did a lot of things to help us feel more confident to bring the that over under, to your point, into a place where, ~you know,~ I could sleep well at night knowing we made the right decision. ~Um,~ and ~so, you know,~ a as if you think about push in the way that news is delivered to consumers and fans, ~You know,~ you publish a story, which you have to be very quick cuz a lot of the metrics in the news business are whose first, ~like,~ first to page publish on a breaking news article.

And if you think about cnn, ~it's,~ it's, there's a lot of value in the breaking news [00:04:00] space. So we spent a lot of time making sure that our time to publish was. ~You know,~ in, in really fast. And then we needed fast purge times to make sure worldwide cash purge times to make sure that it was distributed worldwide ~to the,~ to the edge in a way that, ~um,~ was breaking at the time.

So that, that's where, ~um,~ yeah. But ~I mean,~ some of those players, like I said, the Times, or both times Financial Times and New York Times were folks that we were always looking at, ~um,~ and ~you know,~ had a couple of chats with them through some of our

Noel: ~Mm-hmm.~ Yeah, ~I,~ I guess it makes sense that ~you,~ you'd mentioned CDNs and I feel ~like, you know,~ CDNs have always been important, especially at a huge scale, but ~at the,~ at the time, it was probably less of a thing that everyone, I don't know. I feel like we're ~kind of ~in this era now where a lot of that's taken for granted.

Like people just pop cloud flare up in front of their site and they're like, it'll do its thing. It's fine. I don't need to worry about it. ~Like,~ but at the time I feel like ~that~ that was ~not,~ not the norm at all. ~Like, was that,~ so was a lot of your time ~kind of~ like putting this together and managing this transition was a lot of that ~like at,~ at the CDN layer, making sure ~that was, ~that was humming.

Matthew: ~Well, you know, um, yeah,~ so the CDN was definitely some of our conversations, like I said, with our partners, ~um,~ there's this [00:05:00] concept of things called surrogate keys. So if you change, say an image, you wanna purge that image everywhere, on every page that it's rendered on. ~And so, see,~ these are some of the concepts that, ~you know,~ you don't read the book and.

When you're out at a bar, you're ~kind of~ trying to figure out how you would do this in real time. ~So a lot of, you know, we,~ we spent a lot of time at a local bar called the Park Bar, like even whiteboarding, ~uh,~ how we would do this. And ~so,~ yes. ~Uh,~ at the time, our CDN partner, ~um,~ we were figuring out some of this stuff at scale, ~you know,~ as we went ~with,~ with this, ~uh,~ new render stack.

~You know, it, it,~ it, there was also this other component of the CMS that was also being re. So CMS is in content management system, so we were re-imagining how our content management system went along. And so every piece of the stack in back again in the 2013 days was ~kind of~ under investigation. And, ~uh, you know,~ mu much, much jacked Daniels was used to ~kind of~ figure out ~how the,~ how the site

Noel: Yeah, I'm sure. How, what, when you guys were like doing the CMS transformation, [00:06:00] were you, did you look at other stuff that was out there at the time, like off the shelf? Did you end up writing most of it in house? How did that look?

Matthew: Yeah. ~Um,~ so we had our own in-house cms. That was the origin story on that was, ~you know,~ many years before, ~um,~ it was actually an Adobe Air app because we were, it was early cross platform back in the day. ~Um,~ again, publishing to. An internal, ~uh,~ CouchDB. So we were using couch database, so a lot of different tech.

~Um,~ but we, we kept that in-house cms. For a while. ~Um, you know, CMSs are kind of like religion. Nobody's as perfect and everyone else is sucks. Um,~

Noel: Yeah.

Matthew: ~hopefully, hopefully that'll make the, make the cut. But, uh, um, so we, we did keep our own internal, um, cms, you know, it, this has been some time, so over the last couple of years, I think there's been some, uh, modernization.~

~That's modernization. I put in quotes. Um, but let me re, let me redo that sentence. Um, you know,~ this was a couple years ago when we had our own, when we were building our c m s internally. ~Uh, you know,~ over the last few years I think there's been some other efforts to ~kind of~ change the CMS that we had been talking about.

But yeah, to your question, ~um,~ no. We had our in-house cms ~that was,~ that was working really well for these fast publish moments that we had, as well as a very journalistic. ~Um,~ set of tools that we needed that weren't in the space of like marketing or Droople or WordPress. ~And so, you know,~ when you think about a C M S, you think about [00:07:00] those kind of tools.

~Um,~ those don't always lend itself well to news articles that have, ~you know, uh,~ either carousels or images, ~um, that have,~ that are used across the site. And so there was a very specific reason why we had our internal cms, and again, it was in the journalistic quality that we were going. ~And still go~

Noel: ~Do,~ do you think ~that~ that like. Be different now. I feel like we've ~kind of~ seen the rise of like less opinionated, headless management systems in the past five years. Maybe that's a bit 

Matthew: Yeah. 

Noel: ~but~

Matthew: So this CMS was headless like we were publishing. ~You know,~ JSON payloads that are the no, that node stack rendered and, ~um,~ composited together. So we, ~you know,~ we were headless when headless wasn't cool. But yeah. So ~I mean,~ over the last couple of years, whether it's contemp or grab CMS or some other, ~you know,~ off the shelf CMSs, even strappy, which is an open source cms, big vault, ~kind of~ started to democratize headless CMSs in a way that.

Those of us that were around before, remember what it was to ~kind of like~ rethink tho those problems,

Noel: ~Yeah. Yeah, I think it's, it's like an interesting,~ I think it was [00:08:00] an interesting evolution for them cuz yeah, ~like,~ It felt early, like ~the,~ the Ruples and even like WordPress to some extent, ~like~ had much more of a ~like, you know,~ prescription. ~Like~ this is ~kind of~ how they're trying to solve everything, make it as easy as could be.

But then we ~kind of~ hit this point where ~like,~ developers weren't always reaching for them cuz they wanted to do like the, they didn't always fit the domain that well. ~Right.~ And I think that's ~kind of~ what led to this headless errors, like devs care about how this data flows through the system. So we wanna ~like~ make it so people can use whatever they want.

~Um,~ so it's. I think it's maybe a kind of a common story is ~like~ these things came out and everyone had ~kind of~ built something like this pseudo, ~you know,~ unop opinionated thing internally before, ~um, yeah.~

Matthew: Yeah, and ~you know, I mean,~ even some of the blogs that we were writing ~back,~ back in the day, they were,~ you know,~ either WordPress premiere or we even had DR in house. ~You know,~ at one point, Turner Broadcasting, we probably had a bespoke number in the 15 to 20 different types of CMSs. Some of them quadruple WordPress jummah, ~like, you know,~ you name one, ~uh,~ you could throw a rock and hit one.

And, ~you know,~ there, there was definitely an effort in the, ~you know,~ in the [00:09:00] mid 20 fifteens to start like reducing that footprint just because the overhead of managing these. And then, ~you know,~ through, through that headless CMSs started being the, ~you know,~ the hammer which you ~hit,~ hit that kind of. ~Mm-hmm.~

Noel: ~Right? Like, what, what did,~ what did, ~like~ traffic patterns and stuff. How did that, how'd that stuff change?

Matthew: ~You know,~ there's, ~uh,~ CNN~ is a,~ is a strong video brand as well. Obviously those three red letters ~are,~ are worldwide known. And so we had not only written ~articles,~ articles that had text-based content, but also rich photo galleries. ~Uh,~ and then video to your point. So lot of live and streaming live video, ~um,~ on the web is always a challenge.

~Uh,~ and then also a lot of vod. ~So you,~ we would take, ~you know,~ video program ~and,~ and we would have editors cut them up into video clips and those would become assets as well. So we had our own video CMS that also composed a lot or consisted of a lot of metadata. So you could quickly find an article that, you know, Larry King may have, ~uh,~ covered [00:10:00] prints or ~had~ had, he had, may have had prints on in the nineties.

And then when, ~you know,~ when Prince passed away or, ~you know, any other,~ any other, ~Uh,~ or celebrity that he may have interviewed. We quickly would go to those catalogs ~and, ~and get composition pages of video pages that had our video assets as well as, ~you know,~ legacy, ~um,~ images and then articles. So the composition ~through these, um, you know,~ to those times you had talked about what's really this evolution ~in,~ in making CMSs now be article video.

These composition engines of a ton of multimedia types and then sprinkle in and that scale live events, ~right. So you, uh, you know,~ whether there's an, ~you know, any,~ any, um, chaos events, whether it be a hurricane or a forest fire, ~um, you know,~ we would cover it live and those would also. Routed events through our video player, cuz we had our own custom video player that, that could have, ~um,~ ads injected in it.

~But you know, this,~ this very multimedia rich [00:11:00] site, ~um,~ was ~kind of~ born through those years ~that you~

Noel: Yeah, so ~like was that,~ was that kind of, I don't know, transformation or, I dunno, setup of more video focused content. Was there unique challenges in ~like~ getting the pipeline all configured and smoothed out and working? So ~like, you know,~ the editorial teams could ~like~ work with stuff. Where was most of the challenge in serving the customer?

That was like the internal customer.

Matthew: ~Yeah, I mean, so that, that,~ a very similar challenge to the CMS that I spoke about is, was our video CMS and ~the,~ the tooling around creating, ~um, you know,~ snackable video.

Noel: Yeah.

Matthew: That were in the, ~you know,~ four or five minute. And so if you think about the brand where we have a ton of linear people that are serving the broadcast network are also now starting to contribute to our digital

Noel: Hmm

Matthew: ~Um,~ so we really, we did, there was a lot of work and like figuring out how these assets got cut and edited and then encoded and transcoded into, ~you know,~ HLS streams, ~um,~ that were. Put out on a CDN and then streamed, ~you know, at~ [00:12:00] at scale to our users. And that was another cdn. ~Like, you know,~ like I had mentioned, Fastly, huge partner for us.

We also use Alchemi. And so when you think about how video is, you said, I think you said it, ~you know,~ now it just ~kind of~ works. But again, back in the day when you're trying to do. Live video for, ~you know,~ 2 million folks. It's, those are in ~the,~ the numbers that, ~uh,~ the Super Bowl was covering. ~Our,~ our election coverage in 2016 were in those numbers too.

~So, um,~ and even in 2020 after I had left, it surpassed what I, it was when I was there. ~So, um,~ yeah, ~a lot, a lot of,~ a lot of those challenges we take for granted

Noel: I'm sure.

Matthew: ~Um, and,~ and, ~you know, and even,~ even aws, what AWS has been able to bring to the table. ~Um,~ we. I'm trying to, we were figuring that out on the fly ~in,~ in those years that you had mentioned, we had spun up our own Kubernetes stack, even when Kubernetes wasn't like in the 1.0 version.

This was again in the, I think it was eight. So we were running our own co clusters, managing that and figuring out how to run all of that too. ~So,~

Noel: Yeah. ~Um,~ I feel like, yeah, like the aw w s like ~the,~ the, I dunno, the big cloud fighters were ~kind of like~ coming onto the scene, but they definitely weren't as embedded [00:13:00] everywhere. It wasn't like the assumption that it is today. ~Um, yeah. So~ you mentioned ~several times and you, and you said something, um, at the beginning too, where you said~ you were still ~kind of~ building or coding or something to that effect day-to-day.

~How,~ how was that kind of factored into this? What does that look like? Now? You've mentioned a lot of tech, like a very wide swathe, ~like~ are you getting your fingers wet everywhere or just like ~kind of~ delving deep in a couple areas? Like ~how do you,~ how do you break up that time?

Matthew: Yeah. ~Um, I, you know, if it,~ it's one of those things that every person I've worked for over the, you know, 35, 40 years has said, you gotta quit coding. ~Um,~ because of the, ~you know, the, the,~ the challenge of managing teams, managing strategy and understanding the business side of ~this,~ this industry, ~um, like that, juggle, juggling that those three things on the business side and then,~ For me, it was always how do I, how can I do any of those without keeping my hands?

In the, ~you know,~ the pie to be able to help teams make the right decision, ~um,~ in those, ~in,~ in, with some of these technology challenges. ~So, you know, like I had mentioned, I was at Georgia Tech as a management major, so that's closer to the business side of it. But I always really, I, I had stayed close to tech even as a management major and continue to code.~

~And so, you know,~ the answer to your question is,~ I,~ I keep my hands ~as,~ as in it as possible to one, help the team. [00:14:00] But there's a little bit of, ~um,~ like emotional satisfaction with being able to control my own destiny around building something. Like when you start getting into leadership roles and. You're running a team or managing a team and working with a team, you ~kind of~ yield some of your control over the outcome.

Like you, you get fabulous people to do amazing work and that is always the case. But every once in a while, and I, ~you know,~ I don't think I'm alone in this, you just want to be able to do it yourself. And so I get true satisfaction out of being able to ~like~ help the. Do all of those work things and then also get really down deep and understand like ~what does, what is the, you know,~ how would I build this thing using GraphQL that does X with persistent queries?

~Um, and so, you know, I it's, ~it's a long-winded answer ~to it. It takes a lot of, um, effort and time to make sure I'm like doing the team there.~ My job is to serve the team and then there's parts of it that for me,

Noel: Yeah.

Matthew: like a hobby to make sure~ I can,~ I can

Noel: No, totally. ~I, I, I don't,~ I don't feel like that's an overly long-winded answer at all.~ I feel like,~ I feel like you get that.[00:15:00] 

Matthew: I

appreciate it. 

Noel: It's hard to, 

Matthew: Yeah.

Noel: To capture that, that I don't know, that emotion, ~that,~ that feeling of ~like, I, I,~ I need to have my boots on the ground still to ~like~ have that empathy.

~Yeah. Um,~

Matthew: ~Yeah. I mean, you know, even there, there are,~ there are. And some of the team, ~uh,~ will remember, they still gimme a hard time for this, even though I don't work with them anymore. There was a holiday one of the years that we didn't, we couldn't take a holiday cuz we had to launch the, we had to launch ~our,~ our site and it, ~you know, what,~ what else to launch it other than like January 4th.

So perfect day to launch, ~um,~ if you got everything in the row, ~but you know,~ it's always usually tight. And so we didn't take vacation that. And I say we, because we were all in the room together and it wasn't, and ~I,~ I can't con, ~you know,~ I, ~I couldn't,~ I couldn't do it without being boots on the ground. Not just ~like~ saying I was there, but being in there and, ~you know,~ many of the team members are a member of, ~of a, uh,~ election night where we were testing GraphQL on election night, on the homepage.

And, ~uh,~ yeah, ~we were,~ we were doing those kind of. Because that was fun and we had a good time

doing it 

Noel: ~This kind of, I'm, I'm circling back now, I'm jumping around a little bit, but that does you,~ you talked about like your traffic numbers [00:16:00] on election night, like eclipsing Super Bowl, ~you know,~ eyes

Matthew: Yeah, ~they were close,~ they were close to Super Bowl. It was~ in the,~ in the two millions, ~uh,~ 2 million space. ~You know,~ page views is usually a currency you hear a lot, which is how many pages get served. And ~you know,~ at the end of the day, and this is where like the work that we're doing over in SoundCloud, it's much different ~and,~ and a really interesting change.

But, ~um, when you're,~ when you talk about page views, those are, the CDN is really doing the work for you. Like anyone that's been in this space long enough to. Page vs. Is not the currency you should really care about other than if you have had impressions. But cuz Fastly or aai or ~you know,~ clapper, they'll all do it, even clap.

They'll do it for you. ~Um,~ videos a little bit different when you're doing, ~um,~ but yeah, some big numbers. We were throwing big numbers

Noel: How do you, like, how did you prepare, like test for that kind of traffic? ~Like how do you, how do you test like the, you know,~ making sure everything's working and the thing scales correctly when we go from 10 to 100 or a hundred to a thousand is easy, but how do you test like scaling from a thousand to a million?

Matthew: And ~you know,~ it's funny, and this is where sure, you can run artillery scripts against your [00:17:00] CDN endpoint as long as Akamai's doing their job. Yeah, you're gonna go from a thousand to a million. ~Like, it's,~ it's doing its thing. What starts getting challenging is the quick ways that you needed to purge cash.

~So,~ updates to the pages. So if you just had a static page, serving those numbers over time ~is,~ is a no-brainer. Like you put a long TTL on the page, you just let it get served. And again, as long as ~you know,~ Akamai is doing its thing, or your CDM partner's doing its thing, you're sleeping well in it. But when you have fast churn on pages, ~uh, you know,~ you're purging cash at rates across your site, ~um,~ that start giving you pause, that's where you have to start figuring out how do you load, test that to your question.

So ~it~ it's ~kind of, it's kind of~ twofold. ~And I, I, I. You know,~ we had long testing strategies and run-ups to the election that pretty much played back what happened in the previous election. And we have systems that were able to like, emulate those publishes over time. We're throwing, ~you know,~ load at it via vendors, ~um,~ that help [00:18:00] us like simulate worldwide traffic.

So there were long stretches and, ~you know,~ we would ~some,~ some on the, some of the days leading up to an election. ~Um,~ the weekends we would carve out half a weekend and ~like~ play it back almost in real time. So a lot of these systems grew up at brands, ~you know,~ and even ~like, um,~ I can talk about March Madness cuz we just, ~you know,~ here in the United States ~we,~ we just got done with that and ~not,~ not a known, not a thing that, ~uh,~ Turner Sports and, ~you know,~ Turner Broadcasting now, Warner Media and Warner Brothers Discovery, they actually serve a lot of that digital traffic.

So a very similar. Lot of dry runs, a lot of systems are built to play back at the scale that, ~um,~ you'll see in some of these events.

Noel: ~Yeah, I just, I was. It always, uh,~ has a developer makes me like nervous ~when I,~ when I see these ~like, you know, uh,~ mediums that are gonna have traffic patterns where it's like, you're gonna have 2000 x traffic for ~like~ a day and then everything's gonna go back down. Just, ~you know,~ terrifying.

Matthew: Yeah. And ~you know,~ and this is where AWS ~in,~ in. Whether it be Fargate or Atlanta's, ~uh, you know,~ come into a, come into play where you can elastically scale. But, ~you know,~ again, back in some of the early [00:19:00] days, those weren't around. And so we were, ~you know,~ we were building things for those numbers and so you would do a lot of testing before that to make sure ~your,~ your top, top part of the water line

Noel: ~It was good. Yeah. So is the, as I guess as good a segue as any, is the, is the, is the kind of,~ are the traffic patterns ~and~ and stuff at SoundCloud more constant? Is it more of a constant kind of buzz there?

Matthew: Yeah. ~Um, so, you know, when~ when I left C N I was pretty, I was feeling good. I was like, all right, I, ~you know, I've, I've,~ I've done this thing ~at a,~ at a brand that's seeing huge numbers, and, ~um,~ then I got to SoundCloud, and~ so I'll try to answer this question in, in a, in a different way because I, I didn't,~ I didn't know, but I didn't know.

And that, ~you know,~ you hear that a lot in your career. ~Um,~ but what I have been surrounded with is just these fabulous people that ~have,~ have gone out of their way to explain some things that I thought I knew, ~uh,~ differently to me. And ~so,~ Serving music and so is a much different challenge than it is really to your, to answers to your question, which is what are the traffic patterns and traffic in music is.

SoundCloud is has this distinct and really unique, ~uh,~ proposition to our fans, which are users upload content, [00:20:00] music, and you can find all the new music first on SoundCloud. ~So, ~but even before it gets to the other. Spotifys or, ~um,~ apple Musics. So everything that's on Spotify or Apple Music is also on SoundCloud.

~So~ plus we get close to a hundred thousand new tracks a week, ~um,~ for uploaded content all around the world. So your question around the traffic pattern, some of that really depends on the velocity of how new music is. And anyone can create music. So it is ~like,~ it's democratized. SoundCloud has democratized the ability to be a creator.

~Um,~ and, ~you know,~ with the tools now, whether it be ~a,~ a laptop in a, ~you know,~ an OS station or some other kind, little higher end, ~uh,~ mixing station, and even DJs, ~uh, you know,~ the amount that people are able to listen to independent of what's released at a label. It depends on the, ~you know, like the~ willingness of people to contribute their music.[00:21:00] 

And so sometimes, ~you know,~ we'll see these huge traffic patterns that ~new,~ new drops happen, whether it be a DJ drop, ~you know,~ or an artist drop and you don't know it's coming. ~Uh, you know,~ when you think about a brand like March Manez, ~you know,~ there's a 10 pole event. When you think about a brand like cnn, there are 10 poles events like, ~you know,~ state of the Union or some other ones, and then there's, ~you know,~ breaking.

It has been a much different and super fulfilling, ~um,~ space at SoundCloud to start thinking around how those traffic patterns and how those systems need different solutions that your CDM partner ~can't always,~ can't always, ~like,~ it's not the hammer that you can always use to solve some of these problems, cuz they're ~very,~ very, every stream is usually bespoke to the user because your, what your listening patterns are and what you like to listen to is much

Noel: Mm-hmm.

Matthew: to. ~Hypothetically, I don't know what you like~

Noel: ~One, one listener of~

Matthew: ~I'm an old guy. Yeah.~

Noel: ~Yeah. Is there like, I don't know,~ I dunno if you can disclose this kind of stuff, but I'm curious on ~like~ what the distribution is of like listener data in SoundCloud. Like what percentage of listens are the top 1% of ~like, you know,~ SoundCloud artists?

~Like, is it,~ is it pretty skewed that [00:22:00] way, or is it like flatter than one might expect it to be?

Matthew: ~Um, well,~ so again, it ~kind of~ goes back to when those drops happen, and so a lot of it is very temporal on who you like to listen to and the velocity that they drop music. But when you're fans, like when you're favorite artists or creators drop new music, you'll listen to it. But then throughout, ~uh,~ what I've ~kind of~ coined this term, that's probably not the most accurate one, which is between those spaces of drops, then you listen to other label content or other, ~um,~ other creators that are dropping, and you may then find them to be interesting and now you, and they rise to the top.

So you'll see, ~um,~ a lot of like ebbs and flows. But, ~you know, just like any other, not any other,~ just. ~You know,~ things that you find you start gravitating toward. Once you gravitate toward it, then you wanna ~like~ consume as much as you can, and then you may like slow down a little bit and then something else will pop in there.

~Because, you know, as you start recommending,~ as you start getting recommendations ~and,~ and some of this new music, then you'll find the next [00:23:00] person you want to go like consume for. It or a week. ~Um,~ and so that's really where these, ~it's,~ it's such a hard question to answer because it's very emotional. ~Like, you know,~ consumption ~of~ of music becomes emotional even to like what's going on in your life or one's life.

~Um,~ and those are also like, you can't, you can never gauge those. And I think ~that's,~ that's some of the best part of like, how do you figure out how to do tech around that space?

Noel: ~Yeah. How do you, like, what are the, um, what, what, so~ I think you touched on it a little bit about ~like, you know, how,~ how it feels ~kind of~ has more of this like network social networking feel and there's like a lot, like many and many to many instead of ~like~ one or two to many. What else, like how does that influence your kind of technical decisions ~and,~ and ~the, the,~ the stuff, ~the,~ the ways you guys have been delivering features?

Matthew: right. ~Um, so, you know,~ so SoundCloud is, has been, is, has been around for a very long time, well known brand. ~Um, super scrappy out of Berlin.~ We have a team in Berlin that ~I've,~ I've had the opportunity to, I get to go, ~uh, you know,~ every two months and go visit them in Berlin. I'm, like I said, I'm here in Atlanta.

We have a team in, ~uh,~ la, a team in [00:24:00] London and a team in New York, but most of the engineers are really in Berlin, ~in~ in, ~um,~ LA and, but what we did is we acquired a company called Museo, and this was before I got. That has a really, ~uh,~ strong talent in machine learning to understand genre mood of music. ~So, you know,~ being able to ~like,~ recommend music, like what you've liked to listen to, but maybe even filter in a little bit diverging music, just to see how far you'll go in those guard rails has been, ~you know,~ some of those experiments in tech that we ~kind of~ try to go do is how far away.

From your normal middle of the lane, will you go and then, oh, you've gone this far. Let's see how, ~let,~ let's give you more options in that space. And ~so, you know, like~ the cone of uncertainty for music recommendation starts growing ~as,~ as you, as a listener starts getting a little more inquisitive. ~So the tech to, to answer your tech question,~ it's really in those like machine learning spaces and, ~you know,~ machine learning's one of these overused terms [00:25:00] lately.

But, ~um,~ it is really a space where ~you,~ you start trying to figure out ~how,~ how. Sticky, some of those sides of the rails are, and then you, now you have a wider cone of music you're listening to. ~Um,~ and that's some of the tech, ~you know, we,~ like I said, SoundCloud has been around for a very long time. So we had our own data center in Amsterdam.

We're moving things to the cloud. And so how do you rebuild some of this tech in cloud native spaces? ~Um,~ those, that's some of the day-to-day. ~You know,~ I had mentioned GraphQL we're bringing GraphQL to. ~Um,~ API team, our front end teams are our, ~you know,~ all of the teams really ~are,~ are moving into the GraphQL space.

And, ~you know,~ and ~that's a,~ that's a place where I, ~you know, to,~ to go back to a previous thing, I still get to keep my hands sticky cuz I get to write a service that does some things in GraphQL. ~So,~

Noel: ~I'm sorry. Yeah, you, you talked about it. Oh,~ oh, so hot. Hot off the, hot, off the press. ~Um, you talked about how like there was all these different systems. I think we were talking about, uh, CMSs at cnn, like all these things and there was like an effort to try to bring stuff together just cuz like the overhead of maintaining it all.~

~Was SoundCloud a little bit like leaner when you came in? Was there less or more?~

Matthew: ~Oh, you know, um, there was actually more, and you know, I, I will say that I am standing on the shoulders of giants for the people that came before me at SoundCloud. Like SoundCloud invented Prometheus, the tea. And so Prometheus is as well known, like metricing metric and monitoring. That was created at SoundCloud in Berlin in the mid 2000, mid 2010s because there was an, there was an itch that needed to be scratched in this high scale monitoring system.~

~This is before Datadog, this is before CloudWatch and New Relic. Um, but that kind of scrappy, uh, entrepreneurial build model suited SoundCloud well, in those days. And so, no, there were, I, there is a lot. Self grown systems in this~

~space at SoundCloud, you know, and as companies start maturing. And so this is where, you know, the other side of the, the, my job comes into play is, you know, um, building things is one thing, but then how do you manage the team and.~

~Control the costs of how you know things can happen. Then you wanna start saying, okay, maybe this isn't a place where we need to like own that anymore. We can use a service to help us do it better. You know, whether, whatever, you know, insert service here, it's our job to look at like what does a cloud native story look like and should we still wanna own this tooling or is it time to.~

~Lessen our footprint, lessen what we have to go build. That is not like adding value to a user as a, a fan. Um, but in 2015 it was what we had to do because that's, it wasn't anything else. And you know, like I said, I will, I am standing on the shoulders of giants for all these folks that, uh, had the opportunity to build some of the, you know, some~

~amazing. ~

Noel: ~you think that, I dunno, this is, ~this is kind of a strange question, but do you think that now that we're in ~this,~ this era where there is so much either like off the shelf or available to do with minimal configuration in the cloud, that it makes ~kind of.~ [00:26:00] Your or one's job as a CTO or high level leadership in tech, like it makes it more challenging because you have to make so many more decisions on ~like,~ what do we do ourselves versus what do we buy?

Or before it was probably much more ~like~ there's a few vendors for the three or four things we know we need to buy, and that's it. ~Like,~

Matthew: Yes. ~Um, you know,~ sometimes you, ~when,~ when some of these tools come up, it's, I used to say it was like comparing gasoline. Like you could go to a gas station, Chevron, and you get the same gas you got at Shell. ~You know,~ for the international listeners, those are two domestic or US based, ~uh,~ pumping stations.

So there, there was a time where a lot of this stuff had been commoditized. ~Um,~ and so you really couldn't. That much of a bad decision maybe lately? ~You know,~ I think either with acquisitions and, ~you know,~ the cloud providers, ~you know, the,~ the big three, aws, gcp, and Azure coming on strong and giving their solutions, it makes it really hard because, ~you know,~ whether, ~uh,~ when we have our own solution, you have to take into account how much it [00:27:00] costs to run that both.

The techno, like the engine, ~the~ the compute network and disk that it takes to run it, and then the people that it talk takes to manage it. So then when you it, a lot of the stuff just becomes, ~um,~ a cost benefit analysis to each of the vendors because if they're all ~kind of ~doing the same thing, then it just becomes this negotiation that isn't, it's not, that's not always ~the,~ the best part of your job is trying to negotiate.

It's, some of the stuff is a race to the bottom when it comes to the cost. But there, ~you know,~ there are definitely differentiators in some of the tech that make it, make them, ~uh,~ not that. But ~um,~ yeah, ~it's,~ it's not always the best part. It's not the fun. It's not

Noel: ~Right, right. Again, back to that thing, like~ it's fun to build things, right? Like we always ~like, yeah.~

Matthew: Yeah, it is. ~It's, yeah.~

~So then, you know, it's kinda like, um,~ I'm a Dr. Je, Mr. Hyde in some places where I'm like, I wanna build a thing. And then I'm like, oh, but the other side of ~like,~ don't when thing, ~you know,~ dies. So

Noel: Yeah.

Matthew: ~that,~ that.

Noel: ~No, I think,~ I think ~a lot of,~ a lot of engineers that have been in, in the industry for some time, they ~kind of~ have that understanding or like [00:28:00] they have that pulse on unknowing, knowing the. The potential pitfalls there. ~It like what,~ what percentage of your kind of time ~as,~ as somebody in like technical leadership, ~are you,~ are you chewing on those ~like~ buy build questions?

~Like~ is that a pretty good hunky your work anymore?

Matthew: ~no, it's,~ it is, ~you know,~ like I said, I got to SoundCloud about a year ago. ~Um,~ and ~if,~ if you recall, we had a lot of our own built systems. I spend a lot of time as the contracts a as we're looking through our stack, trying to understand ~like,~ okay, is it time to pivot? If it's time to pivot, what is that gonna take?

~Um,~ and ~so, you know, I I,~ I'm here in Atlanta, so on the east coast of the United States, Berlin is, ~you know,~ five hours ahead of us. So I usually get. ~Um, you know,~ around 5, 5 30, work out and then get on calls with folks in Berlin. And a good chunk of that time is with some of the engineering leaders trying to figure out what is our strategy, whether it be, ~you know,~ are we gonna use RDS or, ~um,~ Amazon's database service to [00:29:00] serve some of our data, my SQL traffic, or are we gonna use Mongo?

And ~so, you know,~ I spent a lot of time going. The tech strategy because that informs the vendors that we're gonna use or not. It informs what we're gonna build. And so this is where ~like tr~ staying really close and being able to ~like~ understand the tech helps to your question, how much time is spent. ~Um,~ and then also trying to reduce our footprint is key.

Footprint of self owned and self-run things. ~Um,~ just because that improves the ability to get new features. To our fans, right? ~I mean,~ some of ~our largest competitor,~ our largest competitor, ~um, you know,~ they have in the 10,000 employee space and our engineering staff is measured in the two 50 to 300. So a lot of those decisions ~are,~ are geared to ~how do we,~ how do we build things, ~uh,~ with quality, but,

~uh,~ quickly, 

Noel: Yeah, that makes a lot of sense. I guess for listeners who like might want to get into like technical leadership, cto, ~like, you know,~ positions eventually would, is that what you recommend they ~kind of,~ the skill they [00:30:00] hone or is there anything you would, ~you know,~ encourage them to focus on?

Matthew: ~You know, everyone's path,~ everyone's path is different. Like I, ~you know,~ would I encourage somebody to stay scrappy in this, ~like~ to become a C T O. ~Uh,~ I never could have done a lot of the things I do now without great people, right? ~So, like, uh, you know,~ I, ~uh, I've,~ I've had amazing friends at work that I've learned a ton from, but ~the, you know,~ my path was, I started with management and so I've been able to use a lot of the management things.

~Um,~ but even ~like, you know,~ in, in eighth grade, I remember we had ~like~ one computer in the, ~uh,~ in our typing lab. And, ~you know,~ I'd stay after school and learn how to. Like I coded a Zork game, if anyone remembers Zurk. It was, ~uh,~ tech space. ~Um,~ this was like 1981. And so I, this kind of started for me back in like eighth grade.

And then, ~you know,~ the journey was management ~in,~ in college. Always stayed close to computers. ~Um,~ worked ~in the,~ in the computer lab at school as well as, ~you know,~ like I said, working at Turner. And, but I, for me, ~it was always, it~ developing and staying close to [00:31:00] technologies was a passion. ~Um,~ I couldn't get enough of.

And so that was my path. ~Um,~ I, the other big part of my journey that I've learned is, ~um,~ humility. ~Like we, I used to say that, you know, we didn't want any smart assholes on the team because as you start getting, um, that's, that was my one swear word for the, for the hour.~

Noel: ~We're good? Yeah.~

Matthew: ~Um, but you know, the. The only real way to, to get people to follow you into the whatever technology choice you're making or to work over a holiday is to be humble.~

~I mean, they'll work for you cuz you're gonna pay them, but they're not gonna love it and they're gonna leave. So then your attrition starts getting, so like~ being humble and understanding and bleeding with empathy, I think is one of those other things that, whether I learned it in school or learned it, By, ~you know,~ not doing it and then people telling me, no, I wasn't doing it.

~Like,~ I think those are some of the real things that, ~um,~ you have to have those skills to lead teams of people that they may not agree with you, but they're gonna, they're gonna follow you. ~Um,~ and ~you know, I it,~ I've been so lucky to be able to like, be surrounded by people that schooled me on some of those skills

Noel: ~Nice. I guess, is there, is there, was there anything you kind of, um, were seeking out in like mentors and people that taught you this or did it just kind of happen by, you know, osmosis?~

Matthew: ~Yeah. Well, it's funny you say that. Like I, I highly recommend folks trying to find a mentor, um, that can give them some of the school of hard knocks without the hard knocks. Um, although there is something to be said for, uh, you know, having to really like, Stay up and work till three cuz you didn't know any, like nobody told you any better.~

~Um, but yeah, I mean, I, I, you know, to the, to the question, my, my mentors were whether friends, friends at work. I made a lot of really strong relationships with friends at work that I learned, I learned from anyone either next to me, you know, above me, below me. They've, they've all been super, um, willing to.~

~Be okay with me screwing up. Right. And, you know, asking for forgiveness is much easier, um, than asking for permission sometimes. So I, uh, you know, I, I have a great partner in crime who was with me at, at, uh, CNN and now at SoundCloud, he's the Chief Product Officer over at SoundCloud. And, you know, finding, I would, so the, the answer to your question is if you find somebody that you can learn.~

~Whether you work with 'em or not, you gotta stay close to them. Cause I, I still stay close to people I've worked with over 20 or 30 years. We don't work together anymore. But, um, you know, you, you, you, you need to find, find your tribe and like it's really strong.~

Noel: ~Yeah, it's good to hear. Uh, and, uh, I like those. No, there's no, there's no special super quick secret answers. Like, do the things that people tell you to do, like, it's good. Yeah.~

Matthew: ~Yeah, I mean if, like I said, I had, they, they started stopping. I had plenty of, plenty of, uh, managers and, uh, fabulous people that I, I still stay close to. They were at some point they were like, well, you can't keep developing if you wanna keep doing this. And then they stopped telling me that cuz they knew I wasn't gonna quit.~

~And, um, so if you love it, you gotta keep doing it, then sometimes it means, you know, 14 or 15 hour days. But I'd rather do that, um, than not. Enjoy my job.~

Noel: ~I guess, yeah. On on, on that note,~ is there anything that you're particularly excited about in the future? Either ~like,~ again, super low level, like specific tech that you're into, or like higher level tr patterns and trends that you're seeing?

Matthew: ~Um, well, you know, I mean, I, you know,~ I had mentioned machine learning's, the overused term. ~You know,~ obviously chat g PT is on everyone's head, ~so, you know,~ but what's what we are testing and I, ~you know,~ again, I [00:32:00] have this fabulous canvas at SoundCloud to be able to play with some tech is like generative art for.

Mood playlists or ~you know,~ creators that want use generator of art to generate album art. And so ~like,~ again, trying to figure out how some of this tech can enrich ev like one of our users or fans' lives. So some of, in some of the space we're using some ai. ~Um,~ some, ~you know, whether,~ whether, ~uh,~ the blockchain is here to stay, ~you know,~ it went through its crypto winter in 21 and 22, but some things around, ~uh, the,~ the blockchain and web three I think is really interesting in the music space for how you have, ~you know, uh,~ if you think about how music is copyrighted and all the royalties that go into music, it's.

If anyone actually knew how that sausage was made, about how royalties are paid out, ~it is,~ it is not a pretty site and I have a great team of folks that I get to work with that have schooled me on it. But this is a place where, ~you know,~ I think blockchain [00:33:00] technology can help understand, ~um,~ the value of a listening experience and then ~like~ the composition of an asset and who.

Royalties around it, on the chain ~is,~ is actually really, ~uh,~ I think something that ~we're,~ we're ~very,~ very interested in. ~Some of the other things, even in the blockchain space are in, um, almost what Four Square did many, many years ago, which is, um, like you got a badge and that badge. Like you were the mayor of a place.~

~And so if you think about it in music, you could be like your Noel Super fan, and you get a badge on the chain that is immutable. Um, that is like, that is a bit of currency that, um, I think is really interesting in this space where, you know, you may have fans that are so passionate, they want, they're very proud about it, and then having something immutable in the N F T space.~

~It's something, you know, we are kind of excited about. I think that's, that's a place for us to, to dabble and not dabble like play, but really be a, a market leader in that because we have all of those fabulous creators that come to our platform first. Um, and many of the artists, you know, today were found on SoundCloud and then went to a label.~

~We're gonna try to keep them with us and get some of these new, new products~

Noel: Nice. ~Yeah, it kind of, it's, it reminds me of a lot of the stuff, um, like Brave was doing pretty early on with like micropayments on the web and like tracking that stuff and like figuring out how to move like assets around in ways that are more conduc. To, you know, the digital world. Um, yeah. So that's cool to hear.~

Matthew: ~You know that if you, you know,~ as our conversation has progressed, ~like, you know,~ there were things that we were doing in 2013 where now are commonplace. ~Right.~ And it's not a, it's not a hard thing. I think now when we look ~in~ back in 10 years in the, ~you know, two. 23 space after Covid,~ people are gonna be like, that's really easy to do what talking about?

But, ~you know, we'll,~ we'll remember the time when ~it was,~ it wasn't as easy and ~you know,~ you didn't just look in book and figure how this

Noel: Right, right. Yeah, I think, yeah, ~that's, that's a, that's a,~ that's a perfect note to end on I think, Matthew. ~Well, um, yeah. Yeah. Thank you so much again for coming and chatting with me. Is there anything you wanna point listeners towards in particular to check out? Give a look.~

Matthew: ~um, you know, I mean, I, I, I.~

Noel: ~That's okay.~

Matthew: ~I ask, hold on. Let me, lemme me, lemme me. Hold on. Let me noodle on that~

Noel: ~a moment.~

Matthew: ~maybe you ask me again. Like, I don't wanna, I don't want to turn into a pimping session for like,~

~SoundCloud. I, you know, cause it's not, if this, I mean, you know, hopefully I've kind, I, this was more not a, not about the brand, but about~

Noel: ~yeah, yeah. I think we had enough depth there, so I think you could do, do a couple~

Matthew: ~no, you know, I,~

Noel: ~Yeah.~

Matthew: ~yeah, no, I think, I think we're good. I don't want it to turn into like, you know, I'm, I'm~

Noel: ~No, no worries. Well, again, um, yeah,~ thank you so much for coming on and chatting with me, ~Matthew. Uh, I think~ you ~yeah.~ Gave us a lot of insight and~ it's,~ it's been a pleasure.

Matthew: I appreciate

it. Thanks for the chat. ~Okay.~