Daniel (00:01.214)
And we're live. Dave, it is fantastic to see you. Ah, you were making such a nice face for that. I should have used that for the thumbnail. We always need like, you know, shocked YouTuber face or like.

Dave (00:07.355)
I really wasn't. It was more like a sort of ah and jazz hands.

Dave (00:17.861)
Yes.

Daniel (00:19.503)
for the stupid thumbnail, which you did the last time. Usually I do it, but somehow my Affinity Designer app just like won't save anything. It would literally just hang for an hour when I try to save files, which is kind of weird. And I thought it was saving in iCloud, but it's not in iCloud, it's just on the disk. And I thought it would be fixed with a reboot, but it isn't, so I'm gonna have to troubleshoot at some point, but I'm gonna.

Dave (00:35.647)
Dammit, Affinity.

Dave (00:44.943)
I've got a suggestion. I have a suggestion. Yeah. Buy pixel made pixel mates, a pro and I will send you over my template from last week.

Daniel (00:47.114)
Yeah? Okay, what's the suggestion?

Daniel (00:56.123)
The template is fine. Mine is also just three minutes of work. But I kind of like Affinity. I was a Pixelmator user for a long, long time. But I assume I don't have the Pro because that's probably a... I bought Pixelmator at some point when the App Store came out for the Mac. So I assume I have Pixelmator, but not Pro.

Dave (01:00.821)
Yeah, yeah.

Dave (01:17.649)
Mm-hmm.

Dave (01:23.975)
Yeah, that's likely.

Daniel (01:25.214)
Yeah, pro says it gives me a buy icon and I can't find a non pro. Hang on, I'm gonna go. Other apps by this developer is nothing basically. Okay, so the Pixelmator non pro is not even a thing anymore. Yeah. I don't like, I'm gonna try it one more time tomorrow and then if it's not working, then I guess I'll, I'll reinstall or something and then if that doesn't work, then yeah.

Dave (01:29.64)
Mm-hmm.

Dave (01:33.151)
Yeah.

Dave (01:42.606)
Yeah.

Daniel (01:54.062)
it's probably time to buy Pixelmator Pro because I'm a pro at the podcast. But I wanna introduce the podcast before you poke me to do it. So hey, welcome to Waiting for Review, a show about the majestic indie developer lifestyle. Join our scintillating hosts, Dave and Daniel, that's me. And let's hear about the tiny slice of their thrilling lives.

Dave (02:00.008)
Indeed.

Dave (02:07.323)
Go for it Daniel, hit it!

Daniel (02:23.33)
Join us while waiting for a review. Getting ever closer to the microphone because it's just so majestic and everything. Ha ha ha.

Dave (02:34.093)
How are you Daniel? How have things been?

Daniel (02:36.118)
Ah, my life is thrilling as always. So I'm gonna work backwards because today and yesterday, I basically spent all my time in server land, which I'm not a native of, and I don't really feel welcome there. I'm just doing my best. Like I'm learning the language, I'm trying to do all the right things, but still server stuff is just massively frustrating to me.

Dave (02:39.019)
scintillating.

Daniel (03:06.078)
Um, and on the other hand though, it is really important and necessary. And it's getting more and more because, um, like we're getting more and more customers, which is really nice. Thank you. Awesome customers. And they're sending more and more data, which is also very nice. Thank you. Awesome customers, but that data needs to be massaged and stored and, and sent around to various services and then queried and like prepared and defragmented. Um, and.

Oh, like, like every time we need a new order of magnitude, like there's just a different set of problems that, that I was like, Oh yeah, like every time I solve the problems and then I'm like, Oh yeah, now we're ready for the next year or so. And then we had another order of magnitude. And then it's like all the same. So we actually, a few days ago, we hit 4 billion signals, which is pretty cool.

And yeah, it's speeding up. And it turns out 4 billion is way harder to manage than 2 billion somehow. And it amounts to about a terabyte a month right now of data. Yeah, it's a lot of stuff just. And the problem is that

Dave (04:06.717)
I remember when it was 2 billion that doesn't feel that long ago

Dave (04:19.531)
Hehe.

Dave (04:25.135)
Right. Wow.

Dave (04:31.323)
Yeah.

Daniel (04:34.382)
I don't know, like the whole, like the data is moving from point R to point B. Like it produces so much bandwidth that like bandwidth costs are going up. Then it needs to be like managed and queried and like stored and defract and whatever in the, in the Apache Druid cluster, which is our database. And that's mostly where I've been spending my time. And it's just so hard to get that thing to be really, really high performance.

And that is, ah, that has like multiple, multiple reasons. Like one of the reasons is, which is really frustrating to me. And I'm going into the weeds now. I'm going to really, I'm like, ah, so listener, if you don't care about, uh, Java on the server doing weird Java things that no one likes, you can just skip this podcast, just like, I don't know, like Raiders, Raiders on iTunes. Thanks for listening.

Dave (05:13.451)
Keep going, keep going into the weeds, I wanna know.

Dave (05:26.987)
I'm sorry.

Daniel (05:30.442)
Read us on iTunes, read us on YouTube, send us emails at contact at waiting for a view and where can people find you Dave? No, okay, so the thing that annoys me the most is the overhead. Like we are paying for our servers, like are we gonna start paying? Like we've been granted a very gracious startup credits

Dave (05:35.167)
Oh

Oh my God, Daniel, don't, yeah, keep going. Don't skip to the end. Let's hear more.

Dave (05:47.443)
Hmm.

Daniel (06:00.546)
thing by Microsoft for hosting on Azure, and that's gonna run out. So now we're gonna actually have to pay for our computing power. And so I'm optimizing for that, but also optimizing just for the new amount of data that we're now managing. And it's so frustrating because we are hosting on Kubernetes because that's the easiest to manage right in the future.

Like, I was going to say, don't write email. But we're not getting so much emails or whatever. If you disagree, write us an email. Write us a toot. But yeah, so I'm using Kubernetes because that's the easiest to manage for me and because there's so many different things. And I can use various shortcuts to make my life easier. Kubernetes does this thing where you can't have swap space, basically.

Dave (06:33.727)
Hehehe

Daniel (06:54.954)
So if your thing uses too much RAM, it will just be mercilessly killed. You will just say, like, oh, for five seconds, you used more than your allotted amount of RAM, like we're gonna kill you. And the allotted amount of RAM is a bit arbitrary. So what happens is that you can have, of course, multiple containers on one machine, right? Because it's Docker. And so one thing that will...

Dave (07:08.991)
Wow. Okay.

Daniel (07:23.842)
just evict containers is if the whole machine says, OK, my total amount of RAM is almost getting full, so I need to throw something out. So that will kill your process. But also, you can have a thing where you request a certain amount of RAM to be reserved for you. And if you do that, for example, to prevent too many containers being scheduled on the same machine, if you do that, then

Also, if you go above that, you can go about, let's say, 10% above that. But if you go more than 10% or more than a few minutes, then you will be mercilessly killed. And so I'm trying, and then I'm going to finish this thought first, and then I'm going to have the other thought. And so on top of that, so what you do is kind of you leave like 10, 20% of space just at the top there. You...

Dave (08:21.727)
Yep. Headroom.

Daniel (08:21.758)
You configure everything to never use the full amount of RAM that you actually have, that you're paying for. You're paying for 128 gigs of RAM, and then you use 110 maybe. Or you allocate 110. But then there's another thing, which is Apache Druid is written in Java. And Java does this super weird thing, where if you launch a Java application on the server, at least, I think it's on the desktop as well. If you launch a Java application,

Dave (08:28.43)
Mm hmm. So you sit you sit under your limit.

Daniel (08:50.21)
you kinda have to supply it with the amount of RAM it's allowed to use.

Dave (08:55.732)
Okay.

Daniel (08:56.938)
And if it hits or exceeds that, it will crash with an out of memory exception. So there's another ceiling where if you hit that, it will crash. So on top of the Kubernetes ceiling, you will like where you actually have to stay, stay underneath. There's another ceiling where I'm like, okay, I'm going to have to have some safety margin to that ceiling as well. So out of the 110 gigabytes of RAM, I'm.

Dave (09:21.616)
Mm-hmm.

Daniel (09:26.654)
maybe daring to go to 100, but today after a few tests, I'm actually going down to 90, because it will spike, and then it will hit the thing, and then it will either be killed by Java or by Kubernetes. And this is just the way it is, for the people who mainly use Kubernetes, a bit of RAM is not so much, but I feel like...

Dave (09:49.566)
part of the job.

Daniel (09:53.902)
there's a machine and I really want to use its performance. Especially since now I can't use the CP or Zoom cores in that machine anymore because I'm reducing my own amount of tasks that I'm doing. So I was like, cool. Now the CPU is only half saturated as well. And before you say, oh, but you shouldn't saturate your servers fully, that's correct. But this kind of server is a worker server that will get a task every now and then and then has to work on that task as well.

Dave (09:57.336)
Yeah.

Daniel (10:23.726)
quickly as possible because I want it to be done as much as quickly as possible. If it gets killed, it's not that bad. The system can retry the tasks and rebuild the server for that. But yeah, it's not something that will continuously run at full tilt. It's something that is.

Dave (10:39.115)
of stunt. It's got a response to like a burst pressure of stuff that it then relieves. But Daniel, I've got a very key question inside of all of this. I thought telemetry deck stack was in Vapor and you're mentioning Java.

Daniel (10:43.063)
Right.

Daniel (10:50.592)
Okay.

Daniel (10:56.87)
Right. So the code that I wrote is written in Swift and Vapor. So that's correct, although there's a tiny bit of Python in there. But our database, our time series database cluster is Apache Druid, and that is written in Java. And so that's not something that I wrote, but there's still some, like, there is still

Dave (11:05.339)
Yep. We forgive you, Daniel, it's okay.

Daniel (11:26.902)
the main, I want to say the main source of administrative work because that is the database that contains all our data. And it's actually really good. It sounds like I hate it. And OK, sometimes I do hate it. But I think it's the best tool for the job. It's really well thought out. It's well supported. It's getting continuous upgrades. And the development is actually moving places. And yeah, so.

Dave (11:42.067)
Yeah.

Daniel (11:55.358)
Nothing against that, but it is written in Java and has the problems that it has. And it's also written by people who want to build something that is highly configurable up to a point where I'm like, come on, please just do it, like pick a default or, you know, like just.

Infer the size of the merge buffers from the amount of RAM or something. Like don't make me actually do that myself and then say on the website, oh yeah, this should actually be the number of processes times the amount of whatever. I'm like, yeah, but if you have that calculation on the website, do it yourself. Please. Sometimes if we leave out the value, it actually will do it, do it itself, but it's kind of trial and error process when it does that.

Dave (12:22.195)
Yeah, yeah, yeah.

Dave (12:31.924)
Yeah.

Dave (12:43.155)
doesn't sound fun. None of that sounds like fun to my swift addled Xcode addled brain.

Daniel (12:44.622)
Ah, and the other thing is, I mean, yeah, but it's also kind of a server admin thing. Like I wrote, like I've been writing server code for half my life, I think. And it's just something that you have to do with to deal with. Like every time I actually switch from mobile development to server development or the other way around, especially before when I did only one thing for one gig and then the other thing for the next gig.

Dave (12:58.812)
Mm-hmm.

Dave (13:12.448)
Mm-hmm.

Daniel (13:15.05)
Um, every time you switch, you kind of have to switch to change gears in your brain, because on a mobile phone or a mobile device or on the desktop, even what you really want to do is safe, or have a, have performance, performance that feels immediate. Like you're going to do a thing once and it has to be quick the first time, but it doesn't have to be perfect or it doesn't have to be, I don't know, like log in or whatever complexity.

Dave (13:32.203)
Mm-hmm.

Dave (13:36.148)
Yes.

Daniel (13:42.454)
Whereas on the server, you're going to do things a million times. It can take five minutes to start. It can take a long time, but if you repeat it a million times, it needs to be efficient. And I think that's actually a pretty, uh, thing to really keep in mind when writing different software for the different platforms. The other thing though is, though is yeah, you're completely right. Like it's hard to find out those things, especially because

Dave (13:52.255)
Yeah.

Daniel (14:10.99)
It's harder sometimes to prototype these things. Like with an app or even with my API server, I can just start it or run the unit tests or whatever. But to find out like what's the correct setting for the buffers or whatever, I have to have a thing where I have to have a cluster with all my data in it. And also that's kind of expensive to host, right? So I don't want to have a duplicate cluster hanging around, but actually I bit the bullet and I have a...

Dave (14:30.698)
Yes.

Daniel (14:40.742)
a tiny second cluster, which is actually kind of fun because I, when I started the config file that actually brings up the whole thing is what's called tiny telemetry cluster because I was just like, yeah, this is from the example file that's called small cluster and I've renamed it to tiny because I was like, okay, this is very small. It's still that you still called that.

Dave (14:59.839)
TT as well. It's sort of alliterative with telemetry deck there as well.

Daniel (15:03.53)
Yeah. So yeah, I have a second cluster now. And I have, actually right now as we are speaking, I am ingesting into it all the data from January and February. And my day so far, my entire day was just like, okay, like provisioning that cluster, like just bringing it up and then making Azure.

Dave (15:18.504)
Mm-hmm.

Daniel (15:30.862)
like asking Azure to, oh, can I have some more CPU cores, please? Yes, I know my quota has exceeded, but I need some more cores. I mean, I have so many credits that's going to go unused in April, so can I please have some core? That's how they get you. They give you all these free credits, and then you can't use them. And then I sped up.

Dave (15:40.011)
Please sir, can I have some more?

Dave (15:52.807)
Hehehe

Daniel (15:59.186)
I basically started a task that says, okay, load all the data for January. And that failed because some memory thing. The process has always failed at some point because of the memory limitations, because I actually used it as, like I managed to get a day or two in, and then I deleted that again. It was like, okay, now I'm gonna get the whole thing.

but fine tune my performance settings because I can't do that on the live cluster because every time I change something, of course, at least the server types, let's think of like five different server types and they're called Overlord and Peon, it's kind of also a Warcraft thing going on there. And the middle managers is what I was working on. So every time I change the configuration, I need to restart the middle managers, which...

Dave (24:52.316)
Yeah, yeah, yeah.

Daniel (25:00.288)
Kubernetes thing does for me. But so of course all the tasks are gone and then like, I can't do that on the life cluster it's it, it causes, it causes chaos and also people, people's queries will sometimes not work and whatever. So now, okay, I'm going to do this on the secondary cluster and then just try out all these things that I can't draw normally try out and that's basically what I've been doing all day and I've just looked and I'm seeing a wall of about

lights. I'm going to send you a screenshot. I post it later and then I can actually reference it in the show notes. And that is good because then this process actually worked and that is good because my tuning is now in a place where this thing is highly, or at least the middle managers are very performant and also don't crash because they touch any of the ceilings basically.

Dave (25:30.923)
Hey.

Dave (25:35.727)
Yeah, yeah, post it and reference it. It's all good. Yeah.

Daniel (25:58.804)
which is also very good. So yeah, that is, so the next step is now to use that cluster that actually has data to then also fine tune the things with the, you know.

to also fine tune the data. Queries and stuff like that. Queries, that's the word I was looking for. Yeah.

Dave (26:27.295)
Wow. So with all of this, it sounds like it's a case of like, well, A, having the secondary box has been a good idea, right? You've let you do.

Daniel (26:37.892)
The secondary box, the secondary box he says. The 16 secondary boxes you mean.

Dave (26:45.847)
I understand. Yeah. It's a cluster. It's.

Daniel (26:48.12)
Eight of which are like high performance 32 cores, 128 gigs of RAM kind of machines.

Dave (26:56.376)
Besides that it's just a box, it's a secondary box.

Daniel (26:59.192)
I mean, it is just a box, you are correct.

Dave (27:04.207)
Sorry, but I don't mean to bismurch the many box and boxes. Box eye? Yeah. But yeah, so that's been a good idea. Staying under your limits for each layer also has got to be a good idea.

Daniel (27:12.834)
Voxen. No, it is.

Daniel (27:19.366)
has.

Daniel (27:25.328)
Yeah. So the three things that this thing has to do regularly is ingestion. That's what I've been optimizing now. Then of course, queries, which is a different type of server. It's called a historical. And that basically gets the, the craze that you send to telemetry deck. And then like one of the, uh, coordinators, I think we'll kind of chop up the query into different, um, historic segments, which are from day to day, like, for example, if you have.

varying a range from January and February, then you have like four historical servers, then like one will get the first two weeks of January, one will get the second week of January, stuff like that. And then they will actually run the queries and they need a lot of disk space, they need a lot of RAM, they need a lot of CPU. So next up is, now that I have example data, is actually optimizing those. And then the third task that this thing really needs to do is compaction, which is basically data diff defragmentation.

Dave (28:03.595)
Mm-hmm. Yeah.

Daniel (28:24.008)
the older people among us will remember as kids, like staring at their computer screens while their hard drive was defract by, I don't know, Norton something or Windows defrag or whatever. And yeah, and basically that process, because that will tremendously improve performance in Apache Druid queries.

Dave (28:33.745)
Yeah.

Dave (28:38.407)
Yep. Wow. That takes me back.

Daniel (28:49.356)
Um, because it can pre-sort the data. Like for example, it can pre-sort the data by app ID test mode, and then a few more properties that are basically in every signal. And so if, and you basically filtering for these with every query. So if it can pre-sort those, then it, the whole, like the server won't have to actually look at a lot of the data that is in there and we'll have, like, we'll just look at your app, for example, and just with the test mode equals.

Dave (28:51.793)
Yeah.

Dave (29:05.779)
Mm-hmm.

Daniel (29:18.632)
false, stuff like that.

Dave (29:20.043)
couple of things that are springing to my mind with this is like when you say pre-sort does that mean it's literally sorting stuff on disk somewhere or is it building up indexes of things to say well okay if I sort this by this then that's where the things live

Daniel (29:36.868)
Um, both kind of. So what is actually happening is like the root format is a binary format. Uh, and it's very much index based, but it does, uh, it has these keys and they're kind of hashes and I'm telling it, all right. Your hash consists of like whatever you decide plus app ID plus, um, test mode, let's say, and then also sort by that hash and what that means, and because it knows that all the, like the properties I gave it are in there.

it actually, whenever I filter by app ID, it will actually go, it can go to exactly that place in the file. And that's the other thing because these are called segment files. And when I just pour in the data, a day will result in about, I don't know, a thousand files on disk, which will have, let's say a hundred thousand or 500,000 or so like signals in them each. Don't check those numbers, like these are just.

Dave (30:13.003)
Mm-hmm.

Daniel (30:34.576)
of the top, like just the example numbers. But that is actually not very performative because of performance, like good for performance because what you really want is like, you wanna have as little files as possible. You wanna have about 7 million entries per file. And so instead like the process will have to look into all these files, then all get the data out there that is relevant, keep it all in RAM. And then actually like once it has collected all the data, then it can actually start. Whereas,

if it's just a few files and then maybe even all your data is in one file, then it can just take a look at that file and then start work there. This is all very low level, of course. Start work there and then go on there. And so, yeah, and because one historical can hold hundreds of gigabytes of data, it can't fit all into RAM. So that's also a reason why these have to, why these needs like reasonably fast hard disks.

Dave (31:13.116)
Yeah.

Dave (31:31.399)
Yep. Damn. Big insight into the telemetry deck cluster, which is huge. I'm sorry, Daniel, I have to highlight this. I remember talking with you probably, what, two years ago when I was interviewing people in season two. And there was a conversation I remember we had a little while ago where you were sort of saying...

Daniel (31:34.099)
Yeah.

Daniel (31:39.249)
Ah.

Daniel (31:42.428)
Hahaha!

Daniel (31:47.496)
Yeah.

Dave (31:56.403)
and just run it on the one server and just keep feeding it as much capacity as it needs until you outgrow up. Then I remember you taking everything to a clustered stack and it sounds like it's exactly the journey you've been describing just earlier, which is that at each stage it's like a different level of both a different level of complexity and a different level of abstraction that sort of kicks in almost as well.

Daniel (32:10.909)
Yeah.

Daniel (32:22.252)
Oh yeah, very, very much so. Like at every stage I need to know, I need to on the one hand go deeper into the abstraction and like go more into the details of stuff that was kind of papered over by the previous layer of abstraction because I need to optimize the buffers and whatever, the segments and the, but at the same time also zoom out and look at a bigger picture and try to automate even more, try to like organize, I'm not organizing individual servers, I'm organizing whole

Dave (32:38.51)
Yeah.

Daniel (32:51.496)
Kubernetes clusters, basically, stuff like that. But yeah, that's kind of the deal where you wanna be very careful to not prematurely optimize, but also you wanna be very, you wanna recognize when is the time to be like, okay, this paradigm doesn't work anymore and it needs a rework. It probably doesn't need a rewrite or recreation, but it needs some kind of work to actually prepare it or bring it into the next.

Dave (33:11.241)
Yeah.

Daniel (33:20.62)
next year basically. And yeah.

Dave (33:22.483)
What drives that for you? What drives that moment?

Daniel (33:28.072)
usually it's a feeling of technical debt building up. Like right now with the existing cluster, I'm like, okay, stuff is working mostly, but compaction tasks are failing about half the time. So performance has been degraded, which is super annoying to me. Every...

every other day or so will send me a message saying like, hey, somehow I can't connect to my database. Oh wait, I can't, like this thing has a Postgres database to know where all its segments are basically. And then it's like, oh, okay, now I'm back because the database is actually super, like it has disk space still, but the Postgres database is actually using 100% CPU in its container. So actually I gave this a bit. So these are all symptoms, like every now and then like something will slightly break, like a few days ago.

Dave (34:12.191)
Ha.

Daniel (34:21.448)
we had an incident where one of those services kind of stopped. And nothing was really, and so like I actually saw in the next morning that it hadn't like, the live streaming of data into the server has kind of had kind of stopped. So I kind of restarted that and nothing was actually lost. Like all the data all kind of came back, but it took the whole day or even more than a day to actually reach, go back to real time, like to catch up basically to real time, which was.

long for my taste. So the short answer is, I kind of feel it, but the long answer is, there's a few indications. Like you want to find the thing, the abstraction level, that feels comfortable right now, where you feel like with your current resources, you can maintain it. And where it's not like a huge...

It doesn't feel like an adventure. Like because an adventure is always something that we want to go on, right? We want to explore, we want to try out new things. And most of the time, like don't do that with your infrastructure or with your database or stuff like that. Like...

Dave (35:27.113)
Yep.

Dave (35:34.028)
No. These things should be boring almost after a point, right? It's like yeah that-

Daniel (35:38.436)
Right. And when your current, even your current stakes feels more and more like, Oh, I'm fighting against this. This is a chore that is like actively unfun right now. Then it's probably time to think about, okay, what is the next step that restores the boring state?

Dave (35:58.027)
That makes sense. It makes a lot of sense. And it kind of reminds me of when we talked a few shows back about the idea of sort of pain driven development, right? It's like when you feel that pain. Yeah.

Daniel (36:06.212)
Yeah, I think that's the same philosophy. There's a conference talk in there somewhere whenever I can find the time to actually write it and find a conference that will have me.

Dave (36:10.771)
sure there is.

PDD.

Daniel (36:17.376)
PDD. Yeah. But yeah, that's definitely the kind of the same. And then you can have adventures, but on your front end, in your user interface, or on the thing that you're actually innovating on, like, for example, with your awesome video pipeline, stuff like that. That's where you want to be innovative. And yeah.

Dave (36:35.252)
Mm-hmm.

Dave (36:40.051)
This is actually reminding me of that a little bit. So just one of the reasons we've spoken more about your server side stuff this show is because I've kind of parked myself recently in terms of the developments I've been doing. I hit a bit of a wall last weekend and then I've not been feeling too good this week. So I've taken a bunch of time out. There was actually an issue over there with the pipeline.

there is an adventure for me to go on a quest for me to pick up when I'm feeling a bit better. Um, but, um, yeah, right now I've got. Mental background processes, as it were, probably thinking about the issues still that will, you know, um, yeah, hopefully come good when I next sit down and look at it, but, um, yeah, but, but what I'm thinking about with that pipeline is that, um,

Daniel (37:10.543)
Mm-hmm.

Daniel (37:26.128)
Yeah, sometimes you need to step away.

Dave (37:34.683)
Yeah, it's been a similar thing, different levels of complexity, different levels of challenge. And you're right, there is a part of me that definitely needs it to reach a boring stage, a non-state, just to highlight what it's been over there for me is I built out this node-based video pipeline about four or five months ago, started working on it and...

brought it to the stage of a test app and I can link the nodes together and make different effects with this. I fast forward to the last month or so and I've now been bringing that into my video mixing app and wiring it up to the UI. I'm actually really glad I've done it this way. So I had a whole moment.

a few weeks ago where I was sort of saying how I felt bad that I'd not been working on the pro version of the app that this pipeline is going to support. One of the reasons that I'm currently, I currently have this sort of side quest to go back to of optimization and issue within it is because of that work, right? So I'm now, because I'm wiring it up, I'm now able to fill the bits that didn't work with lower complexity a few months ago.

Daniel (38:31.421)
Hmm.

Dave (38:52.327)
And now literally adding it back into the app that I've got that has a lot more going on. I'm now hitting that complexity and that is causing me this immediate pain. When everything comes off and works, you won't know it. Like the app will just work. It will just do what it's supposed to do. Right now. I've once again made a very good thermal heater for your hand.

Daniel (39:00.63)
Mm-hmm.

Daniel (39:08.404)
Yeah.

Daniel (39:18.492)
hahahaha

Dave (39:23.584)
Which is something you won't get in serverland unless you've literally got your cluster on your desk and your hand underneath it, right? It's like if I get something wrong with mine, people's devices get very hot. Because the yeah, in this case, it's somewhere to do with how it's passing one layer to the other. And I'm pretty sure there's a loop somewhere in there.

Daniel (39:23.784)
Fantastic.

Dave (39:52.015)
Anyway, I will rejoin my side quest to try and fix the pipeline to try and get this sorted at some point in the next, probably the weekend coming up actually. Yeah.

Daniel (40:00.252)
Yeah. Awesome. Good luck. Like I hope it, uh, it, it will like your rejoin will be swift and, and fantastic. But yeah, like you wanna, you wanna have whatever you're developing and regardless of your size, you want to reach a place of safety and boredom every now and a while where it's like, okay.

Dave (40:13.93)
Ayyy

Daniel (40:24.956)
Like I am on a very good or reasonably good foundation and now can be created from top of that foundation.

Dave (40:33.227)
That's it. That's it entirely. And I think, again, I'm relating this very immediately back to my current issue, right? I don't have unit tests at the moment across this video pipeline, right? Quite deliberately so, because there's been a whole element of like, it's been very, just needing to be built very quickly to do what it does. And I've been able to sort of hold the whole layer, the land in my head.

Daniel (40:47.952)
Mmm.

Daniel (41:02.415)
Mm-hmm.

Dave (41:03.155)
The last weekend kind of proved to me I currently can't hold the layer of the land entirely in my head all at once now. It's kind of outgrown there. And again, that's something else that is sort of a tiered complexity, right? When you're just noodling about with something, it's probably the wrong time to strap loads of tests around it. Now it's being attached to a real app and it's got some real...

Daniel (41:23.282)
Mm-hmm.

Dave (41:29.419)
constraints and things it needs to do. And now I'm hitting these sort of things. Yeah, I'm probably gonna have to put some sort of automated test harness around this. Yes.

Daniel (41:38.62)
Yeah, but that's then the correct time to do it. Like, because while you're still sketching, like putting a test scaffolding or on something that's still like being formed in your brain is like ridiculous.

Dave (41:48.775)
Yes, it's counterproductive. I think that will probably if I can predict a future shows content, then you'll I think there's probably going to be a story a few weeks out from now of me adding some sort of test harness around stuff there. Yeah. But.

Daniel (42:02.576)
Nice, very good.

Daniel (42:06.98)
I have some feedback to our last show by the way. To our, yeah. Because what I do these days is already, I slice up the show and find some interesting tidbits, slightly automated, but not fully, and they post those on Reels and YouTube shorts and TikTok. Which, hi, if you're watching this on TikTok, I don't know if this will land on TikTok. But then of course I go on TikTok with my own account.

Dave (42:09.905)
Oh.

Dave (42:27.993)
Mm-hmm.

Daniel (42:35.716)
and just like mindlessly scroll as I want to do, which is actually why I set myself a time restriction. My phone will actually like lock the TikTok app after an hour. And sometimes I actually run into that and then I'm like, yeah, okay, fine.

but I kind of scroll past and I see a post from both of us because it's like I switched the account back to my account and then of course I follow the Waiting for Review account and I see us both. And you were talking about killing your view models.

Dave (43:09.94)
Oh, yes.

Daniel (43:11.316)
And I don't know if you know this, but on TikTok, you'd never say killing. You never say killing. I think it's to escape a imagined or real censorship. I'm not 100% sure.

Dave (43:21.659)
No, did our account get dinged in some fashion?

Daniel (43:24.388)
No, but what sometimes happens, or like what people sometimes talk about is that accounts will get shadow banned. I don't know if that's actually happens or not, or if that only happens for large accounts and we are not yet large or whatever. Anyway, the correct term for TikTok is you unalived your view models, you unalived them. So please unalive your view models, also unalive your idols.

Dave (43:31.755)
Wow.

Dave (43:44.767)
Wow. Okay, okay, yeah, so.

Dave (43:54.512)
and unalive your previous system constraints. So I'm trying to bring it back to the server topic we started with.

Daniel (44:00.392)
Oh, yeah. So that's the one feedback, like the one thing that I actually had that was a throwback to a previous show and the other one, this is not turning into ATP, but I have follow up, is over the last few shows, we sometimes talked about the European DMA Digital Markets Act that heavily influenced iOS 17.4 that is coming out these days. And

Dave (44:21.252)
Mm-hmm. Yes.

Dave (44:25.917)
Yes.

Daniel (44:28.916)
One topic that we had was that PWAs, web apps on your home screen, would be thrown out, would no longer be allowed. And that has been reverted. Unalived, yes, PWAs have been unalived. And that has been reverted, so PWAs have been unalived. And so if you are in the European Union and update to 17.4, your web apps should be fine. I haven't tried it yet.

Dave (44:37.511)
I'm alive?

Yes.

Dave (44:48.338)
Okay.

Dave (44:57.597)
all good.

Daniel (44:58.196)
I think I'm still on 17.3, but yeah, should be fine.

Dave (45:04.439)
I'm glad, I'm glad because yeah, I use PDAs, people like PWAs even can't even say them. Yeah, but no, I use PWAs all the time. I've got Feedback Bulb, which is our friend Constantine's product, friend and listener of the show. Yeah. And so I use Feedback Bulb for collecting feedback from my apps.

Daniel (45:11.338)
Hehehe, pardon me.

Daniel (45:21.533)
Hmm.

Dave (45:30.655)
The weapons face of that is really lovely and he's set it all up so you can use it as a PWA. And so I have that on my home screen. I want to check my use. I know.

Daniel (45:40.028)
We need a better word for that. Like P W A. It's like too many syllables for a, for a acronym because of the stupid way that English speakers pronounce W and also like, I don't know, like can we just say puver?

Dave (45:46.469)
Yep.

Dave (45:51.144)
Yeah.

Dave (45:55.592)
for that yeah

Daniel (45:57.948)
Poir! Oh, poir is even better. Poir. My favorite abbreviation is, I think, the WSGI server, which is pronounced whiskey. So, poir. Yeah, that's just right in that direction. Fantastic.

Dave (46:07.644)
Yep.

Whiskey, okay. Yeah. Ploy.

But yeah, I use Poire's all the time and I would be very annoyed to not have that and to have to then go to my browser for that just because it's muscle memory now.

Daniel (46:24.596)
Yeah. And I mean, not to turn that into the Apple bashing podcast of the week, but yeah, I mean, especially because sometimes it's hard and frustrating to get things into the app store. Having that escape patch of poise is just really helpful.

Dave (46:39.979)
And it actually serves Apple to have that escape patch as well. It's like, yeah, well, you know, you've always got that route. I'm surprised. I was actually pretty surprised when they went that direction. It definitely felt like malicious compliance to me at that point. And I'm glad they're backpedaling, because I think that could have become a much bigger issue than it needed to be. And

Daniel (46:57.48)
Hmm.

Dave (47:08.987)
Yeah, I don't know. I'm not going to get into apple bashing right now, Daniel. I feel like I would like to see them making more positive decisions around all of this rather than trying to sort of thread the needle on not doing what they're being asked to do. I'd rather we weren't talking about backtracking on plaws. And yeah.

Daniel (47:24.976)
Yeah.

Daniel (47:31.708)
Yeah. And it's, it's kind of sad because Apple is doing so many cool things, like I know many people who work at Apple and, um, like friends and there's going to this, who are really awesome and amazing. And like, whenever I do some nice little Apple bashing, that's, I don't mean those people, like, I mean the people like super high up in charge, who decide, decide on these policies. And I mean, um, so of course, like I love people who are at Apple. So I don't.

Dave (47:59.795)
Yeah, same. Same. And I, and yeah, it ends up being that.

Daniel (48:00.916)
Don't feel attacked, please. So many talented people with the right priorities and who really want to make awesome stuff. And that delights people. And yeah, that's where we should go.

Dave (48:18.939)
Yeah, but Daniel on that note, I am going to have to get on with the rest of my day here on this side of the world.

Daniel (48:23.985)
Yeah.

Daniel (48:29.192)
I already read the outro, I don't know what you want me to do.

Dave (48:32.764)
Oh, go back, read it again. We've got to close this bit of the show. Come on. Chop chop.

Daniel (48:38.264)
Fantastic! Alright, I'm going back to my servers now. So, thanks for listening. Please rate us on iTunes and YouTube. Please send us emails at contact at wait Please write me, toots, at daniel at soci And where can people find you, Dave?

Dave (48:57.855)
people can toot at me at Dave at soci and you can find out all about my apps at lightbeamapps.com. And in fact, if you go to lightbeamapps.com slash govjpro, you'll be able to see a sneak peek of what I will eventually be working on with this video pipeline.

Daniel (49:08.616)
Fantastic.

Daniel (49:19.878)
It's pretty.

Dave (49:21.34)
It is very pretty.

Daniel (49:24.371)
See you soon Dave, bye!

Dave (49:26.303)
Take it easy, Daniel.

Daniel (49:27.88)
Byee