Jason (00:07) That was rad, that was rad. That's really cool. Coming up on this episode of Linux for Everyone, a new poll shows a surprising amount of Windows users are ready to abandon the USS Microsoft and jump over to Linux. The discovery of the week is back and it's all about de-bloating your browser. Joe (00:09) ⁓ Welcome. It's such a catchy song. Jason (00:34) Plus, I dive much deeper down the NAS rabbit hole with a really informative conversation with late night Linux founder and producer, Joe Resington. This is Linux for Everyone, episode 61 for January 19th, 2026. And it starts right now. Hi, this is Lorenzo and I'm listening to Linux for Everyone in Milan, Italy. Welcome home. Hey everybody, welcome back to Linux for Everyone and welcome home. Before we get going here, I just wanted to say thank you for, you know, for taking the time out of your day for devoting some of the most precious resource that we have, which is time, to listening to my rantings and ramblings and my conversations with other people in the community. I appreciate you and I'm grateful that you're here. Okay, let me run down just a little bit of housekeeping, some things I need to tell you about, and then we'll get into just ⁓ a touch of news that I thought was really cool and a discovery of the week and then my conversation with Joe. Okay, if you follow me on Mastodon, you probably know about my alter ego. It's called Seasons of Jason. And that's a website devoted to all of my writing about nature. about camping, hiking, backpacking, ⁓ very introspective and very adventurous. And I also have a podcast that's a companion for that website called The Seasons of Jason Podcast. And the first four episodes that are currently out right now that you can listen to are sort of an origin story about my transformation starting... ⁓ Well, starting back in the nineties, actually, it's about a series of flashpoints that kind of opened the door to me discovering nature and hiking and discovering a whole new side of myself. ⁓ it's about what happens when you start walking away from everything that used to define you and you kind of let curiosity be your compass. So I don't know, I would say if you, if you ever enjoyed any of my writing on Forbes or elsewhere and you're interested in hiking or interested in, getting outside more, think you'll really, really like it. They're short. I think the longest one is 19 minutes. And each episode has this nature soundscape in the background that I record myself. So go check it out at seasonsofjason.com or just, just type in seasons of Jason. into your ⁓ favorite podcast app and it should come up. If you want to support this show, the best way to do that right now is picking up a coffee mug or a shirt or a beanie over at L4E.store. That's L4E.store. There's only about four five items up there right now, but I think they're pretty cool. I try to keep the prices as low as possible while still making maybe three or four bucks per purchase. And I want to be, have some real talk with you. You know, a few people have asked like, well, Linux for everyone is back and the YouTube channel is back. So where's, where's the Patreon? Well, look, I want to show you some consistency first before I want to, I want to establish some trust between us again, before I start just asking for a donation or, you know, a Patreon support from you. That's, that's the honest truth. And the last little bit of housekeeping, if you want to get in touch with me for any reason under the sun, just send an email to letters at Linux for everyone.net. That's letters at Linux for everyone.net. And Hey, if you're up to it, I could use a fresh batch of those welcome home tags that you hear at the beginning of this show. You know, where somebody says, Hey, I'm Jason and I'm listening to Linux for everyone in Merced, California. And just, just. record that and send it my way and I'll try to get it in here. Okay. So you've probably noticed there's a bit of a momentum shift happening in the Linux world. More and more people are paying attention. More and more people are jumping over to Linux, especially since the end of support for Windows 10. Just a few days ago, as I'm recording this, Zorin OS announced that they have now reached 2 million downloads of Zorin OS 18. which I think launched the day after Windows 10 support ended. That's a significant number of downloads. And, you know, as usual, they say that the majority of those downloads come from Windows. The folks over at Bazite are just constantly talking about how they're serving up petabytes worth of ISO downloads. You can't escape people talking about switching from Windows to Bazite, switching from Windows to Cache OS. uh, switching from Windows to Zorin to Mint. Adding to all of this momentum is a new article from Tech Power Up showing their poll results from Q4 2025. They asked their readers if in light of Microsoft ending support for Windows 10 and what is, you know, noticeably arising dissatisfaction with Windows 11, if they would consider moving away from Windows altogether to alternative operating systems. Now they had 34,000 responses. I know that's not a huge number, but it's not nothing. Of those 34,000 responses, 55.4 % voted that they would consider moving away from Windows to Linux. And TechPower up writes that what's even more interesting is that only 4.5 % of those responses say that they're willing to move to Mac OS. And then there was a smaller, much smaller, 1.9 % who say they're willing to give up their PC as their main computing device altogether and switch to a game console. So of those three core responses, Tech Power Up interpreted this as 61.8 % of those people or nearly two thirds of those readers being disillusioned with Windows. And I just thought that was worth sharing. Okay. The discovery of the week is back and I'm really excited to tell you about this one. I'm not going to spend a ton of time on it because I think once you go to the website and just absorb what it's about, you'll, you'll grasp it super, super fast. It's, it's incredible. So Corbin Davenport is a tech writer, a software developer, and a podcaster who is also the news editor at How To Geek. And he has developed. something called Just the Browser. Just the Browser. It's over at JustTheBrowser.com. And this is, as I record this, maybe a matter of days old. What it does is it gets rid of almost all of the annoying stuff that irritates you about your browser. AI features, telemetry data reporting, sponsored content, shopping features, nags about being the default browser. It also prevents browsers like Edge from starting up with Windows and running in the background. And what's really cool about it is you don't have to install any extensions on your browser. You don't have to mess with any code. You don't have to tweak the Windows registry or, or tweak config files. And these overrides should not get reset when the browser updates. And that's, you know, obviously that's, that's been a huge complaint about, about browsers like Edge and about OS's like Windows is a fresh update just kind of, you know, resets all of your settings that you so carefully set up. Because what just the browser does is it uses group policies and not just toggles in your settings tab. So unless that setting itself gets completely removed from the browser, The stuff that just the browser strips out should stick. On Windows and Mac OS, just the browser supports Edge, Chrome, and Firefox. On Linux right now, only Firefox is supported, but Corbin has pledged to add a Google Chrome configuration in the future, as well as support for browsers that have been installed via Flatpak. Because I went to test this out and I did, you know, I launched a few... VMs on my Bazite box and I realized how many, ⁓ how many instances of Firefox are, you know, either pre-installed or by default installed via Flatpak, which is awesome, but probably takes a bit more work on, on Corbin's behalf to get, to get running. Also, I just wanted to give him a, a very, very encouraging pat on the back, not just for this, but for how he writes the instructions. They're so clear and precise. that anyone can do it. And essentially it just involves either, you know, launching terminal or launching a PowerShell window and copying and pasting. And I've used it on a Windows installation and I've used it on a Fedora installation. And wow, it just, it just works as advertised. mean, on Windows, I ran the command, I rebooted and all of a sudden Edge is not. running in the background at startup, all of the annoying stuff that you see when you open a new tab that feed all the news, that's all gone. It's very clean and it's ideally something that you want to run or suggest that other people run on new installations of Windows before they even fire up Edge because it also removes like that seven or eight page ⁓ first run. thing that you have to go through. And it's just, it's just a great little tool. So just the browser.com. I'll stop talking about it. Go check it out. Let's get on with the show. My guest today, you probably cannot listen to more than one or two Linux podcasts without hearing his name. It is Joe Resington, the founder and producer of the Late Night Linux Network, fellow musician and very, very knowledgeable Linux geek. Now this, this conversation we had is really interesting because it's sort of, it, it goes full circle. So a lot of you might not know that in 2019, Joe and I co-hosted a podcast called Choose Linux. And that was over at Jupiter Broadcasting. It was like a year. After I started my Linux journey and started covering, you know, all of my discoveries at Forbes, before Linux for everyone was a thing, Joe and I had Choose Linux. And the first episode that we did, episode number one, I talked about the elementary OS challenge that was happening ⁓ over at Forbes. And he talked about Open Media Vault, about starting your own NAS and using Open Media Vault. And so. And the funny thing is, I didn't even realize that, that connection until, until I started editing this episode of Linux for Everyone. it's just, anyway, it just really made me smile. Aside from going and checking out the entire Late Night Linux network, which I highly recommend you do, Joe, as I said, is a musician and he's got two albums on Bandcamp. I bought the one called Noise with Guitars. And it's fantastic. I hope maybe someday Joe will record more music or maybe we can do a collaboration of some kind because he's got a lot of really good stuff on there. You can check it out at joeresington.bandcamp.com and I'm actually going to play a little clip for you right now. Joe (13:29) ? Jason (13:58) So that was a little clip from a track called Old Order from Joe's album Noise with Guitars. It's it's just grungy. There's the great harmonies, really, really ⁓ impactful, like punchy drums, good guitar. It's just good rock and roll, man. I love it. So go check it out. But I invited Joe on because he has a lot of experience with building and managing a NAS and I really appreciated how the conversation unfolded because, much like we should be doing when someone asks us, Hey, I want to switch to Linux. What distribution should I use? Joe asks a lot of questions and explores like what my needs are, what my skill level is, what hardware I already have to work with and kind of builds a strategy around that. And it was just really, ⁓ a very insightful and educational conversation for me, and I hope that it'll be the same for you. So without any further ado, please enjoy my conversation with Joe. Welcome to Linux for Everyone for the first time. Not the first time that we have co-hosted a show, but... You know, and I may have asked you this back, way back in the Choose Linux days, but since this is the first time that you've been here, Joe (15:15) know. Jason (15:24) I am obliged to ask you your Linux origin story. Joe (15:29) I got into computers through music and recording music and I was pretty broke at the time. So it was windows and I got into modding windows to make it as lightweight as possible. And I got into this thing called N light that did that. And I managed to produce pretty lightweight windows, ISOs, and I got it all like really stripped down, but that meant stripping all the security stuff out and I thought, hmm, this probably isn't a very good idea to like do internet banking and stuff. And so then I decided to give this Linux thing a try and I tried Ubuntu and back then it was GNOME 2. And it's like, yeah, this is okay, but let's see, is there anything lighter than this? And then I found Xubuntu and XFCE and I thought, yeah, this is good. Is there anything lighter than this? And then I tried a bunch of other stuff and I was like, well, that other lighter stuff is kind of all right, but XFCE was pretty much the sweet spot for me. And yeah, so that was around the time of like, Zabuntu 710 maybe. And then 804 came out and that was the first one that I like daily drove. And yeah, I've just been using it ever since really. Jason (16:42) You've been rocking XFCE and Zabuntu the whole time, right? You haven't really deviated from that? Joe (16:47) Yeah, I mean I've tried everything under the sun, I'll- Jason (16:50) So you haven't been lured away by anything else. Joe (16:52) No, no, I've tried it all, but I just keep coming back to Zbuncu. That's what I'm looking at right now. Jason (16:57) Is it just, ⁓ I mean, is it just that it's, you know, light on resources or is there some other element to it that you really like? Joe (17:05) Well, the thing is, the first ever OS I used was Windows XP, and then I used seven, and then 10, and then 11. I've used various Windows versions over the years, but the first one was Windows XP. And so that is what a computer should be like as far as I'm concerned, in terms of the layout and everything. And XFCE makes it really easy to do that, but without any of the baggage that comes along with running Windows. Jason (17:33) gave it a good shot, it just wasn't bloated and pretty enough for me. Joe (17:40) Yeah. You see, always used to set windows to adjust for best performance. like, just, it looked like windows 95 or 2000 or whatever. And even when I was using windows seven, used to do the same thing. And so I've always just set it to be no animations, black background, just really bare bones. Cause I don't really care what it looks like as long as it works. Fair enough. If you like eye candy and all that, get it why you would want to use You know, plasma, one of the other ones. ⁓ Jason (18:11) I'm a plasma. Although I was recently really ⁓ very close to being tempted away by elementary OS again. because I love the skeuomorphic design, know, colorful and not just this flat, boring icon design that everything else is adopting or has adopted by now. yeah. Joe (18:33) XFCE too elementary, I mean, is too ⁓ opinionated for me. how we think you should use a computer and, you know, have at it sort of kind of take it or leave it almost. And so that's why I always say to people, give it a go because it's either going to really click with you or maybe it's not. And for the people it clicks with, it's amazing. Jason (18:58) So we're here to talk about my NAS rabbit hole that I have jumped into at first. Joe (19:07) Okay, well, I need to give you a disclaimer, right? The first thing I'm going to say is everybody, and I mean everybody in the Linux and open source world has an opinion on this. Everybody has an opinion on backups and storage and the way you should do it. And so I'm going to give you my opinions, you know, everyone's got theirs. And I'm not an expert as such, but I am surrounded by experts in my day-to-day working life. So I... run this family of podcasts, Late Night Lennox, and we have various shows and every single person who's a co-host on those shows works in the industry doing various stuff. You've had Graham on your show before, he does documentation for example. I do a show, Two and a Half Admins, where I'm the half admin and Jim and Alan, the guys there are professional sysadmins and they really know their stuff. And so I'm kind of surrounded by experts and that has formed my opinions on this and molded me. ⁓ The guys on Two and a Half Admins, have a very professional outlook, let's say. They look down on shucked drives, for example. That's where you get a USB drive and take it out of the enclosure and put that into a server or whatever computer. Jason (20:19) I would have 16 terabytes at my disposal ⁓ if I chose to take that route. Joe (20:24) Yeah, well they look down on that basically. But then on the other hand, I do the show Linux After Dark where we do these ridiculous challenges where, for example, we did the £20 Linux box challenge. that's £20 or about $30. And that was our budget to go and buy a used machine that could actually do something useful. And it turns out that you can actually buy a reasonable machine to use as a server, know, a home server for that kind of money. And We're all about kind of bodging it and making it as cheap as possible. So I've got kind of got two perspectives on this and I sit somewhere in the middle, I guess. But let me start by asking you a question. What exactly are you trying to achieve? I suppose maybe we should get into why you want to do this, right? Because it was inspired by that guy who lost his Apple account. I think he's got it back now, having created enough media exposure for himself. like that really scared you, didn't it? Jason (21:19) It did. It did. This is someone who jumped into the Apple ecosystem like 30 years ago, never left, and had just a library of memories wrapped up in Apple's ecosystem. 20, I think more than 20 years worth ⁓ of videos and ⁓ photos, not to mention all of the software that he paid for that he instantly couldn't use. As someone who still uses Mac OS a little bit, and primarily, you know, iPhone, I'm in the same boat. My mom recently, earlier this year, ran out storage with hers, so I just said, hey, just piggyback on mine. I'll up my, you know, my iCloud plan, and you can piggyback on mine, and you know, it'll all be quote unquote safe. No, it won't. I just don't believe that anymore. And then, you know, I read other horror stories and people, you know, that lost. data from OneDrive or from Google Drive or from Windows locally or and on top of all that. So I've got another channel that I'm sort of baking right now called Seasons of Jason, like a hiking outdoor adventure type of thing. Joe (22:30) We need more of your origin stories. I was skeptical about your origin stories, but then you told the one about getting injured in the military and then you kind of pulled the twist at the end and stuff. I was really waiting for the next episode, so you need to put that out soon. Jason (22:46) I know, I I know. I revived Linux for everyone and it's just been like tunnel vision for that. anyway, I have so much footage from hiking the entire Oregon Coast Trail and all the various other adventures that I've had this year. And I just am terrified of losing all that. And not to mention future Linux for everyone footage for the YouTube channel or just whatever comes around. And I have a Plex server, an existing Plex server that I've had for like three or four years. And so there's, there's probably eight or nine terabytes on that. And they're just, you know, they're just sitting on an external USB drive. I just want to, I just want to get all of this data back under my control and under my own roof. Not to mention, I also would love to have some things self-hosted like Image, absolutely like Image to take the place of something like iCloud, NextCloud. ⁓ you know, have some space for my, my family to store their, things if they want to, who knows, ⁓ pie hole just, you know, there's so many, it's, it opens the door wide open when you think about what you can do. But like you said at the beginning, everyone has an opinion. And I feel like the more I have researched, the more confused I become and the less clarity I have about like where to start. Joe (24:10) Yeah, it's tricky. It's very tricky. So I suppose the first question is like, how much data are we talking about total and how quickly do you expect that to grow? Jason (24:21) Well, okay. So I suppose I should start by just looking at my existing Plex library, right? That's 7, 8 terabytes. Joe (24:30) Right, but the Plex stuff, ⁓ how can we put this delicately? You're not gonna cry if you lose it, but it would be annoying, right? Jason (24:41) I would probably cry. A lot of what's on there is ⁓ lossless rips of a CD binder that's like four inches thick. So it's, you know, and a lot of Blu-ray rips and things like that. But yeah, no, I mean, that's not, that's all, that is all replaceable. Yeah. Yeah, that's all replaceable. Joe (24:50) Right. headache rather than irreplaceable. So that's good to know. Jason (25:06) You know, if I just keep on taking uploading GoPro footage and iPhone and Canon shots up to iCloud, I'm going to have to go up to their six terabyte, $30 a month plan. My best guess is I could probably easily throw 12 terabytes of data on something right now. Joe (25:30) So we've had various conversations on Mastodon and everything about this. And you threw out a bunch of ideas and you said like you're leaning towards splitting up your storage and your compute. And I'm going to say to you this, keep it simple, right? Keep your main NAS simple. Basically get yourself a desktop PC, essentially, an x86 desktop PC, depending on your budget. You can get something really cheap if it's pre Windows 11. So you're talking seventh gen Intel i5 or whatever, and older. If you want something that is solid, I mean, the reason that the eighth gen i5s and i7s are the cutoff is because that was a huge leap for Intel because AMD were really challenging them at that time. And so if you go for an eighth gen box, you're going to have a reasonable amount of compute for not even that much money. mean, I've got ⁓ an HP mini tower, which I picked up for like 80 pounds, I think. And that had space to put a couple of spinning drives in it, know, three and a half inch discs. So I would say that's the kind of minimum seventh eighth generation. I mean, you could get away with even a fifth or sixth really. Jason (26:48) Let me ask you, Joe, is it more about how many cores you have and whether or not there's, you know, certain like decoding features and, you know, things like that? Joe (27:00) Well yeah, you want something with QuickSync for a start, or the equivalent, but QuickSync has been around for such a long time now that I think even like maybe the third and fourth gen i5s have that, but you basically have to check before you buy anything. But like, I know you moved country not all that long ago and so you don't have a bunch of old hardware sitting around, but like, do you not have any sort of old gaming machine that you could repurpose for this? Jason (27:25) Well, here's, okay, so here's what's interesting. Early in 2025, I bought a $300 old, you know, PC gaming tower on eBay. I was, that was, that was because that was going to be one of the first big projects I did when I, when I came back to Linux for everyone. I'd been thinking about doing this for a year and I wanted to do like this, this gaming time machine where I would kind of install era locked versions of Ubuntu or Fedora and, you know, error appropriate kernel and proton and all that stuff and then kind of track the improvements of gaming across kernels and across distributions and all that stuff. It's got an i5-6600. It's got a Radeon RX 570, which, you know, is pretty decent for like 1080p gaming. It's got 16 gigabytes of RAM. And... Originally, I was all set to go and buy like a Ugreen NAS off the shelf or something. And now, you know, I'm kind of wondering would the right call just be, take this tower that has four SATA ports in it and an NVMe slot. Joe (28:39) ⁓ right, right. Jason (28:41) I didn't even, you know, they're completely unused right now. I just have a 500 gig SSD in there that I was using for distro testing. Joe (28:47) What I would do is take the graphics card out of there because you don't need it. Unless you're to do like very serious graphics stuff with it, you don't need a graphics card in it. If you consider what's acceptable for a Linux desktop use, right? In terms of performance, you take that desktop away and you start just running storage and a few containers for applications headless. And suddenly you can get away with so much less performance. And especially if it's only going to be, or it's primarily going to be you in, you know, your house or whatever, you don't really need to have a hugely powerful CPU. If you're going to have a bunch of users, like in a small office or something, then yeah, that's completely different. But if it's primarily you and like the odd person streaming a bit of Plex, that desktop machine of yours will probably be fine. And if it's got an NVMe drive, yeah, stick the OS on that. Put a couple of disks, you know, big. you know, 16, 18 plus terabyte disks in there. And that is a great starting point. Jason (29:52) Okay. Now how, let's just say, okay, let's just say I get, I fill up the drive base. I've got, get four drives. Right. Is that something that I should start with? And if so, what type of redundancy do I want? Do I want RAID 1? Do I want RAID 5? And then we have to probably start talking about operating systems and ZFS versus anything else. And this is where I just start unraveling because there's so many ways to approach it. Joe (30:20) Well, let me tell you what I've done. So I've got my main NAS, which has got two disks in it, in a ZFS mirror. And that is the source of truth for everything. I've got some old disks that I shucked that are also in there that I do various backups to, but I am not confident in those disks, put it that way. Although as an aside, something you're not going to want to hear. Jason (30:41) Okay. Joe (30:46) is that you have to treat, or at least I treat, every single storage device like it is about to die at any second. Jason (30:54) Does that mean always have a spare? Joe (30:56) Well, it just means always have backups. Jason (30:59) Yeah, okay. Joe (31:01) Raid is not a backup though. Like that is something that I didn't understand straight away. thought, right, well, if I have two drives in a mirror, that means I've got a backup, but that's not how it works. Raid and mirrors and stuff like that, that is about uptime. So when, not if, when one of those drives dies, my NAS is going to be okay for me to quickly buy another one, put it in, re-silver, know, copy all the stuff back onto it and remain up the whole time. And so anyway, I've got my NAS, is the source of truth. then somewhere else I've got my onsite backup. so once a day, everything gets copied onto that onsite backup, which has got another pair of disks in a mirror. And then once a day, that all gets sent to my offsite backup, which is like actually four or five hours away. And this was very expensive to set up. Jason (31:54) Yeah, I can, I'm kind of doing the math in my head as you're talking about. I'm understanding the need for one of those 30, $40 cheapo solutions because my God. Joe (32:03) because the thing is. The machines that you put them in don't need to be that powerful, especially for an offsite backup. if it's only doing storage, it really doesn't need to be powerful. I've got a J1800 machine, is like a passively called Celeron, think. And it was made to be one of those media PCs. And it is super underpowered, but all it does is wake up once a day, receive a bunch of data, and that's it. I'm sorry. Jason (32:37) Could a ⁓ cheap raspberry pie in an external drive do that job? Joe (32:41) I mean, it could, it could. I've got actually next door to me, I've got a Wyze 5070, which is a thin client, a Dell thin client connected to a USB drive. And that gets some of my essential data. It's only a fairly small drive. And you know, that's just... because I had it kicking around and it uses very little power. So I just put it next door. They were fine with me doing that. if something catastrophic happened in my house, then I could kind of just go and ring their doorbell and grab my hard drive and then be back up and running. I'm kind of going a bit overkill there. But do you know about the three, two, one rule of backups? Jason (33:24) Let me think. ⁓ Three total backup, three total locations? Three copies. Joe (33:29) Well, three copies. Two types. you know, maybe ZFS and... Well, yeah, like traditionally it was like hard drives and then burn CDs maybe somewhere like that. for me, it's more like I also have a USB drive that's just EXT4. So that kind of counts as that. And then the one in the 321 is one off-site. So if you want to do it properly. But I understand that this is very expensive and very intimidating. And so... Jason (33:36) medium. Joe (34:00) it can just make you think, well, this is just too much, I'm just going to give up. But it's better to have something than nothing. Jason (34:07) same time, like don't take half measures, right? If you're going to all this effort to protect your data and to have ownership of it, you don't want that one failure point to be the end of everything. Joe (34:18) Yeah. The thing that I would suggest is have your big disks containing everything in, your main backup machine, your main NAS, right? Which it serves as a kind of backup, but also just you've kind of working storage, let's say for all of the clients on your network. And then decide like, how much of this do I actually need to properly back up somewhere else? Like that, that Plex stuff. Do I really need that backed up in three places, including offsite? Do you really need to back that up more than just on your working main NAS? Probably not. I mean, it would be nice to, but you know, this is where ZFS comes in. Now, some people say, ⁓ ButterFS is the way to do it or, you know, traditional RAID or whatever, but all I can tell you is my opinion. And like I said, everyone's got one and I have... learnt about the various options out there. And I'm telling you, ZFS is the way to go for so many reasons. But one of the reasons is you can split it up into datasets or you should split everything up into datasets, which are effectively just like directories or folders as far as the operating system's concerned. You if you installed ZFS on a desktop machine and you had a bunch of datasets, you one for movies, one for TV, one for, ⁓ you know, GoPro footage, one for photos. It would just show up as just folders as far as the operating system is concerned. But you can then choose which of those datasets to replicate or copy elsewhere. So, you you could basically say, well, let's exclude the TV and movies because that's my Blu-rays that I've ripped and let's just send the GoPro stuff and the photos and, you know, my documents, whatever my taxes and all that. so that's what I do for my next door backup. I don't back up any of my ripped Blu-rays and DVDs and everything because I could buy them again. It'd be expensive and inconvenient, but I could buy them again. So that's like one of the reasons to choose ZFS. But one of the huge attractions to me to ZFS, even if you didn't use any of the features like, I've talked about the data sets and everything, even if you just created one ZFS pool, even on just a couple of disks and then just chucked a load of stuff on there, the most attractive feature to me is that You have regular scrubs. Now what a scrub is, is ZFS goes through and checks every block of all the data on the drives and checks that everything's fine. I mean, you must have old photos and old MP3s where you're listening to it and it just, you listen to an MP3 and it's just weird glitches and stuff like that. ⁓ Jason (37:00) do have ⁓ Joe (37:03) Where only half of it loads and then the rest of it's like green or grey. So that's from Bitrot basically. ZFS scans for that. Now if you install it on Ubuntu, you get that scan by default once a month. You can do it whenever you want. could schedule it for every day if you really wanted to, but once a month is kind of sensible. And so once a month it checks all the data. And if you've got a mirror or, you know, at least two, two disks, so two copies of everything. Jason (37:06) I have also seen those, yeah. Joe (37:31) If one of them gets corrupted or bit run, then it will attempt and usually succeed in correcting that, basically replacing it with the good copy that you've got on the other day. And so every month it's checking your data. so, know, like as long as you set up some monitoring or just manually remember every second Sunday or, you know, every second Monday after it's done the whole thing to just run this one command or check in the GUI, which we'll get into. Jason (37:42) ⁓ okay. Joe (38:00) you know, for me, it's just pseudo Z pool status. And it just says, right, scan completed in eight hours and no errors were found. And when I had a single disc that was USB, I used to get errors quite frequently actually. And it kind of compounded and got to the point where I just had to wipe it and start again. But at least I knew, you know, and for me, that was what got me into Z of S in the first place. The fact that those old photos, I mean, I've got photos of my old cat who is no longer with us, you know, hasn't been with us for a long, time. And that was before I got into ZFS and some of them are corrupt and I will never see those photos again. I'll just never see them again. I've got no other copies of it. And like, you know, there's only like a quarter of the image that you can see and then it's just all glitched out. see, Jason (38:48) I didn't even consider that at all. Joe (38:50) Yeah. Now, you know, the various other file systems can offer similar things, but I am of the firm opinion that ZFS is the way to go for storage. It is the last word in storage as far as I'm concerned. Jason (39:02) Is there any downside that you, I mean, if you had to pick one? ⁓ Joe (39:07) Well, if you want it to get like real like Linux guy about it, well, the license is incompatible with the GPL. But like, it's, it's open source. mean, technically it's, some people would argue it's incompatible, years ago Ubuntu started shipping it effectively. It's just one command or two commands to get it installed and it just works perfectly. You know, if you're going to be pragmatic about it, then I wouldn't worry about that. But that's the one argument that the Butter FS sellers will tell you, well that's properly Jason (39:16) Like, where- Joe (39:37) compatible and therefore more integrated and everything. But if you run an Ubuntu LTS or something like TrueNAS or whatever, you're not going to have a problem with ZFS. It's going to be fine. But that's the only real downside as far as I'm concerned. And by default, it uses a fair bit of RAM as well. Jason (39:55) I'm glad you brought that up because since we're still talking hardware, have read, you know, split opinions all over the place in all kinds of, you know, all kinds of different forums and YouTube channels and everyone has an opinion. ⁓ ECC memory or not, because that is quite a contentious topic apparently. And that stands for what error, error correcting code. Joe (40:23) I think so, yeah. And it's, it's a really nice thing to have, but the bottom line is that only really server grade hardware has support for it. And if you're going to do this on a budget, it's just not going to be an option. And it's not a problem as far. I've never had a problem with it. And like, you know, like I said about the scrubs, like ECC could potentially stop BitRot from happening and corruption, but the scrubs there to catch that and hopefully fix that anyway. And if you've got proper backups, it's not a huge issue. Put it this way, I've never, I don't think I've ever owned anything with ECC, because I tend to do things on the cheap. So yeah, I wouldn't worry about it. Jason (41:03) Okay. And you haven't had any catastrophic things happen with your setup, right? That's good. I am sensing this attitude of, you should expect failure, not hope that you won't have a failure. Like just expect it. Joe (41:19) Well, entropy, That's why you need to have sufficient back or you should want to have, let's say, sufficient backups. something you mentioned to me, I think when we were emailing back and forth about this, is you cold storage, right? And cold storage is just a terrible idea because say you copy a bunch of stuff to a hard drive and stick it in the drawer. How do you know that a year later when you've had this catastrophic failure, It's going to work. You have no way of knowing that. It could have seized up. could have been corrupted. You've just got no idea. Whereas if it is in a machine, you know, even if it's plugged in via USB or if it's, know, properly SATA or SAS or whatever, if it's spinning all the time, it's hot storage, it's being scrubbed every month, you know that that is going to be fine because, you know, the longest you can go is a month. And then you're going to find out that there's problems with it. Whereas stick it in a drawer for six months or a year, you've got no idea. And especially with SSDs, when you get into that, mean, SSDs lose their charge slowly, very, very slowly, but surely and get corrupt over time. And to the point where if you leave an SSD long enough, just, the data on it is effectively gone. yeah, I mean, it's, I think that it's a bit over. Jason (42:36) that right? I never knew that. Joe (42:40) blown the issue with that, but it absolutely can happen. if it's in a machine and being checked every month, I don't want to go on too much about this, but that is the biggest attraction to ZFS to me. Jason (42:55) Okay. All right. So, so far we have, ⁓ I don't need ECC memory. probably should use a ZFS setup and I should definitely have at least two drives. Do you have an opinion on drive brands and capacities? ⁓ Joe (43:16) Yes, yes, do. So capacity wise, buy the biggest that you can afford. asterisk on that. Don't buy like the, you know, even if you could afford the very, very biggest, they're not going to be the best value for money. Like you have to look at the price per terabyte, right? And last time I looked, it was a while ago that it was around the kind of 16, 18, 20 ish was the best bang for buck there. I don't know what it is now, but I would say, you know, if you think you only need 10, well, Why would you buy a 10 in 2025, 2026? Like it doesn't make any sense. buy the one that is the most bang for buck that you can afford. I remember, right? My first PC, and this is going back, this must have been, I don't know, 2002, three-ish, something like that. Jason (44:08) before podcasts even existed. Joe (44:10) Yeah, I was late to this man, but my first PC had an 80 gigabyte hard drive and I thought I will never fill this. I'll never fill that 80 gigabytes. And then it kind of started to get full. thought, right, I'm going to go and buy an external drive. So I went to Curry's, which is, know, a big electronics retailer and bought an 80 gigabyte external drive. So I'm never going to fill that. Obviously filled that in a few months. And so, you know, you're always going to need more storage than you think. Jason (44:38) I feel like if you have the space, you're going to fill it up. I never thought, when I bought, I basically have two eight terabyte external drives mirrored with my Plex server right now. And I never thought I would fill up eight terabytes. It's getting there. Joe (44:55) Yeah, exactly. Now, when it comes to brands, Seagate is the only brand that I would buy. Not because they're better than any of the other brands, you know, the other two brands that you can get. It's simply because of fakes. Now, do you know about smart data? Fakes. Fake drives. Well, not fake drives, drives sold as new that are actually refurbished or secondhand effectively. So do you know what smart data is? Jason (45:14) Because of what? I do. Joe (45:26) Yeah, so that's like the data that is, you know, on the drive at a low level kind of firmware level that you can query with various software and it'll tell you, like, you know, how many hours it's been spinning, how many reboots it's had, that sort of thing. Sure. Well, it turns out that that is pretty much trivial to overwrite and fake. you just, you know, it's like on an old car. Jason (45:49) You can overwrite even GPU firmware. So, sure. Joe (45:53) Yeah, exactly. Yeah, it's like clocking a car as they used to call it, you know, like it gets to a hundred thousand miles or whatever, you just take that a hundred off and it's, whatever. so because of everyone's favorite, Chia cryptocurrency, which was all about mining space on drives, some rubbish anyway, loads of people bought a bunch of drives and put them in servers and mined this Chia and then, you know, the bottom fell out of that market. And so they all found their way into the gray market. And now here we are, they're showing up on like proper retailers like Amazon and stuff. And, you know, they've had their smart data reset. so you've got no idea. Jason (46:35) What are they presented on these retail sites? Are they just presented as new? Joe (46:39) Yeah, exactly. just sold out. Yeah, exactly. Like, you know, fulfilled by Amazon, but sold by third party seller. Jason (46:48) I mean, I'll be honest, Joe, that's all I look at. Joe (46:50) Yeah. And so they show up and they look brand new because they've been put in, you know, server. Yeah. Yeah. They do all of that and they look genuinely brand new and you know, they perform fine, but you know, it turns out that they've been on for, you know, two, three, four years, some of them. so anyway, Seagate has a thing called Farm Data. Now I can't remember what farm stands for. It's basically a acronym, but anyway. Jason (46:56) New like cellophane wrap and everything. Joe (47:19) This farm data, at least last time I looked, cannot be overwritten. And so you get one of these drives, you run the smart data on it and it looks brand new. Then you run the farm data on it. Hang on. It's been for this many hours, which works out to be like two, three years or whatever. And I got bitten by that man. I bought some off of Amazon and it was about, I don't know, 40 days later. And I had a battle with Amazon to take it back. And I said, these are fraudulent. don't make me get the police and trading standards involved. And in the end, they managed to take them back. And then I got some from a proper seller. And the first thing I did was run the farm data and sure enough, all was well. So for that reason only, right now Seagate is the only drives that I would buy. Jason (48:04) ⁓ We're talking about just enterprise grade Seagate, right? consumer grade. Iron Wolf or... Joe (48:10) Not enterprise grade, but like NAS grade. Jason (48:17) Now, okay, okay, what is the difference? Joe (48:19) Well, build quality and, you know, warranties and stuff like that. But one thing to look out for is SMR, shingled magnetic recording, without going into too much detail. You know how shingles on a roof kind of overlap each other. One way to save space on drive platters is to, they invented this thing, shingled magnetic recording. And what it means is that in order to rewrite some data, they have to like move it all around. loads and the firmware has to do a lot of work for that. And it means that the performance is really rubbish. It's great for, well, it's useful for, let's say, archiving loads of data. So if it's only going to be written once, then it's just one continuous stream and you can do it in the shingled way and it's fine. But if you're to be rewriting and moving stuff around, then it's just a terrible idea. that is one thing to look out for. Make sure any drives you buy are not SMR. Jason (49:17) Didn't Western Digital get into some trouble for this? Joe (49:20) Yes. So Western Digital, some of their NAS grade drives were SMR. They previously weren't, and then they ⁓ just started shipping SMR drives. Jason (49:29) quietly like without even announcing a change. my gosh. Joe (49:33) And so you need to read the space. Why do you think the likes of Apple are trillion dollar companies, Jason? Because they make this so easy. You just pay them 40 bucks a month for iCloud and they take care of all of this for you. I mean, they stick it in AWS, which they don't like to talk about. yeah, like, you know, same with Google or Dropbox or whatever. Like taking care of your own storage is Jason (49:36) It's just a minefield. Joe (50:02) It's difficult, man. I do it because I enjoy it and I talk about it on shows and everything. And it is ultimately very satisfying, but normal people don't do this for many reasons. think about the expense of even just a basic 321. So you've got two drives in one machine, your main machine, two drives in your onsite backup and two drives in your offsite backup and maybe one USB drive. The cost of all of that plus the computers to actually plug it into and the RAM, which we all know is incredibly expensive, plus the electricity to run it versus 40 bucks, 50 bucks, a hundred bucks a month. Jason (50:39) Yo Over a lifetime, you're paying considerably less, but that initial ⁓ expense is daunting. Joe (50:55) Yeah, but your drives are not gonna last a lifetime. Jason (50:58) Well, then you have to just supplement that with the enjoyment and the satisfaction that you will have of, you know, and I like to tinker, you like to tinker. So, you know, the satisfaction of tinkering with things and learning new things and having that ownership of your data. I know it's not for everybody, but I think there is a ⁓ number of people who are exploring it. Joe (51:21) yeah. So take for example, image, right? So I recently set that up and it's been great. The machine learning stuff is not as good as Google, obviously, because it's basically a Google Photos replacement. Jason (51:34) Machine learning will just go like, it'll identify faces and find people, you know, this is me, this is, this is Judy, this is. Joe (51:37) Yeah. ⁓ Yeah, exactly. so when it does its first, you import all your photos and it does its scan and everything and it comes up with a bunch of faces and then, you know, sometimes it'd show me twice as in, you know, it hadn't recognized so I had to teach it. Okay, well, that's me as well. And so you just add them. But, know, if I search for watch, for example, because I collect Casio watches and I take loads of photos of them and, you know, most of what comes up is a watch, but then it gets a bit loose with it. And, you know, it's not as good as Google Photos, but ⁓ the reason that I switched to it was because Google photos used to be free, right? It used to be completely free. And then they said, well, it's going to be free for lower resolution. thought, well, that's fine. Like whatever they can downscale by photos. And then it was like, well, no, now we're going to start charging. Well, we're going to start counting it towards your 10 gigabytes, whatever for the free tier. And so I thought, well, okay, when it got full, I just created another account. Jason (52:25) That's how starts. Joe (52:41) And then that got full. created another account. And I said, I can't keep doing this. And now one of my account, my latest account is full. And every time I opened the app, full screen pop up on it, you're not being backed up. It's payers money to do this. And I have to dismiss it every single time. And whereas I go to image and it's just like, there's my photos. ⁓ I recorded a big long 4K video on my phone. I uploaded to it. No problem. I don't have to worry about anything. It's just right there. So yeah, it's very, very satisfying to self host your own services, definitely. Jason (53:15) And this may be a bit of a tangent, but since we're talking about image, it relatively straightforward to set it up for remote access too? Like just viewing it on your app up in the mountains or sending, you know, if I'm on vacation, I want to send stuff back to my NAS at home that I've, photos I've taken with my phone. Is that relatively doable? Joe (53:37) It's definitely doable, but it's not going to be in image itself. If you want to access your NAS remotely, you're going to need a VPN. Well, I mean, you could put it on the internet, but that would be stupid. Don't do that. Like directly on the internet. really don't. could, know, forward ports and whatever. Just don't do that. So you're going to need a VPN to do it, right? Jason (53:59) Just for clarification, are we talking about like a VPN you can have in your router or something like tail scale? Joe (54:07) Well, Tailscale is what I was going to say. like people think VPN and they think getting round geo blocks. And you know, that's not what we're talking about. We're talking a proper VPN here. Not, you know, not the kind of thing that sponsors YouTube videos. Although I don't know, Tailscale does sponsor some, but anyway, this is, this is something that you're to have to install on the devices that you want to connect to each other, basically. So you would install it either on a machine, on your network or just on the NAS itself. And so you install that and then you install it on your laptop and your phone. Say you've got your Plex server and you've got a laptop connected to your home Wi-Fi. You can kind of talk to the other devices on your network with a 192 address, right? Yeah. Tailscale just gives you a 100 address, which as long as everything is connected to your tail net, as they call it, your network, you can just talk to everything via this 100 address as if it's on the same LAN, effectively. Jason (55:06) soul. Joe (55:07) Say you've got it, tell us go running on your, they should be paying me for this, but say you've got it running on your now. Well, maybe anyway. So you've it running on your NAS. ⁓ and then you've got it running on your laptop and you're out in the wild somewhere. And you know, either literally in a field somewhere or Airbnb, whatever. Then yeah, you can just connect via this 100 dot IP address to your NAS as if it was on the local network. Obviously it's going to be way slower, but you can still. know, copy stuff on and off, you know, as via Samba or NFS or SSHFS, whatever. Yes. So it's free for like a small number of users. Yeah. It's, plenty for basic use like that. Yeah. ⁓ Jason (55:42) Is tail scale free? So their revenue comes from businesses, enterprise, et cetera. Joe (55:59) Yeah, exactly. I should know this. Yeah, it's 100 devices and three users. Jason (56:05) That's fine. Yeah, that's genera- I think that's generous. Joe (56:09) If you're a home user, hundred devices, you're never going to fill that. But then if you're even a relatively small company, you know, you could easily have, you know, say a hundred employees with two or three devices each, you know, phone, a tablet and a laptop. And it's built on top of WireGuard as well, which means that it's, you know, you can trust it. Cause I mean, I tried to set up WireGuard once and I just realized I was too thick. And then I tried tailscan and it's like, okay, this is really easy. So. Jason (56:37) That sounds like me trying to set up something in Docker versus just going, you know what, I'm going to install Cloud Run on a VPS and then just hit click on two installations of Ghost and I'm good to go. Joe (56:49) Yeah. Well, that's the thing with image. They recommend Docker and it took me a bit to get my head around that because the documentation assumed a lot of knowledge is the bottom line. But which I suppose brings us to like what, what are you going to run on this Nets? Right. So you either use something like open media vault or TrueNAS or you'd like raw dog it as it were, you just stick a Buntu server on there and install it all via the command line. And the question is how much do you want to learn versus how much do want it to just work? Jason (57:26) Huh. 80 % working, 20 % learning. I mean, I would rather, I personally would rather have something that is developed explicitly for a NAS or. Joe (57:31) Alright. Yeah, probably True Now Scale then, I would say. have to change the name again. They always change the... Jason (57:46) I've heard True NAS Scale, I've heard True NAS Community Edition. I don't know what's what, but I do hear True NAS coming up a lot. Joe (57:54) Basically the one that's based on Debian and supports Docker is probably the one that you want. And it's all got a web interface. I mean, I've never actually used it, but I've investigated it. And so you install it on the machine and then you can just open a browser on another machine on the network and then just configure it all via just clicking around. There's still going to be learning involved. I don't know why. I just wanted to learn it like... properly in quotes. And so I just did the Ubuntu server thing and just learned, you know, literally editing config files and stuff like that. you know, making it work and making mistakes and learning and everything. Cause I mean, returning to ZFS, one thing that is really attractive about it is snapshots, right? So you can set it up to do snapshots as often as you like. mean, I do it once an hour. And so say you set it up. and you stick all your data on it and you set it up to do snapshots every hour. then something goes catastrophically wrong. you have a bit of, I don't know, software that's like got a huge bug in it and it deletes your entire folder. Well, it's not a problem with ZFS. If you've got snapshots, you just roll back to the last snapshot from an hour ago. No problem. It's just a few commands. I mean, that's probably a few clicks in true as I've never tried, but Yeah, it's like super easy and Jason (59:20) What kind of impact does that have on your storage space? Joe (59:23) negligible because ⁓ although it takes a snapshot every hour, only uses how can I explain this? The only if things have changed or been added, does that make a difference to the storage use. So say you've got a gigabytes worth used and you make, and then you add a hundred gigabytes, a hundred megabytes to it. Your next snapshot is going to be a hundred megabytes more than if you had 50 megabytes the next hour, it's going to be 50 megabytes more. And so it's It's not like some snapshotting. Well, I signal, for example, takes a backup. Like I've just been having musical phones because my phone broke and Google replaced it and blah, blah. And so I had to go to an old phone and use my signal backup. so I did a signal backup and then people sent me more messages. I was like, I need to do another backup. And each one was like nine gigabytes because it's like encrypted. And so every single one was nine gigabytes. Whereas if it was using this NFS style, would only be like a couple of kilobytes more because it was a few messages. yeah, you don't have to worry about using a bunch of space. Jason (1:00:30) Okay, that's good. So my original plan, ⁓ which I think I've been talked out of now, but my original plan was I've got this, this probably is going to sound, for people who have experience in this space, it's going to sound ludicrous, but I have this framework ⁓ main board and this Cooler Master case that they make for the main boards. And you just kind of have this nice, small, compact ⁓ standalone, I don't know what to call it, a desktop. Joe (1:00:49) Mm. Jason (1:01:00) It's essentially, you know, it's going to be an Intel main board with everything on it. Housed in this nice little slim cooler master case. And you just plug it in. And, you know, I was thinking I'll just plug it in. I'll buy like an external drive bay enclosure and stick some drives in there and I'll have my NAS on there and I'll do all my self-hosting. I'll just do everything on this big powerful or small powerful box. ⁓ And since. Joe (1:01:24) Yeah. Jason (1:01:28) Since then, I've learned that's probably a terrible idea, but is it... is it a terrible idea? Joe (1:01:39) Terrible is a relative word. ⁓ Jason (1:01:41) You have experience with Docker, you know, if you're taking that kind of approach, isn't there a lot less risk involved having everything containerized or... Joe (1:01:51) No, that's not the risk. No, no, it's just physically having drives connected internally, inside a box, connected via SATA, properly screwed in and secure versus even a nice enclosure is still going to be an external cable, which, okay, you've not got kids running around and stuff like that, but it's just, I don't know, it's just a recipe for, not disaster, but like just problems. mean, I have run... various USB external drives. And yeah, some of them have been fine, but some of them have caused me all sorts of problems. You know, you only have to kind of look at it funny. And USB is just a bit of a rubbish protocol generally, you know, compared to SATA, which like, you know, USB is universal. You can connect your webcam, you know, your mouse, your audio device, whatever. Whereas SATA is only for storage. If USB is your only option, Like if you're on a super low budget and you've got a machine, you know, like a NARC or an old laptop or whatever, then sure, go for it. But if you've got the option of this, you know, relatively old gaming PC or, you know, maybe even going and buying another cheap one, then I would do that. Definitely like to have actual internal drives. It can't hurt to have one or two USB drives as well. But for your source of truth, I would highly recommend just doing it properly internal SATA drives. Jason (1:03:22) Okay, that's helpful. So now what I'm thinking is I use my old 2015, 2016 era gaming PC as my NAS, and then maybe set up that framework main board system as a dedicated home server for everything else. Joe (1:03:33) Yep. No, I would... well, maybe. Jason (1:03:49) See, that's where I'm getting hung up is I've got the horsepower and I've got the storage to do it all in one box, but am I better served having maybe image and Plex on a separate device as well as anything else that I want to self-host or... Joe (1:04:10) I wouldn't do it that way. I mean, there's nothing stopping you doing it that way. But think about it like this, right? Presumably you've only got gigabit networking in your house, And so that's going to be a bottleneck. If you have got machines talking to each other over the gigabit network, which you are also using to, you know, stream Netflix and YouTube and send emails and all the rest of it, use day to day. Like it's going to be so much slower than having it Jason (1:04:19) Yeah, yeah, yeah, that's. Joe (1:04:40) inside one box. And so if you've got, this gaming PC that's got the disks in it, and it's also running your Docker containers and your compute essentially, then it can talk to your storage, like, just as quickly as possible. There's no latency there because it's all inside one box. Whereas if you're having to go out to the network, like you've got to go to the NIC first and then to the switch, then to the root, you know, You're basically adding latency and throughput issues as well, because you've only can max out a gigabit per second. And so I would recommend that you stick the applications on the same box as the storage. Like there's nothing stopping you having various clients talking to that storage, you know, and if you find that you run out of RAM on that machine or compute, you could add a second box to it and talk to that storage as well. it was something, you know, something like Plex where it doesn't really matter because it just has to stream like big chunks of big files. Whereas something like image where there's a lot of small metadata and stuff to talk to and, know, to, to compute. That's where really you want to have it as close to the storage as possible. Now, if you had like 10 gigabit networking in your house, it might be different, but if you've got this desktop that's not doing much, you could stick a couple of USB drives. connected to that and have that be your onsite backup. And then you've at least got your main source of truth and your onsite sorted. And then, then you think about your offsite, like at some point down the, down the line. I mean, it's that, what, that framework, is it what the 11th gen Intel one? Jason (1:06:27) Well, it's the Intel Ultra 7155H. mean, it's pretty new. It's 2024. I mean, it's a beast and know, 16 gigs of actually I've got 32 gigs of DDR5. Joe (1:06:33) Alright, so that's new. Yeah. Wow. What you should do is take the DDR5 out, sell it and then buy thousand hard drives with the money. Jason (1:06:52) No doubt, no doubt. was when I was talking to Mike, Mike Kelly about the computer upcycle project, you know, he kept saying I've got just I've got buckets of RAM separated by capacity and all this. I'm like, dude, you better never advertise where you live. Joe (1:07:06) Yeah, exactly. Yeah. So you've got this really powerful machine, but it's only got USB, which is not ideal really. Yeah. It's funny. I've got something similar, but like just imagine that, but like the jankiest version you can imagine. That's what I'm talking to you on now. Right. So one day on Facebook marketplace, I saw a Lenovo think, no, not idea center. Do know what that is? It's an all-in-one like an iMac. Yeah. Yeah. With a broken screen. and it was a 10th gen i5 and I thought I'm having that. And so I went and got it. No power, no cables at all with it. But thankfully it had just, you know, the Lenovo rectangular laptop connector, power connector. So I had my power connector from my X270, which wasn't quite up to the wattage, but I wasn't going to run a screen with it. Anyway, I ripped the guts out of it. I got the back plate from it. and got my grinder on it and cut round and then just screwed it all back together again. And ⁓ yeah, so that's now my recording machine. It's so janky and it looks terrible, but it's pretty much silent and powerful enough to do this kind of thing. ⁓ So yeah, I understand that urge to reuse the hardware that you've got, definitely. Jason (1:08:25) You know, was looking at two, initially looking at two versions of Ugreen boxes and one, I don't remember the model numbers, but one had an ARM chip, the ROK chip, and another had like a weird five core Intel, maybe it was an N100 or a weird five core Intel setup. And I started, I started reading about how, you know, you can install TrueNAS scale on both, but you won't have access to the same types of, ⁓ Joe (1:08:48) Mm. Jason (1:08:54) maybe functionality or apps. Joe (1:08:57) Well, the thing is like most server stuff that's open source is not going to be a problem. Really, it's just ARM has got this weird jankiness to it that I can't explain with data and actual evidence. even when I tried my M1 MacBook Air running Asahi, I think it was Asahi Ubuntu even, right? It just was just a bit weird and janky for ZFS stuff and just server stuff generally. ⁓ I can't put my finger on what it is exactly, but I would just avoid ARM really for this. Especially for, you know, Raspberry Pis and consumer level hardware like that. And cheap stuff like that, Ugreen. Then, you know, if you've got like these proper, you know, servers that have got like ARM server ready certification and everything. Yeah, they're going to be fine. But just you kind of consumer level ARM stuff. I've just had bad experiences with weird jankiness. that I've just not had even with really low end x86. Jason (1:10:01) It sounds like you're definitely a proponent for the DIY approach for building your own. But are there any off the shelf solutions that you would recommend? Like I've heard some people throw ⁓ names like 45 Drives into the mix or ⁓ even Synology. ⁓ Joe (1:10:21) Synology definitely not. They have just done rock pulls basically. Yeah, Synology I wouldn't recommend. I've never used one but from what I've heard. Jason (1:10:30) That was the first brand that I was familiar with ⁓ years ago when I heard the NAS term for the first time. It was always like NAS and Synology and... Joe (1:10:44) QNAP is the other big name there. I personally am all about the DIY. In terms of the pre-built ones, I think the only real reason to use those is if you want it to look nice and be small. Yeah. I mean, you can get a hot swappable case for a PC and put a regular motherboard in it. Jason (1:10:46) Okay. And be hot swappable. Yeah. That's yeah. Joe (1:11:10) You know, where the, if you've got like an older school style case with the five and a quarter base for your optical drives, you can get a cage that's got hot swappable SATA drives in that. But you know, on a kind of home level that we're talking here, I have never needed to spend that money. You know, you can get such cheap old hardware that's got SATA ports in it. Like why bother spending that extra money? Like You know, my friend Gary, he had to put an offsite backup at his parents' house and he sat underneath the router. And so he got an AU Star. I don't know how you spell it. It's like a Chinese brand. And it's like super low profile. It's about the size of two drives and that's it. And so, you know, when questioned about it, or that's the internet, you know, these two books, that's what he told his mom, like, that's the internet. Jason (1:12:13) He got that from the IT crowd. Joe (1:12:15) Well, you know, the thing is, like, if he stuck, like, ⁓ you know, a tower PC in there, it's obviously a computer. what's it using loads of power and blah, blah. And so he kind of snuck this one in there. And like, that makes sense if form factor and size and aesthetics are important to you. But if it's going to sit in the corner somewhere, it doesn't matter. Jason (1:12:39) I think what would be more important to most people perhaps listening to this would be the power consumption. Joe (1:12:46) Yeah. So yeah, so that's a big thing for Arm, right? Is that it doesn't use much juice. Right. if you've got even a relatively modern Intel CPU, or I don't have much experience with AMD, but I assume it's the same. They have these like sleep states that they go into when they're idling and it really can get quite low. And, know, my Wyze 5070 that I have next door, I have seen that plugged into like a little cheap watt meter. I've seen it idle at one watt. Jason (1:13:15) the entire system? Joe (1:13:17) Yeah, like just the actual box. don't think I had a USB drive connected to it at the time, but I couldn't. This was just when I was initially testing. Jason (1:13:25) has to mean that your drives are also going into sleep states, right? Joe (1:13:30) Well, no, not necessarily. like, so bear in mind this one watt that I'm talking about, that was just the Wyze 5070 with no, just, you know, the one internal, um, it's like an M2 SATA, I think, drive in it. But that would idle at about one-ish watts. But X86 doesn't have to use a ton of juice. Like it can idle at like, you know, 15, 20 watts, even with a couple of drives in it. Jason (1:13:43) okay, okay. Joe (1:14:00) So I've got an Anker Solix C300, I think it's called. It's one of those, which is designed for like going camping and stuff. Presumably you like, have you even got one of these maybe? Jason (1:14:12) I have a EcoFlow River 3. It's a 245 watt hour. Joe (1:14:15) Alright, yeah. And it has UPS functionality in it basically, because yeah, uninterruptible power supply for those who don't know. And so you plug this thing into the wall and then you plug your server into that or whatever you want into that. And then if the power goes out, it switches instantly to backup power from the batteries. And then your computer just keeps going for however long the batteries will last for. And so I've got like three routers. and my on-site backup machine. yeah, and all of my wireless access points running power over ethernet. And it like, it idles at about 80 watts, I think, for all of it. But that's part of the calculation here. You know, but if it was ARM based, it might be 10 or 15 watts. And yeah, that's a big difference, right? But at the same time... Jason (1:15:09) If you're someone who's got a four drive setup in there, mean, the larger they are, the larger capacity they are, the more power, I'm assuming that they'll use. Joe (1:15:20) I don't think so. think like a spinning drive is going to use the same amount regardless of its capacity. It's more about the ⁓ RPM. there's, you know, 7200 versus a 54 maybe, you might see a difference. And I would imagine modern drives probably use a bit less than the old ones did. Jason (1:15:37) They're not actually running 24-7 though, are they? They can be idling. Joe (1:15:41) Well, well, this is something you can do is tell them to spin down, right? But I don't recommend it because once a drive is spinning, right? It doesn't, there's inertia there. just spins happily. When you power it down and back up again, that initial ramp up uses a load on almost swore that uses a lot of power. ⁓ it like, you know, it's this surge of power and it's also like, going from nothing to 7200 RPM is a big deal. so it sounds counterintuitive, but at least my understanding is if you spin up a drive and then just spin it for five years, 10 years, it's going to last longer than if you spin it down once a day for five years. Jason (1:16:33) Well, it is mechanical. And so you're, you know, you're stressing it. You're, not stressing it as much on the extreme end of things. It kind of does sound sensible to me. This may be getting in a little bit into the weeds. just a quick question that I had on, ⁓ about drive failure and something like, if you're using something like TrueNAS and you have a drive that fails and you've got, you know, you've got one ready to just pop into the, into the the pool, I'm not even what the technical name is for it. How straightforward is it to add that drive? Joe (1:17:05) That's sh- mean, there's different failure modes, A drive can just slowly start dying and start giving you load of errors, right? And when it starts giving you a lot of errors, like that's it, it's basically curtains. And so you take it out of the pool, take it out of the machine, put in a new drive, and then add that to the pool of the working one. And then it does this process re-silvering. It just copies all the stuff that it needs to onto it. And if you've got a basic two drive mirror, You've kind of got this risky period while it's re-silvering, right? That if one drive is dead and out of the picture and you're copying everything onto the other drive, if you bought them at the same time, there's a fair chance that they'll die within a short period of each other. It's not a problem if you've got adequate backups. Because if you have got a machine somewhere else in your house that is a one-to-one copy of it, so what? If both drives die at once, then you buy two drives and then you copy everything and yeah, it might take you a day or two to copy it all over. It should never be a problem if you mess something up. And I experimented with ⁓ various encryption and stuff, right? And I ended up having to kind of copy stuff back and forth a few times. And it wasn't a problem because I had sufficient backups. And say you realize, hang on, I should have done something different. Like I should have set the record size. won't go into it, but it's like, it affects performance and stuff. I should have set a different record size or I should have set a different compression or whatever. It's not really a problem if you've got backups, because you can just wipe those discs and start again. Jason (1:19:03) Now you've mentioned mirroring quite a bit. ⁓ What are your thoughts on Raid 5 for more than two drives? Joe (1:19:09) ⁓ To be honest, I don't know a great deal about RAID because although we talk about it on the shows, it's not really applicable to me because when I planned my storage setup, it made sense to me to just do two drive mirrors and have three of them. Because if you've got six discs, you could just stick them all in one machine and do a RAID. configuration that makes the most of the space of them. Say you've got six 10 terabyte drives, that's 60 terabytes. But if you do three pairs, three mirrors, then you've only got 10 terabytes. bought 60 terabytes of drives, but you've only got 10 drives. that's something that you have to kind of get your head around and accept that when you buy two 10 terabyte drives, you're not buying 20 terabytes, you're buying 10 terabytes. And even if you buy six, like for a proper setup, or, you know, six times 20, you know, it's, and it's painful, it's really painful to not maximize the space that you've got. Jason (1:20:25) Yeah, I guess that's what a lot of people have recommended so far is, you know, they say if you're, let's say you're sticking four 10 terabyte drives into your NAS. You do that, you do a RAID 5 setup and then you basically allow for one of those drives to fail. It allows for one drive to fail and then you have a backup and you stick that in there. And so you get 30 terabytes of usable. storage, but I guess the reason I'm laboring over this is you really don't, ⁓ you want to make this decision first, right? You don't want to change your mind later on in the process when everything is already set up. Joe (1:21:11) Yeah. But like you see, you know, if you stick multiple drives in one machine, yeah, you're getting a little bit more space, but ultimately that raid is not a backup. It's just, that's one set. so then you've got all that data that you then have to back up somewhere else. And then as many backups as you possibly can of the really, really, you know, the footage from when you walked the Oregon trail, like That's stuff that you want to keep for a lifetime. You'd be devastated to lose that stuff, right? ⁓ Jason (1:21:43) Yeah. That alone is like four or five terabytes. Joe (1:21:46) Yeah, exactly. And but so that's the important stuff. Or maybe it's not, I don't know. You need to make these decisions and what is truly important to you and like is everything important to you? Okay, well that's fine. In which case you're to have to spend an awful lot of money on Whereas, you know, you can get away with buying smaller drives or, you know, filling them up much less quickly by only selecting bits of your data that you actually want to back up properly, with the onsite backup and the offsite backup. Jason (1:22:25) I think one of my final questions this round would be, there any resources that you would recommend? You know, people are getting into this for the first time. And if you go and search YouTube, you're going to get hundreds of different videos with hundreds of different opinions. Joe (1:22:47) It's a very difficult question because it depends on how you want to do it. If you decide to go the ZFS route, Jim Salter, my co-host, he used to write for Ars Technica and he's written some great like ZFS 101 type pieces. So I'd search those out. Obviously I'm going to plug two and a half admins to the podcast, 2.5admins.com. talk a lot about storage and this kind of thing. We talk about other stuff as well and tech news and everything. we have this, at the end of every episode, we do free consulting and we prioritize the Patreon questions, know, people who support us. they are often about ZFS stuff. Recently we did one that was really in the weeds of how to tune ZFS for very specific things like a database and whatnot. But throughout the five... nearly six year run, like we've covered all of this stuff basically. So I would recommend that. Otherwise, don't know, it's difficult because everyone's got their opinion. so you just kind of have to decide, you have to decide who you're to trust, I suppose. Jason (1:23:54) Well, I'm inclined to trust you. You've been around the block. You have the experience and you have a team around you who has the experience. So that's valuable. Joe (1:24:01) Yeah. Yeah, I mean we do get accused of being ZFS zealots on 2.5 admins, but then quietly most of the other late night Linux family hosts of the various other shows use ZFS as well because it is just the best. Jason (1:24:15) Is this a completely ludicrous idea? Shoot it down if you want to. But I have like three, I have a Lenovo Legion Go handheld. I have a Steam Deck LCD. I use my Steam Deck OLED as my main ⁓ Steam Deck. And I have an old ⁓ 1X player handheld. Those are, you know, tiny compact capable desktop computers if you want them to be. Joe (1:24:27) Yeah. Well yeah, it's basically a handheld laptop, right? So yeah. Jason (1:24:46) Yeah, basically. So like, why not just, why not put one of those into service? Joe (1:24:50) Yeah. Why not? Right. Yeah. As long as it's, ⁓ as long as you can run TrueDars on it, or, know, if you're willing to learn the Ubuntu stuff. So yeah, do that and just get, you know, a cheap ish USB dock thing. ⁓ I mean, have you got someone who's willing to let you plug it in in their house? the next. Jason (1:25:09) ⁓ can, can, yeah, I've got somebody. I've got a friend and my best friend is in Sacramento, a few hours away and my mom lives an hour away, so it's... Joe (1:25:19) An hour is kind of the sweet spot for me because an hour away is probably going to be like on a different electrical, you know, part of the electrical grid. Like the weather's probably, you know, if you have like floods and stuff, it's probably not going to be the same. That is what I would do then. 212s in the desktop machine, set the framework up with the two 8 terabyte USB drives. Buy the relevant stuff to set up one of the unused handhelds at your mom or friend's house. with the other 12, TeraBite 1, and that be your offsite and copy everything offsite. Jason (1:25:52) And definitely TrueNAS, TrueNAS scale. Do you have anything positive to say about any other NAS-oriented operating systems? Joe (1:26:01) I have heard good things about Zigmanaz, which is with an X, which is like a fork of an earlier version of TrueNas, think. But I don't know anything about it. yeah, Open Media Vault, I don't know, I think I might have briefly looked at it. But yeah, I don't know. mean, TrueNas is like the name, you know, in Nas in a Box. that's, it's funny, like the people who I know, like my, I have a show, Hybrid Cloud Show, right? And ⁓ they tend to use TrueNAS because it's really easy. Even though they are capable of doing what I do and running the Ubuntu server and doing it all via, you know, config files and all the rest of it and SSH, they just want to click their way around and, you know, make it nice and easy. So yeah, it seems to me that TrueNAS is like the best bet. Jason (1:26:53) Okay. I'm going to give it a shot. I am excited about giving it a shot. I'm just like chomping at the bit here, just waiting for these drives to show up so I can get started. ⁓ Anything else you want to throw out there? Any shameless plugs or ⁓ hot takes about anything going on in the Linux world right now? I know you've got some hot takes on a lot of ⁓ topics like AI and Mozilla. Joe (1:27:02) Second. to give you all the hot takes in the world. I don't even get me started on that. But yeah, check out late night Linux.com or just search your podcast player for late night Linux family. What we do is we produce short podcasts that are to the point about Linux, open source, we've got a development show, we've got a cloud show, we've got two and a half admins, our sysadmin show. And they are all professionals. know, everyone involved, it works in the industry. Apart from me, I'm just the producer and the everyman. And we don't shy away from politics. Like we are, I suppose you might say progressive, put it that way. We're pretty anti-AI. I mean, not everyone's as anti-AI as I am, but you know, we're not hugely fond of the current president in America, one might say, and that whole movement. you know, we are not no politics guys, put it that way. We don't shy away from that stuff. So. Occasionally people show up and say, why are you talking about politics all the time? Keep to the tech. it's like, no, is an inherent, open source as a movement is inherently political thing. So politics can't. Jason (1:28:31) I intentionally try to keep politics out of Linux for everyone, but I have to also acknowledge that it is difficult to do so. You know, it requires completely avoiding certain topics sometimes. It's challenge, especially with tech. Like, tech is becoming very political and very politically driven. And so it's a tough line to walk. Well, Joe, thanks a bunch, man. I appreciate all the knowledge and I hope we can circle back. Joe (1:28:39) Yeah. The thing Jason (1:29:01) maybe in a few months and maybe dig into the weeds a little bit more when it comes to, you know, maybe self-hosting and managing your NAS and things like that. Joe (1:29:13) Yeah, look forward to it. Jason (1:29:14) And that's going to do it for episode 61 of Linux for Everyone. Thank you so much for listening. You can find show notes for this episode at Linux for everyone.net. You can reach out to me directly by sending an email to letters at Linux for everyone.net. And you can get some pretty cool merch over at l4e.store. Also make sure to check out the YouTube channel at youtube.com slash. Linux for everyone and yes, I am absolutely targeting a PeerTube instance in the near future. Until we chat again, you all take care and take care of each other. See ya.