| View previous topic :: View next topic |
| Author |
Message |
lars_the_bear
Guru
Joined: 05 Jun 2024
Posts: 496
|
 Posted: Fri Sep 20, 2024 9:20 am Post subject: |
 |
|
| pjp wrote: |
One thing I've considered using one for is the "generally on and available" stuff so I don't have to keep a PC running 24/7. I'm sure it would do most of what I'd want very well, but then I have yet another environment I have to maintain, so it simply hasn't been compelling enough. |
I don't really feel my Pi NAS has to be maintained. If it were exposed to a hostile network environment, that might be a different story. But, sitting in my house running rsyncd, I'm not bothered.
Also, these are commodity devices. If I cook one of them -- and I do this fairly frequently in my embedded applications -- I just throw it away and buy another on eBay. The entire operating system is on an SD card, so I just plug the card into a new board. And if I blow up the card, I have a backup I can flash on a new card in a minute or two.
Of all the computer stuff I have in my house, the Pi-based stuff is the only part of it that is completely reliable.
But, as I've said, my applications are undemanding. I don't care whether my daily backup takes one minute or two.
I haven't been brave enough to install Gentoo on them yet, though. For my embedded applications, I built the operating system from scratch using scripts. I've re-implemented Portage, badly. I'd like to try Gentoo on the Pi, but it's not come to the top of my to-do list. I think this will be a job for my retirement. For the NAS I just use DietPi, with no X or desktop stuff.
There are specific "NAS" images for RPi, but I haven't tried any of them.
BR, Lars. |
|
| Back to top |
|
 |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9806
Location: almost Mile High in the USA
|
| Posted: Fri Sep 20, 2024 12:44 pm Post subject: |
|
|
I back up my PVR and my NAS/Shell/VM/HTTPD/SMTPD/etc. boxes. Both boxes take about a half to a full hour to rsync -- the PVR has somewhat high churn rate, the shell box VMs take a while to do diffs and depending on how much churn it could take more than an hour to rsync... Both cases I think the bottleneck is the speed of seeks and read rate of the hard drive sets but the PVR backup is real close to being CPU-limited. When it's not cpu limited, it still gets real slow when trying to do diffs between two directory trees with a lot of directories and small files.
The PVR has a 2TB disk (single disk), the shell box is only 4TB (3x2TB RAID5). My rsync backups are to two different machines with encrypted RAID5. I now back up the PVR to a 5 disk 500GB encrypted RAID5 as that's the hard drives I had spare from upgrades or acquired e-waste, and the shell box I back up to more 2TB disks.
Currently thinking about changing my backup strategy after acquiring more e-waste hdd's instead of running so many 500G HDDs... though the RAID works just fine and I now have plenty of 500G disks as cold spares for that array. At least the 500G disk array takes only an hour and a bit to rebuild, my 2T disk array takes 5 hours. Ouch.
I don't think I'll go to 6 disks though that's the limit, mostly to leave one port open for a hot spare if needed on the 6-port motherboards. Need to find sata PCIe cards...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3360
Location: USA
|
| Posted: Sun Oct 20, 2024 5:37 pm Post subject: Re: Redundant local external storage that's not always on |
|
|
| maiku wrote: | I need a local configuration where I can store data with the following considerations.
1) It's really just local.
2) I can easily use it on another at least Linux machine.
3) Has redundancy so it can suffer at least partial failure.
4) Is encrypted.
5) Can be powered on only when I need it (which will not be enough times to justify a NAS in my mind).
6) Does not require an infinite money budget.
In my primitive mind the only thing I know that serves this purpose is USB so I default to getting a dumb USB enclosure that supports 2 drives and buying two rust platter drives and putting them in a ZFS RAID, etc.
Is there a better way? Seems like the pitfalls of ZFS RAID over USB might make things interesting. |
Long, long ago I fell victim to the allure of mounting a USB hard drive on a Pi while I convinced myself it was a viable backup solution. I thought it was great to have cheap/disposable commodity hardware running low-power backup on my LAN. Everything was great until the weak links started emerging... things like transient power losses killing the SD card, limited SD card lifespans killing the Pi, RPi that lacked real disk interfaces forcing me to use less than robust USB data connections that routinely failed in use, other flavor Pi that offered SATA connections but still resulted in a less than robust overall solution. Whenever something went wrong it was always a major asspain fixing it. The power and component savings were illusory and were never justified in light of the repair/downtime frustration they caused.
Between then and now, many other solutions were attempted on temporary basis.
Years later the homelab had evolved to a dedicated multicore ZFS server on BSD with lots of ECC RAM, multiple vdevs comprised of redundant Z2 or Z3 arrays, long-duration UPS power and automated shutdowns in the event of a power outage. Today is years after that, and I don't need BSD anymore b/c I'm able to run ZoL. When a disk fails, I slide in a replacement and issue a command to resilver the drive and I'm done. No headaches.
There's just no way I'd go back to a Pi. If you're serious about your data, then ZFS is the answer. You can power it down intermittently if you want to, but doing that creates it's own problems. Why reduce the robustness of your archive system by only allowing it to take intermittent snapshots of your data. Me? I burn the power.
(first post here in 20 years) |
|
| Back to top |
|
|
pietinger
Moderator
Joined: 17 Oct 2006
Posts: 4987
Location: Bavaria
|
| Posted: Sun Oct 20, 2024 11:41 pm Post subject: Re: Redundant local external storage that's not always on |
|
|
| Bob P wrote: | | (first post here in 20 years) |
... 18 years ... 
Welcome back! 
_________________
https://wiki.gentoo.org/wiki/User:Pietinger |
|
| Back to top |
|
|
lars_the_bear
Guru
Joined: 05 Jun 2024
Posts: 496
|
| Posted: Mon Oct 21, 2024 7:40 am Post subject: Re: Redundant local external storage that's not always on |
|
|
| Bob P wrote: | Everything was great until the weak links started emerging... things like transient power losses killing the SD card, limited SD card lifespans killing the Pi, RPi that lacked real disk interfaces forcing me to use less than robust USB data connections that routinely failed in use, other flavor Pi that offered SATA connections but still resulted in a less than robust overall solution. Whenever something went wrong it was always a major asspain fixing it.
|
FWIW I'd like to point out that in all my years of running Raspberry Pis as NAS systems, I've never, not once, experienced any of these problems. Maybe I've just been lucky -- I don't really have a representative sample of other people's experiences. Still, I leave in a region where power failures are not uncommon. I had three last week. Pis are fine. I wish I could say the same about everything else.
| Bob P wrote: |
The power and component savings were illusory and were never justified in light of the repair/downtime frustration they caused.
|
The energy savings are easy to calculate, if you use a Kill-a-Watt or similar. In my case, the difference between an average power consumption of about 2W for the Pi, and ~60W for the old PC, is worthwhile. The costs saving in components is modest, and depends on the amount of stuff you already have in the junk pile. Since I've never had to carry out any repairs, or experience any downtime, I can't comment on how frustrating these things are 
Whether any of the cost savings is significant or not, depends on your priorities. For me, minimizing energy usage is a matter of principle, regardless of cost.
BR, Lars. |
|
| Back to top |
|
|
maiku
Guru
Joined: 24 Mar 2004
Posts: 593
Location: Escaping from NY
|
| Posted: Mon Oct 21, 2024 1:33 pm Post subject: Re: Redundant local external storage that's not always on |
|
|
| Bob P wrote: | Long, long ago I fell victim to the allure of mounting a USB hard drive on a Pi while I convinced myself it was a viable backup solution. I thought it was great to have cheap/disposable commodity hardware running low-power backup on my LAN. Everything was great until the weak links started emerging... things like transient power losses killing the SD card, limited SD card lifespans killing the Pi, RPi that lacked real disk interfaces forcing me to use less than robust USB data connections that routinely failed in use, other flavor Pi that offered SATA connections but still resulted in a less than robust overall solution. Whenever something went wrong it was always a major asspain fixing it. The power and component savings were illusory and were never justified in light of the repair/downtime frustration they caused.
Between then and now, many other solutions were attempted on temporary basis.
Years later the homelab had evolved to a dedicated multicore ZFS server on BSD with lots of ECC RAM, multiple vdevs comprised of redundant Z2 or Z3 arrays, long-duration UPS power and automated shutdowns in the event of a power outage. Today is years after that, and I don't need BSD anymore b/c I'm able to run ZoL. When a disk fails, I slide in a replacement and issue a command to resilver the drive and I'm done. No headaches.
There's just no way I'd go back to a Pi. If you're serious about your data, then ZFS is the answer. You can power it down intermittently if you want to, but doing that creates it's own problems. Why reduce the robustness of your archive system by only allowing it to take intermittent snapshots of your data. Me? I burn the power.
(first post here in 20 years) |
Glad to see you back! Thanks for your experience.
Not to jump on the "how dare you flame RPi!" bandwagon, but I use RPis pretty consistently for a side gig I do. I have about 300 deployed in production. I must say, uptime issues and SD card corruptions have not been such a big issue over the 10 years I've had doing it. Sure, some have had to be replaced. But that has been the exception to the rule in my case.
That's why I had mentioned using an RPi in my post. I don't have any problems with them in specific. However, it just seems like such a complicated and heavy sledgehammer to me to solve this problem.
100% agree, ZFS is awesome.
_________________
Michael |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo Chat
