Redundant local external storage that's not always on

[maiku]

I need a local configuration where I can store data with the following considerations.

1) It's really just local.
2) I can easily use it on another at least Linux machine.
3) Has redundancy so it can suffer at least partial failure.
4) Is encrypted.
5) Can be powered on only when I need it (which will not be enough times to justify a NAS in my mind).
6) Does not require an infinite money budget.

In my primitive mind the only thing I know that serves this purpose is USB so I default to getting a dumb USB enclosure that supports 2 drives and buying two rust platter drives and putting them in a ZFS RAID, etc.

Is there a better way? Seems like the pitfalls of ZFS RAID over USB might make things interesting.
_________________
Michael

[pingtoo]

If there is no performance requirement. I would suggest use two USB storage and just ext4. redundancy is base on automated file system level copy.

This way you can have long longevity as well as easy to move, plus possible versioning to reduce human error.

[lars_the_bear]

[maiku]

[lars_the_bear]

@maiku

Sorry -- I forgot I had upgraded to a Pi 4, because the Pi 3 doesn't support USB 3 (as I recall). I have an open-frame enclosure made from plywood that I had laser-cut so the bits would fit together nicely. There are three compartments -- in the bottom compartment are the mains transformers, the middle compartment has the Pi, and the top compartment two (at present) 8TB USB 3 disks. The disks have their own cooling. The Pi has a propietary heatsink case, so it doesn't need any additional cooling (or, at least, has not so far).

In earlier incarnations of this set-up I managed the disk spin-down using scripts, and it was pretty aggressive -- I never let them spin idle for more than a couple of minutes. But these new disks take 30 seconds to spin up, so I leave the default power-saving mode, which is to spin down after twenty minutes. The spin-up time is inconvenient, but I consider it a price worth paying, to keep the power consumption low. I'll be moving to SSDs when I can afford them.

It's not a blazingly fast system, but I find it OK for daily backups. And it's cheap. Also, because it's a real Linux, I can run other stuff on it as well, which isn't always easy with a proprietary NAS.

BR, Lars.

[sdauth]

I've built my own "NAS" with decade old parts for this exact purpose. It was way more cheaper than buying usb enclosure since I've had already every parts necessary (not easy to find quality USB enclosure too in the >=4 bay range) with much more capability (proper PSU, proper cooling for drives during long transfer, lots of sata ports, possibility to add a raid card if needed..)
I call it NAS because that's what it is, it runs fully headless, only connected to my router, without any display and fully encrypted too.
I only turn it on for backup / archive purpose since it wouldn't be so good to let that watt monster 24/7 on. (server parts from 2008 are not exactly energy efficient )
It runs gentoo of course with a trimmed down kernel, boots under 10s and use like 80m of ram.
Before that I was long time user of various USB enclosures but there was always an issue at some point (shitty USB chipset, too high temps due to lack of fans..); I still use some USB enclosures though but only with SSD for quick transfer.
I don't know what your storage requirements are but an old laptop (with 2x internal SATA + one extra reusing the optical bay for example) could do the job (assuming you're fine with 2.5" hdd) with the added bonus of embedded UPS (the battery )

[lars_the_bear]

[eccerr0r]

What is your churn rate (how many GB do you write every month) and how much total storage is another question.
I'm using desktop PCs with a pile of hard drives acquired free or minimal cost, connected to them as a software NAS, and shutting them down when not used because buying a pile of USB3 hardware is out of the question. My RPi0W is too slow to do this job as I have a couple disks with enough churn rate that doing rsync backups would take an annoying amount of time to work with USB2 (indeed I think a RPi4 is minimum -- though with the old PCs I get SATA connections -- USB2 with cheap hardware/cables is not reliable enough for RAID).

One thing I noted is that encryption is slow. The first machine I want to get rid of is my Athlon II X2 PC that is backing up my PVR. The full disk encryption plus ssh encryption is double encryption and consumes considerable CPU cycles and does feel sluggish while doing backups - the cpu is pretty much fully consumed to maintain the Gigabit Ethernet plus disk encryption. I also have an old Atom server board used for backing up my main server -- it has AES-NI and this is much faster at full disk encryption but ssh encryption is a bit slow -- at least until I figure out how to get it to use AES-NI as well. I haven't gotten remote power working on the Atom board but that may be a long term solution -- once I can get more SATA ports on it as well as how to deal with FDE that requires a password to unlock the LUKS RAID5 (currently only grub/kernel is not encrypted.)

I've been pondering how to reallocate my machines as of late because of the Atom, might have to use it as my main server but again it's slow and pretty much useless as a distccd machine which the current machine contributes to.

Fortunately for me electricity is cheap and my homemade grid tie solar panels is cutting into my power consumption...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[lars_the_bear]

[eccerr0r]

I don't sweat it as a even a full blown PC, not just a RPi, that's turned on for a few hours a month it's not going to be a huge electrical cost.

I have other PCs running 24/7/365 (they're my distccd/httpd/smptd/nfsd/bind/ssh/... and pvr boxes) and yes those do add up, but that's another thing...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[lars_the_bear]

[maiku]

[eccerr0r]

How fast is the rpi4 copying through scp from Gbit ethernet to usb encrypted volumes anyway, can it keep up 100MB/sec? Will it handle RAID5 at speed (write three disks at 50MB/sec each or four disks at 33MB/sec each)?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[lars_the_bear]

[eccerr0r]

technically energy cost is also cost, so they're the same thing. So basically it's speed or low cost, pick one.

At 35MB/sec for USB3 devices and Gbit Ethernet, that's pretty horrible, that's like USB2 speeds (if the devices were on separate host controllers of course). Satisfies my curiosity of how fast RPi4's are, bottlenecked at about 70MB/sec (with encryption or just mirroring?). Still better than the RPi0W I have regardless.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[lars_the_bear]

[eccerr0r]

The theory is that you would have optimized your own solution.

RPis are not free, and I still would need to buy USB3 hardware which I also don't have -- so I'll have to depend on other peoples testing that weren't involved in marketing them. I have a 2012 laptop that will linefill from ssd 100MB/sec to Gbit Ethernet just fine, so I suspect your 2015 can do the same. So it must be the RPi as the bottleneck.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[pjp]

Is the 100Mbytes/sec benchmark over the network, or internal to the Pi over USB3? You're using RAID on USB connected drives, correct? Also, is the network interface also over USB? If it has an Ethernet port, is it gigabit?

I'm wondering if RAID on USB is the main problem, though a shared USB bus with networking might at least halve the throughput (once across USB, then once across the USB network).

I like the idea of these things, but they still seem very finicky and of questionable practical use for the cost.
_________________
Quis separabit? Quo animo?

[lars_the_bear]

[pjp]

Thanks. That was entirely for me to learn more about the platform. I've considered buying a Pi multiple times, but it just seems like something that wouldn't be sufficiently useful to me for the amount of effort it would take. I'm sure I don't know enough about ARM as a platform, but it seems sufficiently dodgy in that they lack any consistency in how to get them working.
_________________
Quis separabit? Quo animo?

[eccerr0r]

I don't think there's anything wrong with the RPi per-se but the main thing about a lot of ARM platforms are simply not standardized. A custom OS is needed for every ARM implementation, you can't unsolder the eMMC or SD or whatever to one ARM implementation and solder to another, even if both use the exact same eMMC/SD/... chip, like taking a RPi eMMC and sticking into your ARM phone, and expecting it to boot and at least display something on the screen. PC hardware you can usually get some modicum of success when you transplant disks.

This is saying nothing about the speed of these ARM chips. While it seems some of the newer ones do okay, a lot of the older ones are lethargic at best. I would hope that the RPi4 with its GbE and USB3 that it could take advantage of the bandwidth. Perhaps it's only one at a time.

A lot of my PCs, other than the fact I have a lot of slow hard drives, can still copy GbE speeds to the disks and bottleneck at the disks. When I was using USB2 for the disks (BAD decision!) USB2 was the bottleneck for both speed and system reliability, hence I'm a bit hesitant on even using USB3. However as of recent I have some USB3 gear and so far so good, perhaps due to stricter implementation standards. As of now I really don't have much USB3 gear and rely on SATA which is really fast, and RAID5, though it has that penalty of writing parity, is doing a good job spreading the load across the disks so sequential reads/writes are just a little worse than running RAID0 across the disks and get single disk protection.

RAID6 I've not had much luck with. Well, specifically speed. It's horrid.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[pjp]

That's why I referred to the "ARM platform" rather than Pi specifically. Pi has popularity going for it, so it seems more consistent, which makes sense. But I often see threads here where there are issues, but I haven't retained a general perception if they are mostly not Pi.

One thing I've considered using one for is the "generally on and available" stuff so I don't have to keep a PC running 24/7. I'm sure it would do most of what I'd want very well, but then I have yet another environment I have to maintain, so it simply hasn't been compelling enough. Dual-NIC would be nice too, which I don't think they've offered yet (though some other non Pi has, but is more obscure).
_________________
Quis separabit? Quo animo?

[eccerr0r]

You can always add more usb3 NICs ...

Alas, I keep 24/7 PCs running. I'm down to 2, been running more in the past but started merging them as I got multi-core machines that are faster than what I need. Now trying to merge the PVR and the virtual machine box I'm worried about how stable pass-through PCI works, though still a bit concerned about mythtv security.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[pjp]

[eccerr0r]

So rule out ARM/RPi (at least RPi4 that is having problems at 70MB/sec written to 2 disks? I wonder how fast it writes to just 1 disk too...)

At least with USB3 at 20Gbps, that's ridiculous bandwidth compared to 1GbE and 2Gbps head rate HDDs, so sharing that is no problem.

Now if you're dealing with SSDs and 10GbE, then it can get tight...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?