31 May
2019
31 May
'19
6:07 p.m.
On Fri, May 31, 2019 at 06:41:55PM +0100, Gavin Westwood wrote:
With the discussion about RAID 10, that got me back to
thinking about
better/alternative system to my current RAID1+LVM+EXT4 setup on our
Linux home server and I'm looking for advice from other members.
Currently we have 13.6 TB of storage (a lot of which are photos by my
semi-professional girlfriend, and videos from our wildlife cam which
produces about 15 - 20GB of videos a day [email me off-list if you want
to want the URLs of my fledgling Youtube hedgehog and bird channels]).
There is some amount of file duplication, for instance where I have
stuck old backups (copied files and folders, not tar/compressed
archives) on there or photos/videos have been copied to different
folders (e.g. to categorise), so filesystems with built-in deduplication
(like I believe BTRFS has) would be nice. However my main priorities
are: maintaining data integrity, ease of administration, and really a
sub-category of that: ease to expand or shrink and reallocate storage as
required (if necessary - quotas are not required, but crashing due to a
full disk is to be avoided).
For years I have been looking at BTRFS, but it's never sounded 100%
production ready to me (although I remember that at least one distro
made it their default fs). Andy's mention of ceph and stratis were
something new to me, but I'm not sure if they are a bit much for a
single server, and I've no experience with ZFS, but I think I read about
some disadvantages that put me off a few years back, but I forget what
they were now.
Anyway, what do/would you use for this sort of scenario/requirement or
what are your experiences with suitable filesystems for my
requirements? Just to be clear - I want to ensure that a single disk
failure is very unlikely to result in data loss. Also, all disks are
currently the spinning disk type, so any features that takes advantage
of SSDs would be wasted.
I'd go for btrfs. (Disclaimer: I'm a regular contributor to the
btrfs community, and I've written quite a bit of their documentation).
I've been running a RAID-1 btrfs data store for about 10 years. In
that time, I've hit one bug that required me to restore from
backup. That was 9 years ago, when the FS was about as stable as an MP
in a three-way marginal constituency.
I'm now up to 11TB of data on a 13TB RAID-1, and it's been fine for
years, even over power failures and several disk failures.
The primary things you need to ensure is that your hardware is
reliable (no USB storage, and disks and controllers which honour flush
instructions), and that you have backups (because your hardware *will*
fail, eventually).
Scrub regularly, avoid compression and qgroups and parity RAID for
metadata, and you should be fine.
Note that RAID-1 is *not* a backup. You still need backups. RAID-1
is an uptime mechanism. RAID-1 will not save you from an accidental
"rm -rf /mnt/my-preciousss/". RAID-1 will not save you from a fire, or
a double disk failure, or a faulty TRIM implementation, or something
writing zeroes to the device, or running mkfs on the wrong device,
or... (yes, we've seen all of these in #btrfs).
If you can't afford backup *and* RAID-1, use half the disks in the
live system with "single" profile, and half the disks in the backup
machine with "single" profile. It's going to be safer than everything
in RAID-1 all attached to one machine.
For backups of a mostly-append data store such as yours, the
cheapest option in the ~4 TB -- ~150 TB range is BD-R. Over 150 TB or
so, it's cheaper to use LTO tapes. I'll need to update my spreadsheet
if you want more precise figures on that.
Hugo.
--
Hugo Mills | Great oxymorons of the world, no. 2:
hugo@... carfax.org.uk | Common Sense
http://carfax.org.uk/ |
PGP: E2AB1DE4 |