Hello,
On Fri, Oct 14, 2022 at 11:19:05PM +0100, Dom Latter via BitFolk Users wrote:
On 14/10/2022 21:22, Andy Smith via BitFolk Users
wrote:
The risk is a little bit lower if you have been
regularly scrubbing
your array as that forces it to read everything and find any
unreadable bits. Most Linux distributions using MD RAID do schedule
this for once per month.
Could that be why it was doing an unexpected rebuild the other day?
Nothing in /etc/cron.monthly.
On Linux with MD it is usually in /etc/cron.d/mdadm as it is set for
the first Sunday of the month, which is not expressible in normal
cron language (so it fires a script every Sunday to check if it's
the first Sunday).
That's my experience of Debian and Ubuntu anyway. Might be
different on other distributions.
So if I convince my customer to move server to
something with more
than, say, seven, spinny things what RAID should I be looking at?
I think it's quite a hard sell if they're happy with the risk as it
is, but I did not realise there were already so many drives in use.
I didn't see how many devices there are right now, but if we imagine
there are six then at the point where one dies, in terms of
redundancy you effectively have a five drive RAID-0¹. It starts to
make me a bit nervous because of the risk of ANY of those other
devices encountering an error.
At least if you went to RAID-6 you could lose any two at once, which
vastly reduces the risks and if you have lots of drives it doesn't
hugely decrease the capacity or performance.
As for how many drives to have in each RAID-6, opinions are going to
differ. The more you have the higher the performance and capacity. I
might draw the line at about 10 of them. A lot of people with 24
drive bays might go for two 11 drive RAID-6 arrays striped together.
Or 2x10 if two drives were a mirror for the OS to be installed on,
for example.
Here's some thoughts from people asking that question for ZFS
raidz2, which is kind of similar to RAID-6:
https://www.reddit.com/r/zfs/comments/2hvxd4/whats_your_maximum_number_of_d…
Going to ZFS could be an interesting thing to explore because
although ZFS does tend to perform a bit worse than a simple Linux MD
RAID, you do get a lot of nice features included like checksums for
data integrity and self-healing, and compression. And snapshots,
which may or may not be of interest to you.
You can get back some of the performance by having a lot of RAM for
a cache (only used for reads). If you have room for one or more SSDs
then you can put a second layer of cache on that/them (again only
for reads). That doesn't technically need to be redundant since if
it dies ZFS just stops using it, but the resulting decrease in
performance could be problematic, so some people use more than one
SSD for L2ARC. Finally if you did put at least one SSD in then some
or all of it/them could be dedicated to a "ZFS Intent Log" (ZIL,
SLOG, all interchangeable terms). This is for writes only. Again,
loss of the ZIL device means loss of the performance boost² so some
people do mirror them.
A release of ZFS that's coming soon is also meant to have special
data allocation classes, which basically means that you can have a
separate vdev ("array") for file metadata. The point is to put the
small metadata on something fast, and keep the big stuff on the slow
HDDs.
So, lots of interesting ideas there, that were formerly only in the
reach of huge data warehouses with their proprietary solutions for
tiered storage, but clearly a lot to get your head around and
potentially impractical to spec up and test out when you're just
using what can be rented from Hetzner. Simple solutions may be best
at the end of the day.
Cheers,
Andy
¹ Except worse in performance terms because it needs to calculate
parity to read anything, at that point.
² The only time the ZIL is ever read from is if the server crashes
and the filesystem has to replay any writes that went into the ZIL
but didn't yet make it onto the regular storage. So IF your single
ZIL device died AND your server crashed or had power yanked AT THE
SAME TIME, then it would be possible that some data was written to
ZIL and then lost forever.
--
https://bitfolk.com/ -- No-nonsense VPS hosting