31 Jan
2012
31 Jan
'12
4:03 p.m.
Hi,
If you don't currently take advantage of BitFolk's free backups
service then the following doesn't apply to you.
Two extra checks were added last night on the backups that BitFolk
manages for you.
All this info is going to go on a page I will create at
https://tools.bitfolk.com/wiki/Backups but you need to know it now.
- Backup age
This checks that your most recent successful¹ backup is not too
old. The warning threshold is 2.5x the appropriate interval. So if your
backups happen every 4 hours, you'll receive an alert if they are
ever older than 10 hours. If your backups happen daily then
you'll receive an alert if they are ever more than 60 hours old.
And so on.
This alerting replaces the manual process of me being sent
excerpts of log files that say a customer's backups are failing,
then opening a support ticket with the customer to make them
aware.
If you receive this alert then your backups are definitely not
happening.
The alert looks like this:
From: nagios(a)bitfolk.com
Subject: ** PROBLEM alert - backup0.bitfolk.com/Backup age youraccount is CRITICAL **
***** Nagios *****
Notification Type: PROBLEM
Service: Backup age youraccount
Host: backup0.bitfolk.com
Address: 85.119.80.240
State: CRITICAL
Date/Time: Tue Jan 31 12:37:38 UTC 2012
Additional Info:
FILE_AGE CRITICAL: /data/backup/rsnapshot.6-7-4-6/hourly.0/85.119.82.121 is 16842912
seconds old and 4096 bytes
(Those who haven't had a successful backup run in the last couple
of days will have a huge number of seconds listed there because
the backup system was only modified to record last successful
contact recently)
- Backup space usage
This checks that your total backup space usage is not approaching
your current quota. The thresholds are 95% for a warning and 99%
for a critical.
This alerting replaces the manual process of me being warned about
customers who are exceeding their quota and then opening support
tickets with them to discuss what they want to do about it².
If you receive this alert then your backups are still happening,
but you're in danger of (or already are) using more than the
agreed space. If you exceed your quota then we may disable your
backups, so that would eventually cause the above backup age alert
to fire.
Please note that we can't update the measurement of how much space
you're using very often. Backup directories contain hundreds of
millions of files, many of which are copies of each other, but
it's not possible to tell without looking. Adding it all up takes
quite a long time and stresses disk IO quite badly. So we only
update quotas every day at best at the moment.
Also please note that since this is a backup system, deleting
files on your VPS does not immediately result in using less disk
space for backups. It would be a rather pointless backup system if
it threw away deleted files immediately. :) Anything that gets
backed up is going to be kept for as long as your chosen backup
schedule dictates, e.g. 6 months by default. If you have blown
your quota by accidentally allowing large amounts of data to be
backed up, you are still going to have to contact support to get
it deleted.
The alert looks like this:
From: nagios(a)bitfolk.com
Subject: ** PROBLEM alert - backup2.bitfolk.com/Backup space usage youraccount is
WARNING **
***** Nagios *****
Notification Type: PROBLEM
Service: Backup space usage youraccount
Host: backup2.bitfolk.com
Address: 85.119.80.230
State: WARNING
Date/Time: Tue Jan 31 15:52:28 UTC 2012
Additional Info:
WARNING 98.50% (394/400MiB) used
It is possible for the usage to go above 100% because we do allow
you to go over your quota for short periods of time.
Cheers,
Andy
¹ "Successful" as in, rsync connected to your host, did some stuff
and then exited with a non-error exit code. It does not
necessarily mean that what you think should be backed up is being
backed up. As with any backup solution you need to assure yourself
on a regular basis that it's doing what you expect.
² Generally one or more of:
- Buy some more disk space for backups
- Backup fewer files
- Backup less often
--
http://bitfolk.com/ -- No-nonsense VPS hosting