24 Mar
2011
24 Mar
'11
6:29 p.m.
I found the same a while ago on mine. It reduced a many GB hourly diff to a few MB which
including the snapshot was ~1/5 the size.
--
Deanna Earley
----- Reply message -----
From: "Adam Spiers" <bitfolk(a)adamspiers.org>
Date: Thu, Mar 24, 2011 12:02
Subject: [bitfolk] Individual snapshot disk usage report added to panel/backups
To: "Andy Smith" <andy(a)bitfolk.com>
Cc: <users(a)lists.bitfolk.com>
On 24 March 2011 11:38, Andy Smith <andy(a)bitfolk.com> wrote:
Hi Adam,
On Thu, Mar 24, 2011 at 11:26:53AM +0000, Adam Spiers wrote:
Wow, OK - something's not right for sure, as there shouldn't be
anywhere near that much churn on my VPS.
This is great - thanks! However I'm
struggling to understand the
numbers shown. On my VPS I see hourly snapshots consuming 2.5GB -
what does this mean exactly? Presumably not that my VPS is churning
that much data per hour, because it should be idle most of the time
AFAIK.
You refer to the differential report, so there are ~2.5GiB of diffs
per snapshot. Since the per-snapshot report shows each snapshot
coming in at ~9.5GiB that means you have the same amount of files
but with ~2.5GiB changing content every time. Bear in mind that any time a file changes (even if
it's just
metadata), a copy of both the old and new version will be stored in
their entirety.
By metadata presumably you mean inode updates? So I can search for
churn via find -ctime? In that case, could I expect remounting
partitions with the noatime or relatime options to drastically reduce
the size of incrementals?
http://en.wikipedia.org/wiki/Atime_(Unix)#ctime
http://lwn.net/Articles/244829/
I can have a look and see what I can make of it if you
like. I'd
rather not look at customer data without permission.
If you want to look yourself, look for files that change metadata
(mtime, ownership) all the time, like if they're being checked out
of version control or downloaded from somewhere without preserving
mtimes. Something like that.
Ahah. I think I may have located the culprits: huge log files.
-rw-r--r-- 1 root root 67797975 Mar 24 11:47
/home/adam/web/adamspiers.org/logs/error.log
-rw-r--r-- 1 root root 141526573 Mar 24 11:55
/var/www/redmine/log/mod_rails/error.log
-rw-r--r-- 1 www-data www-data 314066833 Mar 24 11:55
/var/www/redmine/log/production.log
-rw-r--r-- 1 www-data www-data 895519982 Mar 24 11:49
/var/www/wordpress/wp-content/debug.log
-rw-r--r-- 1 root root 899657028 Mar 24 11:56
/home/adam/web/adamspiers.org/logs/access.log
If I start rotating these regularly, then I could expect the
incrementals to significantly shrink, right?
For the benefit of anyone else who wishes to sanity check their churn,
I did something like this:
backup_dirs="/boot /etc /home /initrd /root /srv /usr/local /var"
find $backup_dirs -ctime -2 >& /tmp/find_-ctime_-2.out
xargs ls -ld < /tmp/find_-ctime_-2.out | sort -n -k5
If there's a better way then please do share it.
_______________________________________________
users mailing list
users(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/users