18 Apr
2024
18 Apr
'24
10:39 a.m.
Hi,
We had a customer ask about backups of images of their VM disk. This
is not currently a service that we offer.
It has some advantages, so it could be a service that we offer.
The main advantage being:
- High degree of confidence that the VM is just able to be one-click
restored and is then bootable and working, as it's an exact image
of the disk
It does of course also have a heap of disadvantages such as:
- Granularity of backup is "the entire disk". No way to pick and
choose what is backed up except by putting things on different OS
disks.
- It's based off a disk snapshot so nothing in memory gets
preserved; a restore event will be similar to a "boot after
unexpected power cut" event to the restored VM. Though doing the
snapshot while the VM is powered off for a few seconds would be
an option.
- This obviously uses a lot of disk space which we'd charge for at
our archive storage rate: £0.40+VAT/50GB/month. If preferred this
could go onto third party cloud storage but bear in mind that
Amazon S3 is about 3 times the cost of BitFolk's archive storage.
- Limited differential backups are possible though I'm yet to fully
understand how well that works. All I can say right now is that
a differential would use less, but I'm not sure really how much
less.
So, there would be "some charge" for the service itself and
definitely a charge at archive storage rates for the actual amount
of storage involved.
If this sounds like a service that you think you'd actually want,
please get in touch off-list and we'll see if we can develop it.
As it's more the BitFolk way for customers to do this sort of thing
themselves I don't really expect this to be widely desired, but I am
working on it for one customer anyway so if I can make it more
generally applicable then I will.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
18 Apr
18 Apr
10:50 a.m.
On Thu, 2024-04-18 at 10:39 +0000, Andy Smith via BitFolk Users wrote:
Hi,
We had a customer ask about backups of images of their VM disk. This
is not currently a service that we offer.
It has some advantages, so it could be a service that we offer.
The main advantage being:
- High degree of confidence that the VM is just able to be one-click
restored and is then bootable and working, as it's an exact image
of the disk
It does of course also have a heap of disadvantages such as:
- Granularity of backup is "the entire disk". No way to pick and
choose what is backed up except by putting things on different OS
disks.
- It's based off a disk snapshot so nothing in memory gets
preserved; a restore event will be similar to a "boot after
unexpected power cut" event to the restored VM. Though doing the
snapshot while the VM is powered off for a few seconds would be
an option.
- This obviously uses a lot of disk space which we'd charge for at
our archive storage rate: £0.40+VAT/50GB/month. If preferred this
could go onto third party cloud storage but bear in mind that
Amazon S3 is about 3 times the cost of BitFolk's archive storage.
- Limited differential backups are possible though I'm yet to fully
understand how well that works. All I can say right now is that
a differential would use less, but I'm not sure really how much
less.
So, there would be "some charge" for the service itself and
definitely a charge at archive storage rates for the actual amount
of storage involved.
If this sounds like a service that you think you'd actually want,
please get in touch off-list and we'll see if we can develop it.
As it's more the BitFolk way for customers to do this sort of thing
themselves I don't really expect this to be widely desired, but I am
working on it for one customer anyway so if I can make it more
generally applicable then I will.
Thanks,
Andy
Hi,
I would find this useful. I would store such images at home on my cheap
disks. My usecase would be something like this:
1) trigger image via http/wget/curl/API
2) poll for it be completed
3) download image
4) delete image (via some api)
That would be very useful for me indeed and would not require
storage/archive at bitfolk. To restore an image, some sort of upload
would need to be available, but that is secondary for my case, since,
with the image, I can get to the filesystem and restore files if
necessary.
As per 'snapshot' - I am not certain about xen, but did a lot with
qemu/kvm. the "live migration" feature essentially gives you a
consistent view of disk+memory+devicestate that one can store. The
image(s) then are equivalent to starting a running machine. I wonder if
xen does something like that as well.
Also to ponder: It might be possible to send the VM to a "sleep" state
for something like 5s, to get a consistent snapshot of disk and memory.
Conrad
11:19 a.m.
Hi Conrad,
On Thu, Apr 18, 2024 at 11:50:59AM +0100, Conrad Wood via BitFolk Users wrote:
My usecase would be something like this:
1) trigger image via http/wget/curl/API
2) poll for it be completed
3) download image
4) delete image (via some api)
Okay so this is a slightly different set of concerns for us, because
snapshots are at the moment full LVM snapshots which consume disk
space and negatively affect IO performance, which means they need to
be fairly short-lived.
In the solution I am proposing for the one customer who asked,
BitFolk is going to be in control of how long the LVM snapshot stays
around for, so we can keep that burden acceptable. We archive it off
and delete the snapshot quite quickly.
There is always interest in disk snapshots, so possibly it is worth
looking into moving to some other snapshot technology. There is no
getting away from the fact that snapshots use storage though, so
either the storage used will have to be paid for or the lifetime of
the snapshots will have to be limited in some way.
To restore an image, some sort of upload would need to
be
available
Yes it is already possible to overwrite your VM's disk with your own
image from the Rescue VM. Of course if there was some disk imaging
or snapshots feature then it wouldn't be very hard to have some
button in the Panel that restores to it also.
Unless using one of the more advanced filesystems, taking a coherent
snapshot is something that is not easy for a customer to do
themselves, whereas restoring one is, so possibly it is worth
focusing on making just the former a feature, rather than the whole
bit about storing multiple incrementals.
with qemu/kvm. the "live migration" feature
essentially gives you
a consistent view of disk+memory+devicestate that one can store.
There is a save state command but before looking into this I wanted
to see what people's thoughts were.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
12:10 p.m.
On Thu, 2024-04-18 at 11:19 +0000, Andy Smith via BitFolk Users wrote:
Hi Conrad,
On Thu, Apr 18, 2024 at 11:50:59AM +0100, Conrad Wood via BitFolk
Users wrote:
Maybe I misunderstand. my thinking is, that the snapshot mechanism is
the same for backup as well as for my usecase.
Terminology might confuse me. I think of a "disk image" as a file
containing the data of a block-device byte-for-byte. I think of a
"snapshot" as a block-device as created by "lvcreate -s" (in case of
LVM). I understand the snapshot must be short-lived, but the disk-
image need not, right?
I thought the process is something like
1. LVM snapshot of VM backing store,
2. copy snapshot to disk image,
3. send disk-image to some archive-server,
4. drop snapshot on VM backing store.
The _only_ differences I see are:
- api to create, delete and trigger a disk image
and bitfolk charges me for disk-space used of the disk-image (by
hour/minute or as pre-allocated space)
thanks for the clarification on restore - makes sense to me.
Conrad
My usecase would be something like this:
1) trigger image via http/wget/curl/API
2) poll for it be completed
3) download image
4) delete image (via some api)
Okay so this is a slightly different set of concerns for us, because
snapshots are at the moment full LVM snapshots which consume disk
space and negatively affect IO performance, which means they need to
be fairly short-lived.
In the solution I am proposing for the one customer who asked,
BitFolk is going to be in control of how long the LVM snapshot stays
around for, so we can keep that burden acceptable. We archive it off
and delete the snapshot quite quickly.
There is always interest in disk snapshots, so possibly it is worth
looking into moving to some other snapshot technology. There is no
getting away from the fact that snapshots use storage though, so
either the storage used will have to be paid for or the lifetime of
the snapshots will have to be limited in some way. 
2:08 p.m.
On Thu, 18 Apr 2024, 22:10 Conrad Wood via BitFolk Users, <
users(a)mailman.bitfolk.com> wrote:
I thought the process is something like
1. LVM snapshot of VM backing store,
2. copy snapshot to disk image,
3. send disk-image to some archive-server,
4. drop snapshot on VM backing store.
https://github.com/mpalmer/lvmsync
M
12:31 p.m.
On Thu, Apr 18, 2024 at 10:39:51AM +0000, Andy Smith via BitFolk Users wrote:
Hi,
We had a customer ask about backups of images of their VM disk. This
is not currently a service that we offer.
Somewhere in the Bitfolk web site you mention snapshots - are these still
available, and what's the difference?
I'd be interested in something like this occasionally as a short term service:
1. take a full VM disk backup/snapshot
2. Perform a major system upgrade that could go show-stoppingly wrong
3. Restore snapshot if step 2 failed
Even a day or two would be long enough for this, but paying for a month
wouldn't be a problem.
This came up recently when I did an upgrade of Sympl (Virtual host
management system for Debian) simultaneously with Debian upgrade from 11
to 12. In the end I made several offsite safety backups and the upgrade
went OK, but a fiver's worth of peace of mind would have been good value.
--
Anahata
anahata(a)treewind.co.uk -+- http://www.treewind.co.uk
Home: 01535 501017 Mob: 07976 263827
19 Apr
19 Apr
7:18 p.m.
Hi,
On Thu, Apr 18, 2024 at 01:31:33PM +0100, Anahata via BitFolk Users wrote:
Somewhere in the Bitfolk web site you mention
snapshots - are these still
available, and what's the difference?
Sorry, I mentioned snapshots because being able to trigger them on
demand is a relatively frequent customer feature request - that has
not yet been implemented.
We do them on request by support ticket and don't like to keep them
around for long (days, OK; weeks, not so much).
Typically customers ask for a snapshot before they do a major
upgrade, or if they want to do their own coherent image of their
disk. They then let us know it can be deleted once they are done
- or rolled back if this was a failed upgrade.
I'd be interested in something like this
occasionally as a short term service:
1. take a full VM disk backup/snapshot
2. Perform a major system upgrade that could go show-stoppingly wrong
3. Restore snapshot if step 2 failed
Even a day or two would be long enough for this, but paying for a month
wouldn't be a problem.
This is where the tension exists between what Conrad was talking
about:
- A snapshot that gets archived off (takes a while) and then can be
downloaded or hosted by BitFolk indefinitely (for the archive
storage fee). Restoring from that would be quite slow.
To give some context: around one hour per 100GB to archive off,
and around 10 minutes per 100GB to restore back.
versus what you are talking about:
- A snapshot that is available after only a few seconds, but we do
not want it hanging around otherwise we'll need to charge for it
at SSD rates and possibly also a small per-usage charge for doing
it at all.
If the first scenario is something you could find useful and make do
with, then I think that'd be easier and less worrisome to provide.
In the meantime as I say we can do snapshots when asked for, and
provide those snapshots for you to download, or roll them back for
you.
This came up recently when I did an upgrade of Sympl
(Virtual host
management system for Debian) simultaneously with Debian upgrade from 11
to 12. In the end I made several offsite safety backups and the upgrade
went OK, but a fiver's worth of peace of mind would have been good value.
I sympathise and do similar myself, although what I do tend to find
with putting several large apps on one host or VM is that the
complex inter-dependencies make life a lot more difficult when it
comes to upgrading the underlying OS and perhaps finding that
everything is broken all at once.
It is maybe an argument for containers. I don't get on with Docker
much but as I have all these VM hosts I've managed to avoid Docker
mostly by making lots of VMs, which isn't an option open to
everyone. In another life I might easily decide to get to grips with
Docker more, or investigate a competing solution like LXD/Incus.
I guess what I am trying to say is that having a snapshot of the
whole lot isn't always that appealing as rolling it back feels like
a big defeat. Maybe finding a way to compartmentalise things more
such that each bit of them can be upgraded in isolation is something
that would make life easier.
It is for sure still a useful safety net though.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
246
days inactive
247
days old
6 comments
4 participants
participants (4)
-
Anahata -
Andy Smith -
Conrad Wood -
Matthew Walster