Hello,
This is a long email about BitFolk's free backups service. If you
don't use that then you probably want to skip it.
In mid-2010 I asked what you thought should be done about the
backups when a customer accidentally backs up up some large amount
of data that they didn't mean to:
http://lists.bitfolk.com/lurker/message/20100528.061800.6814d4ed.en.html
The TL;DR version of that thread is:
- I don't want customers to have write access to their backups
because
a) It means the backups can't be trusted after a compromise;
b) If they're not writable they can't be trashed accidentally in
some other way;
c) I don't want backup space used for general purpose storage.
- The default backup schedule keeps things for 6 months. That means
that once something is backed up, it is there for 6 months in
backups even if you delete the file from your VPS's disk (or
exclude from backups) as soon as you notice.
When someone blows all their backup quota on some data they didn't
mean to back up[1], they probably don't want to have to buy more
disk space in order to keep their backups working. I imagine they
would rather have the accidentally backed up data deleted. But
they can't because they don't have write access.
- I don't want BitFolk support to be poking around in people's
backup data trying to delete things. The potential for mistakes
and misunderstandings is too great.
- I proposed one method of dealing with the situation which would be
that BF support would delete entire iterations of backups until
the usage went below quota again. If you're quick it might only be
the last 4 hours, or the last day, but this is still rather harsh.
- Other proposals were based around BitFolk developing tools to do
limited file operations on the backups. This could be done, but it
seems like a lot of work for a relatively rare situation. I'm
still hoping for a better idea.
Since that thread took place we've had a bit more experience with
the issue.
In most cases as long as we could see that the disk usage had gone
down and would eventually drop below the quota once some of it
cycled off the end, it was easier to just not enforce quota.
In light of this, one new proposal that I have is that the disk
space for backups should be a lot cheaper. BitFolk has an abundance
of spare disk space that isn't selling. I don't think I want to
reduce the cost of general purpose disk space (it's already
quite low), but it seems only fair that disk space for backups --
which is of limited use -- should be cheaper.
Hopefully this will encourage larger quotas for backups and make
accidental backups a lesser issue, since it will impact you less to
just have them cycle off the end.
My next proposal is that it might be an acceptable risk to allow
customers, via panel.bitfolk.com, to temporarily grant themselves
write access to their backups. I'm suggesting it remains writable
for just a few hours and then it automatically goes read-only again.
Since that would be logged it would be possible to still be
confident about the sanctity of the backups after a compromise.
Do these proposals sound reasonable, or at least better than what we
have now?
Does anyone have any other ideas?
Cheers,
Andy
[1] At this point it is tempting to ask, "why does the backup
facility even let a customer back up more data than their quota
allows?"
The answer is that it is relatively expensive to calculate how
much disk space has actually been used for backups, so it's not
possible to do so at every backup run.
However, even if it were possible, it doesn't really get us
anywhere with this problem: once you back up a huge amount of
stuff that you didn't mean to then you're still in this
predicament whether that is 150% of your quota, 100% or 95%. You
don't want it there, using up space for 6 months.
So let's leave that issue for another day. It is documented:
https://tools.bitfolk.com/redmine/issues/38
Hello,
If you have two VPSes, it's relatively simple to have an IP address
that flips between them.
So say for example you have vps-a on IP 212.13.194.32 and vps-b on
212.13.195.43, and you also have a third IP address 212.13.195.20
that starts off routed to vps-a. You put a web server listening on
212.13.195.20 which hosts some important pictures of your cat which
require high availability.
Some mishap befalls your vps-a, like your web server dies or the
actual hardware it's on dies or something. vps-b notices that vps-a
isn't responding any more and so does something to flip
212.13.195.20 over to itself and continue serving your cat gallery
with many 9's of availability. None of your cat's admirers notice
that one of your VPSes died.
Right now if you buy two VPSes you can already do all of that except
flip the IP address over between them. You need BitFolk's help to
flip an IP address from one VPS to another because normally you
can't ARP for an IP address you don't have routed to you. If you did
bring up 212.13.195.20 on vps-b, all the packets would still end up
going to vps-a.
We can make it so that you can do something to flip that IP to one of
your other VPSes. You take care of all the other details. There
would most likely be no charge for this beyond you having to have
two VPSes.
Is there any interest?
Cheers,
Andy
PS I don't want to get into the idea of failing over an entire VPS
at the moment, nor of automatically migrating a single VPS
between hosts. Both are certainly doable, but the complications
of shared storage and/or reserving unsold resources on other
hosts so that VPSes can be failed over to them are too much. Use
Amazon EC2 or something?
But if you have 2+ VPSes, flipping IP addresses around between
them is quite simple.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
"The electric guitar - like making love - is much improved by a little
feedback, completely ruined by too much." -- The League Against Tedium
Hi all,
My fastcgi process has just stopped twice recently, and I am having
trouble finding out why.
History
Recent upgraded VPS to Ubuntu 10.4. from 8.4 - apparently without
problem.
Running nginx with php via init-fastcgi script
nginx was not changed in the upgrade, neither was the
init-fastcgi script.
Oddities noticed.
/etc/init.d/init-fastcgi stop took 30 seconds but gave no
errors. I could then start the process fine without error.
I can find nothing in the logs that is suspicious, for either time.
This morning Baiduspider was spidering a site running an old version of
Word Press at about the time the problem started, but did not
get a 504 error.
I wish I had run ps -ax to see if the processes were in the machine. I
ran it, and nginx was fine, but did not check the php side. (still asleep).
Has anyone any ideas about what might be happening, what could cause it,
or indeed anything helpful to stopping it happening again?
Thanks
Ian
Hello,
As of last night, the backups source IP and NFS host address which
were both formerly 212.13.194.71 have been changed to 85.119.80.241
[*].
Only a couple of customers appear to have restricted the IP that the
backups can be done from, and I'll be contacting them directly
today.
Those of you who access your backups over read-only NFS will need to
mount them from 85.119.80.241 instead of 212.13.194.71 (look in your
/etc/fstab).
Cheers,
Andy
[*] This is an IP from our new allocation:
http://is.gd/kmhx8
Despite looking nothing like the other one it is actually still
in the same facility. We will be trying to keep all BitFolk
infrastructure inside 85.119.80.0/23.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hello,
This weekend I'm off to OggCamp - http://oggcamp.org/
- if you're there too then do say hello! :)
Whilst there will be support cover and I should be available in
emergencies, this is also awkward timing because the next Ubuntu LTS
release is due for tomorrow.
Firstly I would say if you do have a support issue you need doing
before next week you should put it in now.
Secondly, if you are considering upgrading an Ubuntu VPS to 10.04 I
would recommend waiting until after the weekend. I haven't tried it
yet myself, though I do hope to do so. I think it should work, but
there might be some gotchas, and a lot of people breaking their
VPSes at once will stretch things this weekend. Plus if it does
break all I'll likely be able to do is put you back to 8.04 or
Debian.
Don't forget that I can snapshot your filesystem before you do a
major upgrade so if it all goes wrong we can roll it back to before
you started fairly easily. But you have to ask first.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
faustino.bitfolk.com has a hard disk on its way to death and so I
will be replacing it tomorrow sometime between 17:30 and 18:30.
Rebuild will take a few hours after that.
I'm not expecting there to be any down time or disruption.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
Hi,
Is anyone going to FOSDEM this year?
http://fosdem.org/2011/
I've just booked my tickets so perhaps I'll see you there?
Regards,
@ndy
--
andyjpb(a)ashurst.eu.org
http://www.ashurst.eu.org/
0x7EBA75FF
I'm also on Faustino and experienced the same thing, websites and
mailserver completely down. Things seem to be back to normal now though.
-----Original Message-----
From: users-request(a)lists.bitfolk.com
Reply-to: users(a)lists.bitfolk.com
To: users(a)lists.bitfolk.com
Subject: users Digest, Vol 27, Issue 3
Date: Tue, 04 Jan 2011 10:48:52 +0000
Seems as though it could be happening again. My other VPS (mailserver)
on
faustino is now unreachable. Can't even ping it
Keith
Hello,
Between about 23:39 and 23:46Z last night, 3rd January, multiple
servers became unreachable or suffered extremely poor networking
performance.
Servers particularly affected were:
barbar
kahlua
obstler
urquell
But all networking was affected to some extent, if only through
increased latency.
It appears to have been a denial of service attack of around
350Mbps / 250Kpps. Source IPs were spoofed and multiple destination
IPs were targeted.
I am continuing to work with our upstream network provider on this.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
You dont have to be illiterate to use the Internet, but it help's.
-- Mike Bristow
_______________________________________________
announce mailing list
announce(a)lists.bitfolk.com
https://lists.bitfolk.com/mailman/listinfo/announce
