Hi,
On the evening of Saturday 26th February 2011 starting from about
1700Z we will be taking part in maintenance to completely re-rack
all servers in one suite. Goals of this maintenance:
- Turn the rack around 180 degrees to improve air flow based on the
location of cold air ingress from the under-floor air conditioning.
- Adjust rack posts to allow better air flow up through the rack.
- Re-rack servers in a way that doesn't impede air flow.
- Install new power metering equipment.
This is expected to take several hours. It's difficult to be more
specific because it depends on how fast we are able to work;
obviously we will be working as fast as is safely possible.
Customers on the following servers will be affected because these
servers are in the rack that we'll be working on:
- curacao
- kahlua
- obstler
- urquell
Additionally,
- faustino
which is currently in a different suite will be powered off and
moved. This outage will be much shorter - should be no more than 30
minutes maximum. This will be done after the other work is completed
so it's not possible to specify a time.
How to find out which server you are on:
https://bitfolk.com/customer_information.html#toc_3_Which_piece_of_actual_h…
We will be performing a clean shut down of all VPSes, followed by a
boot again when servers are re-racked.
The following BitFolk services will be unavailable during the
maintenance:
- The old primary DNS resolver, 212.13.194.71
- apt-cacher
- cacti
- tracker
- wiki
The resolver outage means that it's important that all customers
configure use of the new resolvers that were announced here:
http://lists.bitfolk.com/lurker/message/20110124.233915.2c058600.en.html
Failure to do this will lead to very poor network performance during
the outage, because your first resolver will not respond at all.
If you are otherwise unaffected by the maintenance and assuming you
configure the new resolvers, the absence of the above services
should not impact the operation of your VPS.
We can't alter the date/time of the maintenance, but if this timing
is disastrous for you then please let support(a)bitfolk.com know and
me may be able to accommodate a move of your VPS to a server that
will be unaffected.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
You may recall recently that there was a period of poor network
performance because the customer DNS resolver on 212.13.194.71 was
overloaded:
http://lists.bitfolk.com/lurker/message/20110102.221800.b90128dc.en.html
In that thread I promised to provision a new dedicated resolver to
avoid a re-occurrence of the issue.
Instead I took the opportunity to provision several new resolver
hosts in a cluster with fail over for the service IPs.
All customers should change their resolvers from:
212.13.194.71
212.13.194.96
to:
85.119.80.232
85.119.80.233
There's some maintenance coming up in February (details in a
separate email, shortly) which will take 212.13.194.71 offline for
several hours. It's therefore important that you change to using the
new resolvers before this time, otherwise you will experience severe
network performance problems.
If you have any questions please direct to users list or
support(a)bitfolk.com.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
In the last few months there's been an increase in the number of
notices of claimed infringement we have received from various
companies acting on behalf of rights owners such as Columbia
Pictures, CBS, etc..
A number of times this has been traced back to customers running TOR
exit nodes.
Assuming that you do not wish to trigger these allegations, I would
suggest that those operating TOR exit nodes at BitFolk might like to
consider whether they want to all people to be using BitTorrent
through their node.
Or indeed whether they want to be running an exit node at all (as
opposed to a routing node which only talks to other TOR nodes).
Some further information:
http://www.chrisbrunner.com/2006/07/09/why-you-shouldnt-run-bittorrent-over…
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
As of last night, the backups source IP and NFS host address which
were both formerly 212.13.194.71 have been changed to 85.119.80.241
[*].
Only a couple of customers appear to have restricted the IP that the
backups can be done from, and I'll be contacting them directly
today.
Those of you who access your backups over read-only NFS will need to
mount them from 85.119.80.241 instead of 212.13.194.71 (look in your
/etc/fstab).
Cheers,
Andy
[*] This is an IP from our new allocation:
http://is.gd/kmhx8
Despite looking nothing like the other one it is actually still
in the same facility. We will be trying to keep all BitFolk
infrastructure inside 85.119.80.0/23.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
faustino.bitfolk.com has a hard disk on its way to death and so I
will be replacing it tomorrow sometime between 17:30 and 18:30.
Rebuild will take a few hours after that.
I'm not expecting there to be any down time or disruption.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
Between about 23:39 and 23:46Z last night, 3rd January, multiple
servers became unreachable or suffered extremely poor networking
performance.
Servers particularly affected were:
barbar
kahlua
obstler
urquell
But all networking was affected to some extent, if only through
increased latency.
It appears to have been a denial of service attack of around
350Mbps / 250Kpps. Source IPs were spoofed and multiple destination
IPs were targeted.
I am continuing to work with our upstream network provider on this.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
You dont have to be illiterate to use the Internet, but it help's.
-- Mike Bristow
Hi,
Intermittently between approximately 16:00 and 21:00Z today (2nd
Jan), a host which runs one of BitFolk's caching DNS resolvers
(212.13.194.71) was overloaded, causing poor performance for many of
you.
The cause was recent changes in the data transfer monitoring
scripts as previously discussed and detailed in:
https://tools.bitfolk.com/redmine/issues/15
It seems that the more heavyweight method of estimating data
transfer is in some cases slow enough to still be running when the
next scheduled invocation of the script runs, causing both to run
slower, and the problem to spiral.
Since no hosts or critical services were actually reported down, we
weren't made aware of the issue as quickly as might have been
desired.
I have now made some modifications:
- It's now not possible for multiple copies of these scripts to run
at once.
- If system load is too high then the expensive checks will now be
skipped entirely.
There is also the fact that this host is simply doing too much - it
shouldn't be running these scripts in addition to being a resolver.
In the next few days I will be provisioning an additional resolver
and eventually this one will be retired.
Apologies for any disruption this may have caused for you.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
There's been a request for a read-only announcements list for
BitFolk, so I have decided to try this out.
I've created the "announce" list:
https://lists.bitfolk.com/mailman/listinfo/announce
In future any sort of announcement that I would normally send to the
"users" list will instead be directed to "announce", and
automatically copied to the "users" list.
The "announce" list will not accept any discussion, and posts
sent through it will have Reply-To set to the "users" list.
Therefore if you're happy with the content and level of traffic on
the "users" list you need not do anything, as all announcements will
continue to end up there.
If on the other hand you would prefer to only see announcements from
BitFolk and not any user issues, following comments or debate, you
may wish to subscribe to "announce" and remove yourself from
"users".
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting