Hi,
If your VPS lives on snaps.bitfolk.com then you should have by now
received direct email notifying you of planned maintenance on this
server on Saturday 16th April.
If you don't know what host your VPS is on, please see:
https://panel.bitfolk.com/account/
which lists it.
If it's on snaps.bitfolk.com and you haven't received the
notification you should check your spam folders etc.
If your VPS is not on snaps.bitfolk.com then you will be unaffected
and should not notice any disruption.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
Effective immediately the monthly data transfer quota has been
increased to 1,000GB. For those of you on 95th percentile it's been
increased to 2Mbps.
Those of you that pay for additional monthly data transfer may not
now need to do so. If that is the case please contact
support(a)bitfolk.com about dropping back down to the new default.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
I'm pleased to say that the hardware upgrades were completed on 1st
April 2016. That means every single BitFolk customer was moved from
an older piece of hardware to a new one, and in the process gained
some extra memory and an extra virtual CPU core.
The swap provision was standardised to a 1GiB xvdb device with a
single swap partition on it. If you previously had a smaller swap
device (480M customers had 480M swap) or no swap device at all
(older accounts had a swap *file* on their root filesystem) then you
may wish to check if there is some extra swap space you could be
using:
https://tools.bitfolk.com/wiki/Hardware_refresh,_2015-2016#Making_use_of_th…
The higher specs of the new hardware allowed the total server count
to reduce to 64% of what it was. Also, more importantly, the power
usage is now at approximately 24% of what it was before. This should
result in a big saving in hosting fees which are entirely scaled by
power consumption, and represent BitFolk's biggest outgoing.
I will be taking a couple of months to see how things balance out
financially and to allow for some other developments out of my
control¹, but then I will be seeing if I can reduce prices a little
and/or introduce a cheaper 512M memory VPS package.
I am now able to focus on some other areas that need attention.
Those of you who have experienced refused connections to the
apt-cacher² will be pleased to know that this has now been put
behind a high availability proxy so those should be a thing of the
past.
Additionally I have moved the spamd³ service behind there and will
soon do the same with the entropy⁴ service.
These services had previously either not been redundant at all
(apt-cacher) or had merely been multiple hosts behind one proxy
(spamd, entropy) thus having the proxy as a single point of failure.
I've put some work into deploying a new pair of proxies with a VRRP
floating IP (and IPv6) so these services should also now withstand
the loss of one of the proxies.
As a side benefit that now means that we could sell you a floating
IP service, where you could run a command (or, more realistically,
have keepalived or some other monitoring software run a command) to
float an IPv4 and an IPv6 /64 between two or more of your VPSes. Let
me know if you would be interested in that.
Other areas I know are desperately in need of attention:
- Need to look into moving to pvgrub as a booting method.
Current use of pygrub really mandates GRUB 1.x-style
/boot/grub/menu.lst while most distributions are moving to GRUB
2.x /boot/grub/grub.cfg. The lengths we need to go to in order to
force support of older GRUB are getting to be too much, but I
would rather skip over pygrub enhancements that support GRUB 2.x
and move straight to pvgrub.
For those unaware of the difference, pygrub is a Python script that
runs as root in BitFolk's host machines in order to parse boot
instructions from your block devices. This has security
implications as you can probably imagine.
pvgrub on the other hand is a Xen-compatible build of the actual GRUB
that is booted as a virtual machine as you, and then that tries to
boot. This is much safer, and much more reliable since it
actually *is* the real GRUB bootloader rather than a script that
tries to mimim its behaviour.
- Need to modernise Nagios monitoring.
BitFolk's Nagios is now really too long in the tooth, is starting
to have problems doing modern TLS connections for example.
I am looking at replacing it with Icinga, and also hope for
slightly more customer control over what is monitored (i.e. giving
you access to some form of interface that can make changes,
instead of requesting all changes by support ticket). Initially it
just has to work though, because I rely on it for my own
monitoring.
- Support for European bank accounts in GoCardless (Direct Debit).
I know there are few customers with Euro bank accounts who are
grudgingly using PayPal subscription only because BitFolk's
GoCardless integration doesn't yet support them, event hough
GoCardless itself does.
- Support for recurring payments with Stripe (credit/debit cards).
BitFolk payments through Stripe are currently only possible as
one-off payments and I know there are some who would prefer to
have Stripe store their card details and let BitFolk just take a
regular scheduled payment.
- Everything else that is in https://tools.bitfolk.com/tracker/ :)
There's far more things to work on than there is time to do it in,
but there should be some progress at least.
Thanks for your patience! Comments and insight on future direction
are welcome.
Cheers,
Andy
¹ Our colo provider is soon going to be changing their method of
power monitoring and will likely put prices up at the same time
² https://tools.bitfolk.com/wiki/Apt-cacher
³ https://bitfolk.com/customer_information.html#toc_2_SpamAssassin
⁴ https://tools.bitfolk.com/wiki/Entropy#BitFolk.27s_entropy_service
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
If you just have the one IPv4 address on your VPS then you can
safely stop reading as nothing is going to change for you.
As for the 11% of paying customers who do have more than one IPv4
address, read on…
For a long time now I have been unsatisfied with our policies for
granting requests for additional IPv4 addresses.
For the last 9 years we've done what was considered the proper thing
in a world where available IPv4 addresses were running low: we
haven't charged for additional IPv4 addresses, we've only asked
customers to technically justify their requests.
That had a number of problems:
- It's often very difficult to pin a customer down on what exactly
it is they're doing. The ideal scenario where one network
engineer talks to another about requirements is rarely the case
here.
This very often left me in a position where I wasn't convinced
that use of extra IPv4 addresses was technically required, but
it's what the customer wanted and the sale depended upon it.
There is no sane administrative fee that could be charged that
would cover that amount of work.
- Once a request was granted it has been very difficult to audit.
There has been no incentive for customers to return IPv4 addresses
that they are no longer using.
- I don't know of another company in this same space that is
bothering to require technical justification for IPv4 use.
- We're no longer in a world where IPv4 is running out. IPv4 has
already run out.
The point of recording why any given IPv4 address was in use was
for the case where BitFolk had to go back to RIPE to ask for more
IPv4. Those records would have been used to prove that the IPv4
allocation that BitFolk already has really is all used up. BitFolk
won't be going back to RIPE because there's no IPv4 left to go
back for. We just have to manage what we have.
So, from Monday 14th March we're going to start charging for each
IPv4 address after the first one. The charge will be £0.50+VAT per
IPv4 per month (£1.40+VAT per quarter or £5.00+VAT per year).
This will take effect immediately for new signups, but existing
customers will only see it on invoices falling due on or after 14th
March. Those with PayPal subscriptions will have the subscriptions
altered, and those with Direct Debit authorisation will see the
authorisation cancelled (they're for a fixed amount) so will need to
re-authorise.
If you're paying monthly or quarterly and your next bill falls due
some time between now and March 14th you could consider switching to
yearly, as you then won't be charged for this until that year runs
out. You'd therefore save £6.00+VAT per IPv4 per year (£0.50 x 12
monthly payments).
I should point out that (under either this or the previous regime)
BitFolk is not in any realistic danger of running out of IPv4
addresses; this is just about managing what we already have in a
better, fairer and more transparent way which is also more in line
with customer expectations.
IPv6 /64s and /56s remain entirely free and there are no plans to
change that.
If there is something I have forgotten to cover, please do let us
know, either by replying here to to support(a)bitfolk.com.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
"It is I, Simon Quinlank. The chief conductor on the bus that is called
hobby." — Simon Quinlank
Hi,
For those completely committing to containers, I thought I'd just
point out that CoreOS appears to work fine at BitFolk.
Here's some instructions based on a successful test of mine:
https://tools.bitfolk.com/wiki/Installing_CoreOS
If you go further with it then please feel free to edit.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
If you do not operate a Tor exit node at BitFolk then this email
will be of little interest to you.
If you do, then read on…
A number of BitFolk customers operate Tor exit nodes. These generate
a constant stream of abuse reports because quite a lot of abusive
activity is conducted through Tor.
Most of the reports are automated and are informational in nature.
They expect no reply and are therefore ignored on the basis that
"it's a Tor exit node ¯\_(ツ)_/¯".
A minority are real people and in this case BitFolk points out that
the host is a Tor exit node and so neither BitFolk nor BitFolk's
customer has much control of the traffic that goes through it.
A minority of this minority of complainants are for one reason or
another not satisfied with that answer and in those rare cases we
expect the customer to correspond with the complainant.
A recent case was like this and has prompted some updates to our Tor
exit node policy. The updated policy can be found here:
https://tools.bitfolk.com/wiki/Running_a_Tor_node
The changes:
https://tools.bitfolk.com/w/index.php?title=Running_a_Tor_node&action=histo…
Briefly:
- We state clearly that you must use a dedicated IP for your Tor
node. Previously we've insisted customers use a dedicated IP from
the first time an abuse report came in, so this is just
formalising that.
- The correct Tor exit IP must appear in the Tor project list of
exit nodes, for the benefit of remote sites that are using those
lists to construct filters.
One reason why this recent abuse report escalated was because the
customer's exit node was misconfigured to list the VPS's *other*
IP address, so the complainant was seeing abusive activity
(Wordpress brute force in this case) coming from something that
wasn't listed as a Tor node.
- Clarify that when we ask for a response to an abuse report we
expect it within 72 hours.
- Warn that although we prefer correspondence with the complainant
to go through our ticket tracker, if the complainant insists then
you must give them a direct email address that reaches you.
I know that sucks, but it is unfortunately the price that must be
paid for running an abuse magnet like a Tor node: the abuse reports
*must* be answered. In 8 years of having Tor exit nodes at BitFolk
this is the first time that a complainant has insisted upon
corresponding directly with the node operator.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hello,
I realised yesterday after receiving a ticket where a customer asked
what was going on with the VPS specs, that I had not actually posted
anything on this announce list about the hardware refresh and
related VPS spec upgrades that are ongoing.
I have to remedy that now, because to a customer who knows nothing
of this upgrade process it just looks like we are selling better
spec VPSes to new customers and leaving existing customers to pay
inflated prices indefinitely. That means many of you are now going
to receive the same information you already know, so sorry about
that.
Back in July I said that VPS spec upgrades were coming soon:
http://lists.bitfolk.com/lurker/message/20150713.213922.6b419596.en.html
There was more initial delay than I would have liked:
http://lists.bitfolk.com/lurker/message/20150802.112450.2c7d67f5.en.html
…but since September things have been proceeding fairly rapidly and
at this point just over 36% of the customer base has been upgraded.
I made a wiki page which I update regularly:
https://tools.bitfolk.com/wiki/Hardware_refresh,_2015-2016
At the moment because of external pressures on which machines must
be removed from colo first we've had to be more selective about
which customers we offered upgrades to¹, but I think we are quite
close (weeks away) to the point where we can just ask the whole
remaining customer base who wants to be upgraded, and proceed mostly
in order of response.
A big down side of doing things this way is that we have many months
where existing customers are stuck paying the same price for a lower
spec than that which new customers can get, and in many cases
hitting a renewal point and paying again for the lower spec.
We would love to have been able to just cut over to higher specs in
one go. As the majority of customers pick the smallest plan, and the
upgrade from 480M to 1,024M is more than double, that would require
us to have about double the RAM available immediately and it just
wasn't feasible.
We have been able to provide pro rata service credit for those
finding their new spec more than they need and requesting a
downgrade mid-contract.
If you have any questions, please have a read of the wiki link above
and if not answered there then do let us know.
Cheers,
Andy
¹ In fact it is more like which customers we force upgrades upon,
because the response rate is only about 20%, yet we still need the
older machines cleared of customers.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
There's been quite a run of gmail believing that all email from our
Request Tracker was spam and automatically filing it into your spam
folder.
So, if you use gmail please could you have a look in your spam
folder and see if there is any bitfolk email in there, and if so
please do mark it as "not spam".
Meanwhile I have added DKIM, SPF and DMARC to emails from
support(a)rt.bitfolk.com so hopefully this is less likely to happen in
future.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
There's a serious security bug in the Xen hypervisor currently under
embargo until 29th October.
The following of BitFolk's hosts are affected:
bellini.bitfolk.comdunkel.bitfolk.compresident.bitfolk.comsnaps.bitfolk.comsol.bitfolk.com
Some time before the 29th these hosts are going to require their
hypervisors to be upgraded and that upgrade will require the host to
be rebooted, so all VPSes on those hosts will also be shut down and
booted again.
I've not yet fixed exactly when this will be done. Most likely in
the early hours of the morning UK time across three nights close to
the end of the embargo.
To complicate matters further, as you're probably aware we're at the
moment in the middle of migrating customers to new hardware and
upgrading them in the process:
https://tools.bitfolk.com/wiki/Hardware_refresh,_2015-2016http://lists.bitfolk.com/lurker/message/20150927.060438.69cadb3d.en.html
One new host, snaps, has already been deployed and is now at full
capacity, so I'm in the process of deploying the next one now. That
means that the next one can be patched before customers are put on
it, and so the next batch of upgrades will side-step the need for
this other reboot.
So, if:
- You've already been contacted about migrating+upgrading your VPS
but you have so far chosen not to respond, and
- Your VPS is on one of the above listed hosts
then you may wish to consider going back to that email and agreeing
to the migration+upgrade as soon as possible, as otherwise you are
most likely going to experience some down time for the security
patch and ALSO some down time for your eventual forced
migration+upgrade.
I'll follow up as soon as I can with dates/times the patching and
reboots will take place.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Hi,
By now you have probably been made aware of a security deficiency in
the design of SSL 3.0 which has been dubbed "POODLE". Here's some
more info:
http://googleonlinesecurity.blogspot.co.uk/2014/10/this-poodle-bites-exploi…
I am writing to you because, unless this script is flawed:
https://gist.github.com/bitfolk/18e8f48ebe937e802967
then there are over 150 customer IPs at BitFolk that are still
supporting SSLv3 on port 443.
I don't intend to open tickets with individual customers and nag
until this is fixed, because it's very time-consuming to do that.
To check if your server needs reconfiguring:
https://www.tinfoilsecurity.com/poodle
To disable SSLv3 on Apache newer than 2.2:
Add "-SSLv3" to the end of the "SSLProtocol" line which can
normally be found in /etc/apache2/mods-available/ssl.conf on
Debian and Ubuntu.
On Apache 2.2 or older:
You'll need to use "SSLProtocol TLSv1"
Nginx:
Make sure that the "ssl_protocols" line does not contain the
string "SSLv3". e.g.:
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
is good.
Cheers,
Andy
--
http://bitfolk.com/ -- No-nonsense VPS hosting