Hi Andy,
On Sat, 1 Dec 2018 at 21:55, Andy Smith <andy(a)bitfolk.com> wrote:
Hi Luke,
On Sat, Dec 01, 2018 at 09:28:32PM +0000, Luke Taylor wrote:
Over the past week or so my Apache installation
has an issue where it
just.. stops responding and requests just time out
Is only apache affected? i.e. are other things on the VPS responsive
during this time?
Curently, only Apache from what I can tell (can SSH in just fine and other
services seem to be running as expected), VPS is performant with
practically no CPU load. The issues all started off earlier on this week
with MySQL server not responding, however dmesg showed that it was being
terminated by OOM killer caused by high memory usage by both MySQL and
Apache, so I tweaked their settings to limit memory usage which seemed to
solve that issue - not sure if related. In this case as apache is won't
respond I haven't noticed any mysql connection errors. Next time it happens
I'll try to connect to MySQL via command line.
until the server is restarted, 12 hours later
same thing happens,
lather rinse repeat.
When you say "the server is restarted", do you mean the apache
server, or the entire VPS?
Both yield the same result, however I was referring to "sudo service
apache2 restart".
Not sure how else to diagnose the issue :(
There is definitely a bug in Xen which was exposed by the most
recent security patches; that bug most probably only affects 64-bit
guest and can be avoided by upgrading your kernel.
As it involves memory corruption it's hard to predict how that would
manifest itself. If there is a kernel upgrade available then I
recommend you take it.
Do you currently have a file
/sys/devices/system/cpu/vulnerabilities/l1tf ? Is there a kernel
upgrade available? If you upgrade, does that file appear?
Yes, I do have that file containing "Mitigation: PTE Inversion". I only
just recently upgraded kernel to 4.4.0-139-generic and installed all
available updates a few days back.
As far as I am aware, on Ubuntu 16.04 package
linux-image-generic
version 4.4.0.133.139 contains the L1TF fix.
If that doesn't help, I can move your VPS to a test host where the
problematic patch is disabled, and that would then rule out whether
it is that. If your problem persists while your VPS is on that host
then it would be something I'm currently unaware of and would be
suspicious of something inside your VPS. But the timing is
suspicious.
If Apache dies again I'll do a little more investigation and contact you
off list to discuss moving to a test host.
Thanks for your help.
Cheers,
Luke