Hello, On Thu, Apr 23, 2020 at 05:15:03PM +0000, Andy Smith wrote: Some updates and some questions for customers about this. Firstly, there WAS a bug in Xen's LZ4 compression but that was only relevant for direct kernel load. That's when the guest's kernel exists as a file on the host's filesystem and is referenced in the config file for the guest. That's not how we boot guests at BitFolk, so that's not relevant; it isn't the source of the problem here. There's a detailed description of the boot process of a BitFolk guest here: https://tools.bitfolk.com/wiki/Booting The short version is: BitFolk has a grub binary that is compiled as a Xen PV guest kernel. This grub binary acts as your kernel, so it's the first thing a guest does when it's booted. It then looks for grub configuration in your block devices and if it finds it then it loads it. From your configuration it should find the correct actual Linux kernel and initramfs and chain load to them at which point booting proceeds normally. I don't yet fully understand how or why grub as a Xen PV kernel behaves differently to normal grub, but it seems like the Xen PV part of grub requires support for every kind of kernel compression, and no one has yet implemented LZ4 decompression for it. The "not a Xen kernel" error message comes from grub. My enquiries about this on the grub-devel mailing list have so far met with silence: https://lists.gnu.org/archive/html/grub-devel/2020-04/msg00198.html At the moment it doesn't look like anyone is interested in adding LZ4 kernel decompression to grub's Xen PV target. If you are capable of doing so (or know someone who is) but would need financial compensation to do so, please contact me directly; we may be able to work something out. Something that was pointed out to me is that when grub is compiled as a PVH guest it does handle LZ4-compressed kernels. I do want to move from Xen PV mode to Xen PVH mode but am not yet quite ready to do so. It does provide some reassurance though that there is a future where grub can continue to boot these things. A current workaround is to decompress the kernel. This is described here: https://tools.bitfolk.com/wiki/Ubuntu#Decompress_your_kernel Someone on the xen-users list pointed out that an apt hook can be used to automatically decompress kernels at kernel upgrade time: https://lists.xenproject.org/archives/html/xen-users/2020-05/msg00018.html The basic concept seems good though I do think their supplied script needs some security hardening (see my reply). My question to you is, if I made the BitFolk Xen Shell installer for Ubuntu 20.04 install such an apt hook so that it automatically decompressed LZ4 kernels, would that be an acceptable state to consider Ubuntu 20.04 officially supported at BitFolk? What would you be thinking the first time you did a kernel upgrade and saw apt call an extra script to decompress the kernel? At the moment Ubuntu 20.04 is not listed on the BitFolk web site as available, nor is there an installer for it in the Xen Shell, because of the need to decompress its kernel. It is only possible to install it by upgrade from an earlier release or by use of the Rescue VM - and then you need to take care of the kernel yourself. Put another way, is automating the decompression of LZ4 kernels at kernel upgrade time a good enough solution to allow Ubuntu 20.04 to be listed as available on the web site and in the Xen Shell? If the consensus is "no" then I think Ubuntu 20.04 will have to remain as not officially supported here until either there's a pvgrub that supports LZ4 kernels or BitFolk introduces PVH mode. Cheers, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting

On Thu, Apr 23, 2020 at 05:15:03PM +0000, Andy Smith wrote: Ubuntu 20.04 is now supported for new installs both as a new order or by self-install¹. This has required a small workaround be installed by us inside the guest, as detailed here: https://tools.bitfolk.com/wiki/Ubuntu#No_support_for_LZ4-compressed_kernels… Therefore if you intend to upgrade to 20.04 from 18.04 you will have to install that workaround yourself otherwise it won't boot afterwards. If you're looking at this because your VM now doesn't boot, no big drama: just boot the 18.04 kernel and install that workaround. Also please note that Ubuntu are only supporting 64-bit now so those of you on 32-bit will need a complete reinstall. Or you could chance a completely unsupported cross-grade. In theory 18.04 remains supported by Ubuntu until April 2023. Cheers, Andy ¹ https://tools.bitfolk.com/wiki/Using_the_self-serve_net_installer -- https://bitfolk.com/ -- No-nonsense VPS hosting _______________________________________________ announce mailing list announce(a)lists.bitfolk.com https://lists.bitfolk.com/mailman/listinfo/announce