When installing a new kernel (CentOS in this case) I've always done a full shutdown and boot from the console following the advice on the Customer Information page. However, this time I accidentally did a "shutdown -r" but it did actually pick up the new kernel.