2010/1/21 Nigel Rantor <wiggly(a)wiggly.org> Thanks Nigel. Note - ntpdate is happy to adjust this - but dovecot thinks this is bad and kills itself to prevent problems ( http://wiki.dovecot.org/TimeMovedBackwards). This large time delta on boot is a big change from Debian Etch. I am not running NTP - just ntpdate (cron - set clock once a day). Maybe I shouldn't. Cheers, -- Alastair Sherringham

Alastair Sherringham wrote: Yeah, I understand, as long as the ntp rc script starts up before dovecot then by the time dovecot is up and running things should be fine. Lots of things get unhappy if the time apparently changes underneath them. Did you have ntpd running on that machine? Either way, I'd still recommend ntpdate/ntp. For example, it also takes acre of situations where, for example, a desktop machine's battery has died and it is no longer keeping time when powered down. Ah, yes, that might do it. I would suggest you run ntpdate once only on bootup to get the time correct so that ntp can then keep your clock from slewing too far out of alignment from then on. n

Hi Alasdair, On Thu, Jan 21, 2010 at 04:14:13PM +0000, Alastair Sherringham wrote: I'm sorry, I don't know why the host comes up with a large time delta. I'll ask upstream and see if I can find out. On a normal physical machine the host would take its system time from the hardware clock, and most distributions set the hardware clock from the system time when they shut down. On a Xen domU using paravirtualization -- the current situation at BitFolk -- there isn't any hardware clock. That's why there's a message about failing to read the hardware clock during boot up. That should be harmless though, because the host machine should provide the domU with the correct time. The time is correct on all the hosts; this is monitored. As an aside, we can monitor your clock as well if you run an ntpd. Just drop an email to support(a)bitfolk.com; monitoring of things with Nagios is free. I would recommend that you do run ntpd because ntpdate will skew the clock backwards if it needs to, and as you've seen many applications don't like that. ntpdate at (re)boot is a good idea. Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting

On 21 January 2010 19:22, Alastair Sherringham <sherringham(a)gmail.com> wrote: Sorry for following up so late. More information ... To re-cap, I have had problems where the clock on my VP is 30 mins in the future on boot. This behaviour started after a dist-upgrade to Debian Lenny a few weeks ago. Correcting this (ntpdate,ntpd) causes dovecot to die, and since Dovecot is what does AUTH for Postfix, both Dovecot and Postfix are useless without a manual restart. Linux calliope 2.6.26-2-xen-686 #1 SMP Wed Feb 10 11:28:19 UTC 2010 i686 GNU/Linux This still happens and I have been testing all morning - rebooting the VPS numerous times ... ntpd wasn't helping ntpdate wasn't helping What now happens is that *sometimes* there is no clock skew. Sometimes there is. Also - disabling ntpd (update-rc.d) and reinstalling ntpdate has changed behaviour. I now have ntpdate installed again - there is an /etc/cron.daily/ntpdate containing : ntpdate -s ntp0.sfo.bitfolk.com ntp0.lon.bitfolk.com 0.debian.pool.ntp.org 1.pool.ntp.org There are no other crontab ntp calls (or @reboot jobs). However, now, ntpdate *fixes* this before Dovecot dies, so I am happier. ntpdate[1036]: step time server 194.29.130.252 offset -2089.601967 sec It is always ~30 mins. I sit and reboot - login and check - reboot ... OK no skew - dove OK BAD - skew FIXED - dove OK BAD - skew FIXED - dove OK OK no skew - dove OK OK no skew - dove OK (fsck ..) BAD - skew FIXED - dove OK OK no skew - dove OK BAD - skew FIXED - dove OK Odd? Cheers, -- Alastair Sherringham

Andy Smith wrote: I think I have exactly the same problem, though I don't run dovecot, hopefully Alastair doesn't mind me jumping in on his thread. I don't seem to have clocksource=jiffies, and independent wallclock seems to be 1: joseph@button:~$ cat /boot/grub/menu.lst | grep -i jif joseph@button:~$ cat /proc/sys/xen/independent_wallclock 1 I also infrequently get these messages on the console, no idea if they're related or not: [22433.110835] clocksource/0: Time went backwards: ret=6a3118eecc83d3 delta=-14067480 shadow=6a3118c913a34f offset=268faf29 Joseph

Andy Smith wrote: Thanks, I've made that change. I'm not entirely sure I am seeing the same thing as Alistair. At least one of the things I see is two 'Unable to set System clock' messages whilst the VPS is booting - though I now see they're happening before independent_wallclock gets set: Waiting for /dev to be fully populated...done. Setting the system clock. Unable to set System Clock to: Tue Feb 16 19:47:36 UTC 2010 (warning). Activating swap:swapon on /dev/xvda2 [ 13.620255] Adding 262136k swap on /dev/xvda2. Priority:-1 extents:1 across:262136k . Will now check root file system:fsck 1.41.3 (12-Oct-2008) [/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a -C0 /dev/xvda1 /: clean, 43871/2293760 files, 1914874/4587520 blocks . [ 14.529561] EXT3 FS on xvda1, internal journal Setting the system clock. Unable to set System Clock to: Tue Feb 16 19:47:40 UTC 2010 (warning). Cleaning up ifupdown.... Will now check all file systems. fsck 1.41.3 (12-Oct-2008) Checking all file systems. Done checking file systems. A log is being saved in /var/log/fsck/checkfs if that location is writable.. Setting kernel variables (/etc/sysctl.conf)...kernel.core_pattern = /var/tmp/cores/core.%e.%p xen.independent_wallclock = 1 done. Will now mount local filesystems:. The odd thing is the time was 19:16ish when this happened; I have no idea where the 19:47 is coming from, but the ~30 minute difference does seem consistent. Joseph

On 16 February 2010 19:18, Joseph Heenan <joseph(a)heenan.me.uk> wrote: Look in /var/log/syslog - from a boot until the end. Mine starts 30 mins ahead until near the end, then jumps forward (assuming a "bad" boot) : e.g. see attached for an example - notice time jump. I'll try Andy's jiffies, wallclock and ntpd suggestion ... reboot a few times again :-) -- Alastair Sherringham

Alastair Sherringham wrote: ah, yes, I do see exactly the same kind of jump, even with 'jiffies' on the command line: Feb 16 19:47:45 button kernel: [ 20.057553] lo: Disabled Privacy Extensions Feb 16 19:47:49 button postfix/master[1239]: daemon started -- version 2.5.5, configuration /etc/postfix Feb 16 19:12:58 button ntpdate[1030]: step time server 194.238.48.3 offset -2091.700794 sec Joseph

On 16 February 2010 20:11, Alastair Sherringham <sherringham(a)gmail.com> wrote: Here's the syslog from the bad reboot. -- Alastair Sherringham

Hi Alastair, On Tue, Feb 16, 2010 at 08:16:36PM +0000, Alastair Sherringham wrote: I appreciate that time went backwards here which upsets dovecot, but.. You're now synced to an NTP server and have the correct time. Does it later lose sync? Is it possible that what you describe as an "OK" boot had independent_wallclock set to 0 so you were unable to set the time, it remained wrong, and dovecot did not complain? Personally I would rather have the right time and make dovecot start after ntpdate. Having it boot with the wrong time and then have ntpdate/ntp adjust it is the best I can offer I'm afraid, because that's the best I've been able to achieve. Just in case you might be thinking that the host you're on has the wrong time: kahlua$ date && ntpq -np Tue Feb 16 20:47:13 UTC 2010 remote refid st t when poll reach delay offset jitter ============================================================================== +209.237.247.192 212.13.195.4 4 u 770 1024 377 140.306 0.702 0.091 212.13.195.3 212.13.204.4 3 u 900 1024 377 0.280 2.328 0.271 212.13.195.4 212.13.204.4 3 u 910 1024 377 0.364 1.133 7.864 2001:ba8:1f1:f0 212.13.204.4 3 u 764 1024 377 4.560 0.207 1.500 2001:ba8:1f1:f0 212.13.204.4 3 u 767 1024 377 0.336 0.264 0.739 *212.13.204.4 195.66.241.10 2 u 29 1024 377 0.496 2.619 1.066 +212.13.204.5 195.66.241.2 2 u 989 1024 377 0.559 -0.330 0.066 So I don't know why it would be consistently around 35 minutes off. I do note that: kahlua$ date && hwclock --show && uptime Tue Feb 16 20:49:59 UTC 2010 Tue 16 Feb 2010 21:08:23 UTC -0.710159 seconds 20:50:00 up 349 days, 2:02, 5 users, load average: 0.52, 0.89, 1.23 which is obviously about 20 minutes off, but the hardware clock will get set from the system clock next time it's shut down, and I can't imagine why it would be used for anything. I'll sync it from the system clock now, though. kahlua$ sudo hwclock --systohc && date && hwclock --show Password: Tue Feb 16 20:51:53 UTC 2010 Tue 16 Feb 2010 20:51:54 UTC -0.472518 seconds Does that change things? Cheers, Andy -- http://bitfolk.com/ -- No-nonsense VPS hosting Of course not. Everyone knows you can't have your kayak and heat it. -- James Fidell

Andy Smith wrote: Ah, okay - so long as I've not broken anything then I'm happy :-) (and I've not seen the weird kern.log message since adding the clocksource=jiffies.) Thanks Joseph