If you are running your servers in Amazon, it's possible that your VPC doesn't permit access to external NTP peers. In this case, you may follow directions to set up your own NTP server. These were the exact directions my sys admin predecessor followed for the VPC I inherited at my current job.
As it turns out, every one of the servers in our VPC needs external internet access and we configure that to route through a NAT. So, other than the security considerations of allowing traffic over port 123, there was no good reason to not rely on the public NTP peers. Great! So, that meant we could kill our internal NTP and save time and money by not hosting it.
So, how do you safely stop and shutdown your NTP server?
First up, the basics:
- Run a loop across all your servers and `grep 'server' /etc/ntp.conf | grep <ip|name>`. I just used a bash for-loop.
- If you have any matches. Use a similar bash loop to replace the conf file and `sudo service ntpd stop; sudo service ntpd stop`.
- Then run `ntpq -p` to confirm connections could be established with upstream peers.
- Check your image and the scripts that build them. If necessary, rebuild those.
Now you can feel pretty confident that everything you know about is configured properly. ...but what about what you don't know about?? Here's what I did:
- ssh ntp-server
- sudo service ntpd stop
- sudo nc -vulk -w 1 123
This sets up a dumb netcat UDP listener on NTP port 123. It will log all connections to the console. The `-w 1` is the only way I could contrive prevent the first connection from commandeering the process. It will time out the connection after 1 second; then combined with `-k`, it will keep listening for new connections. The screen is simply for the safety of knowing that this will keep running when I close my laptop, a.k.a. screen daemon.
Let this run for a day and if you see any traffic in there, you'll know the IP address of the server trying to get at you for NTP info.