How to safely kill your NTP server

NTP is that service that typically quietly runs in the background on almost every single computer in the world. When a computer boots up for the first time, NTP gets configured and is almost never considered again. Why? Because for most people, it just works. Until the end of this century (Y2K + 100) or the next leap year second, most system administrators will happily not need to think about the wonders that NTP gives us.  There are, of course, always security considerations to make, but again, once those are addressed, NTP again fades to the back of your mind. 

If you are running your servers in Amazon, it's possible that your VPC doesn't permit access to external NTP peers. In this case, you may follow directions to set up your own NTP server. These were the exact directions my sys admin predecessor followed for the VPC I inherited at my current job. 

As it turns out, every one of the servers in our VPC needs external internet access and we configure that to route through a NAT. So, other than the security considerations of allowing traffic over port 123, there was no good reason to not rely on the public NTP peers. Great! So, that meant we could kill our internal NTP and save time and money by not hosting it.

So, how do you safely stop and shutdown your NTP server?

First up, the basics:

1. Run a loop across all your servers and `grep 'server' /etc/ntp.conf | grep <ip|name>`. I just used a bash for-loop. 
1. If you have any matches. Use a similar bash loop to replace the conf file and `sudo service ntpd stop; sudo service ntpd stop`. 
2. Then run `ntpq -p` to confirm connections could be established with upstream peers.
2. Check your image and the scripts that build them. If necessary, rebuild those.

Now you can feel pretty confident that everything you know about is configured properly. ...but what about what you don't know about?? Here's what I did:

1. ssh ntp-server
2. sudo service ntpd stop
3. screen
4. sudo nc -vulk -w 1 123

This sets up a dumb netcat UDP listener on NTP port 123. It will log all connections to the console. The `-w 1` is the only way I could contrive prevent the first connection from commandeering the process. It will time out the connection after 1 second; then combined with `-k`, it will keep listening for new connections. The screen is simply for the safety of knowing that this will keep running when I close my laptop, a.k.a. screen daemon. 

Let this run for a day and if you see any traffic in there, you'll know the IP address of the server trying to get at you for NTP info.