Wireshark is OK, But a Bit Heavy

For sheer absence of dicking around, nothing beats tcpdump(1) which ought to be in your Linux install. I had to find out what systems were still using the old time server, which, happily was an ancient Slackware and, consequently, a Proper Operating System.

tcpdump -c 10000 port 123 > clockies

gathers the first 10,000 NTP packets, and, in another terminal session

gawk '/[0-9] 10/{print $2}' clockies | sort | uniq

gives you a list of the IPs (and you don't have to wait for the first one to finish.) Run wc(1) at the end of the second pipeline from time to time, while the first is still running, and you can see if any new IPs are cropping up.

What I like is the query language: for NTP packets there's just no more intuitive way of writing the search than "port 123"