If you really want to scare the guy write a script to scan the IRC net's for that adress, whois the guy, and do some recon on him by joining the chan if you do find it....
Well, I don't know about scare him, but you could bind nc to a port and have it go to net send or something...
Just in case you don't know about it, here is more info for netcat..
http://netcat.sourceforge.net/
Like everyone sais, make your firewall stop him dead in his tracks...
And also you have to ask yourself, do I *really * *reaally* need an FTP server....
If yes, then make sure you stay aware of the potential bugs and exploits that are being developed for it...Im guessing your running XP or something, cause if *NIX you would most likely know about tcp wrappers and iptables and so on.....Just keep monitoring the firewalls, and disable server if necessary...Later