Yet it's better to be safe then sorry..
What if this person was using your box to do illegal activity's with??
You'd be the one whom get's busted not the person who's using your Box as cover So to speak..
I'd disconnect from the net, format, then reinstall everything from scratch, that's the only sure way of knowing that you got rid of any future attacks through back doors or any other malicious programs..

cheers
Stacy