On another note, depending on the amount of pocket lint, you may be affecting the longevity of the contact points for the USB drive.

I used to carry around a 32MB (whopping size, I know) in my front pocket on my keychain. Eventually there was crud build-up from being there.

I was able to temporarily get the connectivity back by soaking the whole USB drive in a shotglass full of rubbing alcohol.

I'm dead serious. I've used rubbing alcohol on a number of electronics without ill effect. Including cleaning the paths on a TV remote circuit board.

If yours goes that route, try that and make sure the drive is dried out before plugging it in.

If you're still in doubt, see the Wikipedia entry http://en.wikipedia.org/wiki/Isopropyl_alcohol