Well, you have two computers going through a switch to the router - one works fine all the time, the other loses connectivity. A third computer, hooked straight into the router, loses connectivity seemingly at the same time the one behind the switch does.

Very interesting.

The first thing I would do would be to plug all three directly into the router and see if they continue to experience problems. If not, then replace the switch.

If the problems do continue, then replace the router.

I'm assuming you've already tried resetting the router to factory settings right? Sometimes if mine starts acting weird, I'll just do a hard reset and reconfigure the whole thing and voila.