It's been quite a while since I've seen just a 4 port switch for sale in consumer electronics stores. I'd imagine that the cost for the switch would be inline to that of a home router.

As the posts above state, the easiest/cheapest way would be to connect them through a crossover cable. If you have the tools, you can make one. Otherwise, I'd just ask someone you know who is into networking. For some reason they normally cost more in an electronics store than a patch cable. In my opinion, all those stores rip you off when it comes to ethernet cables.

If you do know anyone in computers/networking... ask around for either a switch and some patch cables or a crossover cable. If they're anything like me they probably have boxes of extra networking gear that they could spare.