it'd be best to check with your ISP to see how many IP addresses are assigned to you. if you only get one, then you'll need a router with NAT on it. if you get more than that (one of the ISPs in my area assigns 5), then you can just put one of those on your computers.

if you only get the one IP address, then you'll need the router, and i'll give you a general idea of how you'll configure it, because that's what i'm doing

get the router and its confguration disc. for the config process of mine (Linksys 4-port router), i had to directly connect my computer to my ISP's box to test for connectivity. after the configuration gets past that, it'll ask you to connect the ISP box to the router (mine was by straight UTP cable, not by crossover, as i had initially assumed) in the WAN port. then another straight UTP cable in to the back of the computer. make sure the computer has DHCP enabled until you have everything else set up. finish the install process, then check if you can connect to any websites or whatever. if you can't, poke around in your router's firmware, and i assume for most of them, it's just by typing its IP address in your address bar in IE or whatever. mine's 192.168.1.1, so just do a quick ipconfig in the command prompt to make sure, and your default gateway is your router.

after you've done all that, and you know what address range your router is giving you, you can set static internal IP addresses (192.168.x.x or 172.16.x.x or whatever the router's network address is) just for management purposes. when you set the static IP addresses, you'll need to set static DNS addresses, which either your ISP can provide you with, or your router's firmware will show.

i hope i explained that right