you can definitely use a router to network two computers, you do not need internet access to do this. unfortunately you cannot have a computer online and use your telephone at the same time with a dial up connection.
I use one phone line for the computer dial up and one for the telephone. Could I connect both of them on one line in that case? I use AOL (I persist my parents to switch ISPs), will that be a problem with the internet connection of the network? Also, how would I go about setting up the network so I could file share and stuff.

Thanks for the welcome !