easiest thing to do is get your dsl modem from the dsl company and buy a 5 port hub (routers, switches, and bridges all do the same but cost more), buy some cat5e (i suggest getting the linksys network in a box as it comes with a hub, 2 NICs and 2 15ft cat5e cables to support the 100mb throughput). the dsl modem you'll get will most likely be a crappy USB modem, if they offer you other models, take another one that's not USB. after you have the hub, modem and NICs installed, you have to set each computer's internal IP (usually done already but they all get set to the same IP which you have to change), then you have to set your machine (i assume you're going to be the gateway computer) to be the first IP on the network (ie. 192.168.0.1 and the others are the same except the 1 changes) and you normally have to set the subnet (use 255.255.255.0, it allows for the most IPs on the network, roughly 196 IPs). then you have to set the other computers to use the gateway computer as their DNS and gateway.
then do not use MS's ICS software, it's horrible and sucks. i suggest downloading and using Sygate Home Network. it's easy to use and has little to no overhead on the network. after you have all the IPs and the gateway set up and sygate installed, you're ready to go. all of this can take about an hour to install the modem, NICs, and set up all the software and IPs.