Efficient Network Design

I have been presented with the task of coming up with a couple of budgets to upgrade/revamp our network.

Such things that have been a problem are that the File server, app server is slow to access if you are on the other side of our office due to the fact (as far as I can tell) that there is only two Gbit ethernet ports connecting one side of the office to the other even though over half of our staff is connected through those lines to the main network/server closet.

Since we push a ton of data and I am talking many Gigabytes a day all over the network I was thinking what would be best to resolve these issues and get our network up to a lighting pace.

Such things I have been playing around with are getting switches that can be connected with cascading cables as well as running fiber lines from one side of the office to the other to make a solid and high speed backbone for our network.

Also setting up VLANs, QoS, the DMZ have been ideas I have begun to play around with, but my working knowledge in these areas is pretty novice. If I could divide up the three or four main types of equipment I think that would prove benefitial as well.

Is there someone that has experience with basic networks that need a good high speed overhaul? And if so what have you done or seen done that has really removed the latency problem. Perhaps it is something simpler than upgrading all our equipment to Gbit or greater.

As always thanks for any info you can lay upon me.

What are some basic specs of your network? Some questions:

What standard of cable is used?

How many nodes are on the network (computers, printers, networking devices(hubs, switches, etc), servers, etc)? How many computers are on the network? A ballpark estimate will be helpful.

Is your network connected with mostly hubs or mostly switches?

How is IP addressing being performed on your network? DHCP, static? Private addressing, I assume? Class A, B, or C addresses (the number of nodes on your network will be a large factor here)? IPv4 or IPv6 (I hope to God it's not IPv6). Are all hosts on the same subnet (knowing how many nodes there are on the network will helpful)?

I am not familiar with what your business does, so I will ask a couple questions about the business. Is it divided up into departments? What is the general topography of the way computers are set up (in cubicles, across from each other, etc)?


I may or may not have more questions for you as this discussion goes on.

We use Cat5 for everything (the connection between the switches are Gbit)

We have approx. 250 nodes in the office (a lot of PS3s and Xbox360s that get new game data pushed to them throughout the day). I'd say about 75 computers (4 main servers which do the filesharing, apps, printer, dhcp, dns, exhcange,etc. and some do multiple roles...not good I know). Also a 55 phone VoIP system on the same network.

We use switches throughout the office and a few that are at people's desks for those who don't have enough ports in the floor.

IP addressing done from DHCP server and it is a 172.31.X.XXX scheme. IPv4 of course. Everything is on the same subnet as far as I can tell. The IP range is 172.31.2.XXX to 172.31.10.XXX

We are a small company, but are somewhat divided into depts of Programming, Design, Production, Animation, Art. Everything runs from each persons desk to the switches which are then connected to a main switch that connects to the firewall, servers, etc. The new sections of our office expansion was done quickly and the secondary network closet is only connected by three ethernet connections (one is for the phones only) the other two are for all the computers and consoles.

Let me know if you need more info. Thanks.

Goodness..... I dont even know where to start. At least you are coming to the understanding that it is time to upgrade/fix your network. Are there any standards you are currently complying to in the company? Are there any policies you need to comply with?

If not, this might be a great time to develop some.

I want to recommend bringing in a consultant that is familiar with setting up a network and making recommendations based on experience for you. If that is not an option I am sure this community will offer plenty of suggestions.

Starters, figure out the needs for ports and speeds (include phones, game stations, PC's, etc)
Try to break areas into office sections, also attempt to look at potential growth. Typically when I install equipment we expect 30% growth.

Run Fiber between all your switches, I recommend picking a standard brand as well, Cisco. Manage all your switches.
Once you have it figured out setup your VLANS to segregate your data however you please. I use a different VLAN for: Servers, switches, PC's, etc.

We then go into each port and manually set each speed and duplex on every port and machine whenever possible, label every cable on both ends, where it goes and so forth (eases troubleshooting).

I dont want to touch on your firewalls, QoS appliances and everything else as it is not my specialty. I have used and been happy with Packeteers in the past though.

In short you really need to look at what the needs are for the company, current and future. If you dont have the expertise dont feel bad to bring someone in to shed some ideas and give some advice. Be a great learning experience for you as well.

As you get a more firm picture of what you are looking for I think i might be able to offer more pointed advice as well. Good luck either way. Its a big project

Hrmm, so you only use switches and not hubs? Odd. Some of the more efficient networks I've seen have hubs connecting their computers, and then a switch for every department. Cuts down on the cost of all those extra switches, and still cuts down on net congestion.

So, the Gbit connections are done in cat5 or in cat5e/above?

You use a class B private address for only around 250 nodes? Granted, it is a bit close to the limit for one net for the infamous 192.168.x.x class C, but, I really think subnetting your network up into smaller nets would help you a good deal, especially to get that phone system and those gaming consoles off the rest of your network (I'm assuming you use the phones a lot). Sure, you'll have to add routing capability to your internal network, but switches start to get cumbersome, especially when everything on your network is set up/done by IP addressing and not MAC addressing, which is where switches work.

Going on this information, here's what I would do. WARNING: You will want to be prepared to spend some time here.

You say you have 5 departments. Your company is small now, so I'd use class C addressing (192.168.x.x). Gives you 256 networks, each with 256 hosts (254 usable for each net). You have about 250 nodes (does that include the switches as well?) and 75 computers. I'd do this one of two ways.

1) Subnet the network based on departments. 5 departments, 5 networks. Let's see...my subnetting is a bit rusty here. In a class C subnet mask, you have 24 network bits and 8 host bits. The network bits, you can't do much about. The host bits are the ones you can manipulate. So, you want at least 5 networks to work with (how many hosts are in each department?).

I hope I'm doing my subnetting correctly here.

Everytime you subnet a network, you waste 2 IP addresses. Subnetting is like splitting one network into two or more networks. Each network will need a network address (NA)(the first IP) and a broadcast address (BA)(the last IP). You have one network, you have one NA and one BA. You can't ever use those as legit addresses for your computers and such. You have 2 networks, you have 2 NAs and 2 BAs, one for each network, 4 unusable in total. 3 networks, 3 NA's and 3 BA's, 6 unusable IPs. And so on and so forth.

So you're going to be going into a different network range anyways. Oh well.

The key to doing this lies at the subnet mask. Manipulating the host bits and turning them into network bits. In subnetting, you have to use at least the first two of the 8 bits, and cannot use the last 2 (or the last one, can't remember which).

K, let's say we decide to use a class C private IP address, 192.168.1.1.

So, the default subnet mask for a class C is 255.255.255.0. And, there's 8 bits in the last octet. We HAVE to use at least the first two bits. So, using the first 2 bits, we get 255.255.255.192. How so?

2^7 2^6 2^5 2^4 2^3 2^2 2^1 2^0
0 0 0 0 0 0 0 0 <-----8 bits of last octet

When one of those bits is turned on (a 1), the value for that bit is active. If multiple bits are turned on, the values of those bits are active, and added to each other.

2^7 2^6 2^5 2^4 2^3 2^2 2^1 2^0
1 1 0 0 0 0 0 0

2^7 is 128, and 2^6 is 64. 128+64 is 192. So, the subnet mask would be 255.255.255.192.

Now, how many networks do we have? Everything is done in 2's, because we can only have 2 states for each bit to be--on and off. We add the bits that are on, and put the result as an exponent.

So, we now have 2 bits active. So, the result is 2^2, which is 4. That's the number of networks we would get on a class C, if our subnet was at 255.255.255.192. 4<5. Not enough.

Turn on another bit.

2^7 2^6 2^5 2^4 2^3 2^2 2^1 2^0
1 1 1 0 0 0 0 0

2^5 is 32. So, 128+64+32=224.

255.255.255.224 is our new subnet mask. We have 3 bits now active, so we would have 2^3 networks, or 8 networks. More than enough, right?

Now, let's figure out just how many hosts we can support on each network. We do this by adding the number of bits that are turned OFF (the host bits), and using that as the exponent. 3 bits on, 5 bits off. So, we'd have 2^5 hosts on each network, or, 32 hosts. 32 hosts * 8 networks multiplies to 256 addresses. BUT, the act of subnetting has taken a few of those IPs away from us, to use as NAs and BAs for each new network. So, 8 networks * 2 unusable addresses for each network = 16 unusable IPs. 256 - 16 = 240 IPs. 240 IPs < 250 hosts. Oh well. Some of your departments will have to be under 192.168.1.x, while the other departments will have to be under 192.168.2.x to get enough addresses. If you stick to your current class B address, you probably won't have that problem (but subnetting a class B will be fun as hell, since you have 16 host bits to play around with instead of 8, hehhehhehheh...)


2) Subnet your network based on devices. Your VoIP system on one net, your game consoles on one net, and your computers on a third net. Maybe your main servers on a fourth net. This third net can be further subnetted, based on departments, but then that's just silly. If you plan on subnetting, you will probably want to invest in a router for your internal network, since you will need that to go in between the new networks (since IPs and routers are both at layer3, but switches only work at layer 2, and so won't be able to route anything in between networks).

Maybe my "solution" isn't the most efficient (since you would be wasting a lot of perfectly good IPs), and it probably wouldn't be the right solution, but it's what I'd do in your situation. Lots of documentation is a must as well. Draw out a network diagram, and map out the available IP addresses for each network.


Now, you can either do all that mess above, or it could be just slow harddisk access in your servers, and the above solution will do absolutely SQUAT in your business. But, at least your boss(es) will admire you for that awesome network diagram you're going to draw sometime in the near future. ;)

Oh lord, the forum post screwed up my bits and crap...oh well.

Regular cat5 only supports 100mps. I think the bottleneck would be the NIC's. Cat5e can handle gigabit a it is much cheaper than cat6. The difference would be 125 (potential) megaBYTEs a second from 12.5.

I dont think fiber would be worth the money but it is an option.

My mistake for forgetting to mention it is Cat5e.

Thanks to NukEvil for that good network tutorial and suggestions for a better network. It is a shame I am just the support for this network since it had already been in place for a number of years before I got here.

One thing I would like to eliminate right off the bat is the issues with sluggishness and lag on the other side of the office which I assume a big bottleneck is that just two cat5e connections connected 30+ computers, 30+ XBox 360s, and 10 PS3s to the other side of the office. Not to mention most of the computers and consoles on that side of the office are on 100Mbps switches.

Any ideas in the short term, but I am all for a total network redesign and of course I am working on a full map of our network...which no one ever did at this company so it is quite an undertaking.

If the computers only have 100mps cards it doesnt matter how fast the switches are. The transmission between the switches will be 1000mps but not between the host and the switch. My thoughts:

1. Upgrade all nics to 10/100/1000 cards.
2. Upgrade the rest of the switches to 1000mbps
3. If possible make the network topology as balanced as possible

for example you have 20 computers on 1 switch and there is another with 2 on a switch try to even it out a bit. (if possible)

Most of the computers are new and have 1000Mbps nics.

Since we will upgrade the switches to faster ones would buying ones with more features such as cascade cable enabled switches and ones from Cisco work better than the run of the mill low end ones we currently use from a wide array of manufacturers?

I assume connecting computers and the console they use the most on the same switch would work better than putting all computers on one switch and all consoles on the other...unless of course they had cascade cables between them.

I think so only because it would be one less hop but the only thing that would be changed is latency not bandwidth and that wont even be noticeable. What you want to do is distribute the overall traffic throughout several switches. Each switch should manage about the same amount of traffic. Not all traffic for one switch and little for another.

Another reason why I am trying to figure out where ever single device is connected so I can balance it better and map it out for easy of use and troubleshooting.

If half of our company is in a part of our office where only two cat5e cables connect to our main closet how many connections should be made to the other side? One per switch or perhaps get something that is more high speed to handle the huge amount of data being sent back and forth from the fileserver and app server (I am talking many gigabytes of data every day).

Thanks again guys for helping me figure out the best ways to do this. Much appreciated.

Glad we could help. :-) Just take your time figuring out where all the traffic is concentrated and take it from there. Make sure to keep us all posted on the results too. We love to know how our advice turned out

Good luck and have fun!