Printable View
Recently, after some testing, I builded an NLB "cluster" in one of our webservers (actually, in the one we already had and in a new one ;) ).
It works fine but since that I'm able to sniff the traffic between the members of the nlb and its clients from whatever else computer on the network in promiscuous mode...
Is that normal?
Why does it happens if no one of the destination address (ethernet, ip) is broadcast?
Any haelp would be appreciated.
Thank you.
How's everything connected? Using hubs or switches?Quote:
Originally posted here by DerekK
Why does it happens if no one of the destination address (ethernet, ip) is broadcast?
If it's hubs, it's normal.
If it's switches, it's not. Maybe your switch is in a fail-open state? Switches tend to do this if they see multiple MAC addresses (layer2) using the same IP address (layer3). It freaks out the switch and to prevent connection loss due to the failure it effectively becomes a hub. You may need to configure your switch specifically for the NLB cluster machines.
Ok, let me explain myself:
I've two NLB cluster, both with two nodes with two NIC's each. They are independent with their own ip's and applications.
One of them have the "public" NIC's of both servers connected to the LAN and the "private" ones connected with a crossover cable.
The other cluster have the same configuration for the "public" interfaces but the "private" ones are connected to the same switch to a two ports which are configured as an untagged VLAN.
I can sniff packets between clusters and them clients in both cases.
Colud you explain me little bit more about this fail-open state? In that case, should I be able to sniff the traffic of the other machines attached to that switch?
Thank you very much.
Still not sure but I'm assuming the following (let's concentrate on 1 cluster at a time ;) ):
ClusterA consists of NodeA and NodeB. NodeA and NodeB are connected using a crosscable on interface2. Interface1 of NodeA and NodeB has a LAN IP address. ClusterA also has a virtual IP address?
http://www.microsoft.com/technet/pro...ing/nlbbp.mspx
Good idea, once at a time.
Defs:
Public interface: The one which have configured the cluster IP and another one in the same network and serves the application to the clients. It is attached to the LAN.
Private interface: The one tha's suppose to send the heartbeats between the nodes of the cluster and has another network's ip address. It is attached to the other server's private interface through a crossover cable.
Cluster A:
Public interface: 192.168.69.175, 192.168.69.94
Private interface: 10.29.0.1
Cluster B:
Public interface: 192.168.69.175, 192.168.69.95
Private interface: 10.29.0.2
Am I more clear now?
:D
I hope so.
:(
Ah, yes :)
192.168.69.175 is the cluster's virtual IP address. Clients use this to connect to your cluster. But because 2 different machines (read NICs) respond to this address the switch they are connected to can get confused. A switch usually manages a table with MAC<->IP address translations so it knows which machine is connected to which port. When it gets confused it'll send the traffic to all it's ports. This is called fail-open and it basicly turns your switch into a hub.
Do you use layer2 or layer3 switches? What type/brand?
Found this:
http://www.cisco.com/en/US/tech/tk38...8011b481.shtml
http://support.microsoft.com/default...b;en-us;193602
Fortunatelly we use SSL, but still is quite scary to think that we are broadcasting the sessions between the clients and the webservers to network.
We use 3com switches. By now I'm checking with a portable computer and ethereal that, in fact, the switches are broadcasting that sessions.
One more question: If the switches are in fail-open, should them broadcast all the packets to all ports or only the ones which destination address is the nlb cluster's vistual address?
Because the switch doesn't know which IP address belongs to which port it'll send the packets to all ports.. (don't use the term broadcast in this context, broadcasts are an entirely different beast, it'll get confusing ;) )..Quote:
Originally posted here by DerekK
One more question: If the switches are in fail-open, should them broadcast all the packets to all ports or only the ones which destination address is the nlb cluster's vistual address?
Ok, sorry for the broadcast.
But I meant, which are the packets coming to all ports, the ones which are addressed to the virtual address or every packet that enters in the switch will be redirected to all ports?
And, what should be the solution? Just add the "MAC - port" to the ARP table of the switch?
And... how can I assign more than one port to the same MAC address?
If the switch is in a fail-open state it'll copy any packet from any port to all it's ports..
As for the solution.. It depends on how you've setup NLB (unicast/multicast) and what type of switch you use (layer2 or 3)..
http://support.microsoft.com/default...b;en-us;193602Quote:
If you connect Network Load Balancing hosts with a switch, the switch must be layer 2 instead of layer 3 or higher, because all the hosts share the same IP address (the cluster IP address), and layer 3 switches direct network packets (incoming client requests) according to the IP address of the destination computer.
It's an unicast NLB with layer 3 3com switches....
http://www.microsoft.com/technet/pro...ng/nlbfaq.mspxQuote:
Layer 3 switches need to be specially configured to work with NLB. A VLAN must be established for the hosts in the cluster, and this VLAN must be configured to operate in Layer 2 mode. All Layer 3 switches may not support this capability, and when they do, the mechanism to setup the Layer-2 VLAN is specific to the particular make and model. Consult the documentation for the switch before attempting to configure such a system.
No idea how to do this on 3Com switches.. I've only used Cisco's.
Hey, thank you very much for your help man.
I think I've two possible solutions here:
- Buy a HUB so I connect the servers to that HUB and then uplink it to the switch.
- Create a VLAN, enable routing between that VLAN and the default one BUT the bad news on that one is that a change of IP implies DNS and routing changes that I'm not really sure to be ready for. ;)
Do you see anyone else?
UPDATE: Finally I tested with some old HUB so I connected the two servers into that HUB and the HUB to one of the switches on my LAN... I rebooted both servers.. and... I'm still able to see all the conversations between the cluster and the clients! Actually, that I see is the packets that the clients are sending to the cluster, not the other direction...
Any idea??
Derekk- This really comes down to your network configuration.
Using VLANs we create a dedicated private network for the private NLB connections. If you were to set a port on the switch to be a part of this VLAN you can sniff the traffic on this port and see all of the cluster private communication. If that port is not part of that VLAN you won't see the traffic.
The same is true of our public nics. Multicasting on the subnet must be supported. If you only want the machines that are part of the NLB to see the traffic don't assign any other ports to that VLAN.
So for instance on our catalyst switches there are a lot of ports. If I were to just plugin to any of those ports on the same switch I can't see any of the NLB traffic.
To me it sounds like you are multicasting all of your incoming data to all of the ports on your switch. So even if you put in a hub, you are still sending the incoming data to all of your switch ports...
Perhaps you need to put the machines that are part of the NLB into their own subnet, and make sure that data going to that subnet is only sent to the one port you are using for the hub uplink.
Yes it's true.Quote:
Derekk- This really comes down to your network configuration
I just solved the issue setting to 0 the MaskSourceMAC registry parameter.
From now I can't see the nlb traffic anywhere on the network.
Tnak you very much and I apologize but my first intention was to discuss about nlb security.
Thank you all!
This is all cute but you mentioned you're using SSL. If so, you're only sniffing the packet headers, not the data. If SSL is doing its job, at worst, you're creating a **** storm of unnecissary network traffic because of improper cluster configs (fail open condition that Cisco switches do and many others). Can you clarify what you're seeing? If you are seeing packet headers, this is normal.
To answer your question, yes, NLB is secure when done properly.
--TH13
Since we're using SSL of course I was only able to see the headers, but I was wondering what could happen I we didn't use it. Could you imagine someone in that circumstances download CAIN? ;)
Anyway, I see that the "**** storm" you meant was just a consequence of a missconfiguration (even if that config was exctracted from Microsoft docs), or better, if you build NLB clusters it's important you tune up the config. That's what I learned thank you and one day testing... If someone gets offended because my poor knowledge it's his/her bussiness, I don't really care... :)
I'm happy to see the issue solved and to see that NLB works really fine if you use and configure it properly :)
Who got offended?
As far as I know (is that expression correct?) nobody...Quote:
Who got offended?
I dont know very much english but I'm pretty sure that
the if its conditional ;)Quote:
If someone gets offended
Forget it, probably it was too early to write something....
By the way, I didn't realize it was you... I love your HPING tutorial ;)