February 24th, 2006, 12:04 AM
Two Networks "Accidently" Bridged??
Gotta share this one, this is just too dang wierd.
So one of our client companies opens several tickets with us today after they suffer a power outage for about an hour. Brought the whole network down. After power comes back to the building, they bring all their servers and hardware backup, and all of a sudden network problems abound. DHCP's being goofy as all sorts of IP conflicts are occuring across the network, and they keep gaining and loosing and gaining network connectivity. On this end, we keep losing and regaining VPN connectivity to their network, so obviously something's really screwy over there.
Well, after some phone calls and emails back and forth with the main contact over there, we find out that all of a sudden, the guy in the business suite next door can see their computers from his network! Somehow the network from our client became bridged to their suite mate's network.
So power outage, network comes back up screwy, and they're mysteriously bridged to the guys next door. What the heck could cause such a thing?
Just got some new info -- Both companies share a T1 and have the same subnet. After the outage, they can now ping each other's IPs. Our client company used to be in the suite the other company is in now, and when they moved they evidently left one of their old access points that wasn't working anymore. Their theory is that perhaps the power outage cause the AP to come back to life. So that AP is actually plugged into the other company's network. Now since they unplugged that old AP, they can no longer ping each other. Yeah, "WTF?" abound...[/edit]
[edit 2] Ok, learned a little more -- Both these companies, sharing the same T1, share the same gateway, but are segmented from each other. Basically there are a total of 16 public IPs which are split between the two companies. So behind the gateway are two PIXs, one for Company A and one from Company B, and the each company's network exists independently of the other's behind its own Pix.[/edit 2]
February 24th, 2006, 01:20 AM
just double check the DHCP server has the correct scope ? get someone from the 'other' Co to check theirs too ........
it is more likely that the power outage has corrupted at least one servers network scope, and that as a result they are experiencing network issues because the two servers are using the same IP range
or the leases could have been shot too ..........
for obvious reasons you have to ensure that two DHCP servers on the same network, do not have overlapping scopes ..........
DHCP provides an automated means of managing host IP addresses. The DHCP server is configured with IP address information, including a range of valid addresses and a lease period for those addresses. The range of valid addresses is called a scope.
55 - I'm fiftyfeckinfive and STILL no wiser,
Beware of Geeks bearing GIF's
come and waste the day :P at The Taz Zone
February 24th, 2006, 10:26 AM
One thing that can happen... I've seen it before... Changes made to routers/switches/firewalls.. But "they" forgot to write the config.. Which means the running config was different from the stored one.. Power down, power up.. Hey presto, the "old" stored config..
Experience is something you don't get until just after you need it.
February 24th, 2006, 01:35 PM
Thats what I was thinking, if the routers and switch's Running Configs have not been save the their Startup Configs, after a restart they will revert back to the old startup configs and will lose all config parameters that have not been saved. This also includes any ACL's so there may be a security issue, if this is what has happened.
If they have been admin'd properly they should have a back up on a tftp sever, with a bit of luck!
Foxys post about the DHCP servers is also a valid fault for this issue too. The first thing to check would be the DHCP server IMO as it sounds like an IP addressing problem.
February 24th, 2006, 04:47 PM
Got feeling, they're probably also sharing a switch and using VLANs, and the switch lost its config. Without actually tracing the cables yourself, you'd probably have trouble getting to the bottom of it by proxy.
February 24th, 2006, 04:50 PM
Well, they're still keeping an eye on things, but since they removed that old AP, everything has seemed back to normal.
February 26th, 2006, 09:19 PM
In addition to checking DHCP scopes and leases, might want to pick one of numerous available applications and map your network and see if any machines show up in the list that are foreign.
Of course, both companies should be forwarned of the activity prior to running the scan (get permission prior if need be).