Am I going Nuts or what?????

I have a router with a default gateway at 192.168.3.4. It contains routes to all my private subnets. One of those private subnets also has a gateway to the world via T1. I have built a server that will handle DNS and mail relay in case my primary T! here goes down. It's all rather simple really.

In order to have the secondary mail/dns server function properly I need to be able to route to it's public address from the master router that is 2 hops away. So I merrily palce the following route in the master router:-

ip route aa.bbb.ccc.0 255.255.255.0 192.168.2.10 (an internal router that several other routes point to and function perfectly well).

I write it to memory and ping the box... Bingo. So, I'm happily playing when I get a call from one of my other agencies telling me that they cannot get to a web site that they use frequently. I run an nslookup on it and find it is at aa.ee.ff.gg. The letters that are in this address match the letters in the route I put in so if aa = 10 in the route above then aa = 10 in the nslookup. As you can see the aa network is the only common factor. So I run a tracert on the aa.ee.ff.gg address and my master router tell me that the "destination host is unreachable".... :eek:

So I remove the route and everything is peachy except that I, (obviously), can't reach my aa.bbb.ccc.ddd host through the internal network which I need to do. I re-added the route same thing, I checked it for typos - none, I checked to see if there was any possibility of a conflict - none!!!!! But the host is unreachable... In fact _every_ host in the aa network is unreachable.... I have reloaded and done a hard restart and it is still messed up!!!!

Now correct me if I'm wrong but a network aa.bbb.ccc.0 and a subnet mask of 255.255.255.0 is a collection of hosts from aa.bbb.ccc.0 to aa.bbb.ccc.255 and has no relation whatsoever to the network aa.ee.ff.0.

So why the hell will my Cisco not get this right :mad:

stupid question. Did you clear the routing tables?

//EDIT sometimes when you introduce a new route you have to fluch the table cache. I think it is cisco>clear arp

OK... Stupid answer....

Doesn't a reload, and especially a hard restart, clear the table on a Cisco and start from scratch? I really don't know.... I'm not hugely good at routers... Even so, when I add a route for a Class network with the appropriate subnet mask how in the hell does it kill a whole A class? Inquiring minds need to know..... :o

Yes powering it off should flush it but I always do that as a habit.

cisco> clear arp

I have rarely ever powered off a router or firewall. ;)

I am cranking up my squeaky brain on routing problems. I am not expert either. ACL perhaps?

//EDIT TTL on the secondary route?

Router pointed to correct DNS server

boobs# ip ip-nameserver 12.34.56.78

No ACL's on the router... It's internal and unreachable from the public network....

Powering off is the last resort.... I did it... I couldn't see any other reason other than it wasn't properly parsing the ip route statement and that a restart may "assist" it to understand.... Clearly it didn't help it.... Stupid router... Never had a poblem like this before.... I have a spare router and may try a replacement tomorrow.... the current one is old... very old.... It might just be failing... I have a newer one that I'll copy the config to and slide in there tomorrow.... That's my best guess at the moment... I _know_ the route statement is good.... I have no idea why it messes up the whole A class other then router failure at this point.....

[Edit]

nameservers are irrelevant, all routes are literal....

[/Edit]

Try routing the entire network as a trouble shooting measure. Never did it on a router but the pix let's you route 0.0.0.0 as a catch all.

//EDIT

well you are going to have to get on the router and use trace on all interfaces to make sure they are set up properly. Perhaps an interface failed during the restart.

yo# trace 192.1.10.20

yo# show arp (just make sure they are there)
yo# show interfaces


I am going that way because the route seems ok.

I actually tried the following route:-

ip route aa.0.0.0 255.0.0.0 192.168.2.10

all the while there is the default route of:-

ip route 0.0.0.0 0.0.0.0 192.168.3.4

It worked perfectly for my aa.bbb.ccc.ddd host but the whole A class was still "destination host unreachable"....

I dunno.... I'm beginning to think I don't know s**t about this stuff any more... Alzheimers? Maybe...

[Edit]

I have to leave the satellite office right now..... I'll look after I have dinner and made my sweetie happy.... Other than that, the router is getting replaced tomorrow and we'll see if it works.... if it stays the same i will be appropriately "miffed" :eek:

[/Edit]

LOL that is hilarious, I was just going to ask about the Sat. Office. Have fun um... "working"
I pasted an edit after you gave the latest update.

I'm still at router replacement at this point..... There is nothing wrong with my route statement.... Thus, logically, there is something else throwing itself into this mix.... The router itself has to be a suspect.... We'll see in the morning.....

Without much more in depth info, I doubt anyone online will be able to fix it for you, thus before assuming it must be replaced, I would suggest two things:
1) Is the firmware up to date?
2) Is there anyone you can get to review everything with you? Frequently I've found a second set of eyes can spot very simple problems. :)

Quote:

---

Is there anyone you can get to review everything with you? Frequently I've found a second set of eyes can spot very simple problems.

---

An excellent piece of advice.

I have seen (and been) techs and mechs going round and round on a problem, and some smart a$$ secretary come in and say "hey you guys want this plugged in ?" (over simplified and no disrespct to any secratary meant)

Hey Hey,

If there's no one there to assist you in looking over the setup... could you possible post a sanitized version of the config.... it may be something obscure burried deep inside....

Peace,
HT

[QUOTE]Without much more in depth info[/QUOT

Ipse Dixit

We all live in Shangri La don't we?

:D

OK... It's fixed.....

Oh, you wanna know what the problem was???? You would.... :rolleyes:

I'm not sure I understand this myself but I'll try to give the relevant detail and see if someone can explain why the router was screwing up.

History:

This router was originally set up by my ISP owner some 6-7 years ago when I had a T1 from the net and another to our only other office. It was set as the default gateway for the local network but it had a default gateway of the firewall. Hence, all traffic that was not local was sent to it to decide what to do with. If the traffic was for the other office it would be appropriately routed, if it wasn't it was routed to the firewall to determine what to do with it. The firewall's default gateway was the border router. I never changed that scheme since it worked just fine until the other day.

Example of issue:

Sat at my workstation on the same physical network as this router _and_ the firewall and the firewall having the route to aa.bbb.ccc.0/24 set to go to the remote office I could ping it, term services to it... anything I allow the "southern" firewall to let me do, (it's in the DMZ of the southern firewall so I have to "allow out" tho services I want). But, if I try to go to any address in the aa.0.0.0/8 network from my workstation it would fail.....

The fix:

Standing in the shower... I do my best thinking there.... Don't ask, I don't know!!!! :eek: I wondered about the default gateway and though that before I replace the router I'll see what happens. So I get to work, change my workstations default route to the firewall itself rather then the internal router... Everything functions fine... as it should... So I re-add the route to aa.bbb.ccc.0/24 to the router and ping it.... I works.... I ping random addresses on the aa.0.0.0/8 net and some respond... I tracert random addresses on the aa.0.0.0/8 net and they all tracert out the right way... Hmm... It works!!!!! I go to the DHCP server and change the default gateway to the firewall and test it from some other machines after an ipconfig /release/renew sequence and they all work.... OK.... perfect... It works....

Can anyone explain _why_.... Remember the system worked perfectly before the addition of the route to the router. With the addition of a route to a class C network the tracert to the class C would work perfectly but the whole class A that the class C was a part of ceased working... That's illogical Captain..... Now the default route is changed is all "tickety-boo", (for non-english speakers "tickety-boo" is a good thing.... :D

Any takers?

[Edit]

I dunno.... maybe I am going completely frigging nuts but why did csch's post get negged here?????? That's stupid.... His points were entirely relevant and good advice....

Unfortunately, my attempt to right the wrong was met with "you have to spread.... blah blah blah"....

I don't know who negged him but I have to say that you are a moron... Thanks for contributing....

[/Edit]

Maybe someone has an ax to grind with CSH, I see you made up for it though?


I read your post Tiger and now my head hurts. Then I re-read it. The only thing I can think of and I am very tired, is there was no route built to allow packets back to you through the original default gateway? You would have to run debug on the router to find that out. I'll shut up now because I am clueless and have to draw it on paper. :confused:

Quote:

---

Originally posted here by Tiger Shark
OK... It's fixed.....

Oh, you wanna know what the problem was???? You would.... :rolleyes:

---

Of course. :)

Quote:

---

Can anyone explain _why_.... Remember the system worked perfectly before the addition of the route to the router. With the addition of a route to a class C network the tracert to the class C would work perfectly but the whole class A that the class C was a part of ceased working... That's illogical Captain..... Now the default route is changed is all "tickety-boo", (for non-english speakers "tickety-boo" is a good thing....

Any takers?

---

I think I have the issue digested enough to come up with a relatively educated guess -- but no guarantees. It might be the order the routes were in. Let's use your example and assume aa = 10, bb = 20 cc = 30.
Originally, you had two networks:
A: 10.0.0.0 /8
B: 10.20.30.0 /24

Okay, so traffic hits the router destined for 10.20.30.40, a valid host on network B.
However, the router scans down its list and sees that the traffic is a valid host on network A (listed first), and instead forwards it there.
Now, let's say you reset it all and fixed it. You end up with:
B: 10.20.30.0 /24
A: 10.0.0.0 /8

Same scenario, traffic comes in for 10.20.30.40, but this time it scans down the routes list and sees it's valid for network B, and routes it properly.

Another scenario I came up with is that because the routes conflict it might break one of them (perhaps the original route). Which I'd suggest depends wholely on my interpretation of what you've written. 'Tis quite complicated a situation.

You really don't want those two networks to be "on the same network". You should probably look into subnetting that network. Even breaking it in two -- 10.64.0.0/10 and 10.128.0.0/10 would be sufficient. Of course, that suggestion comes not knowing how many hosts you have on either LAN. :)

Quote:

---

I dunno.... maybe I am going completely frigging nuts but why did csch's post get negged here?????? That's stupid.... His points were entirely relevant and good advice....

---

Just someone's childish response hitting that post of mine, don't worry about it.

Road:

I didn't "fix" it.... I couldn't without running around giving undeserved pos's to people...

As to your head hurting..... Imagine my head.... I know it was right.... I know that the change of default gateway _shouldn't_ matter because the routes on both the router and the firewall were correct, (the packets shouldn't have reached the firewall if they matched the route on the default gateway, which they did), but they still failed with a "destination host cannot be reached" for a whole A class when the route on the router was clearly a C class..... Do you have some _good_ Tynelol.... 'cos mine still doesn't seem to help..... ;)

The router is internal, (behind the firewall)... It just serves the other networks I have... But it was set as the default gateway because the _expectation_ is/was that most traffic would remain internal to the network..... If it wasn't then it was sent to the firewall that had a default route that left my network for the "big wide world"....

I'm still not understanding _why_ the change of gateway makes it all work... but I'm going to sit with my sweetie and forget it until tomorrow...... :D

csch:

That's a fair go I think... The problem I have is that when I added the route the router itself orders them.

To clarify a little on my network.....

Internet -> Border router 1 -> firewall 1 -> internal network 1 -> Router in question -> T1 -> Internal second network -> T1 ->Internal third network -> Firewall 2 -> Border router 2 - Internet.

There are other networks attached by T1's at varying locations but aren't relevant to the issue.

Border router 1: Ser0 IP:- Public on 207.0.x.0 network, Eth0:- Public IP on 207.0.y.0 network.

Firewall 1: Drop in config on same network as Eth0 on border router 1. Secondary IP:- Private network 192.168.3.4

Internal network 1: 192.168.3.0/24

Router in question: Eth0 IP 192.168.3.1 Ser0: 192.168.2.9 255.255.255.252

Internal second network: Ser0: 192.168.2.10 255.255.255.252 Ser1: 192.168.2.13 255.255.255.252 Eth0: 192.168.30.1

Internal Third Network: Ser0: 192.168.2.14 255.255.255.252 Eth0: 192.168.50.1

Firewall 2: Drop in config on 63.x.x.x Secondary IP: 192.168.50.2

Border router 2: Eth0: IP on same 63.0.0.0 as firewall 2 Ser0 on 63.0.0.0 network

Hope that's clearer....

Now, the issue was that there is a new DNS/SMTP server in the DMZ of firewall 2. (Never had one there before so this wasn't an issue). I don't want to open a port on the outside of firewall 2 so I can administer it and also I need it to be able to pass mail all the way up to the exchange server on internal network 1.... So I need to reroute, lets say, 63.1.1.0/24 down through the other two internal networks to firewall 2 which is on that network. There was no issue for internal networks 2 and 3 because their default gateways pointed down to firewall 2 anyway but the router in question had a default gateway of firewall 1 and the internal network 1 used the router in question as their default gateway.

So, logically I add the following route to the router in question:-

ip route 63.1.1.0 255.255.255.0 192.168.2.10, (the internal network 2)

and to Firewall 1

I add the route:-

63.1.1.0/24 -> 192.168.3.1, (the router in question)

Should work fine, once it reached internal network 2 it will be sent on outbound through firewall 2 anyway. A tracert indicates that it does, indeed, route down through the other two networks, mail flows, I can term services to the new server and everything seems fine...... Then I get the phone call..... Being the good admin I am I know that i just changed something and therefore that is the first thing I looked at. I determined that the remote address that was now not cooperating with the users was at, let's say, 63.12.76.10.... So I ping it and get nothing.... OK, fair... I tracert it and the router in question says "destination host is unreachable".... :eek: .... So I tracert 63.1.1.x, (the new server), and it runs perfectly..... I remove the route from the router in question and, obviously, I can no longer see the new server but the remote address at 63.12.76.10 now tracert's just fine out through firewall 1 like it should..... replace the route and the new server is back up but the remote IP is unreachable.

My workstation uses the router in question as it's default gateway. I change it to the secondary IP of firewall 1 and both the new server _and_ the remote IP function just fine.....

I dunno... I still don't see why that change makes a difference. With my DG set as the router in question a packet for the remote IP should go:-

1: My NIC
2. The router in question
3. Firewall 1
4. Border Router 1
5. The internet

A packet for the new server should go:-

1. My NIC
2. The router in question
3. Internal network 2
4. and so on...

The gateway _shouldn't have made a difference.... But it made a really big one.... :(

Last night while downing a Canadian beer with my cat, I was thinking Pix Firewall conduits. I know that's not an answer more a direction but I didn't drink enough beer although hopped enlightenment was close at hand. I printed this out for weekend study.

Road:

Your input would be truly appreciated.....

It's really got me.... It has also resulted in some little "niggles".

I changed the DHCP option for the gateway for the network to 192.168.3.4, (Firewall 1), and got a couple of calls from users about being unable to reach certain web sites. An NSLookup showed they were in the 63.0.0.0/8 network.... Easy fix... trot down and do an ipconfig /release/renew because the lease hasn't expired.... No problem... I tried it and it failed.... SOB... now what?... I go to the network settings and find that even though the setting for "get IP automatically" is set I can still see the 192.168.3.1 greyed out in the default gateway box???? WTF is that all about????? All it takes to fix it is to check the "use this info" radio button and then recheck the "get it automagically" button.... the old DG disappears and the system works just fine because it auto-picks-up the new DG....

I dunno.... this whole thing is "not right".... But I have this horrible feeling it's me... not the system.... :eek:

Tiger Shark, unrelated, but why do you call chsh, "csch"?

;TT

Quote:

---

Tiger Shark, unrelated, but why do you call chsh, "csch"?

---

ROFLMAO.....

'cos, not only am I blind nowadays.... It appears that I am frigging dyslexic too.....

Old age sucks.... :mad:

Really.... no offense was intended and I always thought it was csch...... Oh well.... I'll crawl back into my hole and be quiet for.... er.... um... 30 seconds..... "I'll be baack"... ;)

OK... I'm baack....

Sorry Chris..... never noticed it till it was mentioned..... :eek:

Your DNS server are internal for internet? Or located upstream at ISP?

Like I was saying, I am there petting my cat, right and I think... a firewall isn't a router. It's a filter and will only conduct business that it sees fit to the outside world. You enter a static route but it's not really a route, like in a router. It's a conduit or tunnel. So the problem could lie there. When you had the proble do you remember if you could trace your internal firewall interface?

Cheers I am off now. :)

Road:

I run split or double DNS. My AD servers act as the DNS servers for the domain and request external resolution from my primary or secondary public nameservers in the DMZ. Only the AD servers may request resolution from the nameservers in the DMZ, (firewall rule), and only the nameservers can request resolution from the net, (another firewall rule).

The Firewall I use does act as a router, (Watchguard). I can put routes in it in any way I please to direct traffic internally or out the optional port which I use as the DMZ. Obviously it's routing abiility is limited by the fact that it has only 3 ports but I can direct traffic any way I like to any number of internal routers at the firewall rather than having to send the traffic to an internal router to make the routing decisions.

I could always correctly tracert anything, (including the internal interface of the firewall), as long as it was not in the 63.0.0.0/8 network unless it was in the 63.x.y.0/24 subnet - in which case it worked fine... Everything else returned "Destination host unreachable" from the router in question.....

I dunno.... My head just started hurting again.... ;)

Quote:

---

Originally posted here by Tiger Shark
OK... I'm baack....
Sorry Chris..... never noticed it till it was mentioned..... :eek:

---

Don't worry, I've been called far worse. :D

Quote:

---

Originally posted here by Tiger Shark
I could always correctly tracert anything, (including the internal interface of the firewall), as long as it was not in the 63.0.0.0/8 network unless it was in the 63.x.y.0/24 subnet - in which case it worked fine... Everything else returned "Destination host unreachable" from the router in question.....
I dunno.... My head just started hurting again.... ;)

---

I still think it's related to position in the routing table. It's the only thing that makes sense to me.

Quote:

---

I still think it's related to position in the routing table.

---

Aye, but there's the rub..... The router auto-orders the display for me... so I see it as:-

0.0.0.0 0.0.0.0 192.168.3.4
63.x.y.0 255.255.255.0 192.168.2.12
192.168.100.0 255.255.255.0 192.168.3.3
blah
blah
blah

Now, while I am sure that the display is being ordered that way, (numerically), for reasons of beautification I have no idea how to see the _actual_ order they may be in. Any clues?

Well maybe the actual order is in the way they are entered? That might explain why when you redid the config it worked? I dunno, I'm just thinking aloud. :)

Ouch, everytime you type my head goes dizzy with more info. I thoought you had a pix for some reason and I fixed a similar issues using conduits verses static routes. I too have similar DNS settup. It is nothing but a headache. Hmmm. me runs off to check DNS rulez on the pix!

Chris:

I didn't "redo the config" per se.... I changed the default route on the clients through DHCP provided by my AD servers.... It all seems to be fine now... At least, I haven't had any complaints and the systems is passing mail etc. just like I envisioned it would when I added the darned route.... I've given up thinking aloud about this.... People think I'm nucking futs walking around saying "If the default gateway of the client is 192.168.3.1 and the default gateway of the router at 192.168.3.1 is 192.168.3.4 then......"... ;)

Road: No pix.... I'm a Watchguard kid... Used to be really expensive but now the price seems comparable..... Having never used a pix I can't comment on pro's and cons but the Watchguard is a pretty powerful firewall with lots of "sexy" features.... Try one... You'll like it.... The only thing I don't like is the manuals and the online help.... They frigging suck.... You either already understand _all_ the terminology _they_ use, up front, or you are lost..... It's trial and error time.... 'cos the help is like M$'s... Totally accurate but of no use to anyone.... I'm still finding cool features that some nice documentation would have turned me on to 5 years ago.... :rolleyes:

That is exactly why I like the Pix, it's a defacto standard and the syntax is well documented in countless forums, web sites and books. And if you are inlcined the cost of a SmartNet subscription gets you world class cisco geeks at any time of the day. HOWEVER, if you are using a VPN for remote access... the Watchguard is very cost effective. Especially hardware based. Cisco VPN licensing is way too expensive, at least last time I checked. And to get the same kind of interface you have to license Cisco Enterprise Manager, which is very nice. The newer Pix boxes might even be easier to use, mine are a few years old now.