Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
inignot
Sep 1, 2003

WWBCD?

jwh posted:

Well, I'm no expert, particularly on RIP, but from what I was able to test in a lab, the reason IOS didn't pick up on the fact that you had an interface covered by the 192.168.0.0 statement, is because the RIP configuration is approaching things with a classful mentality. This seems to make a certain sense if you consider that RIPv1 was a classful protocol, and maybe for historical reasons, the IOS configuration was never retrofitted.

So in other words, it looks to me like IOS uses the RIP network statement along classful boundaries. Even though your interface is technically 'covered' by the larger supernet of 192.168.0.0/16, IOS is considering 192.168.0.0 classfully, which would match the 192.168.0 portion of the network, because 192.168 is part of class C or /24 classful address space.

That's my theory anyway, it seems to be true, even if it is a little brain-dead on the part of IOS.

Debugging commands were 'debug ip rip event' and 'debug ip rip database' I think, and then the 'debug ip packet' example that's included in the text I wrote. Be careful with debug ip packet, because I have successfully killed production devices with it.



^^^^^^
This is correct. The Net Master Class CCIE labs will pound this into your head.

RIP sucks, either deal with it's dumbass limitations or run a modern IGP. Pro tip: running a modern IGP is less effort.

Adbot
ADBOT LOVES YOU

Jeff73
Feb 3, 2006

jwh posted:

Be careful with debug ip packet, because I have successfully killed production devices with it.

Seconding this; I killed a 4006 that way. I deleted the ACL constraining output before I turned off debugging - without making sure its very old IOS would automatically disable debugging referencing the list like newer versions do.

Nope.

I hit undebug all quickly and figured that was that, but 20 minutes later it crashed and burned. Judging by CPU usage while duplicating the event, deleting the ACL first caused the undebug command to only stop the output, which left the debugging process free to invisibly continue pegging the CPU.

atticus
Nov 7, 2002

this is how u post~
:madmax::hf::riker:

jwh posted:

So in other words, it looks to me like IOS uses the RIP network statement along classful boundaries. Even though your interface is technically 'covered' by the larger supernet of 192.168.0.0/16, IOS is considering 192.168.0.0 classfully, which would match the 192.168.0 portion of the network, because 192.168 is part of class C or /24 classful address space.

Yep you're exactly right - I wasn't even sure how RIP was allowing him to add a "network 192.168.0.0" as that breaks the class boundary rules for the network statements required of RIP, but I tried it on a 3750, and sure enough it worked, so I got a little confused myself.

RIP does suck. :smith:

Arkady
Jun 18, 2004

Off to work!

jwh posted:

*Jul 12 15:26:27.059: rip_route_adjust for Serial1/0 coming up
*Jul 12 15:26:27.063: IP: s=192.168.4.26 (local), d=224.0.0.9 (Serial1/0), len 52, sending broad/multicast
*Jul 12 15:26:27.067: RIP: sending request on Serial1/0 to 224.0.0.9

Houston, we have a problem. I think I have started celebrating too soon.

I checked this idea in my lab, with the same results as you presented. Except that you didn't show it sending updates to the interface. And it didn't.

Right now the router is sending a RIP request to the 192.168 /16 interface after the interface comes up from shutdown as expected from RIP. The problem is; the router doesn't send RIP updates to that interface. It only sends out a request once, and nothing after.
As always, interfaces with other IP addresses work fine.

Arkady fucked around with this message at 15:01 on Jul 15, 2007

atticus
Nov 7, 2002

this is how u post~
:madmax::hf::riker:
I've done some testing in my own lab. Two 3750's so keep in mind that ports are going to be gigabit ethernet, that should be the only difference here. I've also turned off RIP's auto-summarization so it didn't advertise the major 10.0.0.0 automatically to prevent that route from being possibly aged out of the routing table when we change stuff around. I've also disabled split horizon on the interfaces, even though that won't help anything, but just for similarity's sake.

Here's what I've been able to verify so far (stuff that we already knew):

RIP will not send updates with a non-classful network statement. On Routers A and B it needs to be changed from "network 192.168.0.0" to "network 192.168.4.0"

On Router A, this immediately begins to trigger RIP updates out of the interface that's connected to router B:

code:
2w2d: %SYS-5-CONFIG_I: Configured from console by console
RouterA#
RouterA#
2w2d: RIP: sending v2 flash update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.25)
2w2d: RIP: build flash update entries
2w2d:   192.168.4.24/30 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 1 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/1
2w2d: RIP: sending v2 update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.25)
2w2d: RIP: build update entries
2w2d:   192.168.4.24/30 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 1 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/1
On router B, this results in RIP adding the associated interface to it's database:

code:
2w2d: RIP: add GigabitEthernet1/0/1 to RIP idb list
Now, Router B is receiving RIP updates from Router A:

code:
2w2d: RIP: received v2 update from 192.168.4.25 on GigabitEthernet1/0/1
2w2d:      192.168.4.24/30 via 0.0.0.0 in 1 hops
2w2d: RIP: Update contains 1 routes
Now look what happens when we change the subnet mask on Router A to a /30:

code:
2w2d: RIP: sending request on GigabitEthernet1/0/1 to 224.0.0.9
2w2d: rip_route_adjust for GigabitEthernet1/0/1 coming up
2w2d: RIP: sending request on GigabitEthernet1/0/1 to 224.0.0.9
2w2d: RIP: received v2 update from 192.168.4.25 on GigabitEthernet1/0/1
2w2d:      192.168.4.24/30 via 0.0.0.0 in 1 hops
2w2d: RIP: Update contains 1 routes
2w2d: RIP: received v2 update from 192.168.4.25 on GigabitEthernet1/0/1
2w2d:      192.168.4.24/30 via 0.0.0.0 in 1 hops
2w2d: RIP: Update contains 1 routes
2w2d: %SYS-5-CONFIG_I: Configured from console by console
2w2d: RIP: received v2 update from 192.168.4.25 on GigabitEthernet1/0/1
2w2d:      192.168.4.24/30 via 0.0.0.0 in 1 hops
2w2d: RIP: Update contains 1 routes
2w2d: RIP: sending v2 flash update to 224.0.0.9 via GigabitEthernet1/0/2 (10.31.13.1)
2w2d: RIP: build flash update entries
2w2d:   192.168.0.0/16 via 0.0.0.0, metric 16, tag 0
2w2d:   192.168.4.0/24 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 2 routes
2w2d: RIP: Update queued
2w2d: RIP: sending v2 flash update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.26)
2w2d: RIP: build flash update entries
2w2d:   192.168.0.0/16 via 0.0.0.0, metric 16, tag 0
2w2d:   192.168.4.24/30 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 2 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/2
2w2d: RIP: Update sent via GigabitEthernet1/0/1
Now Router B is sending and receiving updates on both interfaces. When we change the mask on the interface back to a /16, we see the same "route adjust message" :

code:
2w2d: rip_route_adjust for GigabitEthernet1/0/1 coming up
Ok. Let's look at the RIP database on router B.

code:
RouterB#show ip rip database
10.0.0.0/8    auto-summary
10.31.13.0/24    directly connected, GigabitEthernet1/0/2
192.168.0.0/16    redistributed
    [0] via 0.0.0.0,
192.168.4.0/24    auto-summary
192.168.4.24/30
    [1] via 192.168.4.25, 00:00:16, GigabitEthernet1/0/1
Now let's look at the RIP database on Router A.

code:
RouterA#show ip rip database
192.168.4.0/24    auto-summary
192.168.4.24/30    directly connected, GigabitEthernet1/0/1
Before we go and change the mask again, let's try to force Router A and Router B to become neighbors.

code:
RouterB(config)#router rip
RouterB(config-router)#neighbor 192.168.4.25
RouterB(config-router)#end

RouterA(config)#router rip
RouterA(config-router)#neighbor 192.168.4.26
RouterA(config-router)#end
Now we see Router A both sending updates to 224.0.0.9 (multicast address reserved for RIP updates) and 192.168.4.26:

code:
2w2d: RIP: sending v2 update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.25)
2w2d: RIP: build update entries - suppressing null update
2w2d: RIP: sending v2 update to 192.168.4.26 via GigabitEthernet1/0/1 (192.168.4.25)
2w2d: RIP: build update entries - suppressing null update
2w2d: RIP: sending v2 update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.25)
2w2d: RIP: build update entries - suppressing null update
2w2d: RIP: sending v2 update to 192.168.4.26 via GigabitEthernet1/0/1 (192.168.4.25)
2w2d: RIP: build update entries - suppressing null update
Let's look at what's happening on Router B. Interesting. It looks like Router B is still not advertising anything out of that interface, yet the neighbor is still configured.

code:
RouterB#sh ip proto
*** IP Routing is NSF aware ***

Routing Protocol is "rip"
  Sending updates every 30 seconds, next due in 12 seconds
  Invalid after 180 seconds, hold down 180, flushed after 240
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Redistributing: rip
  Neighbor(s):
    192.168.4.25
  Default version control: send version 2, receive version 2
    Interface                 Send  Recv  Triggered RIP  Key-chain
    GigabitEthernet1/0/1      2     2
    GigabitEthernet1/0/2      2     2
  Automatic network summarization is not in effect
  Maximum path: 4
  Routing for Networks:
    10.0.0.0
    192.168.4.0
  Routing Information Sources:
    Gateway         Distance      Last Update
    192.168.4.25         120      00:08:34
  Distance: (default is 120)
code:
2w2d: RIP: sending v2 update to 224.0.0.9 via GigabitEthernet1/0/2 (10.31.13.1)
2w2d: RIP: build update entries
2w2d:   192.168.0.0/16 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 1 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/2
2w2d: RIP: sending v2 update to 224.0.0.9 via GigabitEthernet1/0/2 (10.31.13.1)
2w2d: RIP: build update entries
2w2d:   192.168.0.0/16 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 1 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/2
So! Let's go ahead and remove the neighbor statements since they don't seem to be doing any good whatsoever, and see what happens when we fix the mask on Router B. Once the mask is fixed I'll clear out the route table...

code:
RouterB#clear ip route *
RouterB#
2w2d: RIP: sending v2 update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.26)
2w2d: RIP: build update entries
2w2d:   10.31.13.0/24 via 0.0.0.0, metric 1, tag 0
2w2d:   192.168.4.24/30 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 2 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/1
2w2d: RIP: sending request on GigabitEthernet1/0/1 to 224.0.0.9
2w2d: rip_route_adjust for GigabitEthernet1/0/1 coming up
2w2d: RIP: sending request on GigabitEthernet1/0/1 to 224.0.0.9
2w2d: RIP: sending request on GigabitEthernet1/0/2 to 224.0.0.9
2w2d: rip_route_adjust for GigabitEthernet1/0/2 coming up
2w2d: RIP: sending request on GigabitEthernet1/0/2 to 224.0.0.9
2w2d: RIP: remove GigabitEthernet1/0/1 from RIP idb list
2w2d: RIP: remove GigabitEthernet1/0/2 from RIP idb list
2w2d: RIP: add GigabitEthernet1/0/2 to RIP idb list
2w2d: RIP: sending request on GigabitEthernet1/0/2 to 224.0.0.9
2w2d: RIP: add GigabitEthernet1/0/1 to RIP idb list
2w2d: RIP: sending request on GigabitEthernet1/0/1 to 224.0.0.9
2w2d: RIP: received v2 update from 192.168.4.25 on GigabitEthernet1/0/1
2w2d:      192.168.4.24/30 via 0.0.0.0 in 1 hops
2w2d: RIP: Update contains 1 routes
2w2d: RIP: sending v2 flash update to 224.0.0.9 via GigabitEthernet1/0/2 (10.31.13.1)
2w2d: RIP: build flash update entries
2w2d:   192.168.4.24/30 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 1 routes
2w2d: RIP: Update queued
2w2d: RIP: sending v2 flash update to 224.0.0.9 via GigabitEthernet1/0/1 (192.168.4.26)
2w2d: RIP: build flash update entries
2w2d:   10.31.13.0/24 via 0.0.0.0, metric 1, tag 0
2w2d:   192.168.4.24/30 via 0.0.0.0, metric 1, tag 0
2w2d: RIP: Update contains 2 routes
2w2d: RIP: Update queued
2w2d: RIP: Update sent via GigabitEthernet1/0/2
2w2d: RIP: Update sent via GigabitEthernet1/0/1
2w2d: RIP: received v2 update from 192.168.4.25 on GigabitEthernet1/0/1
2w2d:      192.168.4.24/30 via 0.0.0.0 in 1 hops
2w2d: RIP: Update contains 1 routes
Ta Daa! So now RIP's all like "HEY WOW LOOK AT WHAT I HAVE" and is happily sending updates out of both interfaces. Let's look at the database:

code:
RouterB#show ip rip database
10.0.0.0/8    auto-summary
10.31.13.0/24    directly connected, GigabitEthernet1/0/2
192.168.4.0/24    auto-summary
192.168.4.24/30    directly connected, GigabitEthernet1/0/1
That looks different than when we looked at it before. Here's both before and after:


code:
RouterB#show ip rip database
10.0.0.0/8    auto-summary
10.31.13.0/24    directly connected, GigabitEthernet1/0/2
192.168.0.0/16    redistributed
    [0] via 0.0.0.0,
192.168.4.0/24    auto-summary
192.168.4.24/30
    [1] via 192.168.4.25, 00:00:16, GigabitEthernet1/0/1
code:
RouterB#show ip rip database
10.0.0.0/8    auto-summary
10.31.13.0/24    directly connected, GigabitEthernet1/0/2
192.168.4.0/24    auto-summary
192.168.4.24/30    directly connected, GigabitEthernet1/0/1
Because one end of the link is a /30 and the other end is a /16 - the FIB sees the /16 but the address on the other interface is just too long for the mask. Even if you wanted to add a static route on Router A towards Router B maintaining your addressing scheme, you won't be able to:

code:
RouterA(config)#ip route 192.168.4.26 255.255.0.0 gi1/0/1
%Inconsistent address and mask
So let's change the interface on Router B back to a /16 and change the interface on router A to a /16, change the RIP network statements accordingly and clear the routing table on both sides and see what happens.

Router B is still not sending updates on the interface:

code:
RouterB#
2w2d: RIP: sending request on GigabitEthernet1/0/2 to 224.0.0.9
2w2d: rip_route_adjust for GigabitEthernet1/0/2 coming up
2w2d: RIP: sending request on GigabitEthernet1/0/2 to 224.0.0.9
2w2d: RIP: remove GigabitEthernet1/0/2 from RIP idb list
2w2d: RIP: add GigabitEthernet1/0/2 to RIP idb list
2w2d: RIP: sending request on GigabitEthernet1/0/2 to 224.0.0.9
2w2d: RIP: sending v2 flash update to 224.0.0.9 via GigabitEthernet1/0/2 (10.31.13.1)
2w2d: RIP: build flash update entries - suppressing null update
Neither is Router A

code:
RouterA#
RouterA#
RouterA#
RouterA#
RouterA#
RouterA#show debug
IP routing:
  RIP protocol debugging is on
  RIP event debugging is on
RouterA#
RouterA#
RouterA#
RouterA#clear ip route *
RouterA#
RouterA#
RouterA#show run | b r rip
router rip
 version 2
 network 192.168.0.0
 no auto-summary
Router A might as well not be running RIP, because the network statement on both sides breaks the classful rules of configuring the network statements.

I guess in conclusion, you're pretty much hosed unless you change the addressing.

Jeff73
Feb 3, 2006

Hello Arkady,

Why must the interfaces remain /30 and /16? Are you flatly forbidden from altering prefix lengths or is there a specific functional reason for both or either needing to be left alone, like other devices on the local loop that need the router or VSAT to respond to or send L.3 broadcasts? Do you need rip to advertise the entire 192.168.0.0 /16 range to the device in 10.31.13.0? Are you allowed to add secondary addresses?

If yes and yes, dropping the .26 address down to /30 and adding an otherwise-unused secondary address in the same /16 range to Router B's fa0/0 will accomplish that.

This: (three 2621s, split horizon and auto-summarization enabled to match your config

Cloud
interface FastEthernet0/0
ip address 10.31.13.2 255.255.255.0

router rip
version 2
network 10.0.0.0


Router B
interface FastEthernet0/0
ip address 192.168.4.230 255.255.0.0 secondary
ip address 192.168.4.26 255.255.255.252

interface FastEthernet0/1
ip address 10.31.13.1 255.255.255.0

router rip
version 2
network 10.0.0.0
network 192.168.4.0


VSAT
interface FastEthernet0/0
ip address 192.168.4.25 255.255.255.252

router rip
version 2
network 192.168.4.0


Yielded this:

Cloud
sh ip route
....
R 192.168.4.0/24 [120/1] via 10.31.13.1, 00:00:13, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.31.13.0 is directly connected, FastEthernet0/0
R 192.168.0.0/16 [120/1] via 10.31.13.1, 00:00:13, FastEthernet0/0

debug ip rip database
debug ip rip events

*Mar 1 02:31:52.887: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/0 (1
0.31.13.2) - suppressing null update
CLOUD#
*Mar 1 02:31:58.687: RIP: received v2 update from 10.31.13.1 on FastEthernet0/0

*Mar 1 02:31:58.691: RIP-DB: network_update with 192.168.0.0/16 succeeds
*Mar 1 02:31:58.691: RIP-DB: adding 192.168.0.0/16 (metric 1) via 10.31.13.1 on
FastEthernet0/0 to RIP database
*Mar 1 02:31:58.691: RIP-DB: network_update with 192.168.4.0/24 succeeds
*Mar 1 02:31:58.691: RIP-DB: adding 192.168.4.0/24 (metric 1) via 10.31.13.1 on
FastEthernet0/0 to RIP database
*Mar 1 02:31:58.691: RIP: Update contains 2 routes


Router B
sh ip route
....
192.168.4.0/30 is subnetted, 1 subnets
C 192.168.4.24 is directly connected, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.31.13.0 is directly connected, FastEthernet0/1
C 192.168.0.0/16 is directly connected, FastEthernet0/0

debug ip rip database
debug ip rip events

*Mar 1 02:47:44.811: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/1 (1
0.31.13.1)
*Mar 1 02:47:44.811: RIP: Update contains 2 routes
*Mar 1 02:47:44.811: RIP: Update queued
*Mar 1 02:47:44.811: RIP: Update sent via FastEthernet0/1
*Mar 1 02:47:52.311: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/0 (1
92.168.4.26)
*Mar 1 02:47:52.311: RIP: Update contains 1 routes
*Mar 1 02:47:52.311: RIP: Update queued
*Mar 1 02:47:52.311: RIP: Update sent via FastEthernet0/0


VSAT
sh ip route
C 192.168.4.0/24 is directly connected, FastEthernet0/0
R 10.0.0.0/8 [120/1] via 192.168.4.26, 00:00:12, FastEthernet0/0


debug ip rip database
debug ip rip events

*Mar 1 01:36:41.163: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/0 (1
92.168.4.25) - suppressing null update
*Mar 1 01:36:42.059: RIP: received v2 update from 192.168.4.26 on FastEthernet0
/0
*Mar 1 01:36:42.059: RIP-DB: network_update with 10.0.0.0/8 succeeds
*Mar 1 01:36:42.059: RIP-DB: adding 10.0.0.0/8 (metric 1) via 192.168.4.26 on F
astEthernet0/0 to RIP database
*Mar 1 01:36:42.059: RIP: Update contains 1 routes

If that's not precisely what you need each to see in their routing tables, alter it by switching auto-summary off and/or adding a distribute list or two.

If you aren't allowed to add secondary addresses or have no extra addresses within 192.168.0.0 /16, I'm out of ideas pending elaboration on your addressing restrictions.

Jeff73
Feb 3, 2006

Wait, no secondary addresses are needed. This also worked with my equipment:
On routerB:
int fa0/0

no ip add 192.168.4.230 255.255.0.0 secondary
ip add 192.168.4.26 255.255.0.0

...puts us back where we were:

debug ip rip data
debug ip rip event

*Mar 1 03:15:22.063: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/1 (1
0.31.13.1)
*Mar 1 03:15:22.063: RIP: Update contains 2 routes
*Mar 1 03:15:22.063: RIP: Update queued
*Mar 1 03:15:22.063: RIP: Update queued
RouterB#
*Mar 1 03:15:24.847: RIP: received v2 update from 10.31.13.2 on FastEthernet0/1

*Mar 1 03:15:24.847: RIP: Update contains 1 routes

However, adding a static route to 192.168.4.24 255.255.255.252 via fa0/0 causes Router B to do this:

RouterB(config)#ip route 192.168.4.24 255.255.255.252 fa0/0
RouterB(config)#
*Mar 1 03:17:23.151: RIP-DB: redist 192.168.4.0/30(metric 0, last interface Fas
tEthernet0/0) to RIP
*Mar 1 03:17:23.155: RIP-DB: redist 192.168.4.24/30(metric 0, last interface Fa
stEthernet0/0) to RIP
*Mar 1 03:17:23.155: RIP-DB: Get redist for network 192.168.4.24
*Mar 1 03:17:23.155: RIP-DB: adding 192.168.4.24/30 (metric 0) via 0.0.0.0 on F
astEthernet0/0 to RIP database
*Mar 1 03:17:23.155: RIP-DB: add 192.168.4.24/30 (metric 0) via 0.0.0.0 on Fast
Ethernet0/0 (donot_age)
*Mar 1 03:17:23.155: RIP-DB: Adding new rndb entry 192.168.4.24/30
*Mar 1 03:17:23.159: RIP-DB: Created rip ndb summary entry for 192.168.4.0/24
RouterB(config)#^Z
RouterB#
*Mar 1 03:17:23.159: RIP-DB: Adding new rndb entry 192.168.4.0/24
*Mar 1 03:17:23.159: RIP: sending request on FastEthernet0/0 to 224.0.0.9
RouterB#
*Mar 1 03:17:24.575: %SYS-5-CONFIG_I: Configured from console by console

And then begin doing this:

*Mar 1 03:17:32.367: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/0 (1
92.168.4.26)
*Mar 1 03:17:32.367: RIP: Update contains 1 routes
*Mar 1 03:17:32.367: RIP: Update queued
*Mar 1 03:17:32.367: RIP: Update sent via FastEthernet0/0
RouterB#
*Mar 1 03:17:44.583: RIP: sending v2 update to 224.0.0.9 via FastEthernet0/1 (1
0.31.13.1)
*Mar 1 03:17:44.583: RIP: Update contains 2 routes
*Mar 1 03:17:44.583: RIP: Update queued
*Mar 1 03:17:44.583: RIP: Update sent via FastEthernet0/1


Checking the routing tables shows:

Cloud
sh ip route
....
R 192.168.4.0/24 [120/1] via 10.31.13.1, 00:00:18, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.31.13.0 is directly connected, FastEthernet0/0
R 192.168.0.0/16 [120/1] via 10.31.13.1, 00:00:18, FastEthernet0/0

Router B
sh ip route
....
192.168.4.0/30 is subnetted, 1 subnets
S 192.168.4.24 is directly connected, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.31.13.0 is directly connected, FastEthernet0/1
C 192.168.0.0/16 is directly connected, FastEthernet0/0

VSAT
sh ip route
....
C 192.168.4.0/24 is directly connected, FastEthernet0/0
R 10.0.0.0/8 [120/1] via 192.168.4.26, 00:00:05, FastEthernet0/0

If this won't do either, I'll have to wait for elaboration on the addressing situation. :confused:

atticus
Nov 7, 2002

this is how u post~
:madmax::hf::riker:
Nice work Jeff.

It makes sense now that I look at it... RouterB now has an exact route in its FIB to send data to that network. I think I was putting my static route in the wrong place. :smith:

Jeff73
Feb 3, 2006

^^^ Actually, it was re-reading your experimentation that made me think of it. :) "Oh yeah, a static to the /30 would be inside the /16."

GodofLint posted:

Crappy Situation.

I have a couple questions:
1. What model(s) are the Netscreen boxes?
2. How feasible is gaining control of your DHCP service? Is it a turf war or do the central folks have a legit reason?
3. What are the address ranges being given to your remote clients? (disguised if public - just looking for prefix lengths, contiguity, etc.)

Ninja Rope
Oct 22, 2005

Wee.
You know, I'm usually not one for hacky one-off solutions, but you could set up a *ix machine with 1:1 NAT as your new gateway. Have the "outside" interface of this box run DHCP and always grab X number addresses and have the inside then redistribute an equal number of (different) addresses to clients. Basically, a second level of NAT and DHCP that you control, but have the *ix box do 1:1 NAT on top of the real DHCP server's leases.

Probably too complicated and hard to support to be realistic, but still sounds like a fun project.

Edit: Do you have control over the Juniper boxes? I was assuming not...

Ninja Rope fucked around with this message at 05:15 on Jul 17, 2007

Jeff73
Feb 3, 2006

That's close to what I was thinking, but with a Pix/ASA it's pretty straightforward to set up. I want his Juniper part numbers to see if I can grab config guides and translate Cisco into Junipese. There are a few ways he could solve the DHCP problem using his firewalls (assuming they rival a Pix), but none might be doable for political or local technical reasons. If one of them is, add a Lan to Lan VPN for server traffic and a route to the firewall's local router for all else and he'd be set.

At the least, he'd have a bargaining chip for getting his own DHCP server: "Gimme DHCP before I get hit by a truck and leave you to deal with this monstrous kludge."

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Jeff73 posted:

I have a couple questions:
1. What model(s) are the Netscreen boxes?
2. How feasible is gaining control of your DHCP service? Is it a turf war or do the central folks have a legit reason?
3. What are the address ranges being given to your remote clients? (disguised if public - just looking for prefix lengths, contiguity, etc.)

The netscreen box in the main building is a NS50, while the external sites are all NS5G's. I was talking to the guy who installed them, and it seems like they have the capability to do what we need with the only big problem being that the NS50 might not be able to handle the load and we will need to move to something beefier. I guess the real question is how to get those boxes to push the external traffic out to the building routers we don't control and keep the traffic we do want moving along the VPN link to the main building.

As for DHCP, it is run through the university networking people and all clients must have their MAC registered through an abomination of a web interface. Works great for kids in the dorms, but hampers our operations severely. The end goal would be to cut off DHCP traffic at the building firewalls and drop in a DHCP server with IP blocks that we will be assigned to control from the campus people. That would be done after the sites are linked correctly and that traffic only has to be cut off from a single point.

One of the big messy parts of this is that we are using fully public IPs and there is no way in hell I will be able to change that in to a fully NATed environment at this point. IPs are in the 111.x.x.x/255.248.0.0 range, with each building typically getting 1 to 3 111.x.x.0 DHCP assigned IP blocks to work with depending on the size, along with a static range for weird crap that needs it. Those would obviously change to IPs assigned to the main building once the DHCP traffic is routed through the VPN.

BangersInMyKnickers fucked around with this message at 19:13 on Jul 17, 2007

atticus
Nov 7, 2002

this is how u post~
:madmax::hf::riker:

GodofLint posted:

One of the big messy parts of this is that we are using fully public IPs and there is no way in hell I will be able to change that in to a fully NATed environment at this point. IPs are in the 111.x.x.x/255.248.0.0 range, with each building typically getting 1 to 3 111.x.x.0 DHCP assigned IP blocks to work with depending on the size, along with a static range for weird crap that needs it. Those would obviously change to IPs assigned to the main building once the DHCP traffic is routed through the VPN.

Jesus christ what a big lovely mess

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

atticus posted:

Jesus christ what a big lovely mess

Big Ten universities only hire the very best and brightest.

One day I hope they can get someone in here far better than me to kick some rear end, create a DMZ for servers that need to be exposed, and put everything else behind a NAT boundary like it should be. For my purposes I would be happy if I could just control our DHCP server and VPN links to push out machine images through a PXE environment. University bureaucracy will most likely keep that from happening until it all blows up in some poor smuck's face and he gets fired.

Jeff73
Feb 3, 2006

GodofLint posted:

Full NAT not possible.

When you say full NAT is not possible, do you mean the university's network policy requires that all hosts use public addresses? If not, I'm surprised they're resisting given that you'd be completely out of their hair except for the bandwidth used by your tunnels and the remote sites' web traffic.

GodofLint posted:

Need to divert non-serverbound traffic to the local router.

Let me finish a couple 1-2 day local projects and I'll hit Juniper's site. I've never gotten to play with their stuff and it sounds interesting. Are you looking at just hub-and spoke or full/partial mesh? Do the remote sites have their own local file/other heavy-traffic servers or are they all in the center?

BangersInMyKnickers
Nov 3, 2004

I have a thing for courageous dongles

Jeff73 posted:

When you say full NAT is not possible, do you mean the university's network policy requires that all hosts use public addresses? If not, I'm surprised they're resisting given that you'd be completely out of their hair except for the bandwidth used by your tunnels and the remote sites' web traffic.

It is more not possible due to internal bureaucracy and the old "ain't broke don't fix it" mentality. Except things ARE broken and people are too busy sticking their heads in the sand because they don't want to take the incentive to fix things or not having any concept of how very hosed up our current operating model is. The university network people tend to be pretty accommodating when it comes to things like this (mostly all they need to do is allocate us a large enough IP block for the extra machines that will be "inside" our building), but they are expressing their doubts as we would be the first ones to successfully pull it off. Other divisions or departments have tried and failed in the past.

It is going to be completely hub and spoke. All servers are located at the central site with just printers and clients at the remote offices on campus.

And I can't tell you how much I appreciate the help here. Thank you.

MrZodiac
Jul 19, 2005

Dinosaur Gum
I'm troubleshooting a very weird website performance issue. The tie in to Cisco is that both ends of the equation sit behind ASA 5510's [7.0(4)]. My Wireshark dumps have lead me to possible TCP checksum problems across the board. But in researching that I found that 1) it's very rare, and 2) Wireshark has trouble dealing with NICs that do Checksum Offloading. So, how would I verify if I am having checksum problems? Both ASA units show zero checksum errors with the tcpstat output. TCP offloading seems very common on modern NICs, especially gigabit.

Arkady
Jun 18, 2004

Off to work!

Jeff73 posted:

RouterB(config)#ip route 192.168.4.24 255.255.255.252 fa0/0

Hey guys,
I have tested this idea in the lab and it works perfectly. Jeff, Atticus, thank you very much for your help.

It is still very interesting why the IOS looks to send out a /16 (or even /24) RIP update from a /30 network. I'll try to do some tests tomorrow and try to figure this out.
The only two options I see are at the moment are; the other side of the network being /30 or the address on the Interface being somehow in in the /30 network. Both seem very unlikely.
I'm probably missing something.

Korensky
Jan 13, 2004

MrZodiac posted:

I'm troubleshooting a very weird website performance issue. The tie in to Cisco is that both ends of the equation sit behind ASA 5510's [7.0(4)]. My Wireshark dumps have lead me to possible TCP checksum problems across the board. But in researching that I found that 1) it's very rare, and 2) Wireshark has trouble dealing with NICs that do Checksum Offloading. So, how would I verify if I am having checksum problems? Both ASA units show zero checksum errors with the tcpstat output. TCP offloading seems very common on modern NICs, especially gigabit.

If you are capturing from the local machine using wireshark and using TCP checksum offloading the calculation is performed on the NIC so the checksum at the driver/wireshark level will be 0 (it's not until the packet is hitting the wire itself that the NIC will calculate this). Try using the capture command on the ASA (if you can still do this) or capture through a mirrored port on the switch if you want to see what it looks like on the wire.

What is the particular website performance issue you're looking at? MTU/MSS problems the the most common ones I can think of if you're looking at the network level.

Jeff73
Feb 3, 2006

Hokay, my servers are clean of viruses (Thank you, Guy Who Infected The Entire Domain With An Unsecure XP Box And Domain Admin Access!)

Arkady, which OS version(s) are the firewalls running? Do you have a CSC account? I lack one, so the Juniper knowledge base is playing cagey.

Arkady
Jun 18, 2004

Off to work!

Jeff73 posted:

Arkady, which OS version(s) are the firewalls running? Do you have a CSC account? I lack one, so the Juniper knowledge base is playing cagey.

I think you mean GodofLint? :confused:

Wicaeed
Feb 8, 2005
I'm gonna be starting my CCNA soon, do you guys have any suggestions as to which books/routers I should be buying for my labs?

Jeff73
Feb 3, 2006

Arkady posted:

:confused:


Hah, yes, sorry; I was writing a post to him and a PM to you in tabs simultaneously and got my wires crossed.

More topically, I wish Juniper's site were like Cisco's, which reserves only software and web apps for CCO members. I've found plenty of GUI VPN-setup guides and documents in Juniper's databse, but the CLI info invariably asks for my nonexistent corporate login. I've set up a site-to-site with a couple lab 515e's that's similar to what God of Lint might need, but after lurking in the juniper.com forums I think that the way Netshields split traffic between the tunnel and the local router is different from how Cisco handles it. If he could snag a complete version of the CLI config guide and command reference for his version of the firewall O/S, it'd be awesome.

wicaed posted:

CCNA books/tools
Initial study: The Cisco Press CCNA 1-2 and 3-4 Network Academy companion guides, Third Edition, cover everything in the CCNA Netacad program module-for-module, plus a little - but beware of the numerous errors. Try to follow every chapter with a read-through of related Cisco white papers and guides + other online sources. Actually, never trust anyone no matter how far you progress; even the CCIE written prep by Odom is full of crazy mistakes. Odom is the next thing to Yoda, but, for instance, one of the chapters asserted three times (including in the quiz) that a newly-added switch in VTP client mode can't overwrite vlans in the domain regardless of its config revision number. :downs:

Review close to exam time: The Cisco Press exam Intro and ICND exam prep books are pretty good and come with the Boson practice exam engine. The Boson exams seemed trickier than the actual cert tests to me, so they might be a good measuring stick to determine readiness.

Router to play with: jwh gave solid advice a few pages back: grab a used 3640 and use the IOS to run Dynamips/gen. Cisco technically ought not to like people doing that, but it's openly discussed on the Cisco forums as a good student's tool. You might also want to grab a 2950 switch if you can afford it, as well as something old'n cheap with CatOS so you can learn at least its basic syntax for legacy equipment.

atticus
Nov 7, 2002

this is how u post~
:madmax::hf::riker:

Jeff73 posted:

one of the chapters asserted three times (including in the quiz) that a newly-added switch in VTP client mode can't overwrite vlans in the domain regardless of its config revision number. :downs:

annnnnd how is that incorrect? :confused:

I'm almost positive that he's correct, as only switches in VTP server mode can modify VLAN information. Wikipedia confirms...

From wikipedia:

quote:


VTP operates in one of three modes: server, client and transparent.

* Server – In this VTP mode you can create, remove, and modify VLANs. You can also set other configuration options like the VTP version and also turn on/off VTP pruning for the entire VTP domain. VTP servers advertise their VLAN configuration to other switches in the same VTP domain and synchronize their VLAN configuration with other switches based on messages received over trunk links. VTP server is the default mode.

* Client – VTP clients behave the same way as VTP servers, but you cannot create, change, or delete VLANs

* Transparent – When you set the VTP mode to transparent, then the switches do not participate in VTP. A VTP transparent switch will not advertise its VLAN configuration and does not synchronize its VLAN configuration based on received messages. However, in VTP version 2, transparent switches do forward VTP messages that they receive out their trunk ports.

Jeff73
Feb 3, 2006

T'aint so. From http://www.cisco.com/warp/public/473/21.html#vtp_ts_rec_ins

Cisco.com posted:

How a Recently Inserted Switch Can Cause Network Problems

...The configuration revision number of the switch that you inserted was higher than the configuration revision

number of the VTP domain. Therefore, your recently introduced switch, with almost no configured VLANs, erased

all VLANs through the VTP domain.

This happens whether the switch is a VTP client or a VTP server. A VTP client can erase VLAN information on a

VTP server....

To verify, here's the result of testing with a couple 2950s running 12.1:

2950-1 initial state, not connected to 2950-2:
2950-1#sh vtp s
VTP Version : 2
Configuration Revision : 0
Maximum VLANs supported locally : 250
Number of existing VLANs : 5
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0xA2 0xDB 0xD8 0x3B 0x97 0xC2 0x54 0xD5
Configuration last modified by 0.0.0.0 at 0-0-00 00:00:00
Local updater ID is 0.0.0.0 (no valid interface found)
2950-1#sh vlan
VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa0/2, Fa0/3, Fa0/4, Fa0/5
Fa0/6, Fa0/7, Fa0/8, Fa0/9
Fa0/10, Fa0/11, Fa0/12, Fa0/13
Fa0/14, Fa0/15, Fa0/16, Fa0/17
Fa0/18, Fa0/19, Fa0/20, Fa0/21
Fa0/22, Fa0/23, Fa0/24, Gi0/1
Gi0/2
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup


2950-2 initial state, not connected to 2950-1:
2950-2#sh vtp stat
VTP Version : 2
Configuration Revision : 0
Maximum VLANs supported locally : 250
Number of existing VLANs : 5
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0x57 0xCD 0x40 0x65 0x63 0x59 0x47 0xBD
Configuration last modified by 0.0.0.0 at 0-0-00 00:00:00
Local updater ID is 0.0.0.0 (no valid interface found)


Test setup:
1. Set the vtp domain to Lowtax on both switches
2. Add VLAN 10 and 20 to 2950-2, then fiddle with settings until its config revision is 4-5.
3. Change 2950-2 to client mode.

Test start state verification:
2950-1, still disconnected, configuration unchanged except 'vtp domain Lowtax'

2950-2, still disconnected
2950-2(config)#do sh vtp stat
VTP Version : 2
Configuration Revision : 4
Maximum VLANs supported locally : 250
Number of existing VLANs : 7
VTP Operating Mode : Client
VTP Domain Name : Lowtax
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0x96 0x80 0x79 0x9F 0xEC 0x96 0x3E 0xF1
Configuration last modified by 0.0.0.0 at 3-1-93 00:13:03
2950-2(config)#do sh vlan

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa0/2, Fa0/3, Fa0/4, Fa0/5
Fa0/6, Fa0/7, Fa0/8, Fa0/9
Fa0/10, Fa0/11, Fa0/12, Fa0/13
Fa0/14, Fa0/15, Fa0/16, Fa0/17
Fa0/18, Fa0/19, Fa0/20, Fa0/21
Fa0/22, Fa0/23, Fa0/24, Gi0/1
Gi0/2
10 VLAN0010 active
20 VLAN0020 active
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup


Now, connect them and watch what happens to 2950-1's vlan database:

2950-1#debug sw-vlan vtp events
vtp events debugging is on
2950-1#debug sw-vlan events
vlan manager events debugging is on
2950-1#
00:21:13: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
00:21:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, chang
ed state to down
00:21:14: sw_vlan_process: add trunk port Fa0/1
00:21:14: VTP LOG RUNTIME: switchport trunk mode on Fa0/1 has changed
00:21:14: VTP LOG RUNTIME: delaying first flood on new trunk
00:21:15: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, chang
ed state to up
00:21:19: VTP LOG RUNTIME: Summary packet received, domain = Lowtax, rev = 4, fo
llowers = 0
00:21:19: VTP LOG RUNTIME: Summary packet rev 4 greater than domain Lowtax rev 0
00:21:19: VTP LOG RUNTIME: Domain Lowtax currently not in updating state
00:21:19: VTP LOG RUNTIME: Summary packet with followers field zero
00:21:20: VTP LOG RUNTIME: Transmit vtp request, domain Lowtax, start value 0
00:21:20: VTP LOG RUNTIME: Summary packet received, domain = Lowtax, rev = 4, fo
llowers = 1
00:21:20: VTP LOG RUNTIME: Summary packet rev 4 greater than domain Lowtax rev 0
00:21:20: VTP LOG RUNTIME: Domain Lowtax currently not in updating state
00:21:20: VTP LOG RUNTIME: Subset packet received, domain = Lowtax, rev = 4, seq
= 1, length = 244
00:21:20: VTP LOG RUNTIME: Transmit vtp summary, domain Lowtax, rev 4, followers
1
2950-1#sh vlan

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa0/2, Fa0/3, Fa0/4, Fa0/5
Fa0/6, Fa0/7, Fa0/8, Fa0/9
Fa0/10, Fa0/11, Fa0/12, Fa0/13
Fa0/14, Fa0/15, Fa0/16, Fa0/17
Fa0/18, Fa0/19, Fa0/20, Fa0/21
Fa0/22, Fa0/23, Fa0/24, Gi0/1
Gi0/2
10 VLAN0010 active
20 VLAN0020 active
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup


I agree that it ought not to work this way. I also think we might want to start linking config dumps as hosted text files or something, because they are chewing some serious page real estate. :geno:

ragzilla
Sep 9, 2005
don't ask me, i only work here


Jeff73 posted:

T'aint so. From http://www.cisco.com/warp/public/473/21.html#vtp_ts_rec_ins


To verify, here's the result of testing with a couple 2950s running 12.1:

I think this is something with DTP (switchports being set to switchport mode dynamic) than anything else, I just had a customer's switch (Foundry MLX) wipe the VLAN database on one of my cust aggregation switches, the switch was configured with just VLANs and it was in server mode. Nobody had ever configured the VTP domain/secret since we stopped using VTP. That's now in our standard configs (set VTP domain/secret).

Jeff73
Feb 3, 2006

Girdle Wax posted:

Possibly DTP causing a server-mode switch to overwrite VLANs on a switch without a set domain

If the Foundry had a domain name set, what happened to your VLANs is standard Cisco VTP behavior.

Test:
I wiped the two 2950's config and vlan.dat files, then reloaded them and set FA0/1 of both to trunk, disabling DTP to remove it from the equation. I then configured several VLANs on one of them, but left the domain blank. The second one I gave a domain name and a single VLAN.

No-domain switch before:
No-Domain#sh vlan
VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa0/2, Fa0/3, Fa0/4, Fa0/5
Fa0/6, Fa0/7, Fa0/8, Fa0/9
Fa0/10, Fa0/11, Fa0/12, Fa0/13
Fa0/14, Fa0/15, Fa0/16, Fa0/17
Fa0/18, Fa0/1
00:07:59: %SYS-5-CONFIG_I: Configured from console by console9, Fa0/20, Fa0/21
Fa0/22, Fa0/23, Fa0/24, Gi0/1
Gi0/2
10 VLAN0010 active
20 VLAN0020 active
30 VLAN0030 active
40 VLAN0040 active

No-Domain#sh vtp stat
VTP Version : 2
Configuration Revision : 4
Maximum VLANs supported locally : 250
Number of existing VLANs : 9
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0x73 0x62 0x0D 0x68 0xFB 0x8E 0xE9 0x2D
Configuration last modified by 0.0.0.0 at 3-1-93 00:04:55
Local updater ID is 0.0.0.0 (no valid interface found)


Cisco-Domain switch before:
Cisco-Domain(config)#do sh vlan

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa0/1, Fa0/2, Fa0/3, Fa0/4
Fa0/5, Fa0/6, Fa0/7, Fa0/8
Fa0/9, Fa0/10, Fa0/11, Fa0/12
Fa0/13, Fa0/14, Fa0/15, Fa0/16
Fa0/17, Fa0/18, Fa0/19, Fa0/20
Fa0/21, Fa0/22, Fa0/23, Fa0/24
Gi0/1, Gi0/2
47 VLAN0047 active
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup

Cisco-Domain(config)#do sh vtp stat
VTP Version : 2
Configuration Revision : 1
Maximum VLANs supported locally : 250
Number of existing VLANs : 6
VTP Operating Mode : Server
VTP Domain Name : Cisco
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0xE0 0x7E 0xAB 0x17 0xC4 0x84 0x62 0xCB
Configuration last modified by 0.0.0.0 at 3-1-93 00:10:06
Local updater ID is 0.0.0.0 (no valid interface found)


The config revision on No-Domain is higher, but watch the change when I connect the two:
No-Domain#debug sw-vlan vtp events
vtp events debugging is on
No-Domain#
00:02:12: VTP LOG RUNTIME: switchport trunk mode on Fa0/1 has changed
00:02:12: VTP LOG RUNTIME: delaying first flood on new trunk
No-Domain#
00:02:14: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
00:02:15: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, chang
ed state to up
00:02:17: VTP LOG RUNTIME: Summary packet received in NULL domain state
00:02:17: VTP LOG RUNTIME: Summary packet received, domain = Cisco, rev = 1, fol
lowers = 0
00:02:17: VTP LOG RUNTIME: Transitioning from NULL to Cisco domain
00:02:17: VTP LOG RUNTIME: Summary packet rev 1 greater than domain Cisco rev 0
00:02:17: VTP LOG RUNTIME: Domain Cisco currently not in updating state
00:02:17: VTP LOG RUNTIME: Summary packet with followers field zero
00:02:17: VTP LOG RUNTIME: Transmit vtp request, domain Cisco, start value 0
00:02:18: VTP LOG RUNTIME: Summary packet received, domain = Cisco, rev = 1, fol
lowers = 1
00:02:18: VTP LOG RUNTIME: Summary packet rev 1 greater than domain Cisco rev 0
00:02:18: VTP LOG RUNTIME: Domain Cisco currently not in updating state
00:02:18: VTP LOG RUNTIME: Subset packet received, domain = Cisco, rev = 1, seq
= 1, length = 224
00:02:18: VTP LOG RUNTIME: Transmit vtp summary, domain Cisco, rev 1, followers

No-Domain#sh vtp stat
VTP Version : 2
Configuration Revision : 1
Maximum VLANs supported locally : 250
Number of existing VLANs : 6
VTP Operating Mode : Server
VTP Domain Name : Cisco
No-Domain#sh vlan

VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Fa0/2, Fa0/3, Fa0/4, Fa0/5
Fa0/6, Fa0/7, Fa0/8, Fa0/9
Fa0/10, Fa0/11, Fa0/12, Fa0/13
Fa0/14, Fa0/15, Fa0/16, Fa0/17
Fa0/18, Fa0/19, Fa0/20, Fa0/21
Fa0/22, Fa0/23, Fa0/24, Gi0/1
Gi0/2
47 VLAN0047 active
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup


A VTP server that sees a client or server with a higher config revision in the same domain will dump its data in favor of the newcomer's. A VTP server without a domain that sees a domain name in a message will assume that it's a member, switch to that domain, drop its revision to 0, compare its zero to the other switch's 1+, then adopt the other's VLAN data because it's 'better.'

Edit: That's what I get for correcting a sloppy copy-paste by hand.

Jeff73 fucked around with this message at 06:16 on Jul 23, 2007

Boner Buffet
Feb 16, 2006
I'm having a strange issue here at work. For the second time in as many days part of our phone system has been almost unusable. It's a cisco VOIP system(call manager, unity vmail). We have two types of phones, this cisco 7960, and the 7905.

Through highly unscientific processes, I think I've determined that it's the voice vlan where this storm is occuring. The data vlan seems unaffected All the ports on all of our Cat 3960 switches, which provide the POE and obviously access to the phones, are blinking at a highly absurd rate. Phone quality is almost completely useless. Making a call itself works roughly 50% of the time. Pulling the uplink cable on every 3950 switch one at a time calms the switch itself down, but I haven't been able to do anything about the possible storm. The core switch is a 4506.

Here's the really strange part. Only the 7905 phones are showing symptoms, not the 7960s.

So I guess my question is, how do I go about calming this storm down? Do broadcast storms work with a head; in other words if I unhook the troubled node how fast does the storm subside? Could my core be causing the issue?

Jeff73
Feb 3, 2006

InferiorWang posted:

Voip traffic issue.

1. Did you change anything in the network around the time the behavior began, like adding network hardware or servers, adding more phones, altering the voice VLAN's QoS settings or changing configurations elsewhere?
2. Are all the phones showing the issue, or is it only happening to those downstream of specific switches?
3. When you reconnect the uplinks after quieting a storm, does it resume immediately? Does it ramp back up gradually? Does it not appear at all at first, then suddenly come back in force later? Does the behavior vary?

If you haven't yet, I suggest dumping SPAN sessions of a couple ports and your voice VLAN, then checking the output with Wireshark to see exactly what and where the excess traffic is. The problem might stand out immediately. If not - and you're allowed to do so - please link private-info-masked /sh run outputs from a representative 3960 and the 4506 in a text file.

Boner Buffet
Feb 16, 2006
Would a rouge NIC possibly cause this? While I was trying to get this sorted, I couldn't remote to our Unity server. I could get into the call manager servers fine. I had to re-seat the patch cable and bounce the box to get it to talk on the network correctly. Once I did that, the storm seemed to subside. I'm not sure if that was the problem or if it was just a coincidence. But to answer your questions:

1. No, I haven't touched anything. As far as I can tell, nobody has added any hardware or done something like plug both ends of a patch cable into one switch.

2. All 7905 model phones. Intermixed 7960s work fine on the same switches.

3. It's pretty much, plug the uplink back into the switch, and within 5 seconds the ports go bonkers again. I noticed there is no convergence time though. I didn't put the switches in myself but I'm going to check the GBIC port configs to see if they have port fast enabled. If they do, could that be part of the problem?

Thanks for the heads up. If the issue continues, I'll give wireshark a whirl.

Korensky
Jan 13, 2004

If it's a storm then wireshark will quickly be overwhelmed. I suggest you turn spanning-tree portfast off everywhere to start with, enable CDP if you haven't already and use "sh cdp nei detail" to locate any switches that may be patched in a topology loop (which is REAL easy to do if your cabling isn't organised). I prefer the CDP command because I can quickly use it to see things like "why is there another switch plugged into gi0/12 when all my GEC trunks are gi 0/44-48" etc.

Edit: and if those switches are 3560s, turn on autoQOS or some form of QOS :)

I mention it because I had a topology loop eventually running over a million pps for about 6 hours one day until it finally caused any noticeable impact on the phones (mess up where one switch wasn't running MST properly).

Korensky fucked around with this message at 22:44 on Jul 23, 2007

Jeff73
Feb 3, 2006

Korensky posted:

Wireshark can't hang. Try CDP, no portfast. Use QoS.

Upon testing, Wireshark was able to handle a SPAN feed from a saturated gig link long enough to capture a slice of the data decent enough to spot the nature of the excessive traffic I put on it to test. Mirroring the port leading from his Unity or a 3560 uplink shouldn't be a problem. I'd only skip it if the switch's CPU is close enough to pegging that SPAN might crash it.

Regarding the rest, making sure QoS is configured and checking for rogue switches via CDP sounds good. :)
I can't tell though: are you suggesting he leave portfast off and CDP on for all ports indefinitely, or only during testing?

Korensky
Jan 13, 2004

Jeff73 posted:

Upon testing, Wireshark was able to handle a SPAN feed from a saturated gig link long enough to capture a slice of the data decent enough to spot the nature of the excessive traffic I put on it to test.

Sorry - this was more to do with most PC NICs being unable to handle the interrupt-fest that is capturing line-rate GigE rather than a jab at wireshark itself. And I guess it's true - he doesn't really need an accurate capture - just a snapshot of the crap (which should be easily visible).

quote:

I can't tell though: are you suggesting he leave portfast off and CDP on for all ports indefinitely, or only during testing?

Personally I leave CDP on everywhere internally (no cdp enable on outside facing interfaces of course) and put portfast back on once you've determined the source of the fault and fixed it. Then lock the racks your patch panels are in so the users can't tamper with your cabling :)

Boner Buffet
Feb 16, 2006
I'm hoping that I pinpointed the issue to the NIC in our Unity server acting up. I haven't heard anything since I bounced it.

We do have QOS enabled to favor the voice VLAN. I'm not sure on CDP. I'll have to double check the configs. I didn't install the gear myself. To be honest, I know enough to be dangerous with cisco gear at this point, but I wouldn't call myself an expert by any stretch!

Jeff73
Feb 3, 2006

Korensky posted:

Then lock the racks your patch panels are in so the users can't tamper with your cabling :)

I wish. :( This is one of my IDFs:

There's no cover for the front, the room is open to the public two doors from a wino-infested street, and those are 4003's with engine I. Anyone with an old mac serial cable can grab control by typing enter-enter within 30 secs of pulling and replacing the power cords. :smith:
I'm thinking of fabricating a steel plate that wraps around the back and covers the power/console areas, but I'm open to suggestions.

Wang, when you inspect the config, configuring [CatOS](enable) set port host [IOS](config-if-range)#switchport host and bpdufilter/guard will help protect against loops - though maybe not the kind Korensky encountered. Korensky, how did MST fail and what was the fix?

Ninja Rope
Oct 22, 2005

Wee.

Jeff73 posted:

I'm thinking of fabricating a steel plate that wraps around the back and covers the power/console areas, but I'm open to suggestions.

Not to alarm you, but a company I worked for a few years back once had the wiring closet in a remote office broken into by an angry heroin junkie. Unfortunately, the wiring closet looked a lot like yours (but with an 1u server, cisco 2600, and a netscreen 5 in place of your 4003's), and before anyone could stop him he managed to completely rip out all of the cabling, pry the 2600 and ns5 free and throw them to the floor, pull the t1 smart jack off the wall, ruin the patch panel, bend the rails on the 1u, etc. Fortunately, the netscreen and the 1u server survived the attack, but the wic on the 2600 was trashed, along with the rest of the patch panel and a lot of the cabling.

From then on, the corporate office mandated steel cages (datacenter style) or at least locking 4 post racks in every office.

Jeff73
Feb 3, 2006

Ninja Rope posted:

Not to alarm you with TALE OF TERROR

I said I was open to suggestions, not heart attacks. :psyduck:

mezoth
Aug 7, 2006

Jeff73 posted:

I said I was open to suggestions, not heart attacks. :psyduck:

Disgruntled employees are worse - I worked for a company that laid a bunch of people off during the bubble-pop a few years back, and a line tech walked into a room and cut around 60 voice DS3 connections. It was hell to get fixed from where I was (4 states away) because none of the remaining line-techs cared and the manager was tech incompetent. This helped the company go into bankruptcy even faster!

Honestly, if you do not have a seriously reinforced door on that closet then get a cage installed. Its not as expensive as you think, and keeping junkies/thieves/etc several feet back normally will save your equipment. I cant help you about the 4003's with sup1s however, that is just pushing your luck at this stage. :)

Jeff73
Feb 3, 2006

mezoth posted:

Honestly, if you do not have a seriously reinforced door on that closet

I wish it were a closet. It's a large, open room that connects to other rooms and is heavily traveled. The room's hallway door has a window through which the equipment is clearly visible and past which local derelicts occasionally stagger on their way to use the semi-public bathroom as a makeshift shower facility. I arrive every day expecting to find it covered with urine or the cables scattered and their insulation chewed. The department itself has an uncertain future, so solutions to this situation that cost more than $0 are verboten by the powers that be.

I'm buying a doberman. :colbert:

Adbot
ADBOT LOVES YOU

jwh
Jun 12, 2002

Jeff73 posted:

I'm buying a doberman. :colbert:

How about some of these? http://www.speedysigns.com/signs/DANGER_High_Voltage.asp

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply