Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
unknown
Nov 16, 2002
Ain't got no stinking title yet!


Drighton posted:

I believe it means its refurbished.

No - it's literally a "spare" for your network. Basically it comes without _any_ software licenses included, since you've obviously already got them for the device that you're assuming will break.

It's basically for those people that need to replace gear without going through TAC first so your downtime is lowered.

Adbot
ADBOT LOVES YOU

Paul Boz_
Dec 21, 2003

Sin City
Yeah, basically it's treated as a clone by Cisco, for the purpose of replacing equipment with equipment already on hand rather than suffering from downtime while waiting for replacements. I had to buy two spare OC12 cards at work the other day :coolfish:

conntrack
Aug 8, 2003

by angerbeet
Sounds reasonable enought. Might hop on that bandwagon later on then.

Drighton
Nov 30, 2005

conntrack posted:

Sounds reasonable enought. Might hop on that bandwagon later on then.

Having a spare for important equipment is the best career/business decision you can make. Seriously, I've had people tell me how crucial a specific device is to their network, and with it dead they are losing "millions of dollars", and poo poo hits the fan with their coverage. I won't name names, but this has even happened with city utilities.

Boner Buffet
Feb 16, 2006
code:
duplex mismatch discovered on GigabitEthernet0/1 
(not full duplex), with Switch GigabitEthernet0/1 (full duplex)
I'm trying the whole learn as I go thing. I've got a 2950 sitting on my desk I'm farting around with and I'm getting the error message above on the console. Are the two Gig0/1 ports on different switches all together?

ragzilla
Sep 9, 2005
don't ask me, i only work here


InferiorWang posted:

code:
duplex mismatch discovered on GigabitEthernet0/1 
(not full duplex), with Switch GigabitEthernet0/1 (full duplex)
I'm trying the whole learn as I go thing. I've got a 2950 sitting on my desk I'm farting around with and I'm getting the error message above on the console. Are the two Gig0/1 ports on different switches all together?

Yes. Cisco devices typically talk across links using a protocol known as CDP (Cisco Discovery Protocol) to tell each other their hostnames / ip addresses / settings they're using on the link.

In this case, the switch you are looking it has it's Gi0/1 port set to half-duplex, but the switch on the other side of the link is reporting in it's CDP packets that it has the link set for full-duplex. Check your int Gi0/1 configuration, and remove any commands like half-duplex or switchport nonegotiate.

Boner Buffet
Feb 16, 2006
Cheers. I checked the Gi0/1 interface and it was set to auto duplex which in turn put it as half duplex.

ionn
Jan 23, 2004

Din morsa.
Grimey Drawer

InferiorWang posted:

Cheers. I checked the Gi0/1 interface and it was set to auto duplex which in turn put it as half duplex.

That's what happens if you connect an auto-duplex port to a full-duplex one. Since it can't autonegotiate, it goes to half-duplex, and you get a shitload of unnecessary duplex collisions (because the other end still sends as if it were full duplex).

ragzilla
Sep 9, 2005
don't ask me, i only work here


ionn posted:

That's what happens if you connect cisco equipment to cisco equipment.

Fixed.

I still can't believe how horribly unreliable autonegotiation Cisco-to-Cisco is. It's crazy that in this day and age we still have to go through and manually force speed/duplex on ports to stop Cisco gear from doing something stupid.

vvv You've probably never had the fun of using a 7200 with an IO-FE or PA-FE-TX, they're both absolutely terrible at autonego, I didn't even bother letting them autonego after the first one I had to turn up.

ragzilla fucked around with this message at 19:36 on May 10, 2007

Herv
Mar 24, 2005

Soiled Meat
/\ /\ /\ /\

I had a 7507 with a fast ethernet controller on it, but it was about 6 years ago, couldn't for the life of me remember what type it was. I do remember I had 2-100mbit connections trunked to the router with autoneg on the link speed/duplex but once again, not sure on the actual card. I guess it's a mileage may vary thing there.

Girdle Wax posted:

Fixed.

I still can't believe how horribly unreliable autonegotiation Cisco-to-Cisco is. It's crazy that in this day and age we still have to go through and manually force speed/duplex on ports to stop Cisco gear from doing something stupid.

Wierd, I have rarely had problems with auto negotiation with both sides being in auto. Having said that, 3Com 905x and the Cisco 6509/3xxx LAN switches certainly have made my day go poorly.

Herv fucked around with this message at 20:44 on May 10, 2007

ate shit on live tv
Feb 15, 2004

by Azathoth

Herv posted:

Wierd, I have rarely had problems with auto negotiation with both sides being in auto. Having said that, 3Com 905x and the Cisco 6509/3xxx LAN switches certainly have made my day go poorly.

I work in a Cisco SP Lab. It's been my experience that if both sides are set to Auto-neg, than there is no problem. However if one side is set to half-duplex, or it can only do half-duplex like on a FastHub 400 or something, then auto negotiate won't work. So normally for our default configs we set all interfaces to full-duplex, and there isn't a problem.

Kashekya
Dec 11, 2003

Pillbug
I'm having some trouble with my PIX501, v6.3(5). We have a pretty small office here where I work and we use the pix501 as an outside router.

It is a pretty simple config with the outside interface of the pix configured with our static IP from our ISP. The inside interface is 192.168.10.254, with our internal server that handles everything being 192.168.10.1

My problem is that we want people to be able to be able to be remote clients using pptp vpn dialin. I have everything configured so a client from the outside can dial in and connect no problem. Dialin users are given an ip from the pool of 192.168.5.10-192.168.5.25

The problem is, even though they can connect no problem, they can't go anywhere from there, cannot connect to our internal server(192.168.10.1) or anything.

I have tried a few things with access lists but I must be missing something easy I think.

dwarftosser
Sep 3, 2002

PLEASE LET ME SUCK YOUR COCK, BRETT!

The Fecal Jesus posted:

I'm having some trouble with my PIX501, v6.3(5). We have a pretty small office here where I work and we use the pix501 as an outside router.

It is a pretty simple config with the outside interface of the pix configured with our static IP from our ISP. The inside interface is 192.168.10.254, with our internal server that handles everything being 192.168.10.1

My problem is that we want people to be able to be able to be remote clients using pptp vpn dialin. I have everything configured so a client from the outside can dial in and connect no problem. Dialin users are given an ip from the pool of 192.168.5.10-192.168.5.25

The problem is, even though they can connect no problem, they can't go anywhere from there, cannot connect to our internal server(192.168.10.1) or anything.

I have tried a few things with access lists but I must be missing something easy I think.

Give your remote clients addresses on your same subnet instead of 192.168.5 addresses.

Also PIXs often have alot of NAT-traversal issues, especially when you have clients behind a different PIX connecting into your pix. If you add this command to your config it will resolve those issues.

code:
crypto isakmp nat-traversal 20

Korensky
Jan 13, 2004

Girdle Wax posted:

I still can't believe how horribly unreliable autonegotiation Cisco-to-Cisco is. It's crazy that in this day and age we still have to go through and manually force speed/duplex on ports to stop Cisco gear from doing something stupid.

All the modern gear is fine when it comes to auto-negotiating but I generally lock everything down with hard settings everyway since it's just best practice.

jwh
Jun 12, 2002

Isn't autonegotiation required for 1000base-t? I know you can force it off on fiber interfaces, but I don't think I can turn down autonegotiation on my X6548-GE-TX modules.

edit: I think I'm wrong about the module. I seem to remember there was something out there that would only take a 'speed auto', but now I can't figure out what it was.

jwh fucked around with this message at 02:21 on May 11, 2007

ragzilla
Sep 9, 2005
don't ask me, i only work here


jwh posted:

Isn't autonegotiation required for 1000base-t? I know you can force it off on fiber interfaces, but I don't think I can turn down autonegotiation on my X6548-GE-TX modules.

edit: I think I'm wrong about the module. I seem to remember there was something out there that would only take a 'speed auto', but now I can't figure out what it was.

GBIC/SFP based ports can only take 1000/full afaik, fixed modules may vary by platform.

multiprotocol
Sep 16, 2004
label switching is fun. i can relate to that.

Girdle Wax posted:

GBIC/SFP based ports can only take 1000/full afaik, fixed modules may vary by platform.

1000baseT always requires autonegotiation to work properly. If you have auto on one side, and something else on the other, you usually end up with the AUTO side negotiating to 100/half. This has been my experience with PIXs, 4948s, 6509s, and 7604s.

Tremblay
Oct 8, 2002
More dog whistles than a Petco

jwh posted:

Isn't autonegotiation required for 1000base-t? I know you can force it off on fiber interfaces, but I don't think I can turn down autonegotiation on my X6548-GE-TX modules.

edit: I think I'm wrong about the module. I seem to remember there was something out there that would only take a 'speed auto', but now I can't figure out what it was.

Were you thinking the PA-4E modules for VXRs?

jwh
Jun 12, 2002

Tremblay posted:

Were you thinking the PA-4E modules for VXRs?

I can't seem to remember. I think I'm confusing memories of one thing with something else. I just checked most of the copper modules I have around here, and they all seem fine, but didn't check any of the PA's.

In other news, we finally pulled our 'haunted' 7206VXR out of production today. If anybody wants an early PA-8T1-IMA that is possessed by evil spirits, you know who to talk to. I won't be sad to see it go, that's for sure.

GhostSeven
Apr 23, 2005

Yesterday Was A Million Years Ago
I have a SoHo 97 with the god awful web admin. I have limited experience with cisco gear but basically just want to get it to do is work as a Ethernet ADSL modem to replace my linksys wireless router that is doing that at the moment.

I have an 8 IP block so would like to set it to unnumbered, my setup is as follows


Internet <---> CISCO GOES HERE <--> IP Cop Box <--> Rest Of Network

Currently a linksys modem/router/wirless bit of kit is sitting where the SOHO 97 would go, it is doing the job fine but it's a little overkill and has most of its features turned off.

This may be a little rambly and hard to understand but hope I have got the details across.

I have a basic understanding of the cicso config files and I can Hyperteminal into the unit etc.

Any tips etc would be fantastic (Or a config example of how to do this)

Oh also is the SOHO 97 ok on 8Meg ADSL (ADSL Max In The UK)

GhostSeven fucked around with this message at 16:37 on May 11, 2007

karttoon
Apr 11, 2006
-?-

jwh posted:

If anybody wants an early PA-8T1-IMA that is possessed by evil spirits, you know who to talk to. I won't be sad to see it go, that's for sure.

If it's free or cheap I'd love to get it. If you're serious jwh, hit me up via aim or icq (profile).

On another note, anyone else get the e-mail for the Cisco 'myspace' thing? Basically a networking site for networkers where you can upload your photos and have a friends list. I signed up :(

Herv
Mar 24, 2005

Soiled Meat

GhostSeven posted:

I have a SoHo 97 with the god awful web admin.

I have an 8 IP block so would like to set it to unnumbered, my setup is as follows

Internet <---> CISCO GOES HERE <--> IP Cop Box <--> Rest Of Network

Ok when you say you want IP unnumbered you want your block of 8 public IP addresses to be where?

Would the outside interface of the IP Cop Box hold some/all of the public ip addresses? If so then you just want to bridge the DSL interface to the ethernet interface and then let IP run on the IP Cop Box.

Here's an example of a SOHO97 with bridging, for Verizon DSL (ISP settings may vary). You will need the DSL specific information as well. (e.g. pvc 0/35)
code:
interface ATM0
     no ip address
     bridge-group 1
     PVC 0/35
          protocol bridge broadcast
          encapsulation aal5snap

interface Ethernet0
     no ip address
     bridge-group 1



bridge 1 protocol ieee
Disclaimer: Hand typed it, don't have a config handy as I am running IP on my DSL interface.

What the above will do (sans mistakes/syntax) is bridge the DSL circuit to the front of the IP Cop Box so to speak, transparently providing the first useable IP address to a device behind the soho97 router.

Good luck, any specifics just ask away, should be able to get you up.

I have used a SOHO97 bridged and running IP on an ATM subinterface for what its worth.

jwh
Jun 12, 2002

karttoon posted:

If it's free or cheap I'd love to get it. If you're serious jwh, hit me up via aim or icq (profile).

The card is seriously screwy; trust me when I say you don't want it. I think the company is going to try and sell the whole 7206 "as-is", haunted IMA card and all, by tucking it in with a lot of Passport 8600's. I pity the poor bastard that ends up with that IMA card. And the Passports.

What is this Cisco myspace thing you mentioned?

karttoon
Apr 11, 2006
-?-

jwh posted:

The card is seriously screwy; trust me when I say you don't want it. I think the company is going to try and sell the whole 7206 "as-is", haunted IMA card and all, by tucking it in with a lot of Passport 8600's. I pity the poor bastard that ends up with that IMA card. And the Passports.

What is this Cisco myspace thing you mentioned?

Well if the chasis is good I'd still take it.

http://www.academynetspace.com/

It was made for students/alumni/instructors etc of the Cisco Networking Academy but I suppose anyone could register. It has a nifty little interactive 'browser' that lets you find people by area. Apparently on May 15th they will also be releasing some kind of like Cisco test on the site where people can compete to get the most correct answers and their names will be on some leader board.

Boner Buffet
Feb 16, 2006
How about some theory?

Lets say you have a stack of 2950s. They are all layer2. You want vlan2 to attach to some public kiosks for example. To be able to have those vlans extend beyond each device, you would want them trunked, correct?

2950 -trunk- 2950 -trunk- 2950

To further that, you have an http proxy server attached to vlan1. To be able to access that proxy server so your kiosks can have net access, you would then want that trunk to extend to a router which will have 2 interface cards? One on a vlan1 switchport, and one on a vlan2 switchport? And then from there, you can use ACLs to let only ports 80/8080 route through?

Do I have that all right or am I missing something?

jwh
Jun 12, 2002

karttoon posted:

Well if the chasis is good I'd still take it.
Chassis and NPE is still worth a good chunk of change; I think the company is going to try and sell it. If they are planning on tossing it, and for some reason nobody gets their paws on it first, I'll keep an eye on it for you.


InferiorWang posted:


How about some theory?

Lets say you have a stack of 2950s. They are all layer2. You want vlan2 to attach to some public kiosks for example. To be able to have those vlans extend beyond each device, you would want them trunked, correct?

2950 -trunk- 2950 -trunk- 2950
Yes, at least, that's one way of doing it. You're better off with a hierarchical distribution than a daisy-chain, but yes. Alternatively, you could look at something like qinq to extend vlan2 over a another switched backbone, or l2tpv3 to extend vlan2 over an IP backbone, or VPLS to extend vlan2 over an MPLS backbone.

quote:

To be able to access that proxy server so your kiosks can have net access, you would then want that trunk to extend to a router which will have 2 interface cards? One on a vlan1 switchport, and one on a vlan2 switchport? And then from there, you can use ACLs to let only ports 80/8080 route through?

Do I have that all right or am I missing something?
You could instead present an 802.1q trunk to the router device that carries both vlan1 and vlan2, and configure dot1q subinterfaces, if you're trying to cut down on interface consumption.

Arguably, a better idea would be to invest in a layer-3 switch, and apply access-lists to layer-3 SVI's, or use vacls.

ragzilla
Sep 9, 2005
don't ask me, i only work here


Girdle Wax posted:

GBIC/SFP based ports can only take 1000/full afaik, fixed modules may vary by platform.

Correcting myself here, GLC-T SFP modules are capable of 10/100/1000 on certain platforms, listed at http://www.cisco.com/en/US/products/hw/modules/ps5455/products_device_support_table09186a0080446625.html

Boner Buffet
Feb 16, 2006

jwh posted:

Yes, at least, that's one way of doing it. You're better off with a hierarchical distribution than a daisy-chain, but yes.

By that do you mean each switch has individual "home run" uplinks back to a "core" switch?

jwh
Jun 12, 2002

InferiorWang posted:

By that do you mean each switch has individual "home run" uplinks back to a "core" switch?

Generally, yes. If tends to be easier to manage, and results in a more predictable point of congestion. The ability to then solve bandwidth problems from access switch distribution/core switch by use of etherchannel is compelling. It really depends on how much traffic you're doing, and whether you intend to build distribution/core redundancy. Like anything else, I guess it depends on what you want to do, but I'd bet that the SH/SC cisco crowd consensus would be for a hierarchical switching infrastructure. If cost is prohibitive due to long-distance leased fiber, for instance, you may want to instead buy layer-3 switches and move to an IP model.

If you do daisy-chain, be aware that you shouldn't expand wider than seven devices (switches, spanning-tree aware bridges, etc) and also use spanning tree. Spanning-tree will flip out on you.

Boner Buffet
Feb 16, 2006
Money is always an issue, but only in terms of getting proper gear. I desperately need servers and we still run old cabletron chassis switches circa 1997 which need to be replaced. That continues to fall on deaf ears, but I guess that's a whole different thread!

We're pretty well sorted with cabling and we have our electrician who can pull cable anywhere it's physically possible. His only limitation is tipping fiber.

After reading your post, I'm wondering what the hell the designers of the network in our new K-5 school were thinking. They ran more cable in that building than we'll ever really need. But, they ran it to areas where we don't need it. There's no reason why they couldn't pull more fiber to the IDFs. Each closet has a 3650 for POE VOIP, and below that are 3-5 daisy chained 2950s. In hindsight, there's no need to daisy chain them when you're building brand new.

GhostSeven
Apr 23, 2005

Yesterday Was A Million Years Ago
Just read your reply again and I think the config you posted will do exactly what i want by the sounds of it, sorry for this post as it may have not been required!

Thanks again



Herv posted:

Ok when you say you want IP unnumbered you want your block of 8 public IP addresses to be where?

Would the outside interface of the IP Cop Box hold some/all of the public ip addresses? If so then you just want to bridge the DSL interface to the ethernet interface and then let IP run on the IP Cop Box.



Actually I think im being a bit of an idiot, I dont really need to use the other IP addresses (not really have a use for them any more) just need to bridge(?) the connection between the outside world and the ipcop box. I just think i used to use unumbered on the linksys (sorry if im not being clear)

My current IPCop Box uses one of my external IP block and my router uses another, can i set the cisco to have the same IP on both sides of its interface i.e WAN same as local IP on the cisco. Again sorry if this is not clear, but hope this helps.

GhostSeven fucked around with this message at 22:02 on May 11, 2007

Herv
Mar 24, 2005

Soiled Meat

GhostSeven posted:

Just read your reply again and I think the config you posted will do exactly what i want by the sounds of it, sorry for this post as it may have not been required!

Thanks again

Yep, sounds like it will work for you. The Firewall (IP Cop) will have to have the ISP gateway as the default route if it isn't defined already.

Cheers

Ivor Biggun
Apr 30, 2003

A big "Fuck You!" from the Keyhole nebula

Lipstick Apathy

IBM posted:

I always use Hyperterm and didn't know that there were better options. What do you guys recommend instead of Hyperterminal?

I like PuTTY - It was upgraded a while back with support for serial connections.

jwh
Jun 12, 2002

I'm reworking our qos framework for low-speed branch sites, and as part of the process, I've started second guessing myself about the recommended application of service-policies on frame-relay interfaces.

If you go back to about 2002-2003, it looks like the recommendations was frame-relay traffic shaping and frame-relay map-classes, after which the recommendation was a nested hierarchical policy-map. Ok, I have nested hierarchical policy-maps configured already, no big deal.

But, literature from 2005 lists best-practices as frame-relay traffic shaping! I thought MQC got rid of all that garbage?

Truth is, I don't really care where I put the policy-map, but I do want to know recommendation is. Because it's GTS and CBWFQ in a nested policy-map, I'm inclined to think I'm correct in applying this to the sub-interface, but now I'm having serious doubts.

Herv
Mar 24, 2005

Soiled Meat
code:
C827(config-subif)#service-policy output esp
CBWFQ : Not supported on subinterfaces
I was able to get it on the pvc though, but it's not showing up as cbwfq.

jwh
Jun 12, 2002

Herv posted:

code:
C827(config-subif)#service-policy output esp
CBWFQ : Not supported on subinterfaces
I was able to get it on the pvc though, but it's not showing up as cbwfq.

That's normal; or at least, that might be normal. If you nest the cbwfq policy-map under a generic traffic shaping (GTS) policy-map, it'd probably let you apply it.

Funny aside, while it will work for frame-relay sub-interfaces, it won't work for ATM pvc's, because they shape natively, and it won't let you do it. But, cbwfq can be applied directly to an ATM pvc.

Good times.

obsidian440
Apr 15, 2004

Don't question god's choices.
I have very very little cisco knowledge and find myself more and more working with the company firewall/router. The info I get in the pix manager about is
Cisco PIX Firewall Version 6.3(4)
Cisco PIX Device Manager Version 3.0(2)

I would like to setup logging to a server within our network, which I have done, but now need to clean up the messages I am recieving. Is it possible to log only denied VPN connections and system peaks, CPU utilization, memory, bandwidth etc ? I tried looking around for info on the different facility levels but didn't find much help, admittedly I didn't look very hard. Is this type of setup possible ? If not, what would be a good setup that would get info on possible attacks or unauthorized connections ? Thanks guys!

Korensky
Jan 13, 2004

Hot tip for those who manage multiple devices and didn't know about this one (you'd be surprised how obscure it is):

How to schedule commands (ala cron) on your IOS device:

http://www.cisco.com/en/US/products/sw/iosswrel/ps5187/products_feature_guide09186a00801b0695.html

They also use a CNS IOS sync as an example. CNS is another awesome feature for managing large amounts of devices running the same IOS version.

ragzilla
Sep 9, 2005
don't ask me, i only work here


obsidian440 posted:

I have very very little cisco knowledge and find myself more and more working with the company firewall/router. The info I get in the pix manager about is
Cisco PIX Firewall Version 6.3(4)
Cisco PIX Device Manager Version 3.0(2)

I would like to setup logging to a server within our network, which I have done, but now need to clean up the messages I am recieving. Is it possible to log only denied VPN connections and system peaks, CPU utilization, memory, bandwidth etc ? I tried looking around for info on the different facility levels but didn't find much help, admittedly I didn't look very hard. Is this type of setup possible ? If not, what would be a good setup that would get info on possible attacks or unauthorized connections ? Thanks guys!

If you're stuck on 6.x, you'll need to look into setting the levels of the messages you're trying to pick out, then setting the pix to only send messages of that severity or higher, which might help your signal-to-noise.

If that doesn't help you could look at moving to another syslog server like syslog-ng which has built in filtering capabilities so you could direct your 'interesting' logs to a special log file. If you feel like really going over the top you could then setup SEC (Simple Event Correlator) to watch that log file and take actions on the messages, like warning you if someone fails to log in too many times.

Another alternative if you have the hardware, and can get the software, to do it would be to upgrade to 7.x and use the built in message list filter feature.

As far as logging CPU/Memory, I'd have to recommend setting up cacti on a Linux host somewhere on your network and using that to graph the CPU/Mem/Interface Traffic OIDs.

Adbot
ADBOT LOVES YOU

ragzilla
Sep 9, 2005
don't ask me, i only work here


Cisco goons, you're my only hope (although I've been thinking of typing this up on c-nsp, but it'd probably degenerate into a 6500/7600 whine). All of this (mostly) takes place inside a single facility, the few services that extend outside the facility are not of a big concern / are another topic entirely. I really wish layer2 production networks wouldn't leave the building but that isn't too feasible, yet.

My datacenter, over the past few months, has had a couple of service affecting issues, some of them out of our control (UPS failure), some of them our direct fault (accidental spanning-tree loops (thanks portfast) which freaked the hell out of the core network- plugging new switches into the core at 4pm is a bad thing). Due to various reasons, our network is a mixture of relatively old (3500XL access switches), and relatively new (3750 stacks for distribution, 2950G access switches on newer rows) gear, due to this we're somewhat limited in recovery time due to being stuck on PVST for the legacy equipment. Waiting 40 seconds for spanning tree to reconverge because a gig link dropped, really, really sucks. Doubly so because we have over the last year started offering more products, namely voice (as SIP out of our softswitch). And people really hate it when they pick up the phone and get a fast busy. Thankfully we've been able to provide decent QoS inside our network for the voice so people don't have quality issues, mls qos on xx50 series switches is pretty great, as is CoS based priority queuing in the 12k platform. Ideally I would like to completely eliminate reconvergence times after link failures, a lofty goal but I think we can get pretty close.

Because of the issues, and the pressure placed on us by the customers, we've been looking at what we do in our network, and what our options are going forward (we're just about to start building out the 'second phase' of our existing space- a good a time as any to decide what you want to do for the next 2 years right?).

Currently, we run fairly large layer2 domains, with layer3 terminating at our 2 'cluster'/'distribution' stacks (A & B), consisting of 3750G-24T's and 3750G-12S's. We have 2 /24 colo customer VLANs which span pretty much all of our access switches in production (7 I think at last count), 1 per row of cabinets- end of row switching. The distribution switches talk OSPF with our customer access routers (t1/ds3/oc3/external ethernet aggregation), and core routers, giving them knowledge of all our internal customer prefixes and enabling fairly optimal routing in the distribution-access layers. Since they're 3750 stacks though, they have a snowball's chance in hell of doing anything decent BGP wise to get them to make smart routing decisions at distribution, so I get all sorts of fun crap like packets coming up from a customer, up to 1 of my cores, back down through distribution and up to the other core- this is happening because the distribution switches have a default pointed up to an HSRP between the 2 core routers (12008s), so if the destination isn't bestpath in the router it ends up in, across it goes!

So far, this has worked fairly well (up until a few months ago), and is a hell of a lot better than what we had a few years ago before we rebuilt during a move into our new facility, but this poo poo with spanning tree has got to stop, we can't be sitting with our thumbs up our asses waiting for the network to reconverge.

So enough with the problem, my main point here is to present my solution (which I and my direct report both really like- but we still have to sell to management and his direct report), and see if anyone here can provide me some "hey did you think of X" or "I tried this and this really bad thing happened".

This new design is heavily influenced by a Cisco implementation document- High Availability Campus Network Design— Routed Access Layer using EIGRP or OSPF.

My Goals:
- First a simple/sample diagram, all links here are point-to-point layer3 links. Currently addressed using /30s but I'm considering using /31s to converse address space.
code:
    [ Core ]
    /      \
   /        \
  /          \
[DistA]===[DistB]
  \          /
   \        /
    \      /
    [Access]
- Get rid of spanning tree. How? Everything coming out of the distribution switches (pondering between 6509/Sup720 and 7609/RSP720 for this- arguments for/against these would also be welcome) is a layer 3 link, layer 3 links up to the core routers. layer 3 links down to the customer access routers. layer 3 links down to the access/aggregation switches. layer 3 links 'across' to my existing distribution switches to get the legacy network uplinked through the new network.

- To accomodate getting rid of spanning tree, I want to move the layer3 termination down into the customer access switch (probably using 3560s). 2 routed uplinks, 1 to each distribution switch. Then a handful of SVI's with /28s or /29s serving a handful of customers. These switches will participate in the OSPF loopback program, and limited iBGP as listed below. Spanning tree will be turned off on the customer aggregation SVIs. To try to prevent things from breaking we'll try to implement as much layer2 security as possible:
-- Turn on portfast
-- Set portfast bpduguard (shut your port off if you plug a switch in- I really hate it when people try to take my layer2 network out of the building)
-- port security max-mac-count 1
-- switchport protected (just started considering this one tonight, by doing this and local-proxy-arp on my SVI I can stop the customers from talking directly to each other, and force all traffic to be processed and policed by the switch).

- Reduce OSPF database size to get rapid OSPF convergence. Currently my OSPF database probably has ~3k routes in it. This kind of hurts on my slower processors like the GRP-B's in my GSRs. My plan to address this is to reduce OSPF to carry loopbacks only, then exchange customer prefixes via BGP only.

--'Load Balancing' and failover between layer3 links will be handled natively via OSPF equal cost multipath. Since everything will have 2 layer3 links, every device should, under normal conditions, have 2 equal cost paths to reach any other loopback in the network. So long as I don't lose both links, my iBGP sessions should never drop.

--The distribution switches will essentially act as route reflectors, cores peer with distribution, distribution peers with customer access routers (also 12008s).

--My internal networks will probably be originated and 'pulled up' in the distribution switches, thus letting me simplify my routing tables at the core and customer access routers by only advertising aggregate prefixes to them instead of everything.

--The new colo access switches will participate in limited iBGP with the distribution switches, the distribution switches will originate a default, and only a default, down to the access switches (to keep the routing table nice and simple) - I am also debating maybe doing the default in OSPF if I can figure out why it's not working out in my lab using stub areas. The colo access switches will advertise their SVI /28s and /29s as well as any subnets routed to customers, via BGP. While moving this functionality to iBGP instead of a traditional IGP will result in a 1 minute delay in the routes getting into the distribution table, I feel this is an acceptable delay since it is only once off (when the switch reloads), and it gains faster OSPF convergence which gets my ECMP back faster.

- All internet traffic in the access/distribution switches will be in an internet VRF (vrf-lite in all the switches basically). This is mainly a management thing- I plan to use the main routing table for management access only- haul a single eth from every switch back to an isolated switch for a new management network. So I can actually get into the switches over ssh/telnet when the poo poo hits the fan. Part of this project will also be making sure we have working consoles in case that switch fails *grumble*. This introduces somewhat of a learning curve (remembering to type 'vrf internet' all the time), but I feel the management advantages outweigh the disadvantages.

- OSPF is optimized as described in the Cisco doc above (200ms hellos for fast failure detection- in the future we will probably also look at doing BDF if the 3560 ever supports it), in addition to ispf to speed up spf calculations.

- A major design goal is to keep customers out of the drat core/distribution layers (hey you kids, get off my lawn). We've been pretty poor about this up until this point, we have a couple of customers taking gig (copper mainly) out of distribution, and we tend to (ab)use the network for our own purposes too. Need to get 20 VLANs to an ESX box? Plug it into distribution! We also carry some of our internal network traffic (ugh) over VLANs in distribution. But that's another project.

All of this up to this point seems to work fairly well in the lab. I get pretty much unnoticeable fail over (after a few tweaks) with my stack of 3550s (Core, DistA, DistB, Access). Running an Asterisk Echo Test with music playing into it results in pretty much uninterrupted music unless I do something terrible like pull out both gig links to the core or access. Reloading and hard powering down switches you're hard pressed to even hear a blip.

I've come across a couple of problems, the biggest one was resulting from a distribution switch reboot and ECMP. Basically since OSPF comes up faster than BGP, my core was sending traffic for an access switch down to DistA, which didn't know how to handle it, so it sent it back up to the Core, back down to DistA etc. I managed to resolve this by adding a high metric static in each of the distribution switches, pointing to each other over the link between them. This way when a switch is still 'becoming active' after a reload, it will continue to pass traffic across to the other distribution switch which is still up, which will be able to forward the traffic as normal.

My full table BGP customers I can continue to serve out of my customer access routers like I do now. I'll probably hang a dedicated switch off each of those GigE interfaces just for that, and keep them isolated from the distribution/core. There was a proposal to perhaps hang some these customers off 10/100/1000 aggregation switches, terminate L3 in the agg switch then have them to eBGP multihop to the distribution switches but this bugs me in 2 ways.
-- we need to redistribute the customer's routes down to that agg switch via BGP. So if they drop their session it'll be 2+ minutes until the path back to them is good.
-- Customers in distribution, get off my lawn.

We do have some colocation customers who do HSRP with us for failover on our and their equipment, to continue to offer this I came up with the following solution:
-- The customer will 'purchase' 4 switchports (in 2 adjacent switches), and a dedicated SVI/VLAN in 2 switches. 1 port out of each switch will go to the customer, the other 2 ports will be a dedicated 'tie' link to connect the 2 islands. Since I don't have any trunks up to the core this is the best I can think of, plus it gives the customer a dedicated link at their purchased speed to carry traffic which comes in over the secondary switch in the unlikely event that we lose all uplink on their primary switch.

I'm sure there's more but this is starting to look like a pretty huge wall of text and it's getting late, if it helps I may have some Visio diagrams to look at tomorrow depending on if I'm motivated enough to work on them.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply