Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
uhhhhahhhhohahhh
Oct 9, 2012


Feel like i'm being hella dumb with this one but would like someone to double check me on some basic HSRP poo poo if possible before we commit to something bad.

I've got 4 Nexus 9300s in 2 vPC domains in our new data centre - 2 fibre (our 'core' switches), 2 copper (extra 'core' poo poo that doesn't use fibre/management/whatever). The vPC peer links are connected via 100gig on both pairs. The copper switches are connected to the fibre ones on 2 vPCs over 20gig each. We're getting a managed (ugh) IPVPN MPLS from our service provider. They're providing us 2 ASRs with a single 10gig port each. We're also going to have 8 VRFs on our Nexus switches and the ISP routers. They're going to give us subinterfaces presented on a .1q trunk with HSRP on each subif.

Originally my bosses wanted to buy another pair of 10gig switches to run the HSRP through, so ISP Router -> 10gig switches -> Nexus. I don't want to pay for anything so i said we could just make all the layer 2 vlans on the copper switch and connect it to that. Now i'm second guessing myself and thinking is that even going to be necessary? couldn't I just patch one ASR into a fibre switch and the other ASR into the other fibre switch and the HSRP information will be exchanged over the vPC peer link? All the SVIs and VLANs are going to be on there already in their relevant VRF Contexts.

Theoretically it should work but I'm doubting myself and getting thrown off here because the Nexus fibre switches are going to be doing routing and that's making me think it might mess with the HSRP info. Typically i'd just throw a blank 2960 in the middle but that's not going to work here and I don't have any spare routers lying around that I can lab this on.

uhhhhahhhhohahhh fucked around with this message at 19:31 on Dec 24, 2018

Adbot
ADBOT LOVES YOU

uhhhhahhhhohahhh
Oct 9, 2012


BaseballPCHiker posted:

I am definitely not a licensing hardware guy so please forgive my ignorance. Is the only difference between DNA Essentials and Advantage is that I can make a device a layer 3 switch?

Besides some like obscure inventory benefits for whatever DNA is? I feel like I'm missing something major, and also feel like Cisco is jumping in full steam ahead to the byzantine world of Microsoft licensing.

Both can do dynamic and static routing. There might be some features missing from Essentials for EIGRP for example but I couldn't tell you what they are. If you were doing full MPLS you'd need Advantage for certain though.

We didn't like this DNA garbage and bought the perpetual NX-OS Essentials 'old style' license for our 9300 switches. I've set up VRF Lite on them with EIGRP and OSPF so I know it works.

It took us way longer that it should have to get our heads around this DNA nonsense but the takeaway is, unless you have basically nothing and you're starting from scratch - I mean no Wireless and no ISE mainly - it's not worth being on the hook forever. If you've already got an ISE deployment with xxxx amount of base licenses and wireless controllers with licenses it's just not worth it. I suppose if you were doing a refresh you could consider it but even then you're going to be on the hook for a lot of revenue every 3 or 5 or 7 years because even if you stop paying the DNA Essentials for example, your switches keep Essentials features perpetually but you lose all the SDN access and you lose your wireless and ISE licenses.

Your licensing costs will go through the roof and you still have to pay up front for the hardware as well. The backend appliance for their new management software for the Catalyst switches is like £20k on it's own.

uhhhhahhhhohahhh
Oct 9, 2012


Woof Blitzer posted:

Are optical transceiver modules plug n play?

Yes, so long as you bought the right one for your fibre and that they're Cisco branded.

You can use non-Cisco ones uses a hidden command but TAC won't help you if you have problems (even if they're unrelated, from what I've heard)

If possible, look at Direct Attach Cables as well. They're less flexible because the 100gig ones, for example, have fixed length limits, but they're much cheaper than SFPs.

uhhhhahhhhohahhh
Oct 9, 2012


BaseballPCHiker posted:

I've got a very annoying problem that will be resolved with a code upgrade and reboot soon but in the meantime messages are spamming my syslog like crazy.

Specifically this message:
Local7.Notice x.x.x.x COUNTER: DATE TIME: %DUAL-5-NBRCHANGE: EIGRP-IPv4 10: Neighbor x.x.x.x (VlanX) is up: new adjacency
Then the same thing with a is down: Peer Termination received

I've tried making a logging discriminator but cant seem to get the drat thing to stop filling up my syslog.

I tried this:
logging discriminator EIGRPMSG facility drops DUAL severity drops 5 mnemonics drops NBRCHANGE
logging console discriminator EIGRPMSG
logging monitor discriminator EIGRPMSG
logging buffered discriminator EIGRPMSG

If I do a show log I dont see the messages anymore but I still see them in my syslog. Do I need to do something else to stop these from getting sent out?

EDIT: Added logging host x.x.x.x discriminator EIGRPMSG thinking that would stop it from sending to syslog server but still no luck.

I had trouble doing my own discriminator using mnemonics, and had better luck with msg-body instead. I did this for blocking dot1x logs instead:

'logging discriminator AuthFail msg-body drops Authentication failed for client'

Although I never tested this on logging to a syslog server, just on the buffer.


Couldn't you also just set your syslog server to drop that message?
You can also disable logging of neighbour changes and warnings under the eigrp process but you might lose useful information doing that

uhhhhahhhhohahhh fucked around with this message at 23:53 on Mar 13, 2019

uhhhhahhhhohahhh
Oct 9, 2012


Yeah I'm struggling to figure out where that screencap is from. If you're making a Flexconnect ssid it doesn't technically need an interface on the WLC, the management one can be set fine from what I remember. You set the native vlan on the switch port and under the Flexconnect tab on the AP for it's IP, then set the Flexconnect vlan for the ssids for the AP (or with a Flexconnect group). That VLAN needs to be on the switch the AP is on and have access to wherever your DHCP is of course

uhhhhahhhhohahhh fucked around with this message at 13:51 on Mar 23, 2019

uhhhhahhhhohahhh
Oct 9, 2012


adorai posted:

A little research led me to the command peer-gateway, which I entered on my 93180 pair under vpc domain 1. This appears to have solved my problem. It is a little wishy-washy as to whether this is best practice or just an unsupported crutch.

I have that on all 3 of my vPC pairs. I think I also threw peer-switch on it for good luck as well.

uhhhhahhhhohahhh
Oct 9, 2012


Having some weird packet loss/routing/layer 2 problems last week with my Nexus switches & managed MPLS (), I semi-fixed it and I think i might be overthinking things super hard here.
Backstory:
We've got a (brand new) primary DC and a (old primary) secondary DR DC, connected via a trunk port () with only a server VLAN allowed across that trunk at the moment. I explained it wasn't the best idea, but my suggestions of using VXLAN or just having a separate server subnet at each DC and using DNS for DR, instead of static IPs, for our 100ish and growing VMs was shot down by my managers. Things are extra messy because we're still migrating from our old unmanaged EVPN on a different EIGRP process to this new IPVPN, and the EVPN core is at our secondary DC, along with still moving poo poo from one DC to another, so there's constant changes. We control the routing at our DCs using our Nexus switches, we have an EIGRP process for each VRF that the ISP redistributes into their BGP MPLS. the other ~70 non-DC sites just have the managed router and our switches. The server subnet is advertised out at the primary DC at the moment, but can be advertised out of both if we want - the idea is that, in the event of a disaster, we move the SVI for the VMs (25.1/24) from the primary nexus to the secondary, add it to the EIGRP and then bring all the servers up there and we don't have to involve the ISP and wait 4 hours for them to change the routing.

We have a SIM APN that we use for remote access, and they were dropping a ton of packets to the servers and anything on different ranges that's at the primary DC, and the traceroutes had lots of drops and unresponsive hops that shouldn't have been there - traces to devices at the secondary DC were fine. They come in from a leased line to the SP's router, then an interconnect router (i didn't set this up), that has a GOLR to the secondary fibre nexus which then would've been layer2 to the servers at the primary DC. The nexus at the primary DC has a static for the SIM subnet (30.0.0/20) via the secondary nexus fibre (25.4/24), which has a route for that subnet in it's routing table from the old EVPN EIGRP process via the interconnect router (25.254/24). Trying to figure out what i'm missing here/why i'm very dumb, because the routing for this should work fine? despite being inefficient because there's an extra 'hop' in the middle. The fix for it was changing the static route for the mobile device subnet on the primary nexus to point directly at the APN interconnect router interface.

an additional, similar problem: dropping a ton of packets from a non-dc site (12.0/23) to the secondary nexus copper switch management IP, 27.2/24. Trying to keep it as simple as possible, it only has the management SVI and only layer2 vlans. All the SVIs for the other subnets there exist on the fibre switch. it only has a GOLR under this VRF to the fibre switch - 27.1/24 - and that knows the route back to the non-dc site from the IPVPN EIGRP via the MPLS router (27.254/24). Again, to me, this should work fine? the fix to stop it dropping packets was to set a static for the non-dc subnet via the MPLS router's IP. doesn't make much sense why it worked though, since the GOLR sends everything to the fibre switch, which then knows the route back to the subnet. This poo poo is making me feel hella dumb, and it doesn't help the weirdness started happening at the end of a 14 hour day because we were doing some other work.

made a quick & bad drawing to show how this specific stuff is connected, if it makes my rambling easier to understand:

Data centre link is 10gig p2p.

uhhhhahhhhohahhh fucked around with this message at 20:41 on May 4, 2019

uhhhhahhhhohahhh
Oct 9, 2012


Opps, forgotten to mention. The link between data centres is a 10gig point to point.

I wouldn't have done this either, I mentioned having 2 different server subnets and using DNS but they shot me down

uhhhhahhhhohahhh
Oct 9, 2012


Thanks Ants posted:

Are you doing anything to filter broadcast or proxy MAC discovery between the two sites?

It's just a trunk port with a single allowed VLAN statement.

I still haven't figured out why the static routes for intermediary hops were causing problems. I removed the route I'd done before the weekend for the management of the copper switch at the secondary DC, it's gone back to use the EIGRP route like before and it now it works fine, no dropped packets who knows.

I half got my way anyway, they've said once we've moved over everything on that server vlan from the secondary to the primary, I can make 10gig point to point layer3. They still won't let me do VXLAN though, too bad. There won't be any automatic failover if our primary dc gets exploded, and I'll have to move the default gateway IP for the severs over manually, but all the VMs will need to be brought up manually as well anyway.

uhhhhahhhhohahhh
Oct 9, 2012


My boss is too cowardly to let me use non-Cisco transceivers. We've probably spent £20-30k in the last year on 10gig and 1gig SFPs. I've got about 25 1gig transceivers that our ISP sends out with their NTUs that they never bother to collect sitting in a drawer.

uhhhhahhhhohahhh
Oct 9, 2012


Tetramin posted:

Hmm maybe I’ll try to replace the syslog server I had our system guys build with a Linux box then. They built me a windows box but if it’s free I can probably change that. I have no attachment to OS for this.

Thanks

We're testing out Graylog at the moment, seems pretty good.

uhhhhahhhhohahhh
Oct 9, 2012


MF_James posted:

So, I'm mildly perplexed by this, the situation is that I have an ASA5515 pair that is logging to a syslog server, I have moved the syslog server to a new machine (from a win7 desktop to a server 2019 VM), since doing that I'm no longer getting netflow logs.

We don't have netflow specifically set up, but we are doing debug logging; the only configuration change that was made on the ASA was removing the old server and adding the new in its' place.

I confirmed the server is not seeing those logs via pcap on the server, but it does get other debug level logs without issue.

I confirmed that those logs appear on the ASA by logging to the console directly.

It does not (appear) the traffic is being dropped by the switch, I looked at counters and they aren't going up.

I ran a cap on the ASA itself to try to confirm 100% that it's sending the logs to the syslog server, but the capture only shows date, time source IP/port, destination IP/port and packet count, doesn't show the actual content of the packet.

Anyone have any ideas about where I can look next? I was really hoping to validate the ASA is sending the logs with more verbose capture but I don't see how to do that. Even if it is, I don't see what the hell could be happening to the packets, they seem to be disappearing into the ethernet

Did you add the new IP in both places? If you're doing it through ASDM, you do it on the device management page and on the Service Policy page.

uhhhhahhhhohahhh
Oct 9, 2012


MF_James posted:

Both places? We don't have netflow explicitly configured, it (should) be sent because logging level is set to debugging and debug-trace is on; if that's what you mean. Doing this all through the CLi, I don't really use ASDM.

Perhaps I am mis-using netflow, but I assumed that's what they were... here's an example log that I'm not getting now that I was before:

<166><1566403763000>ASA-6-302014:Teardown TCP connection 24157561 for outside: X.X.X.X/443 to inside: X.X.X.X/21135 duration 0:00:00 bytes 0 TCP FINs from inside

I don't think that's technically netflow, just the standard ASA log about connections being created, ended or denied.

This is the guide I used to get started with it: https://community.cisco.com/t5/security-documents/configuring-netflow-on-asa-with-asdm/ta-p/3119466

uhhhhahhhhohahhh
Oct 9, 2012


On the netflow summary page, using the flow navigator on the left, you can change to Detailed in the first section. Then choose the endpoint that your screenshot is from and you can filter by that protocol, and the time range it was in. This should give you a page that has Endpoints that generated the traffic and Conversations between devices if there was a source and destination

uhhhhahhhhohahhh
Oct 9, 2012


Cool ASA poo poo: upgraded a 5525X to 9.8(2)35 last week and now the ASDM logs are spammed 30 times a second with ICMP logs that are supposedly coming from the next hop for the management interface - a Nexus 9k - which definitely isn't doing any tracking or icmps to this device.

uhhhhahhhhohahhh
Oct 9, 2012


Prescription Combs posted:

That's pretty old code, might wanna consider something in the 9.12(2) interim train

I'd love to but they won't pay for a support contract here so I've got to make do with what I've got. I've got an ASA cluster on the same version in a different DC that isn't doing the same thing.

Unrelated, but has anyone used/deployed Catena on Nexus switches?

uhhhhahhhhohahhh
Oct 9, 2012


Prescription Combs posted:

Well that sucks. Logging config any different between the one spamming vs the cluster?

Both the same as far as I can see. The workaround would be to disable those syslog messages but then I'd lose ICMP messages for troubleshooting.

uhhhhahhhhohahhh
Oct 9, 2012


I'm currently in the process of setting up a new HA pair of PA-3220s, going to be migrating from some Huawei NGFWs. I've used Cisco ASAs as well and the PA is so much easier to work with compared to both of them. The documentation is incredibly well written, has a bunch of examples and it's easily googleable when working out the quirks. Most of the basics carry across between all the firewalls but there's always some stuff they do differently, like no VRRP on the PAs. I fuckin hate having to do anything on the ASAs, cli and asdm both loving suck, and our ASA pair is in a cluster because they bought a gig internet and our model only has 650mbit throughput.

Not done much on cli yet though so can't say how good it is.

Expensive as poo poo though, cost us like £24k each for 3 years of licensing. The Huaweis were like £3k.

uhhhhahhhhohahhh fucked around with this message at 13:08 on May 21, 2020

uhhhhahhhhohahhh
Oct 9, 2012


The DHCP option never did anything for me. Have you done the DNS entries for CISCO-CAP whatever? Even with them we'd still get random ones that just didn't seem to want to join. Manually putting controller IPs from SSH or Console always seemed to sort it out, mine are all 2x00 series APs though.

uhhhhahhhhohahhh
Oct 9, 2012


Time to start looking up phone numbers and emails for division/senior managers/CEOs on LinkedIn and start spamming them about the lovely service you're receiving.

uhhhhahhhhohahhh
Oct 9, 2012


Thanks Ants posted:

I understand people have QoS requirements or compliance issues but I cannot be arsed dealing with WAN stuff from ISPs any more, they always gently caress the routing up or you get a different configuration built depending on which tech builds the changes, with each change request just compounding the disaster of a state that the configurations are in. I don't care about any of the potential cost savings or the carrier diversity, for me the best thing about SD-WAN overlay using VPN tunnels is that I can see the config that's running and get it changed without five days notice.

If your MPLS is just a massive L2 domain and you run the stuff at the edge yourself then you should be OK

We moved from an unmanaged VPLS to a managed L3 MPLS and it's total dogshit. My managers tried to sell me on it because 'we wouldn't have to worry about routing' and that it would be someone else's problem (????) and it saved £10k out of a contract that's over a mil. I still have to worry about routing, it's still my problem because I'm expected to fix it, except now I can't fix it when there's a problem and were buried under layers of jank trying to make poo poo work because it turns out the ISP doesn't have dozens of network gods waiting to implement the perfect network for us. We'll be putting up some new DHCP servers soon and we'll have to pay like £300*50, one for each site to have its IP helper reconfigured, because they poo poo on my idea to have our own L3 switch or router at each site that we peered with the ISP router so we could at least be in control of things like that.

When we were transitioning our backup DC to the new network and obviously, there was a problem with the VRFs. We got a call booked and an engineer and I asked him to check the config on their router, he was silent for a minute or two and then said, "sorry, i don't know what a VRF is."

Felt super bad for him. What a bunch of pieces of poo poo booking that guy onto the call in the first place.

uhhhhahhhhohahhh
Oct 9, 2012


Kirk Byers seems to be a good resource. Every time I try and get through his free beginners course some bullshit project comes up so I can't say if it's actually good or not. He's involved in lots of other stuff though, and the #networktocode Slack is semi-linked to him/netmiko/napalm/nornir/eNMS I think.

His website: https://pynet.twb-tech.com/

uhhhhahhhhohahhh
Oct 9, 2012


Kazinsal posted:

I've got a couple dozen Cisco devices that I need to regularly back configs up for but can't change the configs on to have them automatically push their configuration anywhere. Does anyone know of a tool or expect scripts or something that I can just drop into a cron job that'll basically ssh in, show running-config, and copy the output to a file?

If you can enable scp on them, you can automate pulling the running or startup config from them in any way you please

uhhhhahhhhohahhh
Oct 9, 2012


I have to look after a cluster of 5525Xs and they're such loving garbage. It just inexplicably stops forwarding traffic if one specific firewall becomes the master, but it works fine when it's not. Also gently caress ASDM forever.

uhhhhahhhhohahhh
Oct 9, 2012


Jedi425 posted:

Check the syslog settings. Are you forwarding syslog to a TCP port? If the host is unreachable, the ASA stops forwarding. Could be you have some kind of issue reaching it from one and not the others?

Syslog is UDP and it's done over the OOB management, so it's always reachable. We also only recently got a SIEM anyway and it was happening before.

Kazinsal posted:

I think the only reason I would use an ASA or a FirePower running ASA code over a more competent firewall is if I needed to handle a few tens of gigabits of L3/L4 traffic and needed to programmatically change non-persistent firewall rules in a dumb and quick manner via clogin or something like that.


Haaaaaaaave you considered opening a TAC case

All the documentation for clustering on ASAs is filled with notes saying TAC don't support this configuration at all. My boss wanted it this way for zero reason, we gain nothing doing it this way over a HA pair because our internet connections are 1gig and we can't active/active them either. He knows that because he was on all the same phone calls as me with our ISP saying we couldn't do that, but acts surprised 1+ years later when it's ever mentioned they aren't active/active or I have to correct him on a phone call.

Also, the audacity on you to assume we pay for TAC, or even software updates, on our edge firewall.

uhhhhahhhhohahhh
Oct 9, 2012


I've got a turbojank setup but I can't do anything about the jank. Trying to make some OSPF <-> EIGRP redistribution work cleanly.

Baiscally, in a DC, I've got a HA pair of WAN routers (that I have no access to), two Nexus 9300 in a vPC, and a HA Palo Alto firewall pair - all with HSRP, etc. - The Nexus and WAN routers peer using EIGRP to learn the routes for all our sites, and we redistribute (static) routes into EIGRP for the other sites to learn, from both our DCs. That bit's fine.
I've set up OSPF on the Nexus and PAs so I don't have to put static routes in 9 fuckin' places (because can't do EIGRP obviously), that bit all works fine too. Then I redistribute EIGRP into OSPF with tags, redistribute OSPF into EIGRP, tagged again, and block those tags from being re-advertised back in on each process. That bit all seems fine, too. The problem is OSPF has a lower AD than EIGRP, so instead of installing routes for all the other WAN sites via the primary WAN router, one of the Nexus switches will install those WAN routes via the other Nexus switch or the PA firewall, which is very Not Ideal. I can't just increase the AD for the OSPF learned routes, because I'll be advertising the same routes at my other DC also, for fail-over purposes, but this one should be preferred until the route doesn't exist anymore. My workaround for this was making a prefix, route-map, and table-map on each Nexus, for it's peer IP, and adding that to OSPF. It works how I'd hope... but this seems super hacky. Is there a better way I should be doing this? Or is this just the expected amount of jank with a design as poor as this?

uhhhhahhhhohahhh
Oct 9, 2012


Methanar posted:

Sounds pretty jank.

If you have decent config management, is it really that much harder to populate a static route entry in 9 different places rather than maintaining and documenting a complicated dance of prefix, route-map and table-maps out there? I suppose this might be a pretty big if.

Why was EIGRP ever used if there wasn't a solid commitment to only using cisco hardware?

You read all that and then assumed we have config management? Absolutely everything is done manually. I'm going to start using eNMS/ansible/Netmiko soon to do some stuff but my junior and me will need to keep it secret because if my bosses find out they'll make us turn it all off. They'd sooner have someone sit there and spend 2 months upgrading switches one by one than having something to do it automatically... And they won't pay for something like Prime either.

It's more like static routes will have to go on 6 switches, 4 Palo Alto firewalls and 2 Cisco firewalls. I feel like once the redistribution config is nailed down it shouldn't ever need touching again?

It was originally all OSPF. We'd already bought some non-Cisco firewalls. the config was all written up and then my boss decided at the last minute, right before implementation, he wanted it to be EIGRP. I think probably because he doesn't know OSPF (he barely knows EIGRP tbh) even though he's done zero of the implementation or config or daily management of it. Pre- this WAN setup when we had a VPLS with EIGRP and only Cisco devices they were still using static routes in the DCs for no discernible reason. I vaguely remember overhearing somebody saying it's unsecure to use routing protocols on your firewall/DMZ because the hackers can get to all your of network. I didn't bother to argue they could do that anyway since there was static routes for every internal network on them anyway.

uhhhhahhhhohahhh
Oct 9, 2012


Too late in the game to change it, they'll never go for it. Especially because we'd have to pay since it's a managed WAN contract.

The guy didn't even want any automatic failover for our routing/DCs. I had to slip that bit in too. He genuinely believes it's better for our staff to lose their access to patient information until someone gets called out and changes it manually by putting the static routes in the other DC so he can send an email out saying "IT fixed it ".

uhhhhahhhhohahhh
Oct 9, 2012


My DR and route tagging part works fine. I can advertise a route at both DCs with one of two tags, and it'll be prioritised towards the primary or secondary DC based on which tag I set.

The main bit I'm unsure about is the table-map and if there's a better way of avoiding that, but still having one Nexus not learn WAN routes from the other. The other issue with the table map is, if another SVI is added onto these switches and gets advertised into OSPF, I'm probably going to have to block the Nexus peers' SVI IP on each of them on the table-map too.

uhhhhahhhhohahhh
Oct 9, 2012


Updated our WLCs last night to fix an ARP bug with AP2802 access points. Only to be met with another bug where locally switched devices can't get an IP after a successful CoA on the same model. I can't upgrade any further because we have 300+ 2600 APs that aren't supported on any other version.


Oh and 30 access points disappeared and they're probably stuck in recovery mode, so someone has to drive out and unfuck them manually. I want to cry.

uhhhhahhhhohahhh
Oct 9, 2012


I actually did look before hand, that's how I knew not to update to 8.10 or whatever they're up to now or we'd have lost 300 APs, and this new bug is not listed as an open caveat on this version, either.

Adbot
ADBOT LOVES YOU

uhhhhahhhhohahhh
Oct 9, 2012


You can also just buy Nexus 9300 fibre and copper pairs and vPC them but that's extra management overhead vs stacking. We paid less than £8k per switch for our 93180s, and bought the 10/25/40gbit SFPs and full copper ones.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply