Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
falz
Jan 29, 2005

01100110 01100001 01101100 01111010
Tik is ghetto, but useful in some applications. Not well tested, tons of bugs. Ipv6 probably still doesn't actually work, I chimed in on this thread 9 years ago and can't see a resolution yet.

https://forum.mikrotik.com/viewtopic.php?t=51124

Basically it's ok for home use and some wisp stuff but I would never use it for a "real" isp if you actually want stability.

(Yes I just shat all over them but my home router is a tik)

Adbot
ADBOT LOVES YOU

wolrah
May 8, 2006
what?
I think of MikroTik as the Eastern European equivalent of Ubiquiti.

They both made their names in the WISP space, they're both well known for offering excellent bang for the buck on network gear, but if you intend to use their hardware you have to understand what you're getting and what you're not getting.

Impotence
Nov 8, 2010
Lipstick Apathy
I have heavy v6 deploy/use internally, mikrotik ipv6 either dies or works questionably, afaik they don't support BGP large communities 5 years later (which is a necessity in this day and age)

My home router is a tik also. But I don't run BGP and have IPv6 at home.

Thanks Ants
May 21, 2004

#essereFerrari


All hardware and software is poo poo

doomisland
Oct 5, 2004

Thanks Ants posted:

All hardware and software is poo poo

This is correct

SamDabbers
May 26, 2003



Honestly, it's amazing any of it works at all.

jeeves
May 27, 2001

Deranged Psychopathic
Butler Extraordinaire
Mikrotik is okay as long as you arn't doing a lot of vlans nor BGP.

And you're okay staying on top of updates at least once every 6 months.

They reboot hecka fast though, it spoils me esp when Cisco/Junipers take 10 min to boot or so it seems.

wolrah
May 8, 2006
what?
I bought some cheap Mellanox 40G cards to screw around with in my home LAN and I had some questions regarding cabling, I figured this was probably a more appropriate place than the home networking thread for this class of hardware.

1. Can a QSFP+ passive DAC really run 7 meters reliably? I see all the major cable vendors sell them, but it seems like the stuff the network equipment vendors sell themselves tend to be limited to shorter distances. Curious in case my housemate wants to connect his PC from the other side of the room, since it ends up being about $50 cheaper than fiber based on some quick checking.

2. How durable is the cable portion of a DAC? I'm going to be patching a desktop that sits on top of a motorized sit/stand desk to a rack on the opposite side of the desk. I could probably get away with 2 meters but I feel like it might be close when I have the desk in standing mode so I''d probably be looking at a 3 or 5 meter cable where I'd coil the excess and hang that coil from the underside of the desk.

3. For now apparently both the Mellanox NICs I have and the Brocade switch I'm eyeing aren't picky about transceivers so it doesn't matter immediately, but if future expansion makes it an issue has anyone messed around with reprogramming the EEPROMs in one end to match what their hardware wants? I've seen people build adapters to do it with SFP format devices attached to a Raspberry Pi somehow, not sure how relevant that is to QSFP+ gear.

4. Should I just suck up a few extra bucks in the up front cost and pick up some SR4 optics and MPO patches instead of bothering with DACs? If I do that is there any benefit to using the "SR4 Lite" modules rated for 30 meters over the ones rated for 150 meters if they're the same price?

Kazinsal
Dec 13, 2011



1. I wouldn't go beyond 5 metres with a passive. Switch to fibre for anything longer.

2. The minimum bend radius of a QSFP+ DAC is usually something around 60 mm. You generally want to be a fair bit looser than that, call it 80 mm, so a coil of DAC should probably have a diameter of minimum 16 cm.

3. Transceiver flashing tools can get a bit pricey and considering how cheap generic, pre-flashed transceivers are these days I would honestly just recommend buying specific brand-flashed transceivers if you need one instead of trying to mess around with a reflashing kit. Unless you want to do tunable wavelength stuff but that's kind of verging on advanced optical physics voodoo and I think you should avoid getting near that.

4. Maybe? Depends on how far you're really planning on going. Obviously there's the distance factor, which sounds like something you might be coming up against. Also, MMF has a bend radius of 30 mm so if being able to bend in tight spaces matters to you, definitely go for glass instead of copper. The SR4-Lite stuff as far as I know is a Mellanox proprietary thing so... godspeed if you go with that.

wolrah
May 8, 2006
what?

Kazinsal posted:

1. I wouldn't go beyond 5 metres with a passive. Switch to fibre for anything longer.
Sounds reasonable. 7 was where the prices started to get pretty close anyways, and I think this run would be borderline at that length so having the ability to go longer easily makes it a pretty easy decision if we go that way.

quote:

2. The minimum bend radius of a QSFP+ DAC is usually something around 60 mm. You generally want to be a fair bit looser than that, call it 80 mm, so a coil of DAC should probably have a diameter of minimum 16 cm.
Ok, ballpark 6.5", that's not too bad. A bit looser than I'd usually coil a cord but still definitely an option.

quote:

3. Transceiver flashing tools can get a bit pricey and considering how cheap generic, pre-flashed transceivers are these days I would honestly just recommend buying specific brand-flashed transceivers if you need one instead of trying to mess around with a reflashing kit. Unless you want to do tunable wavelength stuff but that's kind of verging on advanced optical physics voodoo and I think you should avoid getting near that.
My interest in reflashing would be primarily to be able to future-proof DACs if I ever end up with hardware that cares about what's plugged in to it. I figure when I'm scrounging for whatever high speed network gear is cheap it's a reasonable possibility I'll end up having to deal with that at some point. I don't want to have to buy new cables if I change hardware. Obviously for fiber modules I'd just get them preconfigured and leave them where they are.

quote:

4. Maybe? Depends on how far you're really planning on going. Obviously there's the distance factor, which sounds like something you might be coming up against. Also, MMF has a bend radius of 30 mm so if being able to bend in tight spaces matters to you, definitely go for glass instead of copper. The SR4-Lite stuff as far as I know is a Mellanox proprietary thing so... godspeed if you go with that.
At least within this house, where I'll be for at least another year, the high speed network is not likely to leave a single 12x20 room so the longest plausible run would be maybe 15m if we needed to get a cable over a doorway. That said, wherever I end up next all bets are off so the flexibility is definitely keeping it in the running.

I had been thinking the SR4-Lite stuff might have been better suited for short range use, but after looking in to it and finding out that normal SR4 was good down to 0.5m with the price being the same I obviously have no reason to go with the low power version if I go with optical.

On that note I can't really figure out the market for AOCs, they seem to be a "worst of both worlds" between fiber and DACs and aren't even really significantly cheaper than a pair of SR4 optics plus the appropriate patch fiber in between.

falz
Jan 29, 2005

01100110 01100001 01101100 01111010
I have the opposite opinion on 3. I'd *always* have some tools available to fix optic programming of any type, it's always an issue.

Also let's you function in a multi vendor environment with stocking only single part numbers.

The "easy button" for this is using someone like Flexoptix as a vendor as you then self code with an easy to use box that doesn't have you editing hex to fix (although we have one of those too)

Thanks Ants
May 21, 2004

#essereFerrari


What you need is to convince someone at work that you need a box to flash optics with and then just borrow it when required.

I have been doing some cabling work while I'm on this extended WFH period and realised there's a lot of tools I use fairly often that I was just taking home from the office when I needed them.

wolrah
May 8, 2006
what?
I did a bit more research in to reflashing transceivers and as far as I can find it's the same basic principle for QSFP+ as it is for SFP/SFP+, just no one makes a convenient Raspberry Pi hat to give you an easy interface to the transceiver's I2C lines. It seems like if I wanted to go down that path the cheapest option might actually be to buy another cheapo QSFP NIC and hack it up.

I ended up buying a 3m DAC from FS that identifies as a Mellanox cable. It'll work for my immediate needs and was the cheapest option by a substantial amount, which is nice considering this is purely for me to gently caress around rather than any practical purpose.

Thanks Ants posted:

What you need is to convince someone at work that you need a box to flash optics with and then just borrow it when required.

I have been doing some cabling work while I'm on this extended WFH period and realised there's a lot of tools I use fairly often that I was just taking home from the office when I needed them.
That's my usual strategy for sure, my work has paid for or otherwise provided basically everything in my network except for a few desktop switches and the computers themselves, but unfortunately we deal with fiber so rarely that I have no reason to need anything related.

wolrah fucked around with this message at 23:44 on Jun 5, 2020

wolrah
May 8, 2006
what?
Followup:

Got my DAC, installed and flashed the other card, connected everything together, and success.

The 3m DAC was definitely the right choice, it's the perfect length for my current setup. It just barely touches the ground when I have my desk in the low position and doesn't stress the cable at full height either. Any idea of using DACs to get any further than my desk next to my rack is gone though.

FatCow
Apr 22, 2002
I MAP THE FUCK OUT OF PEOPLE
Oh now this makes me tingly.

https://www.fs.com/products/96376.html

Charliegrs
Aug 10, 2009
For a Cisco 3702 AP the default mode is lightweight AP right? I have an RMA unit Cisco sent me hooked up the same switch port as the old unit and it has a pingable IP but the drat thing will not talk to the WLC. I ran capwap and cert debugs from the controller and I see no traffic whatsoever from this AP. I'm wondering if maybe the unit isn't in LAP mode?

BaseballPCHiker
Jan 16, 2006

Charliegrs posted:

For a Cisco 3702 AP the default mode is lightweight AP right? I have an RMA unit Cisco sent me hooked up the same switch port as the old unit and it has a pingable IP but the drat thing will not talk to the WLC. I ran capwap and cert debugs from the controller and I see no traffic whatsoever from this AP. I'm wondering if maybe the unit isn't in LAP mode?

Lightweight sounds correct to me.

I've had some success consoling into those APs and running a clear capwap ap all-config to get them to reset and finally talk to the WLC.

Also now that I think about it, check for any weird DHCP options you may have set for your network the AP is going into. I feel like there were some phone DHCP options we had set for a subnet that didnt seem to play nice with a couple of 3702s. I'll have to go back and dig up the tickets and look.

Charliegrs
Aug 10, 2009

BaseballPCHiker posted:

Lightweight sounds correct to me.

I've had some success consoling into those APs and running a clear capwap ap all-config to get them to reset and finally talk to the WLC.

Also now that I think about it, check for any weird DHCP options you may have set for your network the AP is going into. I feel like there were some phone DHCP options we had set for a subnet that didnt seem to play nice with a couple of 3702s. I'll have to go back and dig up the tickets and look.

That's actually the next step I'll be taking ( consoling into and wiping the AP) as far as DHCP options we just have option 43 to point the AP to the controller. That's the thing that bugs me about this AP replacement. It's literally just a hardware replacement we've made no config changes on anything not the switch, the switch port, or the WLC. So my guess is maybe this RMA unit isn't exactly "brand new" it might be a refurb with some remnant of an old config or it's on like scanner mode or something.

uhhhhahhhhohahhh
Oct 9, 2012
The DHCP option never did anything for me. Have you done the DNS entries for CISCO-CAP whatever? Even with them we'd still get random ones that just didn't seem to want to join. Manually putting controller IPs from SSH or Console always seemed to sort it out, mine are all 2x00 series APs though.

unknown
Nov 16, 2002
Ain't got no stinking title yet!


Partycat posted:

Would you care to pass along any major trouble you had with mikrotik in this sort of application? I’ve been unclear on high throughput perf on their x86/CHR platforms

Don't use mikrotik in a core internet router situation. While it can do it, it can't deal with that many routes.

In regards to their hardware (so not x86/CHR), they are single threaded processes, so RIB/FIB updates for peers that drop can take _minutes_.

Another fun thing is doing an IP route lookup can take a couple of minutes (!) in the CLI with a full table as it doesn't look it up in hardware and has to go through the entire routing table entry by entry seeing if it matches..

Buffers? What buffers? :haw:

There's lots of little things that just add up to a big no.

falz
Jan 29, 2005

01100110 01100001 01101100 01111010
Imo Mikrotik should only be used for

* SOHO stuff
* MDU stuff where shaping is required (shockingly good at that)
* maybe some wifi stuff

I say maybe on that last part because Ubiquiti does a better job for almost all wifi things.

Also note I have a tik as my home router / Nat box.

SamDabbers
May 26, 2003



I think that Mikrotik CCRs could possibly be decent bang for the buck in a BGP-less MPLS core as simple label switches, but haven't had an opportunity or reason to lab that out.

Thanks Ants
May 21, 2004

#essereFerrari


MikroTik is also good if you want an LTE modem that you can hang on a wall and power off a PoE switch

MF_James
May 8, 2008
I CANNOT HANDLE BEING CALLED OUT ON MY DUMBASS OPINIONS ABOUT ANTI-VIRUS AND SECURITY. I REALLY LIKE TO THINK THAT I KNOW THINGS HERE

INSTEAD I AM GOING TO WHINE ABOUT IT IN OTHER THREADS SO MY OPINION CAN FEEL VALIDATED IN AN ECHO CHAMBER I LIKE

So, I'm trying to do some ACL stuff on JunOS and I'm getting unexpected results, and results that directly conflict with their KB articles.

My end goal is to allow a VLAN to talk to other members of its' VLAN (Subnet1), to a second VLAN (Subnet2), the internet and nothing else.

I have the following configured:


set firewall family ethernet-switching filter Test-Filter

set term Test-Term-1 from destination-address Subnet1

set term Test-Term-1 then accept

set term Test-Term-2 from destination-address Subnet2

set term Test-Term-2 then accept

set vlans vlan_subnet1 filter input Test-Filter


With this configuration, Subnet1 cannot talk to other members of subnet1, but can talk to subnet2; subnet2 can talk to subnet1 just fine as well. Subnet1 cannot talk to the internet (which I expect at this point).

Juniper's own documentation says that this should work (https://www.juniper.net/documentation/en_US/junos/topics/example/firewall-filter-ex-series-configuring.html), even loving around with this and having only Test-Term-1 in the filter does not allow it, changing order etc does nothing.

This is a 3400ex switch with JunOS 14 or 15 I can't remember the exact revision. Any hints here as to what I'm doing incorrectly?

Thanks Ants
May 21, 2004

#essereFerrari


Aruba are buying Silver Peak

https://news.arubanetworks.com/press-release/hpe-accelerates-edge-cloud-strategy-acquisition-sd-wan-leader-silver-peak

unknown
Nov 16, 2002
Ain't got no stinking title yet!


SamDabbers posted:

I think that Mikrotik CCRs could possibly be decent bang for the buck in a BGP-less MPLS core as simple label switches, but haven't had an opportunity or reason to lab that out.

I know a few people who have done that and haven't complained about it.

You run into the standard mikrotik issues (buffers, interoperability, etc) if you need to move large amounts of data.

BaseballPCHiker
Jan 16, 2006

Anyone have any advice on how to get a provider (Comcast) to fix their ENS connection for a few sites? Im having a hell of a time trying to get them to do anything about it, besides saying their device is up and the connection is fine.

Basically about 3 days ago at 4am three of our sites went down all at the same time. I rush out to the sites, and get a few of them back up either over old dark fiber or cellular backups. Looking into it more it seems that EIGRP is failing, the l3 switches will receive hellos, form adjacencies, then fail to send out Hellos successfully and collapse, over and over.

At first I thought maybe it was a cisco bug or something, so I update code and still nothing. Next thing I did was to grab a new l3 switch, right out of the box, and configure it with the same setup to take out to the site. Had the exact same issue. So this time I plug it in to our Comcast provided Cienna device and take a Pcap to send to them. It basically just shows EIGRP failing over and over. Whats interesting is if I run a show CDP neighbors I only see like 3 out of our 20+ ENS sites.

Because this is an ENS point to multipoint setup I should be able to run a show CDP neighbors and see all of my other sites. I bring this up to Comcast and they shrug and go yeah thats weird but dont really have an answer for me.

The other interesting wrinkle is that we cut off service at two other ENS sites the day before all of this happened. When I brought this up to Comcast to see if they may have accidentally terminated service to the wrong locations I got literally nothing out of them.

At this point our sites have been down for 3 days and I'm running out of patience. Any advice on what I could do to help move this along? At this point I am 99% convinced its on Comcasts end, I just dont know how to help them along.

uhhhhahhhhohahhh
Oct 9, 2012
Time to start looking up phone numbers and emails for division/senior managers/CEOs on LinkedIn and start spamming them about the lovely service you're receiving.

BurgerQuest
Mar 17, 2009

by Jeffrey of YOSPOS
Call them out on NANOG maybe? Might get some support.

falz
Jan 29, 2005

01100110 01100001 01101100 01111010
Do you have an account rep or anything? Try to get an escalation tree.

Then shop another provider or diversify providers.

Then switch igp away from eigrp to ospf or something standard so you'll be ahead of the game if you ever want to mix in another non-cisco vendor.

BaseballPCHiker
Jan 16, 2006

It gets better!

The Comcast tech I sent all of the info to just decided to close the ticket randomly apparently without notifying us!!! So I got to open another ticket for another one of the sites that are down and go through the same thing. Started spamming our account reps too to see if that gets us anywhere.

Partycat
Oct 25, 2004

The account reps better be able to escalate, otherwise you got a 30 site WAN to shop for a better deal from another telco.

unknown
Nov 16, 2002
Ain't got no stinking title yet!


falz posted:

Do you have an account rep or anything? Try to get an escalation tree.

Then shop another provider or diversify providers.

Quoting because it's true..

Always always ALWAYS always get the latest escalation chart for your wan providers as a quarterly thing to do.

And then don't let the ticket close until you get it solved to your satisfaction/specs, and then file a billing complaint to get a credit on the ticket for the outage as a gently caress you.

Basically if you deal with a monolithic company they've got processes to deal with issues and you need to use that against them. Challenge #1 is getting that process (also called an escalation chart). You'd be surprised how fast things get solved properly when you're allowed to go up two levels or more of management and that manager is not able to say no/gently caress off to you because then they are in breach of contract.

I've actually called an EVP at 5am before because people lower on the escalation chart wouldn't pick up their phones like they should have. Let me tell you, poo poo got fixed fast as he started calling people to ask why people weren't doing their job and he was getting woken up and he couldn't poo poo on me.

Of course this is assuming you're buying a circuit that has a real sla (like 4 hour mttr).

BaseballPCHiker
Jan 16, 2006

Our account rep is useless. And his sales engineer guy, I dont even know where to begin. Its like he devoted his life to AppleTalk or something and lives in some bizarro networking world because nothing ever seems to sink in with them.

We opened another ticket for our other down site, CC'd reps demanded to know why our other ticket was closed without notifying us and now my boss is asking about SLAs and service credits. We also started saying that one of these sites is a fire station and that this could effect public safety. We'll see what comes out of it.

Thankfully between dark fiber and cellular routers no site is hard down. First time I've ever had an outage with Comcast like this. They've been great and solid for the 5+ years we've had them until now.

Thanks Ants
May 21, 2004

#essereFerrari


I understand people have QoS requirements or compliance issues but I cannot be arsed dealing with WAN stuff from ISPs any more, they always gently caress the routing up or you get a different configuration built depending on which tech builds the changes, with each change request just compounding the disaster of a state that the configurations are in. I don't care about any of the potential cost savings or the carrier diversity, for me the best thing about SD-WAN overlay using VPN tunnels is that I can see the config that's running and get it changed without five days notice.

If your MPLS is just a massive L2 domain and you run the stuff at the edge yourself then you should be OK

uhhhhahhhhohahhh
Oct 9, 2012

Thanks Ants posted:

I understand people have QoS requirements or compliance issues but I cannot be arsed dealing with WAN stuff from ISPs any more, they always gently caress the routing up or you get a different configuration built depending on which tech builds the changes, with each change request just compounding the disaster of a state that the configurations are in. I don't care about any of the potential cost savings or the carrier diversity, for me the best thing about SD-WAN overlay using VPN tunnels is that I can see the config that's running and get it changed without five days notice.

If your MPLS is just a massive L2 domain and you run the stuff at the edge yourself then you should be OK

We moved from an unmanaged VPLS to a managed L3 MPLS and it's total dogshit. My managers tried to sell me on it because 'we wouldn't have to worry about routing' and that it would be someone else's problem (????) and it saved £10k out of a contract that's over a mil. I still have to worry about routing, it's still my problem because I'm expected to fix it, except now I can't fix it when there's a problem and were buried under layers of jank trying to make poo poo work because it turns out the ISP doesn't have dozens of network gods waiting to implement the perfect network for us. We'll be putting up some new DHCP servers soon and we'll have to pay like £300*50, one for each site to have its IP helper reconfigured, because they poo poo on my idea to have our own L3 switch or router at each site that we peered with the ISP router so we could at least be in control of things like that.

When we were transitioning our backup DC to the new network and obviously, there was a problem with the VRFs. We got a call booked and an engineer and I asked him to check the config on their router, he was silent for a minute or two and then said, "sorry, i don't know what a VRF is."

Felt super bad for him. What a bunch of pieces of poo poo booking that guy onto the call in the first place.

Impotence
Nov 8, 2010
Lipstick Apathy
I don't understand the whole WAN or site to site or whatever via ISPs, why would you not run your own overlay network / how is there any larger QoS problem than can be enforced by regular SLAs? The compliance thing I can understand but that seems to imply that the two points should be connected by some private link, but that almost certainly won't physically exist at an ISP and it'll just be on shared transport anyway?

Kazinsal
Dec 13, 2011



BaseballPCHiker posted:

It gets better!

The Comcast tech I sent all of the info to just decided to close the ticket randomly apparently without notifying us!!! So I got to open another ticket for another one of the sites that are down and go through the same thing. Started spamming our account reps too to see if that gets us anywhere.

Yep, you're definitely in NANOG callout post territory now.

Thanks Ants
May 21, 2004

#essereFerrari


Just make really sure you're right before the nuclear callout, it's a loving terrible feeling when you go ballistic in a ticket and then five minutes later realise you hosed up.

Adbot
ADBOT LOVES YOU

Partycat
Oct 25, 2004

Yes but sweet when you’re completely right. Unfortunately poo poo reps just move off your account , they never seem to leave the company

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply