Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
teethgrinder
Oct 9, 2002

I can live without internet, but 8+ hours without meaningful communication about what is going on is lol.

My Freedom phone internet has been decaying all day to where I can barely even send IMs.

I'm going outside!!

Adbot
ADBOT LOVES YOU

Methanar
Sep 26, 2013

by the sex ghost
The removal of internet from canadian society is a good thing

Kazinsal
Dec 13, 2011



The Yukon fibre cut the other day was a loving mess as well. Highway washout broke Northwestel's line to BC and cut them off from the outside world for twelve hours. There's currently only the one trunk along the Alaska Highway and if that thing goes down the whole territory loses connectivity.

Mister Facetious
Apr 21, 2007

I think I died and woke up in L.A.,
I don't know how I wound up in this place...

:canada:
I hope thousands of Canadians are learning a valuable lesson about using smart home products that need to contact an outside server to be useful

MatchaZed
Feb 14, 2010

We Can Do It!


And yet this will not lead to proper regulations since the CRTC and the Libs and Cons are owned by the telecoms.

gently caress I hate this country.

teethgrinder
Oct 9, 2002

Mister Facetious posted:

I hope thousands of Canadians are learning a valuable lesson about using smart home products that need to contact an outside server to be useful

Meh a lot of them seem designed to work around this. I.e. the connected lights will power on manually if you cycle the power switch.

The down side of this is that those lights power on when there's a brief power outage overnight lol.

Chris Knight
Jun 5, 2002

me @ ur posts


Fun Shoe
https://twitter.com/crtceng/status/1545421218534359041

Mister Facetious
Apr 21, 2007

I think I died and woke up in L.A.,
I don't know how I wound up in this place...

:canada:

The government watchdog (that is headed by retired executives of Rogers Bell and Telus) is out too???

Mister Facetious
Apr 21, 2007

I think I died and woke up in L.A.,
I don't know how I wound up in this place...

:canada:

MatchaZed posted:

And yet this will not lead to proper regulations since the CRTC and the Libs and Cons are owned by the telecoms.

gently caress I hate this country.

Bet you any money that Rogers will use this outage as a reason IN FAVOR of letting them buy Shaw

Dallan Invictus
Oct 11, 2007

The thing about words is that meanings can twist just like a snake, and if you want to find snakes, look for them behind words that have changed their meaning.

Mister Facetious posted:

The government watchdog (that is headed by retired executives of Rogers Bell and Telus) is out too???

only our landlines/main contact number is out, internet and work cells are fine

long weekend for the call centre agents, I guess

Chris Knight
Jun 5, 2002

me @ ur posts


Fun Shoe
Cross post

CLAM DOWN posted:

the entire Rogers IP block is officially off the internet lmao, this will be a while



Cloudflare sez
https://blog.cloudflare.com/cloudflares-view-of-the-rogers-communications-outage-in-canada/

quote:

Based on what we’re seeing and similar incidents in the past, we believe this is likely to be an internal error, not a cyber attack.

Cloudflare Radar shows a near complete loss of traffic from Roger’s ASN, AS812, that started around 08:45 UTC (all times in this blog are UTC).

Cloudflare data shows that there was a clear spike in BGP (Border Gateway Protocol) updates after 08:15, reaching its peak at 08:45.

A BGP update message informs a router of changes made to a prefix (a group of IP addresses) advertisement or entirely withdraws the prefix. In this next chart, we can see that at 08:45 there was a withdrawal of prefixes from Roger’s ASN.

Since then, at 14:30, attempts seem to be made to advertise their prefixes again. This maps to us seeing a slow increase in traffic again from Rogers’ end users.

The graph below, which shows the prefixes we were receiving from Rogers in Toronto, clearly shows the withdrawal of prefixes around 08:45, and the slow start in recovery at 14:30, with another round of withdraws at around 15:45.

MustardFacial
Jun 20, 2011

Fucker in charge of you fucking fucks




It's definitely not a cyber attack. Rogers pulled a Facebook and misconfigured some core router somewhere to tell the rest of the internet "Hey, this block of IP Addresses no longer exists. Don't send traffic here."

You can watch all of the connections drop from Rogers in this video I recorded. All of the circles represent some kind of network operator (ISP's, CDN's, etc.), Rogers is the red circle. All of the lines moving around are routing paths calculated by all of the network operators trying to find alternate ways to communicate with Rogers as they drop each individual connection. When the lines disappear is the connection dropping from Rogers either because Rogers is announcing "this no longer exists" or no alternate path can be calculated to the destination.

https://www.youtube.com/watch?v=aRNcdYejaqw

MustardFacial fucked around with this message at 21:52 on Jul 8, 2022

shadow puppet of a
Jan 10, 2007

NO TENGO SCORPIO


They should make a new Canadian heritage moment about the recent grad from Herzing college that managed to shut down half of canada

Mister Facetious
Apr 21, 2007

I think I died and woke up in L.A.,
I don't know how I wound up in this place...

:canada:

MustardFacial posted:

It's definitely not a cyber attack. Rogers pulled a Facebook and misconfigured some core router somewhere to tell the rest of the internet "Hey, this block of IP Addresses no longer exists. Don't send traffic here."

You can watch all of the connections drop from Rogers in this video I recorded. All of the circles represent some kind of network operator (ISP's, CDN's, etc.), Rogers is the red circle. All of the lines moving around are routing paths calculated by all of the network operators trying to find alternate ways to communicate with Rogers as they drop each individual connection. When the lines disappear is the connection dropping from Rogers either because Rogers is announcing "this no longer exists" or no alternate path can be calculated to the destination.

https://www.youtube.com/watch?v=aRNcdYejaqw

Needs more Benny Hill in the background

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Mister Facetious posted:

Needs more Benny Hill in the background

Looks like withdrawing their ASN also took their DNS with it so they very likely Facebooked themselves

some kinda jackal
Feb 25, 2003

 
 

Number19 posted:

they very likely Facebooked themselves

CRTC really should mandate that more of our disasters are cancon, not just what we import from America.

Shumagorath
Jun 6, 2001

shadow puppet of a posted:

They should make a new Canadian heritage moment about the recent grad from Herzing college that managed to shut down half of [Canada]

w00tmonger
Mar 9, 2011

F-F-FRIDAY NIGHT MOTHERFUCKERS

Blown away that they still don't have an ETA and cause

Oxyclean
Sep 23, 2007


You still down? My rogers internet came back sometime overnight, but some stuff has been a bit slow.

Lobok
Jul 13, 2006

Say Watt?

My cable TV came back but it's choppy.

St_Ides
May 19, 2008
My Rogers-network Primus came back on sometime between 11 and 3 AST last night in NB

Zaroff
Nov 10, 2009

Nothing in the world can stop me now!
My cellphone came back yesterday evening, and the internet (through Fido) came back this morning, however it’s not been 100% yet.

Snapshot
Oct 22, 2004

damnit Matt get in the boat
My Rogers internet is still out. Tried calling as it shows no problems at my address and the wait time was over three hours. Was hung up on a couple of hours in. Live chat not much better.

Lobok
Jul 13, 2006

Say Watt?

So the cable TV feed is smooth now... but the audio is doubled and out of sync so every sound and voice is repeated two seconds later, lol

:doom: ROGERSSSSS!

Lobok
Jul 13, 2006

Say Watt?

^ if only we had a Red Skull version of that

Oxyclean
Sep 23, 2007


Current I cannot load a youtube video by some Canadian LPers/streamers, but I can watch many youtube videos from American creators. Rogers failing CanCon.

Tippecanoe
Jan 26, 2011

Any advice on getting a good deal on bell fibre? I'm thinking I'm gonna switch over pretty soon. My internet's still down and my connection's been flakey enough lately that I already wanted to switch

Shumagorath
Jun 6, 2001

Tippecanoe posted:

Any advice on getting a good deal on bell fibre? I'm thinking I'm gonna switch over pretty soon. My internet's still down and my connection's been flakey enough lately that I already wanted to switch
They would only go as low as $80/mo for 1Gbps when my condo fibre is $50. They made some argument about it being a dedicated line as if they're not doing carrier-grade NAT for every tower too. Of course I was going to have to call in and renew that bargain every year.

Just say you have a current plan / competing offer and they can make $X or they can make $0.

infernal machines
Oct 11, 2012

we have sealed ourselves away behind our money, growing inward, generating a seamless universe of self.

Shumagorath posted:

They would only go as low as $80/mo for 1Gbps when my condo fibre is $50. They made some argument about it being a dedicated line as if they're not doing carrier-grade NAT for every tower too.

lol. Bell Fibe is GPON, so it's very much not a dedicated line. It's good though, if you don't have any other fibre options.

ToxicFrog
Apr 26, 2008


Nitr0 posted:

What amazes me most about this is actually Interac. How could a national service only have one ISP? Interac is literally down across the country because Rogers went down? Come on guys.

IIRC individual terminals use whatever ISP the host business does but the actual Interac servers are hosted in a Rogers datacentre somewhere, so :rip:

teethgrinder posted:

I can live without internet, but 8+ hours without meaningful communication about what is going on is lol.

My Freedom phone internet has been decaying all day to where I can barely even send IMs.

See, this confuses and infuriates me -- my internet went down, ok, I'm not on Rogers but they own the last mile here so that's no surprise. But usually when that happens I can tether to my phone and keep on going. Instead the Freedom Mobile network was so hosed I couldn't even get phone calls through, let alone IP traffic.

One of the reasons I signed on with them back when they were WIND is that they had their own towers and backbone infrastructure independent of Bell and Rogers, and weren't just a Rogers reseller. If they're now dependent on Rogers infrastructure why the gently caress am I still paying them rather than signing on with loving Bell so there's at least a chance they won't both be offline at once?

acetcx
Jul 21, 2011

ToxicFrog posted:

IIRC individual terminals use whatever ISP the host business does but the actual Interac servers are hosted in a Rogers datacentre somewhere, so :rip:

See, this confuses and infuriates me -- my internet went down, ok, I'm not on Rogers but they own the last mile here so that's no surprise. But usually when that happens I can tether to my phone and keep on going. Instead the Freedom Mobile network was so hosed I couldn't even get phone calls through, let alone IP traffic.

One of the reasons I signed on with them back when they were WIND is that they had their own towers and backbone infrastructure independent of Bell and Rogers, and weren't just a Rogers reseller. If they're now dependent on Rogers infrastructure why the gently caress am I still paying them rather than signing on with loving Bell so there's at least a chance they won't both be offline at once?

My cell phone is with Telus and during the Rogers outage it was dogshit slow. No videos and even images took like 15 seconds to load. I assume it was overloaded from all the people using it since their home internet was down.

Nolgthorn
Jan 30, 2001

The pendulum of the mind alternates between sense and nonsense
Am I understanding this correctly. Rogers is first of all apparently the only reason half of Canada is connected to anything. Rogers announced to the entire internet that it no longer exists, so all traffic that used to go to Rogers stopped. Now they're trying to figure out how to get Canada connected again?

shadow puppet of a
Jan 10, 2007

NO TENGO SCORPIO


I spent the whole workday tethered to a vpn and voip on freedom and while it wasn’t fast like normal it was perfectly fine for what I needed from it.

I still sometimes can’t send texts to people 10 feet away from me, but that’s business as usual with freedom.

Shumagorath
Jun 6, 2001

ToxicFrog posted:

IIRC individual terminals use whatever ISP the host business does but the actual Interac servers are hosted in a Rogers datacentre somewhere, so :rip:

See, this confuses and infuriates me -- my internet went down, ok, I'm not on Rogers but they own the last mile here so that's no surprise. But usually when that happens I can tether to my phone and keep on going. Instead the Freedom Mobile network was so hosed I couldn't even get phone calls through, let alone IP traffic.

One of the reasons I signed on with them back when they were WIND is that they had their own towers and backbone infrastructure independent of Bell and Rogers, and weren't just a Rogers reseller. If they're now dependent on Rogers infrastructure why the gently caress am I still paying them rather than signing on with loving Bell so there's at least a chance they won't both be offline at once?
I'm with Freedom too, and I got my best service in the subway where it's 100% their towers with no Rogers roaming.

Nolgthorn
Jan 30, 2001

The pendulum of the mind alternates between sense and nonsense
Cloudflare is reporting back to normal does all of your internet work again?

https://radar.cloudflare.com/asn/812?date_filter=last_7_days

MasterBuilder
Sep 30, 2008
Oven Wrangler

ToxicFrog posted:

See, this confuses and infuriates me -- my internet went down, ok, I'm not on Rogers but they own the last mile here so that's no surprise. But usually when that happens I can tether to my phone and keep on going. Instead the Freedom Mobile network was so hosed I couldn't even get phone calls through, let alone IP traffic.

Well everyone else on freedom wanted to do the same as you so throughput went to poo poo. I got a haircut yesterday and my barber said he was hotspotting for everyone else. I went through 2gb to cover work stuff just by myself.

Stanley Pain
Jun 16, 2001

by Fluffdaddy
I dropped something. What was it? All of my BGP routes. LMAO.

Methanar
Sep 26, 2013

by the sex ghost
The insane thing is how long it lasted.

What exactly did they do they it took like 12 hours to undo. Like, did they somehow nuke their oob management network, and then have no ability to even access their own devices?

Was there a logistical nightmare of even being able to contact anybody else with the whole network down?
Was there some trainwreck of authenticating the staff to physically enter colo locations with a serial cable in hand?
Did they have to do like a coordinated dance with multiple engineers in different physical locations manually removing a config change over console.

what even the gently caress. I want to read the post mortem for this

infernal machines
Oct 11, 2012

we have sealed ourselves away behind our money, growing inward, generating a seamless universe of self.
not everything is done to the standard of pirate streaming sites on homebrew hardware, i'm afraid

Adbot
ADBOT LOVES YOU

Methanar
Sep 26, 2013

by the sex ghost

infernal machines posted:

not everything is done to the standard of pirate streaming sites on homebrew hardware, i'm afraid

You make fun of me but I only nuked my network like 3 times in 3 years.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply