Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
George H.W. Cunt
Oct 6, 2010





I brought eggnog today. It’s going to be a nice afternoon

Adbot
ADBOT LOVES YOU

Japanese Dating Sim
Nov 12, 2003

hehe
Lipstick Apathy
Hello, page 1337.

Someone please kill me for noticing and feeling obligated to comment on it, TIA.

silicone thrills
Jan 9, 2008

I paint things

Japanese Dating Sim posted:

Hello, page 1337.

Someone please kill me for noticing and feeling obligated to comment on it, TIA.

It's ok, 2002 was the peak for alot of people.

Japanese Dating Sim
Nov 12, 2003

hehe
Lipstick Apathy

silicone thrills posted:

It's ok, 2002 was the peak for alot of people.

:hfive:

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer
Today I continue to deal with a sev1 outage due to Christmas Eve production changes

chin up everything sucks
Jan 29, 2012

I believe it is now time for Working in IT 4.0 to be created

Thanks Ants
May 21, 2004

#essereFerrari


Kashuno posted:

Today I continue to deal with a sev1 outage due to Christmas Eve production changes

How many bodies have you had to hide so far?

ChubbyThePhat
Dec 22, 2006

Who nico nico needs anyone else
Maybe only had to kill one and display it for all to see. I feel that would send the message.

LochNessMonster
Feb 3, 2005

I need about three fitty


Worked from home today. Wanted to sleep in but got paged at 8 that our development swarm cluster was having issues. After that breezed through the day and had 0 questions or interrupts. I need more days like today.

12 rats tied together
Sep 7, 2006

Judge Schnoopy posted:

Yes, scripts have very specific single use cases, and aren't meant to be interactive. Modules are tools that can be distributed and used in a variety of ways. None of this has anything to do with powershell.

Powershell specifically excels in this area (modules) when compared to something like Python, though.

It's hundreds of times easier to push a cmdlet to a module with a fancy argument specification, default values, a help file, shell autocomplete, and other poo poo like that and then have your coworker be able to use it interactively from powershell.exe than it is to do the same with python.exe.

I don't think "the defining feature of the language" and "a shell more than a scripting language" are particularly contentious statements in this light.

It being a garbage scripting language though was intentonally contentious. As you said, gently caress IWR, but gently caress System.Web.WebClient too, along with pretty much every adjacent namespace in the framework.

Farking Bastage
Sep 22, 2007

Who dey think gonna beat dem Bengos!
Poor Centurylink...



quote:

Update:
On December 27, 2018 at 02:40 GMT, CenturyLink identified a service impact in New Orleans, LA. The NOC is engaged and investigating in order to isolate the cause. Field Operations were engaged and dispatched for additional investigations. Tier IV Equipment Vendor Support was later engaged. During cooperative troubleshooting a device in San Antonio, TX was isolated from the network as it was seeming to broadcast traffic consuming capacity, which seemed to alleviate some impact. Investigations remained ongoing. Following the isolation of the San Antonio, TX device troubleshooting efforts focused on additional sites that teams were remotely unable to troubleshoot. Field Operations were dispatched to sites in Kansas City, MO, Atlanta, GA, New Orleans, LA and Chicago, IL. Tier IV Equipment Vendor Support continued to investigate the equipment logs to further assist with isolation. Once visibility was restored to the site in Kansas City, MO a filter was applied to the equipment to further alleviate the impact observed. All of the necessary troubleshooting teams in cooperation with Tier IV Equipment Vendor Support are working to restore remote visibility to the remaining sites at this time. We understand how important these services are to our clients and the issue has been escalated to the highest levels within CenturyLink Service Assurance Leadership.

A fiber multiplexer flaked out and broadcast stormed their entire network.

Tetramin
Apr 1, 2006

I'ma buck you up.

Farking Bastage posted:

Poor Centurylink...




A fiber multiplexer flaked out and broadcast stormed their entire network.

I was confused as hell looking into toll free outages when me and everyone in the office could reach the numbers from our cells, but nobody in MO could.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer

Thanks Ants posted:

How many bodies have you had to hide so far?

Can I hide my own body?

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


All of our west coast newrelic synthetic monitors died for about a half hour today, giving me panic that I thought our sites were down. I can only presume it was Centurylink causing that.

Ham Equity
Apr 16, 2013

i hosted a great goon meet and all i got was this lousy avatar
Grimey Drawer

Farking Bastage posted:

Poor Centurylink...




A fiber multiplexer flaked out and broadcast stormed their entire network.

My forums access is as fast as ever, but this is preventing us from connecting to the Fed. Which, as an FI, seems maybe slightly more important than shitposting.

But only slightly.

Gucci Loafers
May 20, 2006

Ask yourself, do you really want to talk to pair of really nice gaudy shoes?


I use Powershell with...

- Active Directory
- Active Directory Federation Services
- SQL Server
- Windows including my own “de-crapifyer” for my workstations via Chocolatey
- Azure AD
- Azure: Networking, VMs, Post-Deployment Scripts on VMs.
- Exchange and Exchange Online
- SharePoint and SharePoint Online

I guess you could say I use Powershell quite a bit and I have usually have the ISE on my screen over half of the day.

Slowly moving to VS Code but the vendor refuses to push out a version that isn’t nearly a year behind.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer
Power shell.....is good

The Fool
Oct 16, 2003


Tab8715 posted:

I use Powershell with...

- Active Directory
- Active Directory Federation Services
- SQL Server
- Windows including my own “de-crapifyer” for my workstations via Chocolatey
- Azure AD
- Azure: Networking, VMs, Post-Deployment Scripts on VMs.
- Exchange and Exchange Online
- SharePoint and SharePoint Online

I guess you could say I use Powershell quite a bit and I have usually have the ISE on my screen over half of the day.

Slowly moving to VS Code but the vendor refuses to push out a version that isn’t nearly a year behind.

All of this, except I don't do anything with SQL.

I also put a fair amount of time in to automation via Azure Devops. Using CI/CD pipelines as a scheduler and launcher for various tasks works very well.

Basically, why give a ps1 to some monkey when I can automate the running of the script too?

Schadenboner
Aug 15, 2011

by Shine

Farking Bastage posted:

Poor Centurylink...




A fiber multiplexer flaked out and broadcast stormed their entire network.

Im teh Tier IV Support.

Farking Bastage
Sep 22, 2007

Who dey think gonna beat dem Bengos!

Schadenboner posted:

Im teh Tier IV Support.

drat dude. That's gotta suck. Here's something I got from reddit when I posed the question of why in the gently caress was everything on the same broadcast domain:

quote:

Nah. I've heard rumors that this is a problem with their long haul DWDM network since there are a lot of transport circuits impacted in addition to Layer 3 services. My guess is the broadcast storm is on the management plane of that caused by lots of attempts to replicate with other nodes.

Edit: To expand a little bit. Most long haul fiber is lit with transport equipment like Ciena or Infinera. I believe CenturyLink is a Infinera network. They're big, so they're probably using something like the Infinera DTN-X XTC to light fiber at 1.2Tb/s and then sell subrate interfaces off of it. So you could buy 10G or 100G on a link between Denver and Chicago. Packets that go in Denver show up in Chicago. Since there's no direct fiber from Denver to Chicago, they map you to a channel on Denver to Kansas City and then a different channel from Kansas City to Chicago.

Everything is going SDN, even transport, so the definition of how to route traffic from Denver to Chicago is managed in software. My guess is the 'broadcast storm' is impacting the management layer of the transport network and preventing them from making changes to different nodes to restore services. That would explain the need to go touch each node to restore services.

Now when it comes to providing internet access, most ISPs will actually 'buy' transport from their long haul division. So there's a MPLS/L3 network layered on top the transport equipment to provide IP services. For example, CenturyLink will have a core router in Denver, connected to their Denver DWDM transport equipment at 400G, with a connection to Chicago where it connects at 400G to a core router in Chicago. All of the internet services you use come from that additional layer that uses circuits provisioned by the transport network. Sitting right next to that 400G path are other circuits for other carriers (Cogent, HE, Comcast, basically anyone that isn't operating their own national fiber network)

Usually the thing that fails is the MPLS/L3 layer, transport gear is extremely reliable outside of random fiber cuts and line card failures. When you hear about a network problem it's almost always the L3 layer. In this case, I'm guessing that the L3 layer is fine, but there's no or reduced connectivity between nodes due to the DWDM transport layer failure.

Luckily none of my sites in FL were affected on AS209 and AS2379

Farking Bastage fucked around with this message at 03:20 on Dec 28, 2018

AreWeDrunkYet
Jul 8, 2006

Schadenboner posted:

Im teh Tier IV Support.

I once worked a desktop support job and some networking change basically cut off the building we were in. There was nothing we could do, but people in charge decided that we should be doing something. What they finally came up with was printing off hundreds of pages of a generic statement about the outage and having us go to door to door handing them out.

Docjowles
Apr 9, 2009

AreWeDrunkYet posted:

I once worked a desktop support job and some networking change basically cut off the building we were in. There was nothing we could do, but people in charge decided that we should be doing something. What they finally came up with was printing off hundreds of pages of a generic statement about the outage and having us go to door to door handing them out.

This is lol, but also not totally bananas. Who among us hasn’t heard “hey is the internet down for you too?” spread like wildfire throughout the office as you get a sinking feeling in your gut and pull up your monitoring tool of choice to see all red?

Printing off hundreds of fliers is a little dramatic but I support the general idea of spreading info during a lovely time. What are you gonna do, email everyone? (Just kidding, our parent company IT totally emails people in the event of an email outage, which we can enjoy reading hours later :v:)

GreenNight
Feb 19, 2006
Turning the light on the darkest places, you and I know we got to face this now. We got to face this now.

We can broadcast messages via our Cisco phones which tends to scare the poo poo out of people. We only use it when email is down. Which is also extremely rare. Maybe twice in the last 15 years.

Internet Explorer
Jun 1, 2005





We text people, just like we would if there was any other emergency. Out of band communication is important. :)

mllaneza
Apr 28, 2007

Veteran, Bermuda Triangle Expeditionary Force, 1993-1952




I'm really enjoying getting into Powershell. I just keep running into places where our corporate overlords have locked down scripts; "-ExecutionPolicy override" is something I use way too much.

Last month I wrote my first line of Powershell. Next month we're running my code on machines in the Research OU. Yikes and gently caress yeah at the same time.

tortilla_chip
Jun 13, 2007

k-partite
It's not the same broadcast domain. The filtering config on their ROADMs got borked, and apparently there's no OOB path for a fix (my guess being their OOB network rides the OSC/DCC). That's a lot of truck rolls.

tortilla_chip fucked around with this message at 04:52 on Dec 28, 2018

Wibla
Feb 16, 2011

A similar issue took down the Oslo subway system a few months ago :v:

Lets Get Patchy
Aug 8, 2006

mllaneza posted:

I'm really enjoying getting into Powershell. I just keep running into places where our corporate overlords have locked down scripts; "-ExecutionPolicy override" is something I use way too much.

Last month I wrote my first line of Powershell. Next month we're running my code on machines in the Research OU. Yikes and gently caress yeah at the same time.

Hell ya. We barely touched on powershell in my advanced sysadmin class last semester so I decided to get a few udemy classes and a book to study over the winter break. I really enjoy working with it so far.

Farking Bastage
Sep 22, 2007

Who dey think gonna beat dem Bengos!

Lets Get Patchy posted:

Hell ya. We barely touched on powershell in my advanced sysadmin class last semester so I decided to get a few udemy classes and a book to study over the winter break. I really enjoy working with it so far.

It really comes in handy when you can script an O365 tenant for a migration. Before I went full network, I could fully build out a tenant down to the last note or contact in about 30 minutes.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer
It’s friday night

Going into day6 of the sev1 outage with it being still a pretty severe problem but downgraded to sev2. Sounds like a good enough reason to celebrate with a beer

Methanar
Sep 26, 2013

by the sex ghost

Kashuno posted:

It’s friday night

Going into day6 of the sev1 outage with it being still a pretty severe problem but downgraded to sev2. Sounds like a good enough reason to celebrate with a beer

What can break that it stays broken for 6 days

Docjowles
Apr 9, 2009

Something that was not actually a sev 1 tier service, if it's down for a week and the company remains in business :pseudo:

Zil
Jun 4, 2011

Satanically Summoned Citrus


Methanar posted:

What can break that it stays broken for 6 days

At this point I would guess everything.

Methanar
Sep 26, 2013

by the sex ghost

Docjowles posted:

Something that was not actually a sev 1 tier service, if it's down for a week and the company remains in business :pseudo:

Yeah like thats my question.

Okay someone does something really really dumb and now you need to do an 8 hour mysql dump restore. Fine. What happens that takes over 6 days to fix

SeaborneClink
Aug 27, 2010

MAWP... MAWP!
Spoiler: The OP works for CenturyLink :ssh:

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


When you own the code sev1 can last a while.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer

Methanar posted:

What can break that it stays broken for 6 days

Long story short, a non reversible change was made to a thing that caused our older in house customized EDI solution to be hosed permanently and we’ve had little to no EDI with our main 3rd party storage facilities this entire week.

Sev1 for us doesn’t make it Sev1 for our new EDI provider the week of Christmas or the 3rd party.

They are managing manual entry for most of the stuff but poo poo is not shipping on time or is being missed entirely, which since it’s food causes shelf life issues etc. one of the orders that didn’t ship of course was the CEOs partner’s company so that has been fun.

Like it’s not that it takes 6 days to fix, it’s that it’s not just an internal issue at this point with mapping requirements and changes etc, but it’s sev1 for us because the CEO is very angry and also we lose money on everything that doesn’t ship

Kashuno fucked around with this message at 05:19 on Dec 29, 2018

Docjowles
Apr 9, 2009

What impact is this having on the value of Fishcoin?

SeaborneClink
Aug 27, 2010

MAWP... MAWP!
Oh yes I forgot the "how inconvenienced is management" column in the incident response severity matrix.

Not a SEV1 because 19% of orders were picked, packaged or shipped incorrectly, but one order out of several hundred or thousand a day caused a personal inconvenience for the one person that can shout internally.

Adbot
ADBOT LOVES YOU

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


Lol. Have him killed

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply