Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
Shut up Meg
Jan 8, 2019

You're safe here.

larchesdanrew posted:

She did not last the day.

I feel pity joy and relief.

Adbot
ADBOT LOVES YOU

nexxai
Jul 17, 2002

quack quack bjork
Fun Shoe

larchesdanrew posted:

I feel pity and relief.
:gizz:

terrenblade
Oct 29, 2012

larchesdanrew posted:

I can't be arsed to go back and dig up the old post, but a few years ago I was involved in some drama involving the wicked Property Control Clerk, a staggeringly poorly documented equipment disposal, and the most lies I've ever seen a human being tell. I almost lost my job and got into serious legal trouble over the property clerk throwing mountains of buses on top of me to protect her own rear end.

Since then, I've made sure to never communicate with her unless it is through email or text. There is no such thing as too much CYA with this woman. I've dotted my Is and crossed my Ts.

I've waited.

And today, the wicked Property Control Clerk was finally slain. She was not slain by a massive theft. She was not slain by the Department of Education catching her shredding legal documents. She was not slain by a carefully laid trap from yours truly.

She was slain by a $120 pocket projector that was purchased three years ago, found wanting, and left forgotten in a desk.

The director suddenly remembered its existence yesterday and set out to find it, but it did not turn up. He tried to blame me, but I am blameless. He tried to blame the previous owner, but he is blameless.

So he blamed the only one truly at fault: The Property Control Clerk. It was a small screw-up. It wasn't labelled as it was under the inventory price threshold. It was left in a drawer and accidentally discarded when they replaced the furniture in that office last year (as best as I can figure).

What happened, and what the director is infamous for, is that suddenly inventory was on his radar. He went over to her office and demanded to see every scrap of paperwork she has on all of our inventory, suddenly discovering that there is no paperwork. There has been no paperwork for years. Just hastily scribbled notes on various notepads and post-its. He came to ask me what my experience with her was, to which I responded that it has been resoundingly negative and that I fear for my job every time inventory is involved as there are documented cases of her altering paperwork and destroying documents to protect herself.

I reminded him of the botched disposal that almost cost us half a million in penalties.

She did not last the day.

I feel pity and relief.

I remember the previous interaction, and I feel triumph. but it isn't mine so I'm letting it go. Glad that one doom is off your shoulders.

Johnny Aztec
Jan 30, 2005

by Hand Knit
Allow yourself a few fist pumps, and some private mockery. You've earned this, friend.









as an aside, "Private Mockery" would make for a good rum label

Arquinsiel
Jun 1, 2006

"There is no such thing as society. There are individual men and women, and there are families. And no government can do anything except through people, and people must look to themselves first."

God Bless Margaret Thatcher
God Bless England
RIP My Iron Lady

larchesdanrew posted:

She was not slain by a carefully laid trap from yours truly.
You sure about that?

larchesdanrew posted:

He came to ask me what my experience with her was, to which I responded that it has been resoundingly negative and that I fear for my job every time inventory is involved as there are documented cases of her altering paperwork and destroying documents to protect herself.
:commissar:

Pyroclastic
Jan 4, 2010

Renegret posted:

That's what I was thinking. Every time I had a computer running without an hsf, it just shut down within minutes.

When I've had the HSF fall off in other computers in my school, that's what happened. They'd be working slowly, then just shut off. That'd usually clue me in pretty quickly that it's a thermal problem.
I'm guessing since I wasn't trying to actually do anything but wait for whatever the computer was trying to do to finish, and it was only on for less than 10 minutes, and there's a decent amount of airflow in the case, it wasn't quite enough to reach the shutdown threshold. Still, running at 3.2ghz processor at 7% its rated speed... Never experienced a Pentium II 266 running Windows 10!

I decided I couldn't be bothered cleaning Chromebooks today and instead helped put together a lovely filing cabinet and disassembled a large laminator to unjam it. Did deliver all those iPads, though.

Re: Inventory stuff. Our Accounting person is responsible for inventories, and she's pretty paranoid about it. Spent weeks trying to find three Toshiba docking stations, then since they were under the $200 threshold, she relented and just removed them all from our inventory. Our original inventory was hackneyed when I started there, and lots of stuff wasn't inventoried at all, and what was, was done strangely. I took it upon myself to start my own excel inventory, and a year later the district finally decided to get asset tracking software.

We're about to move to our third inventory system in three years. Samanage (it literally duplicated our entire inventory last year somehow), then Follett Destiny (it's a great library checkout system, but the tech inventory is clearly tacked on), and now we're moving to Asset Panda.

Pyroclastic fucked around with this message at 02:23 on Aug 23, 2019

SyNack Sassimov
May 4, 2006

Let the robot win.
            --Captain James T. Vader


A few hundred tickets came in two days ago...to the lobby of the theater our worst client uses for their shows because they're a theater nonprofit. And then those tickets got torn because the ticket scanners (which are jumped up Android phones with a barcode scanner glued on and cost $1k each) wouldn't connect to the WiFi we just upgraded so they couldn't scan tickets.

More specifically, we just put in Meraki MR45s to replace the MR42s that were there, and it does seem like the 45s are much pickier about what devices they allow to connect (or I guess the devices have a problem negotiating with them). From a quick Google I see some references on Reddit to old Intel chipsets having problems but not a lot else.

I eventually got the scanners to work yesterday by setting the SSID to "full compatibility" in the Meraki controls, which amusingly states "compatibility back to pre-1999 devices", but we're also seeing issues with their laptops because of course they have incredibly ancient machines trying to connect.

Anyone else seen connection problems with MR45s, and/or have any suggestions beyond upgrading drivers? I'd like to think Meraki will release a firmware update at some point to help fix these issues but my experience with them is that firmware updates are not released quickly. In the meantime apparently the scanners aren't connecting at the same distances they used to (possibly switched to 5 GHz for some reason, and the theater is all concrete so that's fun) so I guess I'm heading over to the theater today again - normally one of our techs would but I live very close to the place - to gently caress around with Android 7 phones, thanks Zebra.

Thanks Ants
May 21, 2004

#essereFerrari


Aerohive used to have very specific SSID profiles if you needed old handheld inventory devices to work - stuff like turning off any 'advanced' features like power saving, letting 802.11b devices connect etc.

Sheep
Jul 24, 2003
Our older Meraki stuff seems to work but anything purchased post-2016 seems to have issues, either sporadically or just all the time. None of our brand new HP PCs and no iPhones can connect to any of our SSIDs in one of our new facilities, but my Dell computer and Android phone connect just fine. I'm far enough removed from dealing with that stuff that I don't care any more but yeah for anything where you have to deliver stable wireless, Meraki is not going to fit the bill unless you have total control over the devices connecting and can do a shitload of proof of concept testing before deployment. Dealing with Meraki issues isn't a constant thing but it happens often enough that I'm bitter about the whole experience.

We actually have a bunch of lovely Linksys wireless APs running in production right now because of this.

Edit: AFAIK there is no fix other than to swap hardware and pray if you've already made the obvious compatibility-related changes on the dashboard and that didn't resolve it. We did have very minor improvements turning off band steering and splitting 2Ghz and 5Ghz to separate SSIDs but that's the only not-totally-obvious suggestion I can make.

Sheep fucked around with this message at 20:59 on Aug 23, 2019

SyNack Sassimov
May 4, 2006

Let the robot win.
            --Captain James T. Vader


Hrm interesting. We haven't really had issues with Meraki wireless before the MR45s, and even with the 45s it seems like it's old stuff having issues, not newer (<2 years old) equipment.

I ended up fully "updating" the ticket scanners to their most recent patch level, which was July 2019 except they're still running Android 7
But whatever, that seemed to help the disconnection issues. Now of course there's the problem that we need another AP in the building because there's a solid 2 feet of concrete between the first and second floors (with giant steel beams embedded in the concrete). But of course the same construction also makes it difficult to run new wires.

Anyway it's now 100% beer o clock so yeah no fucks given until Monday.

Maigius
Jun 29, 2013


A ticket did not come in, instead a giant pile of printouts appeared on my desk. Changes for one of the web pages, one that notoriously has had errors when it's been updated.

Thanks Ants
May 21, 2004

#essereFerrari



It really sounds like you need a proper wireless survey and then to design your network around the results of that and the radio specifications of the terminals you're connecting to it.

Space Skeleton
Sep 28, 2004

Maigius posted:

A ticket did not come in, instead a giant pile of printouts appeared on my desk. Changes for one of the web pages, one that notoriously has had errors when it's been updated.

You get printouts? Not a photocopy of scrawlings made on a legal pad?

SyNack Sassimov
May 4, 2006

Let the robot win.
            --Captain James T. Vader


Thanks Ants posted:

It really sounds like you need a proper wireless survey and then to design your network around the results of that and the radio specifications of the terminals you're connecting to it.

Yes, I know, and in fact will be conducting such a survey real soon now. The annoyance here is that there was an existing MR42 AP that we replaced (same exact spot), and that worked fine with everything including these ridiculous scanner devices, and I was inquiring if anyone had had specific experience with the relatively new MR45s as to whether they were worse than the 42s. Specifically at listening, since the transmit signal level is low but not non-existent and my supposition is that the extremely weak response from the scanners was picked up well enough by the 42 but isn't by the 45.

But frankly a wireless survey won't really help much in that it will almost certainly just confirm my suspicions of just how low the signal level is (I mean, I can get an idea using my phone's signal analyzer, the location with problems is around -78 dB) and will tell me I need to put an AP in on the second floor where the return signal won't be blocked by two feet of concrete and steel beams. Which, I already know. And which I can't do jack poo poo about because it's not feasible from a monetary standpoint to run the wires because of the lack of access through the building (and I really would rather not mesh APs).

Bigass Moth
Mar 6, 2004

I joined the #RXT REVOLUTION.
:boom:
he knows...
I have 1500 alert emails. A guy on my team was doing some reboots this weekend. Excited to see if it went well!

my cat is norris
Mar 11, 2010

#onecallcat

If you'd like to read the post-mortem from my company's ransomware incident, shoot me a PM and I'll send the link your way. I don't think there are any problems with me posting it right here, but I'm playing it safe! :shrug:

Wizard of the Deep
Sep 25, 2005

Another productive workday
If the postmortem is publicly posted, the only problem I can imagine with sharing the link is we'll know where you work.

Which, fair.

Laranzu
Jan 18, 2002

Zero VGS posted:

Seems to be something having to do with OneDrive; deleting a folder in the O365 web portal will cause OneDrive to delete the folder on my end. The few folders I have that aren't synced to OneDrive can be deleted normally. At least now I can get rid of the huge iCloud backup I took for a user.

I had this happen on a fresh Win10 install when my time zone synchronized and put the system clock way ahead of where it was on initial install. I'd whack delete on local folders and nothing would happen as the one drive integration freaked out over timestamps from the future.

Did anything happen to the system time by chance?

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


NOC calls me at 3am: Hey, <teamA> needs a few servers restarted, can you assist with that?
Me: Sure, that's what I'm here for. Join me to the Teams room
Me in Teams room: Ok, so who are we hitting with the hammer tonight?
Team A Person: So, we're getting a lot of connections from <Environment>
Me: ...
Team A Person: ...
Me: And we believe rebooting some of the <TeamA> servers will mitigate/resolve this?
Team A Person: Ok
Me: That was a question.
Team A Person: Oh, yeah. That's what we've done in the past to resolve this
Me: Ok, which servers are we looking to reboot
Team A Person: All of the prod hosts for <TeamB>'s environment
Me: So... yeah. I'm not gonna go rebooting <TeamB>'s servers at 4am at the behest of another team. Let's think about engaging that team first, eh?

But yeah, chief, I'm gonna just go hitting someone else's servers with a hammer. Not to mention, it's 99.999% more likely that a reboot isn't necessary, just cycling a service or 5.

BlankSystemDaemon
Mar 13, 2009



Rebooting demonstrably doesn't fix anything, it only masks the problem.

System-level tracing with minimal performance hit to do in-place troubleshooting, or even just capturing the whole operation to do post-mortem analysis.
DTrace is capable of doing this, and on top of only taking ~2% of cputime it only does it for the code that's being actively traced - and best of all, it's available on all platforms: Windows, Linux, macOS, FreeBSD and NetBSD, Illumos, Solaris.

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


D. Ebdrup posted:

Rebooting demonstrably doesn't fix anything, it only masks the problem.

System-level tracing with minimal performance hit to do in-place troubleshooting, or even just capturing the whole operation to do post-mortem analysis.
DTrace is capable of doing this, and on top of only taking ~2% of cputime it only does it for the code that's being actively traced - and best of all, it's available on all platforms: Windows, Linux, macOS, FreeBSD and NetBSD, Illumos, Solaris.

Sure, but I leave that to the app developers to handle after the fact. I'm not in the business of debugging someone else's code, especially at 4:30am now.

And yeah. Rebooting, in the *nix world in particular, is a last resort to get poo poo moving if all else fails.

RFC2324
Jun 7, 2012

http 418

AlexDeGruven posted:

Sure, but I leave that to the app developers to handle after the fact. I'm not in the business of debugging someone else's code, especially at 4:30am now.

And yeah. Rebooting, in the *nix world in particular, is a last resort to get poo poo moving if all else fails.

reboot a N*X server means a hardware or kernel issue.

or lovely java is leaking uncontrollably again

BlankSystemDaemon
Mar 13, 2009



RFC2324 posted:

reboot a N*X server means a hardware or kernel issue.

or lovely java is leaking uncontrollably again
Java belongs in a jail, with resource control via rctl. :colbert:

RFC2324
Jun 7, 2012

http 418

D. Ebdrup posted:

Java belongs in a jail, with resource control via rctl. :colbert:

I don't get to engineer these lovely platforms, I just get to keep them alive

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


RFC2324 posted:

reboot a N*X server means a hardware or kernel issue.

or lovely java is leaking uncontrollably again

Java is never not leaking or lovely.

RFC2324 posted:

I don't get to engineer these lovely platforms, I just get to keep them alive

Samesies :smith:

BlankSystemDaemon
Mar 13, 2009



RFC2324 posted:

I don't get to engineer these lovely platforms, I just get to keep them alive
Then stuff the developers in jail too? :v:

PremiumSupport
Aug 17, 2015

AlexDeGruven posted:

Java is never not leaking or lovely.


Tell me about it.

I play a lot of Minecraft in my off-time, and because I use mods, I'm stuck on the Java version. Every week or so I have to reboot my PC because over half my memory is "in use" due to Java not releasing it properly.

klosterdev
Oct 10, 2006

Na na na na na na na na Batman!

D. Ebdrup posted:

Rebooting demonstrably doesn't fix anything, it only masks the problem.

The moment it goes from "system hiccup" to "the problem happened again" looking into the cause should be mandatory. If it happens a second time, it's going to happen a third.

BlankSystemDaemon
Mar 13, 2009



klosterdev posted:

The moment it goes from "system hiccup" to "the problem happened again" looking into the cause should be mandatory. If it happens a second time, it's going to happen a third.
Yep. It should be so commonplace I'm tempted to verb it and call it rootcausing, just so it's easier to include in every description of troubleshooting.

Dirt Road Junglist
Oct 8, 2010

We will be cruel
And through our cruelty
They will know who we are
Co-worker who recently transitioned to a different team: Did we ever post-mortem the most recent spaghetti incident?
Me: Post-mortem? POST-MORTEM? Do we EVER post-mortem anything?
Co-worker: So that's a no.
Me: I mean, I asked for one, but you know no one in HQ can hear voices that are higher pitched than their own.

Meanwhile, ridiculously short handed because of recent staff transitions and several people burning out and quitting, and I've got a bunch of net new sites crawling up my rear end about getting imaging online. This is fairly trivial on my end, in terms of effort, but the problem is trying to unfuck an automation script written by a former teammate who had never written code before while also getting the prod environments online. (He wrote a function...wherein all the variables get declared. Send help, I'm officially dead.) Again, trivial effort, but time consuming, so I need everyone to just get out of my loving way and let me do it. Instead, the most senior member of the senior team says, "We can dogpile this and get it done faster!"

Like...

My dude.

I just told you that's the opposite of helpful.

Plz stop.

monsterzero
May 12, 2002
-=TOPGUN=-
Boys who love airplanes :respek: Boys who love boys
Lipstick Apathy
Wheeeeee. It's busy season again and our dining operation's credit card processor has been up and down since yesterday. Can't even access their status page and their support line has been busy for an hour. Thank god kitchen people will just nod when you tell them "<processor>'s poo poo is hosed" and not expect me to fix a major company in another state's systems with magic like some other departments.

Gerdalti
May 24, 2003

SPOON!

monsterzero posted:

Wheeeeee. It's busy season again and our dining operation's credit card processor has been up and down since yesterday. Can't even access their status page and their support line has been busy for an hour. Thank god kitchen people will just nod when you tell them "<processor>'s poo poo is hosed" and not expect me to fix a major company in another state's systems with magic like some other departments.

I'm sure this is why I had to go to three different liquor stores to buy whiskey last night. Credit card machine down in both the first two.

porktree
Mar 23, 2002

You just fucked with the wrong Mexican.

RFC2324 posted:

reboot a N*X server means a hardware or kernel issue.

or lovely java is leaking uncontrollably again

Or the find command forks itself to eternity and fills up 256gb of memory and 512gb of swap.

stevewm
May 10, 2005

monsterzero posted:

Wheeeeee. It's busy season again and our dining operation's credit card processor has been up and down since yesterday. Can't even access their status page and their support line has been busy for an hour. Thank god kitchen people will just nod when you tell them "<processor>'s poo poo is hosed" and not expect me to fix a major company in another state's systems with magic like some other departments.

I work for a chain of retail stores.. Its always fun when Verifone's gateway service goes down (Verifone Payware Connect) Last time it happened it was down for nearly an hour, in the middle of the day.

And it never fails there are always several transactions in progress when this happens. We end up with the problem where our POS software submitted a transaction, but never received a response back. So we have no idea if a customer's card was actually charged or not. Sometimes we find they where double charged. Always fun cleaning those up the next day.

Mr. Fix It
Oct 26, 2000

💀ayyy💀


mood:

https://twitter.com/mrfixit42/status/1166874014540677120

mllaneza
Apr 28, 2007

Veteran, Bermuda Triangle Expeditionary Force, 1993-1952




D. Ebdrup posted:

Java belongs in a jail, with resource control via rctl rm. :colbert:

BlankSystemDaemon
Mar 13, 2009



porktree posted:

Or the find command forks itself to eternity and fills up 256gb of memory and 512gb of swap.
And again I want to mention jails and rctl, because those resource limits can apply to not just processes, users, or login classes but also jails. It can even warn you long before anything gets that big.
If you're doing a proper job of it, FreeBSD need only be rebooted for kernel upgrades.
Although I do wish someone would finish the kload project (like ksplice for RHEL/Oracle Linux, as FreeBSD has stable ABI/KBI by design), since that'd make it so that you would only need to reboot every 5 years when a major supported branch goes EOL.

mllaneza posted:

A lack of :words: about killing Java
Once upon a time I had the distinct pleasure of retiring a Java implementation of some server side software that should never have been made in Java, after the company had finally hired a developer to write it anew (it got rewritten in Erlang to make it easy to distribute load across many servers, and still runs today although neither of us work there anymore).
I don't think I've ever felt so good about decommissioning a server and bringing it up again for the new setup, but it was so cathartic to finally get rid of it after fighting with it for over 3 years.

AlexDeGruven
Jun 29, 2007

Watch me pull my dongle out of this tiny box


porktree posted:

Or the find command forks itself to eternity and fills up 256gb of memory and 512gb of swap.

As an AIX person, the idea of that much swap gives me the shivers.

I run 16gb on my 256gb boxes. If AIX is paging, something is wrong.

suuma
Apr 2, 2009
Our hosting provider of 6ish years is closing a datacenter this Sunday and I've spent the last 9 months migrating what is probably 1/3rd of our customer base to Azure or another datacenter with the same provider.

There is only one fucker left and they are insisting we let them continue using it in-place until noon Saturday.

I want to die

suuma fucked around with this message at 02:55 on Aug 30, 2019

Adbot
ADBOT LOVES YOU

kensei
Dec 27, 2007

He has come home, where he belongs. The Ancient Mariner returns to lead his first team to glory, forever and ever. Amen!


suuma posted:

Our hosting provider of 6ish years is closing a datacenter this Sunday and I've spent the last 9 months migrating what is probably 1/3rd of our customer base to Azure or another datacenter with the same provider.

There is only one fucker left and they are insisting we let them continue using it in-place until noon Saturday.

I want to die

Oh no they shut it off at Noon on Friday, I migrated you by 4 PM you're welcome!

  • 1
  • 2
  • 3
  • 4
  • 5