|
Negromancer posted:The best thing to take away from this is gently caress iptables. As long as you have your security groups setup correctly, you really should not need to run a software firewall. The future is now!
|
# ¿ Nov 14, 2013 21:27 |
|
|
# ¿ Apr 26, 2024 04:30 |
|
I had to explain to some super-aggressive DBAs in a meeting that while I supported their log-scrape witch hunt to discover why oracle poo poo the bed over the weekend, they simply could not forward me 5 pages of /var/log/messages starting 30 minutes before the incident, with dozens of lines they thought "were concerning" bolded and underlined. Lines that included our puppet runs updating various unrelated files on the system and a ntpd restart 25 minutes before the crash. They demanded I explain line by line what each item meant and also provide documentation on everything puppet does so they could examine it for "any possible conflicts". I told them that puppet runs enterprise wide and has been running for over 2 years without issue. I then, sadly, made a cardinal sin: I told them that oracle was simply another application, it was not special, and that while it may have more specific OS requirements like kernel parameters, in general when an app crashes, you need to look inside the app not outside of it. Furthermore, without a foundation in operating systems, I was unable to explain to them the intricacies of common system functions, much like I'm not qualified to evaluate database architecture decisions. WOW, were they not happy. I could have absolutely been more helpful had they not been desperately trying to point the finger at my group for the outage. The idea that someone has access and control over their beloved servers has made them resist automation on many servers for months and months. "Why no you can't just stamp out an oracle server! It is NOT just another app! It requires endless customization and tweaking and DBAs logging in to keep running! Oracle servers are like beloved pets that require care and attention!" Bullshit. Get with the program. You are not special. Welcome to the cloud, motherfuckers. We shoot servers in the head and bring up identical ones every single day. Oracle is no exception. Bhodi fucked around with this message at 07:23 on Nov 23, 2013 |
# ¿ Nov 23, 2013 06:52 |
|
dennyk posted:Also, the DBAs wanting to know what puppet is doing on their system is perfectly reasonable. They don't care how puppet works on a code level, they just want to know what stuff it's managing/changing/adding on their system, which should be easy to figure out and document for them. When they shifted a fact-finding meeting into an impromptu automation grill session, that was pretty much my limit. I could have handled it better, since honestly the hostility doesn't help the company at all, but without personnel changes or someone above us putting his foot down, nothing's going to be resolved and we'll just continue to go back and forth. Neither my boss or the DBA team is willing to budge, and the capacity problems are still far enough out that management can ignore the issue... for now.
|
# ¿ Nov 23, 2013 19:19 |
|
There are definitely support benefits beyond being able to point a finger at someone. With Red Hat, For example, we used an extremely long resolv.conf search entry, up to the limit of the RFC (256 chars), during kickstarting through PXE. However RHEL 5.3 (?) didn't support the length. We opened a ticket and they actually patched the issue for the next release and then backported the patch into a custom PXE image and forwarded it for us to use. I'd have a hell of a time trying to replicate the fix. I could have probably managed it, but it'd have probably taken me days to get it right. There are downsides, too, though. Like the way they Red Hat forces servers to connect to RHN (external internet access) for licensing purposes. You can't simply download security patches for servers unless you buy a $XXk "Satellite Server" which will internally mirror the repositories, or "Proxy Server" which is basically a proxy passthrough. There is no quick, convenient rsync solution to provide local, easily updated repo mirrors for your systems like there is with CentOS, Debian, and basically every other linux distro. Unless, of course, you hack together some awful solution like one "patch" server that downloads and installs every single RPM available while using the "save the the RPM" flag, then shoves all those RPMs into some repo. (Don't do this!) Edit: for the record, Satellite Server AKA spacewalk is a barely-functioning turd of a tomcat application that was created in-house and some bright guy realized they could sell to customers for lots of money because some companies don't actually want every server to be able to have internet connectivity but still want patches and updates. AVOID! I must have submitted a dozen bugs I found in the first 3 months we used it, to which the reply was "This will be fixed in the next version, there is no workaround, sorry". Hate. Hate. Hate. Bhodi fucked around with this message at 23:26 on Jan 23, 2014 |
# ¿ Jan 23, 2014 23:18 |
|
dogstile posted:So where's a good start for SQL that isn't loving oracle? I tried oracle before and it was loving horrible. For a traditional relational DB, considering Oracle owns mysql now, postgres is your most common other option.
|
# ¿ Jan 23, 2014 23:53 |
|
Well, the last time I touched it was almost 3 years ago, at my last job, but I was pretty sure RH disallowed reposync of the updates repo back then due to licensing issues (as our rep explained it to us). We looked REALLY HARD for alternatives. I'd like to say Sat Server was so bad RH desperately looked for something, anything else. I haven't used katello yet, but I'm just glad SS is on long term support. Edit: 3 years, not 2. Yikes. Time is passing, passing... Edit2: just remembered our issue with reposync at the time, it only allows (allowed?) you to sync repos of the same arch and OS type as the server it's running. Not good for a heterogeneous environment where we literally just wanted a local yum mirror of all OS rpms, and own repos, to be updated with createrepo. Plus random 3rd party stuff like rsyncing vmware tools RPMs. Bhodi fucked around with this message at 00:57 on Jan 24, 2014 |
# ¿ Jan 24, 2014 00:44 |
|
Funny, because net connect is heads and shoulders above the next best offering (Cisco any connect, I guess), at least viewed from the user side.
|
# ¿ Jan 28, 2014 05:17 |
|
Don't worry, she's just going to be overruled, told her timeline is unacceptable, and to "make it fit, that's your job".
|
# ¿ Jan 29, 2014 18:23 |
|
Devops from the other side is "Encourage dev to release and test a stable product by waking them up at 4am when their lovely application goes down, rather than have them just throw it over the fence to ops with a 'good luck!' every single release" Also, devops produces people with a basic understanding of the OSI model and what happens outside their little container. You know, useful stuff like cpu/memory usage, network bandwidth, performance, monitoring, all that stuff baked in at the dev level rather than slapped on at the ops level. Maybe even the ability to understand why they don't just slap together 3 outer joins or why they can't just throw more memory at their app. Edit: Which is awesome. In theory. I've never worked at a place it happens, but I can dream! Bhodi fucked around with this message at 21:31 on Feb 20, 2014 |
# ¿ Feb 20, 2014 21:27 |
|
MLP, pretty sure.
|
# ¿ Feb 22, 2014 09:44 |
|
|
# ¿ Apr 26, 2024 04:30 |
|
Che Delilas posted:Pretty much. What sticks out in my memory the most is just how FAST it happens. The second you step out the door, BAM, flop sweat over every inch of you, and I'm not exaggerating for effect here - it was literally one second. I'd much rather have the occasional aggressive dryness that we get up here in winters, you can put lotion on if you need to. Friends don't let friends live down south.
|
# ¿ Mar 1, 2015 17:03 |