Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
spanko
Apr 7, 2004
winnar

mayodreams posted:

Our O365 is down. MS posted an incident:


Literally as I walked out the door for a 4 day weekend. :negative:

Ours is down too.

Adbot
ADBOT LOVES YOU

Inspector_666
Oct 7, 2003

benny with the good hair

spanko posted:

Ours is down too.

At least three clients are having issues as of end of business today for us too. Here's to not being on call!

Zero VGS
Aug 16, 2002
ASK ME ABOUT HOW HUMAN LIVES THAT MADE VIDEO GAME CONTROLLERS ARE WORTH MORE
Lipstick Apathy
Oh okay I was wondering why the enterprise Windows thread hadn't mentioned it, we're all bitching in here.

J
Jun 10, 2001

How do you folks on O365 deal with management and C levels freaking out when it goes down? We are in the process of trying to make a case to get switched over to O365, but for the life of me I can't figure out a response to "What do you mean you have to wait for microsoft to fix it? Why can't YOU fix it? You fixed it before on the old system! Get on the phone and call somebody and get it fixed!"

JewKiller 3000
Nov 28, 2006

by Lowtax
WELCOME TO THE CLOUD, MOTHERFUCKER.

Down for our customers too.

CLAM DOWN
Feb 13, 2007




Zero VGS posted:

Oh okay I was wondering why the enterprise Windows thread hadn't mentioned it, we're all bitching in here.

We're too busy crying about the end of server 2003

Gucci Loafers
May 20, 2006

Ask yourself, do you really want to talk to pair of really nice gaudy shoes?


How long and is it still down?

Super-NintendoUser
Jan 16, 2004

COWABUNGERDER COMPADRES
Soiled Meat

J posted:

How do you folks on O365 deal with management and C levels freaking out when it goes down? We are in the process of trying to make a case to get switched over to O365, but for the life of me I can't figure out a response to "What do you mean you have to wait for microsoft to fix it? Why can't YOU fix it? You fixed it before on the old system! Get on the phone and call somebody and get it fixed!"

I've had good success showing my customers the cost of the service vs the cost of paying for me to rebuild the database over night, as well as the cost of servers, licensing and cals. Hosted exchange comes out clearly ahead.

FRINGE
May 23, 2003
title stolen for lf posting

CLAM DOWN posted:

You should feel bad for holding any kind of generalized, stereotypical, or ignorant opinion.
Except for HR trolls. In which case its a generalized, stereotypical, and informed opinion.

Daylen Drazzi
Mar 10, 2007

Why do I root for Notre Dame? Because I like pain, and disappointment, and anguish. Notre Dame Football has destroyed more dreams than the Irish Potato Famine, and that is the kind of suffering I can get behind.
Friend of mine gave me a verbal referral to his company, and after sending in my resume they immediately wanted an in-person interview. I was a little surprised at how fast they were moving, especially as there wasn't even a job posting. I thought maybe I had missed it or something and asked my friend about it, and his response was "they're making one for you right now". The position is for a system support analyst, but I'm not sure that the position will be something I'm interested in. Plus it doesn't sound like the pay will be anywhere near to what I'm making now. I suppose I'll just have to go to the interview and see where things stand.

mayodreams
Jul 4, 2003


Hello darkness,
my old friend
Our Tenancy is back online as of about 20 minutes ago.

We had just over 3 hours of outage, which is our first in the year since migration. I told our C-Levels that 3 hours of downtime is the minimum required for scheduled maintenance and is insignificant relative to the costs of maintaining on-prem.

Microsoft posted:

Current Status: Engineers have reverted the update in all regions and affected customers should be experiencing service restoration. Service teams are validating that the configuration change has been applied correctly and that service health is recovering as expected.

User Experience: Affected users are unable to send or receive email via the Exchange Online service when using multiple protocols including Outlook, Outlook Web App (OWA), Exchange ActiveSync (EAS), and Exchange Web Services (EWS). Some users may also be unable to access the Exchange Online service.

Customer Impact: A higher than average number of customers are reporting this issue. Analysis indicates that customers will likely have some users experiencing this issue.

Incident Start Time: Wednesday, July 15, 2015, at 9:25 PM UTC

Preliminary Root Cause: Engineers have determined that the root cause of this issue is a recent update that is causing higher-than-normal CPU usage on Active Directory infrastructure that handles authentication within and to the service.

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer
where do you guys find the line of demarcation is for o365 being cheaper than running your own exchange? For our ~750 users, I cannot imagine it being less expensive to move it to the ~*cloud*~.

mayodreams
Jul 4, 2003


Hello darkness,
my old friend
My company is very cheap, and the storage requirements alone dwarf the cost of the just email plan (EoL) at around $4/mo per user. Even at 1000 users, $48k a year for no bullshit email is amazing compared to just getting Exchange, CALs, Windows Server instances, storage, backup, rack space, heat, power, backup, etc. Not to mention the man hours we don't waste maintaining that infrastructure. I have 2 Server 2008 R2 (soon to be 2012 R2) VM's that run ADFS and AAD Connect and that is extent of our footprint for Exchange, Skype for Biznass, and our future Azure instances.

The idiots running the company a few years ago dropped $100k on Exchange 2010 and CALs that we literally never used because they poo poo their pants at the cost of backing it with the amount of storage necessary to grow our environment. We are lucky to be buying basically branded supermicro boxes that run Nexenta for all of our SAN needs because the other players are way out of the range that the leadership wants to spend.

Microsoft is aggressively pricing everything so you do everything via subscription, and its working. Hell, I bought a Office 365 Home sub and can't be happier with it. Gave me Office 2016 for Mac on day one. :smug:

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
It's often worth it just to have everything about it be somebody else's problem politically rather than technically. That way, you don't waste two weeks of engineer-hours when the head of HR makes it her pet project to take up two hours of an engineer's time every time she gets a spam email.

(I had this happen to my team at a prior job.)

Inspector_666
Oct 7, 2003

benny with the good hair

J posted:

How do you folks on O365 deal with management and C levels freaking out when it goes down? We are in the process of trying to make a case to get switched over to O365, but for the life of me I can't figure out a response to "What do you mean you have to wait for microsoft to fix it? Why can't YOU fix it? You fixed it before on the old system! Get on the phone and call somebody and get it fixed!"

We have a few clients on O365, and while it's kind of a no-brainer for non-profits (holy balls it is cheap for them,) it really is miserable when anything goes wrong. The large majority of our clients are on another hosted Exchange provider who is much smaller and has a "flatter" support structure, so we can usually get a competent person on the phone quick, or at least get to a supervisor quickly.

If anything goes wrong on O365 it's at least several days of troubleshooting with people who have a wildly variable grasp of both technology and English. My co-worker is on week 6 or 7 of troubleshooting and issue and the MS people only just yesterday said "OK why don't we try turning logging on?"

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Inspector_666 posted:

If anything goes wrong on O365 it's at least several days of troubleshooting with people who have a wildly variable grasp of both technology and English. My co-worker is on week 6 or 7 of troubleshooting and issue and the MS people only just yesterday said "OK why don't we try turning logging on?"
Are you signed up directly through Microsoft, or through a VAR? One of the benefits of going through an MS partner is that it's usually easy to find someone to grease the wheels and run the conference calls when the poo poo hits the fan in situations like this.

Honestly, if your co-worker is 7 weeks into a high-priority support issue and you haven't at least had a conf call with your sales rep, you're all doing something really wrong.

Antioch
Apr 18, 2003
Man I wish we could get O365. Stupid financial industry "oh no we can't trust the cloud what if our data leaks? Mail quotas, the hell is that? Just buy another SAN, it'll be fine."

Inspector_666
Oct 7, 2003

benny with the good hair

Vulture Culture posted:

Are you signed up directly through Microsoft, or through a VAR? One of the benefits of going through an MS partner is that it's usually easy to find someone to grease the wheels and run the conference calls when the poo poo hits the fan in situations like this.

Honestly, if your co-worker is 7 weeks into a high-priority support issue and you haven't at least had a conf call with your sales rep, you're all doing something really wrong.

We're direct to MS I think.

And yes, I don't know why this hasn't been made into a big loving deal already, although the issue is really only an issue because the user is an old guy who doesn't like any change at all.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


So, how many of you are dealing with the clusterfuck that is TLS 1.0 and PCI?

All our execs in a panic right now because over half our paying traffic comes from browser/OS combos that don't support TLS 1.1 out of the box or at all. It doesn't have to be officially disabled until June of 2016, but still that seems like a short timeframe to essentially rid the web of anything lower than IE11/Win7. I know you can enable TLS 1.1 support on IE8,9,10 on Windows 7, but users aren't going to do that.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

bull3964 posted:

So, how many of you are dealing with the clusterfuck that is TLS 1.0 and PCI?

All our execs in a panic right now because over half our paying traffic comes from browser/OS combos that don't support TLS 1.1 out of the box or at all. It doesn't have to be officially disabled until June of 2016, but still that seems like a short timeframe to essentially rid the web of anything lower than IE11/Win7. I know you can enable TLS 1.1 support on IE8,9,10 on Windows 7, but users aren't going to do that.
It's like Flash -- once everyone else disables it too, you can be pretty sure that you're not going to have that problem anymore.

skipdogg
Nov 29, 2004
Resident SRT-4 Expert

adorai posted:

where do you guys find the line of demarcation is for o365 being cheaper than running your own exchange? For our ~750 users, I cannot imagine it being less expensive to move it to the ~*cloud*~.

You've got to take a mile high look at everything from a business perspective and not a strict IT perspective. We've been with Microsoft since the old BPOS days. I guess you could say we were 'early adopters' as we moved to the service in 2009.

When we made the decision to move to BPOS, the predecessor to O365, some of the key points that swayed our decision

- OpEx instead of CapEx. Businesses like free cash flow and being able to pay for services on a monthly basis instead of investing a massive amount of CapEX money into on premise solutions make the accountants happy. We were looking at about 100K a year in BPOS costs, vs about 500K in CapEx to fully build out a new Exchange 2007 environment that would last 3, maybe 4 years. Hosted solutions also scale as needed. When we acquired a company we didn't have to worry about our existing Exchange environment being able to handle the load. We just added more licenses to our plan, and our monthly bill went up correspondingly. Our current O365 spend is around 1 million a year right now, but we went with the E3 licenses instead of keeping Office licenses on our EA with SA.

- Take a look at all the costs involved. Exchange costs aren't just hardware and CAL's. You have to pay for anti-spam, backup solutions, and other associated costs. You have to mitigate on premise dependency issues. Our main DC is in San Jose, and rolling blackouts can happen in the summer. We're unable to get a generator due to permit issues, so we either had to host everything in a cage at our colo, or deal with the possibility a good portion of the company could lose email access. We have a cage at Equinix in San Jose. It is NOT cheap. I think our space runs in the 20K a month range.

Sophos Anti-Spam was running us north of 50K a year plus appliance costs. Backup software, tapes, and offsite storage was another good chunk of change. We ran Trend-Micro ScanMail on the boxes which was another licensing cost. All these costs went away with the hosted solution as they were included.

Anyway, any good sized businesses decision to move something to the cloud or not isn't strictly about money. Sometimes it's worth it to pay a little more to have a solution hosted for you from a business perspective. I understand most IT people want full control over their systems, but each situation should be evaluated as each one is different.

The company I work for prefers to use hosted solutions when possible. We find they make a lot of sense for our environment. We use quite a few SaaS services.

We moved Oracle from on premise to their hosted On Demand product
We moved our internal MS Project environment to the hosted Project Online service
We use Office 365 and all it's associated services
Helpdesk software was moved to a SaaS platform
several other departmental services moved to SaaS subscriptions.

No more CapEx for those services, and IT administration time has been reduced letting us focus on other areas. We also don't have to worry about one of the 3 datacenters we have going offline for any reason and killing a global service. We have 20 offices across the globe. There is never a moment in the day where employees are not working. The cloud/SaaS model isn't right for everyone, but it can make a lot of sense.

CLAM DOWN
Feb 13, 2007




skipdogg posted:

Anyway, any good sized businesses decision to move something to the cloud or not isn't strictly about money. Sometimes it's worth it to pay a little more to have a solution hosted for you from a business perspective. I understand most IT people want full control over their systems, but each situation should be evaluated as each one is different.

It's not just about cost or having control over systems, even though those are important ones. Something like O365 could be free for all the good it does us - all the O365 datacentres in North America are in the USA, which makes it a useless service for us unfortunately.

Gucci Loafers
May 20, 2006

Ask yourself, do you really want to talk to pair of really nice gaudy shoes?


CLAM DOWN posted:

Something like O365 could be free for all the good it does us - all the O365 datacentres in North America are in the USA, which makes it a useless service for us unfortunately.

Im pretty confident its Worldwide unless you somehow provisioned you tenant in the wrong geographical region.

Even then, it wouldn't and shouldn't be that slow.

CLAM DOWN
Feb 13, 2007




Tab8715 posted:

Im pretty confident its Worldwide unless you somehow provisioned you tenant in the wrong geographical region.

Even then, it wouldn't and shouldn't be that slow.

http://blogs.technet.com/b/uspartner_ts2team/archive/2013/03/29/where-is-my-office-365-data-located.aspx

Your primary datacentre is always in the region where your headquarters address is, and for non US customers your secondary datacentre might still be in the US. I get it's not an issue for a lot of people but it's an instant dealbreaker for some.

Cool PDFs about their locations: http://www.microsoft.com/online/legal/v2/?docid=25

Inspector_666
Oct 7, 2003

benny with the good hair

Vulture Culture posted:

Are you signed up directly through Microsoft, or through a VAR? One of the benefits of going through an MS partner is that it's usually easy to find someone to grease the wheels and run the conference calls when the poo poo hits the fan in situations like this.

Honestly, if your co-worker is 7 weeks into a high-priority support issue and you haven't at least had a conf call with your sales rep, you're all doing something really wrong.

Follow-up: we don't have a sales rep/account manager apparently.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Vulture Culture posted:

It's like Flash -- once everyone else disables it too, you can be pretty sure that you're not going to have that problem anymore.

It's not quite the same thing.

You can take a step on your end to ensure that you have a substitution in place for Flash. With SSL, if the client doesn't support the protocols you are using, no connection at all happens. That's it.

What complicates matters further is you don't have a level playing field.

PCI is a risk mitigation step. It is not law, it is not required to be applied equally to all parties. If Amazon decides they want to keep TLS 1.0 running past June 2016, they are large enough and have enough sway that Visa and Mastercard aren't going to say "Too bad, you are cut off."

That's where things get a bit hazy in this. Smaller to mid level companies cannot set their own terms with processors like the big guys can. There's uncertainty there. If a large entity like Amazon does the shift, then the crowd will move with it. If they decide they don't want to (and they are perfectly capable of doing that), there's going to be less of a push to sunset older browsers and OSs that don't support these security items.

In the end, Google, Facebook, and Amazon are all going to need to sunset TLS 1.0 to really force change and it's very likely that they will all keep it enabled past the deadline because they can.

beepsandboops
Jan 28, 2014
These sales emails are the worst:

quote:

Hi beepsandboops,

What is a cloud made of? A cloud is made of WATER. And what is our “cloud” made of? IT Professionals who WALK on water!

in a well actually
Jan 26, 2011

dude, you gotta end it on the rhyme

beepsandboops posted:

These sales emails are the worst:

Ask them to demo it.

kensei
Dec 27, 2007

He has come home, where he belongs. The Ancient Mariner returns to lead his first team to glory, forever and ever. Amen!


beepsandboops posted:

These sales emails are the worst:

Hey, pop back into IRC. Let's do lunch soon.

Xik
Mar 10, 2011

Dinosaur Gum

Daylen Drazzi posted:

The position is for a system support analyst, but I'm not sure that the position will be something I'm interested in. Plus it doesn't sound like the pay will be anywhere near to what I'm making now. I suppose I'll just have to go to the interview and see where things stand.

For what it's worth, that's my current job title and the role is literally level 1 help desk.

I would personally still go to the interview to find out more, especially if they are creating the role for you.

22 Eargesplitten
Oct 10, 2010



beepsandboops posted:

These sales emails are the worst:

I wouldn't want to make up a butt, even if it meant I could walk on water.

Dick Trauma
Nov 30, 2007

God damn it, you've got to be kind.
The brewery that was interested in building on land we own has stated that they've now approved spending money to continue exploring this option. It will be a long road because it's not some little microbrew, it's a giant, old company that asked if we could handle an astronomical amount of waste water. Something like 2 barrels of waste water per barrel shipped.

Zaepho
Oct 31, 2013

Dick Trauma posted:

Something like 2 barrels of waste water per barrel shipped.
That's actually a really good number. Micro's have been known to hit as high as 6BBL of water per BBL of bright beer. Very cool business to be tangentially involved in either way!

hihifellow
Jun 17, 2005

seriously where the fuck did this genre come from

Zaepho posted:

That's actually a really good number. Micro's have been known to hit as high as 6BBL of water per BBL of bright beer. Very cool business to be tangentially involved in either way!

Sometimes it isn't! Ask our 1099, who is currently working on starting a brewpub, and from the dirtbag neighbors who tried to sue him out of the property so they could have it as parking space for their drug den, the contractor that gave him a low estimate then almost tripled the cost when it came time to start work, and finally the city council that has no clue what it's doing and is making him jump through so many impossible hoops because all they know how to do is follow the strict letter of some very asinine regulations, he's at the point where he only keeps going because he's made it this far and it's something he's always wanted to do. It'll be very cool when it's done at least!

Dick Trauma
Nov 30, 2007

God damn it, you've got to be kind.
These guys make at least 80 million gallons of beer per year so I think they probably have the technical aspect mastered. :cheers:

Bigass Moth
Mar 6, 2004

I joined the #RXT REVOLUTION.
:boom:
he knows...
I don't know man I bet Busch light makes more than that.

siggy2021
Mar 8, 2010
Tonight is our big Internet cut over from our poo poo ISP and a 50 Meg line to a 100 and from a VPN between sites to an MPLS network.

Everything seemed great after we swapped cables. We were immediately able to phone conference in from each site, remote into our servers at HQ from each site, and hit everything else.

Then we lost connection to the AS/400. Then HQ dropped off the conference call. Then a storm took out power at one of the remote sites.

Looks like something caused DHCP to die at HQ when they switched over. It's going to be a long night.

Contingency
Jun 2, 2007

MURDERER

siggy2021 posted:

Tonight is our big Internet cut over from our poo poo ISP and a 50 Meg line to a 100 and from a VPN between sites to an MPLS network.

Everything seemed great after we swapped cables. We were immediately able to phone conference in from each site, remote into our servers at HQ from each site, and hit everything else.

Then we lost connection to the AS/400. Then HQ dropped off the conference call. Then a storm took out power at one of the remote sites.

Looks like something caused DHCP to die at HQ when they switched over. It's going to be a long night.

Is the DHCP server at HQ? If not, you may have IP helper set up and pointing to the old WAN IP or device.

siggy2021
Mar 8, 2010

Contingency posted:

Is the DHCP server at HQ? If not, you may have IP helper set up and pointing to the old WAN IP or device.

DHCP server is on the DC at HQ. We theorized that the old router had something in it's config to be a helper. We rooted through the old config files and found them. Unfortunately we have no control over our previous or current routers. The configs were supposed to be copied over, the new isp had the old ones. Someone hosed up.

They are trying to get our new cyber roam firewalls to take over that functionality right now. They've got about ten minutes left before we call it, revert back, and go home.

I'd honestly rather it just get done and be here until God knows when tonight over having to come to this shithole satellite branch late at night in a few more weeks and do it again.

Adbot
ADBOT LOVES YOU

GreenNight
Feb 19, 2006
Turning the light on the darkest places, you and I know we got to face this now. We got to face this now.

siggy2021 posted:

DHCP server is on the DC at HQ. We theorized that the old router had something in it's config to be a helper. We rooted through the old config files and found them. Unfortunately we have no control over our previous or current routers. The configs were supposed to be copied over, the new isp had the old ones. Someone hosed up.

They are trying to get our new cyber roam firewalls to take over that functionality right now. They've got about ten minutes left before we call it, revert back, and go home.

I'd honestly rather it just get done and be here until God knows when tonight over having to come to this shithole satellite branch late at night in a few more weeks and do it again.

You sound like my job. We have MPLS to a remote site but kept a VPN tunnel as backup.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply