Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
GrandMaster
Aug 15, 2004
laidback

Vanilla posted:

So I have a ton of perfmon stats from a certain server.

What tools do you use to analyse these? I know there's the windows Performance Monitor tool but i've found it a bit 'hard'.

Do you know of any third party tools for analysing permon outputs?

we use this:
http://pal.codeplex.com/

its brilliant, you feed it the log files in either binary or csv format, it churns the logs and spits out a html report, makes pretty graphs, highlights any issues and has explanations for what each counter means.

Adbot
ADBOT LOVES YOU

GrandMaster
Aug 15, 2004
laidback

are there many emc users here?
we just had a nightmare day at the office - emc engineer was in to install 3 new dae's to our cx3-40. first tray went in with no problems but when the second was plugged in, the bus faulted and took the entire array down. the "non-disruptive" upgrade pretty much brought down our entire call centre

after the emc engineering lab did all their research it looks like a dodgy lcc in the dae was the cause.

has anyone else seen anything like this happen before?

GrandMaster
Aug 15, 2004
laidback

paperchaseguy posted:

yes. Once a few years ago saw a bad lcc in a clariion when adding DAEs. Don't think it took down the whole array though.

just heard back from support, they will be replacing the cabling on the SPA side bus 0 as there were some other strange bus errors. it looks like SPA crashed, SPB didnt so i'm not sure why the luns didnt all trespass and stay online

GrandMaster
Aug 15, 2004
laidback

conntrack posted:

Anyone using datadomain? We got quoted a price that would buy us a petabyte of raw disk for the same price as a data domain box.

We could probably buy half a petabyte, compress it with standard gzip and come out paying less money.

Going back to tape and tape robots is starting to sound good again.......

yeah, we had a similar quote.. decided to go with a sun thumper instead, zfs inline dedupe is out in the next release of solaris

GrandMaster
Aug 15, 2004
laidback

Serfer posted:

He goes over there, and immediately ejects drive 1.

lol, dont failed drives normally have a red/amber light on them instead of green?


does anyone know of any utilities for merging emc navisphere analyzer logs?
every time i want to look at perf stats of the array over the last week or so, i have to download all the logs, merge them 2 at a time with naviseccli. then merge the merged files, then merge those merged files until i'm left with one big log.

its a massive time consuming pain in the rear end and i want something where i can just point it at the directory of logs and it can merge them all and spit one out for me. it could probably be scripted, but i'm not great with scripting.

GrandMaster
Aug 15, 2004
laidback

Vanilla posted:

(edit: then again not sure if the CX3 could have gone there)


It cant, unisphere came out with FLARE 30 and cx3 only goes up to FLARE 29
I believe you can run it off array though, will have to get around to doing that some day as our cx3's still have about 2 years of maintenance left

GrandMaster
Aug 15, 2004
laidback

I've had great experiences with the EMC techs that have come onsite to relocate arrays/add disk trays etc. They were nice friendly guys, knew their poo poo, kept everything neat and worked quickly. I suppose it varies from region to region though.

Phone support is a different story altogether

GrandMaster
Aug 15, 2004
laidback

Aren't EMC supposed to do these upgrades for you? For every CLARiiON/Celerra upgrade we have done, they have sent a tech out unless it was a low end AX or VNXe.

EMC also needed a whole bunch of change control done from their end to make sure they don't run into any compatibility issues with hosts or things like recoverpoint. I'm based in Australia so it might be different here..

GrandMaster
Aug 15, 2004
laidback

It's been a while since we bought storage, but we just got a quote for 9x 600GB 15K disks for an EMC NS120.. Just the disks, no DAE.
The quote was $21k AUD, we're being bent over right? It seems reaaaallllly expensive, I'm sure we bought a dual controller AX4-5i for about $25k around a year ago.

GrandMaster
Aug 15, 2004
laidback

Yeah that's exactly what I'm asking, like I said it's been a long time since we bought any storage so I'm totally out of touch with what enterprise stuff costs these days..

GrandMaster
Aug 15, 2004
laidback

1000101 posted:

Realistically speaking most end hosts would be fine with just 4gbps of storage bandwidth and even then few come close to the upper ceiling of that.

We still run a lot of 2G FC (not for much longer thankfully) and we still don't come close to maxing out. The biggest problem we have are the SFPs burning out because they are so old.

Then again, we are still on crappy old CX3 arrays which probably aren't even capable of pushing data out faster than that so YMMV.

GrandMaster
Aug 15, 2004
laidback

Misogynist posted:

Until you hit a firmware bug that trashes your filesystem and replicates the changes downwind.

Depends on your replication system.. We run EMC Recoverpoint, it journals all of the replication block changes and we can roll back to any point in time on the remote system.

GrandMaster
Aug 15, 2004
laidback

We've been looking at nimble boxes too, but I was surprised at how expensive they were considering it's full of lovely SATA. We are looking at ~100TB of storage, and it came in more expensive than Compellent, VNX5500 & FAS3250 boxes with similar capacity - the other boxes take up more space but I've got much more confidence around the performance since they all have truckloads of 15K SAS & SSD caching.

I'm concerned about how some of the workloads would perform on a Nimble like some of our OLTP/OLAP etc apps. I'm sure VMware/VDI would run pretty quick though.

GrandMaster
Aug 15, 2004
laidback

I can't be too specific, commercial in confidence and all that, but the first quote we got was over $400k, after some discussion we got it below $300k (AUD).

GrandMaster
Aug 15, 2004
laidback

Does anyone have experience with Compellent? We are considering making the move from EMC, but I'm a little hesitant as EMC is all I've ever known and we've had a pretty good experience from them overall. Cost is obviously the driving factor here, but since it will be much easier for us to migrate onto another EMC system and they have already been proven then they are still in the game.

I've seen a few gotchas in this thread about the VNX series, but not a lot around Compellent so I was wondering if anyone could share their good/bad experiences?
On paper they look pretty good.

We are running ~100TB, workloads are typically: Oracle OLTP, MSSQL OLTP & OLAP, ESX & vCloud Director for self service vm provisioning.

GrandMaster
Aug 15, 2004
laidback

By gotchas I mean catastrophic failures/not living up to performance expectations.. anything like that.

We only need block over FC, but I think EMC might have have beat compellent by a fraction, mainly based around the ease of migration. We have a very short timeframe to migrate everything to a remote datacenter, and recoverpoint will make my life much easier than double handling (migrate to loan swing box/replicate remotely/cut over). The only other thing I need to know is how well the emc snap integration works with commvault.. netapp and compellent kept saying that their integration was the best, not sure why they are different but i'll get more details from our CV account manager tomorrow.

I also LOVED the netapp oncommand balance app that we demoed, i'm going to try to pick that up too if it fits in the budget even though we aren't using netapp storage. Being able to easily trace performance stats from VM->Physical Host->FC ports->LUN->RAID group was pretty fantastic.

Thanks for the feedback too amishpurple

GrandMaster fucked around with this message at 13:18 on Dec 13, 2012

GrandMaster
Aug 15, 2004
laidback

skipdogg posted:

Looking for a good intro to SAN book. I know the absolute basics, but I guess I'm looking for more detail. I know what a LUN is, but what purpose does it have, why are they created. I know that iSCSI and Fibre Channel are connection protocols, but what makes them different and the advantages and disadvantages of each. Basically a good foundation book with some general best practices. It doesn't have to be vendor specific, but we're an EMC shop if it matters.


http://www.bltrading.com/pdf/featuredwp/emc.pdf
This was my bible when I was learning EMC. I just googled this one, but theres probably a newer version on powerlink if you are running something newer than the CX series.
It covers everything from RAID types & performance characteristics, workload types and why some might need different raid levels, clariion architecture, fc/iscsi fault tolerant networks & multipathing etc. Highly recommended reading.

GrandMaster
Aug 15, 2004
laidback

Stugazi posted:

EMC will give us a Networker license for really cheap.

If you value your sanity, you will give Networker a wide berth.. Awful piece of software.

GrandMaster
Aug 15, 2004
laidback

I'm trying to figure my way around an IBM V7000 array at the moment, can anyone tell me if they have a feature similar to EMC Storage Groups / IBM DS Host Groups?
If I have to configure each cluster node individually, ensuring my lun numbers are consistent then I'm going to punch this lovely array in the face

GrandMaster
Aug 15, 2004
laidback

IBM that is weak. Thanks for that, better get reading...

GrandMaster
Aug 15, 2004
laidback

Does anyone know if the VNX2 is released yet or have a release date? We are about to pull the trigger on a smallish VNX5300, but I'd say it's worth waiting a couple of weeks to get the new one..

GrandMaster
Aug 15, 2004
laidback

Dilbert As gently caress posted:

Is there any comparable competitor to the Cisco MDS-9148? Looking to hook up about 4 hosts to some DP 8Gb/s FC adapters to a VNX 5400, loaded with Flash/15k.

Do you really need 48 ports? with a VNX & 4 hosts you'll only be using 8 ports on each switch.
Theres the MDS-9124 or Brocade 300 which are kind of equivalent (8GB FC/24port).
I find the brocades a bit easier to work with (zoning etc), but Cisco's FC gear tends to be a bit cheaper.

GrandMaster
Aug 15, 2004
laidback

El_Matarife posted:

HORRIBLE VNX2 bug ETA 175619 https://support.emc.com/docu50194_V...e=en_US
SPA and SPB panic within minutes of each other, and their associated LUNs and DMs go offline. This problem occurs every 90-99 days in the following systems: VNX5200 VNX5400 VNX5600 VNX5800 VNX7600 This problem occurs in a VNX8000 system every 80 days.


Ahh awesome, heard about this last week from our EMC TAM but the patch wasn't out yet. Their temporary fix was to reboot one of the SPs to stagger the uptime so you don't lose both at the same time.

GrandMaster
Aug 15, 2004
laidback

Nitr0 posted:

How old of the VNX? We have an old mofucker and we couldn't do anything because EMC wanted 30k for monitoring tools and the dumps we could do were encrypted and could only be read by EMC.

thats only for performance stats (navi/unisphere analyzer) and yeah, it's stupidly expensive.
you can still send fault alerts via email without that, just configure under the system/monitoring section in unisphere.

GrandMaster
Aug 15, 2004
laidback

Vanilla posted:

In a box the size of a TV

Haha yes! EMC sent me a physical entitlement certificate for a classroom training course in a giant box, padded with bubble wrap. I then had to jump online and use the code on the certificate to register.
Ridiculous.

GrandMaster
Aug 15, 2004
laidback

Strife posted:

So, maybe a stupid question, maybe not, but should I have each host hba be in a zone with both 3PAR controllers, or should I make a separate zone for each host to each controller? Either way the host sees twice as many paths back to the storage as it did earlier.

Check with your array vendor.. EMC best practice is 1 HBA, 1 controller port per zone.
So as a minimum each host would have the following added to the zone set:

Fabric A
HBA0 -> SPA P0
HBA0 -> SPB P0

Fabric B
HBA 1 -> SPA P1
HBA 1 -> SPB P1

I'm not an FC expert, but i believe this reduces the impact of RSCN notifications for fabric changes, and also VNX/CX arrays crack the shits and spam error messages as their controller ports can either be an initiator or a target if they are using mirrorview/sancopy and they keep trying to log into each other.

GrandMaster
Aug 15, 2004
laidback

Do we have any 3PAR nerds here? I'm coming from EMC, and trying to make sense of the 3PAR LUN presentation - it looks really weird.

I would expect each host to have 4 paths to each LUN based on the zoning, but there appears to be 15 duplicates for each path - it's showing 60 paths per host per LUN. WAT? VMware isn't showing duplicate paths, just the 4 that should be there.


GrandMaster
Aug 15, 2004
laidback

Does anyone here know 3PAR well?
We've just had a bunch of new disks installed, and i figure it's probably not best practice to mix different capacity disks in the same CPG as the performance profiles will be different (450GB & 1.8TB, both 10K). Our R5 CPG is picking all disks as the filter is just type=fc.

My plan is to create another CPG with a disk type filter, modifying the disk filter on the existing CPG and then using AO to tier.

T1 - CPG_R5_450
T2 - CPG_R5_1800

Is this a bad idea? Can I modify the filter on the existing CPG even though VV data will then be on disks that aren't included? I figure a tune/system maintenance job will get data onto the right CPG disks or something?

GrandMaster
Aug 15, 2004
laidback

goobernoodles posted:

Being in the construction industry, our file servers are filled up by 70% images. While I've done one-off image resizing and compression runs, I have to talk with departments before and after in order to avoid situations where they get resized to something too small. They need to be able to zoom in and see minor details.


Hah, ours sounds the same - tons of civil engineering related photos, saved at the highest possible resolution.

I use this powershell script:
http://poshcode.org/621

You set the minimum horizontal resolution that will be resized, anything smaller gets left alone and larger gets resized to that minimum. Reduced our total image size usage by about 70%.

GrandMaster
Aug 15, 2004
laidback

goobernoodles posted:

Hell yes, that's what I'm talking about. Thanks man!

No worries!
Just a note, I had issues running it on Windows 2012 powershell - it would run until it consumed all available memory on the box then crash. Under 2008 it's fine.

GrandMaster
Aug 15, 2004
laidback

I dunno if any other vendors do anything similar to RecoverPoint, but synchronous replication with journalling to allow recovery to any point in time is pretty great in situations like this ATO thing. I really miss working on EMC gear after doing EVA/3PAR for the last year or so.

GrandMaster
Aug 15, 2004
laidback

Not sure why everyone is bitching about emc support.. I've had very few bad experiences, and they were in the olden days (clariion).
We've got vnx/unity/vmax/isilon/centera/data domain, support handle all our code upgrades and and hardware faults are rectified quickly..
Compared with other vendors like HP 3PAR (urrrgh) and VMware their support has been great.

GrandMaster
Aug 15, 2004
laidback

I dunno, but we've had a VMware support case open for over a year with no end in sight. They just keep asking for logs over and over again. Our TAM has even been kicking them and it's still going nowhere.
Luckily it's just an annoyance and not causing us major issues.

Adbot
ADBOT LOVES YOU

GrandMaster
Aug 15, 2004
laidback

You're not getting it, regardless of whether they can fix the problem, you always need someone else to point the finger at. This is blameshifting 101 and keeps you employed haha

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply