Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Nebulis01
Dec 30, 2003
Technical Support Ninny

So I have a stupid question. I'm new to administering a SAN so please bear with me.

We just recently got a Dell Equalogic PS4000X. I configured it out of the box for RAID6 since it's mainly going to be used for read access. However I was exploring some of the volume options and it appears you can assign a raid type for the volume, but it won't change any of the member drives to fit this configuration.

My question is this:

Is it possible for a member to have more than one raid type or do all 16 drives have to be allocated to the same raid type? It seems like that would be a stupid design but I've not found anything in the documentation to answer it, or I'm blind and can't locate it.

Any help would be much appreciated.

Adbot
ADBOT LOVES YOU

Nebulis01
Dec 30, 2003
Technical Support Ninny

Intraveinous posted:

I'm not sure if that's how EQL works it or not, but I hope it wasn't a waste of time.

Not a waste of time at all, went back through it and still couldn't find an option for it. I submitted a support ticket to Dell yesterday and this what they said

EquaLogic Support posted:

Dell Storage Support received your request concerning your EQLC PS4000X stating that you have questions concerning multiple raid types on a single array.
Each member in your array group can be only 1 raid type. You can change raid types in subsequent members, but they can be only 1 type a piece.

Which is fine, just would be nice to do multiple RAID types with the same member (physical unit). Apparently this requires you spend more than $30,000 or go with a different vendor

Nebulis01
Dec 30, 2003
Technical Support Ninny

Nomex posted:

Any server HP sells that's larger than a DL380 has 4 10GbE links. I've got a few customers using all four, though probably not to saturation. It's more for fault tolerance. We generally present each 10 gig link as 4 2.5 gig links to VMWare.

I'm curious, I've never worked with 10GE but do you run in the queue depth issues like this? I'm assuming it would split the queue to appropriate levels?

Nebulis01
Dec 30, 2003
Technical Support Ninny

I could see the 45mbps limit with a a large queue depth depending on the read/write size, and array setup perhaps? We get poo poo performance with a QD of 32 in CrystalMark but since our SAN is a glorified file store anyway, I'm not really worried about it.

EDIT:
I was curious I ran an ATTO 2.46 run with 4KB blocks and a Queue Depth of 10.
1GB Length - Read 42MB/Sec, Write 1.1MB/Sec.
2GB Length - Read 52MB/Sec, Write 5.2MB/Sec

As a point of reference

These are from the following configuration:

Dell Equal Logic PS4000X SAN in RAID6 (400GB 10K RPM SAS, 16 spindles). Controller on the SAN has a 2GB cache and 2 GigE iSCSI connections to a dedicated GigE switch. Jumbo Frames and Flow Control are enabled. Server has 4 GigE Broadcom iSCSI NICs w/ Jumbo Frames and offloading enabled, and the dell multi path driver balances between them.

code:
Below is with MPIO v3.4.2, Firmware 5.0.2

-----------------------------------------------------------------------
CrystalDiskMark 3.0 x64 (C) 2007-2010 hiyohiyo
                           Crystal Dew World : [url]http://crystalmark.info/[/url]
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

           Sequential Read :   170.694 MB/s
          Sequential Write :   100.999 MB/s
         Random Read 512KB :    56.960 MB/s
        Random Write 512KB :    76.950 MB/s
    Random Read 4KB (QD=1) :     1.086 MB/s [   265.0 IOPS]
   Random Write 4KB (QD=1) :     7.727 MB/s [  1886.4 IOPS]
   Random Read 4KB (QD=32) :    24.852 MB/s [  6067.5 IOPS]
  Random Write 4KB (QD=32) :    24.209 MB/s [  5910.3 IOPS]

  Test : 1000 MB [I: 0.0% (0.1/499.9 GB)] (x9)
  Date : 2010/09/29 14:36:01
    OS : Windows Server 2008 R2 Datacenter Edition (Full installation) [6.1 Build 7600] (x64)


Below is with MPIO v3.4.1, Firmware 4.3.7

-----------------------------------------------------------------------
CrystalDiskMark 3.0 x64 (C) 2007-2010 hiyohiyo
                           Crystal Dew World : [url]http://crystalmark.info/[/url]
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

           Sequential Read :   180.898 MB/s
          Sequential Write :   101.380 MB/s
         Random Read 512KB :    60.362 MB/s
        Random Write 512KB :    89.607 MB/s
    Random Read 4KB (QD=1) :     1.514 MB/s [   369.6 IOPS]
   Random Write 4KB (QD=1) :     6.854 MB/s [  1673.3 IOPS]
   Random Read 4KB (QD=32) :    29.304 MB/s [  7154.3 IOPS]
  Random Write 4KB (QD=32) :    17.736 MB/s [  4330.1 IOPS]

  Test : 1000 MB [I: 0.7% (3.3/499.9 GB)] (x9)
  Date : 2010/09/23 16:53:14
    OS : Windows Server 2008 R2 Datacenter Edition (Full installation) [6.1 Build 7600] (x64)

Nebulis01 fucked around with this message at 14:59 on Oct 12, 2010

Nebulis01
Dec 30, 2003
Technical Support Ninny

The only thing I can think of (not a SAN/Storage guy myself) would be that you're having issues using just one link and having such small reads/writes. Is there any way you can utilize the dell MPIO driver and add another link or two to that box?


Also, I've got our setup running on Broadcom and they seem to run just fine. What firmware revision are you running? There have been some pretty decent fixes in the last few firmware/MPIO driver releases

Nebulis01 fucked around with this message at 16:30 on Oct 12, 2010

Nebulis01
Dec 30, 2003
Technical Support Ninny

I would say that the MPIO driver is not balancing those NICs the same way, it looks like its weighing eth0 (intel?) pretty heavily. Is there a reason your round robin instead of Least Queue Depth?

In Perfmon (2008R2) you can log iSCSI by connection in both bytes sent/received and requests

Nebulis01
Dec 30, 2003
Technical Support Ninny

I'd venture this 'consultant' is also a NetApp certified partner/preferred vendor or similar.

Nebulis01
Dec 30, 2003
Technical Support Ninny

You could use StarWind's iSCSI initiator for that. http://www.starwindsoftware.com/iscsi-initiator It's free and would do want you want.

Nebulis01
Dec 30, 2003
Technical Support Ninny

Misogynist posted:

He doesn't need an initiator (Windows has had one built in forever), he needs a target server. I don't know of any free ones on Windows.

My bad. He does need a target server, StarWind makes one but it isn't free.

Nebulis01
Dec 30, 2003
Technical Support Ninny

three posted:

Equallogic also works this way.

Not the P4000 series. I wish it did, but apparently the P6x00 series does.

Nebulis01
Dec 30, 2003
Technical Support Ninny

szlevi posted:

AFAIK all EQL boxes, regardless of generation or model numbers, run exactly the same firmware, same features, same everything - are you sure you are not in some manual mode?

Well it's quite possible I'm retarded. It's my first SAN. I put a ticket into EQL support and they state it couldn't be done. I can't find anything in the documentation to let me do it either.

Nebulis01
Dec 30, 2003
Technical Support Ninny

Yes, the box I'm talking about is the EqualLogic PS4000X (sorry about the make/model confusion)

It's limited to 2 members in a group, 256 Volumes, 2048 Snapshots, 128 Snapshots/volume, 32 volumes for replication, and 128 replicas per volume. The PS6x00 series has a substantial increase in all of those metrics.

Nebulis01 fucked around with this message at 19:30 on Jan 3, 2011

Nebulis01
Dec 30, 2003
Technical Support Ninny

szlevi posted:

...but Storage Server 2008 vs Server 2008R2? Tough call... are they both Enterprise and is SS2008 x64?

WSS2008 is available in x86 or x64. WSS2008R2 is available only on x64. Unless you really need the iSCSI or De-duplication features, Server 2008R2 would serve you quite well.

Nebulis01
Dec 30, 2003
Technical Support Ninny

I had a 'my first SAN' growing experience today.

We've got a Dell PS4000X that's been running since September nice and peachy. We got another since we're running low on space. I went and added it to the group, but wanted to make sure nobody would use it so I disabled eth0/1 as soon as possible. Little did I know that the 'Enable Performance Load Balancing in Pools' check box is checked by default. Our entire production environment that uses SAN as storage came to a screeching halt because the SANs had already started load balancing before i disabled eth0/1.

It was a learning experience, thankfully we didn't lose any data and had about at 5minute downtime while I figured out what the gently caress I did.

Thought I'd share my idiot story

Nebulis01
Dec 30, 2003
Technical Support Ninny

Crackbone posted:

Is there any resale value in a Dell MD3000 (bare, or with 15 176G 10K SAS drives inside)? I inherited this from a company buyout, and it's honestly more a hassle than it's worth in our environment. I checked ebay and there appears to tons of them not selling at $2500 or higher.

If you feel like disposing of it. I could really use it for a SQL server we're planning on standing up :)

Nebulis01
Dec 30, 2003
Technical Support Ninny

Moey posted:

Why is no one mentioning Dell? :ohdear:

When we went for shared storage my boss went with a Dell MD3200. Is there a reason no one seems to be touching these (in these forums) for smb storage solutions?

I have two MD3000i and a 1220 and like them. We also have two EqualLogic PS4000X (one purchased in dec 2010 for $29,999.99 inc tax. One purchased in March 2011 for $27,800.00 inc tax )

Nebulis01
Dec 30, 2003
Technical Support Ninny

what is this posted:

Apple switched cleanly from OpenFirmware to EFI without most people noticing any change...


You can do poo poo like this when you control the entire spectrum of hardware your software runs on, and your customers don't mind upgrading every two releases. But when you're forced to support hardware that is sometimes going on two decades old you have a few less options.

Nebulis01
Dec 30, 2003
Technical Support Ninny

XMalaclypseX posted:

Hi folks. I'm looking to get a bit of advice on a small SAN setup for a non-profit.

I'm going to be replacing a small blade sql array with vmware and I need a new SAN. Here are some of my musts:

ISCSI 1GB or greater
1500 IOPS or greater
12TB usable with RAID 50
Hot Spare
No replication or snapshots
Dual controllers

Nothing major or special, however price and reliability is a factor. Whatever devices and vendors you can point me at would be appreciated.

We have a pair of Dell Equallogic PS4000X boxes and are very happy with them. They just introduced the PS4110 series that bumps the interface form 2x 1Gbps ISCSI to 2x 10Gps ISCI. The 12TB capacity is going to be a stickler though you can't get anything like that until you step up into the PS6510 range and that's going to run you 60-75K.

Nebulis01 fucked around with this message at 16:22 on Mar 27, 2012

Nebulis01
Dec 30, 2003
Technical Support Ninny

Nukelear v.2 posted:

You can spec a 4110E with up to 36TB raw, 12 x 3TB NL-SAS. Which will you run you in the 20-25k neighborhood. 10GE switch infrastructure however...

I'd forgotten they offered a 3.5" variant, good call! and yea the 10GE switches and cards :(

Nebulis01
Dec 30, 2003
Technical Support Ninny

adorai posted:

I figured it would probably be $2000 plus travel, lodging, and meals since we'll want to do it after hours for obvious reasons. Seem reasonable?

Negotiate it down to nothing, your spending $200k if they want the business they will eat the cost.

Nebulis01
Dec 30, 2003
Technical Support Ninny

Aniki posted:

I think it is the PS4100E. That's good to know. As far as how their snap shots work, are you able to browse them and recover individual files or do you need to mount the snapshot in order to retrieve the file? With NetApp standing pat on pricing at the moment, I need to start seriously considering some other options. I'd like NetApp to work out, but they need to start making some concessions on their end to make it a more fair deal. I hate getting stuck doing negotiations, but with a small IT department, I guess that's the reality of the situation. It's sad when getting back to programming and designing databases sounds so refreshing right now.

With our PX4000X we have to mount set the snapshot to online, mount it, do what we want with it, un mount, set to offline. This is firmware v5.0.7 and I don't see anything in the changelog that makes this process change

Nebulis01
Dec 30, 2003
Technical Support Ninny

NippleFloss posted:

This will always be the case with snapshots on LUNs. It had to be mounted to a host that understands the filesystem on the lun snapshot. To the array its just a bunch of bloks in a container.

NetApp can expose files from a snapshot directly when it's an nfs or CIFS share because those are WAFL files.

Well that's pretty sweet, the more you know :)

Nebulis01
Dec 30, 2003
Technical Support Ninny

NippleFloss posted:

Any major storage vendor can provide a system, or scale out systems, to provide a petabyte or more. You'll need to give a lot more detail regarding what kind of data you're collecting, how it's accessed, what level of protection you want, backup requirements, etc.

How do you even back up that much data? I'd assume just mirror the disks to another location? LTO-5 drives only write at 140MB/s off site for that much data would take a staggering amount of tapes and drives to be effective.

Nebulis01
Dec 30, 2003
Technical Support Ninny

Nomex posted:

Sorry guys, I can't go into to many specifics. It's a pretty large environment though.

If you go by the sizing guide here: http://technet.microsoft.com/en-us/library/bb125019%28v=exchg.65%29.aspx

Assuming medium performance on all 336 15k disks (150 IOPS/disk) and average (0.75 IOPS/user) workload he'd have roughly ~37,800 users. That's a pretty big exchange 2003 environment :)

Nebulis01
Dec 30, 2003
Technical Support Ninny

Internet Explorer posted:

I think I'm the resident Equallogic cheerleader.

I'll cheer alongside you. I have a pair of Equallogic PV4000Xs and love them.

Nebulis01
Dec 30, 2003
Technical Support Ninny

evil_bunnY posted:

Yeah there's no way 6TB on a equal4k is $35K US.

Our PS4000X was $29,999 (450GB 10K Drives, 3yrs 24/7/365 2hr support)

I should note we bought that in the 2nd half of 2011. The first one purchased in 2010 was $35k I think.

Nebulis01
Dec 30, 2003
Technical Support Ninny

BnT posted:

What are considered "class 2" drives?

Nearline SAS I believe, stuff that's supposed to be used for bulk storage of infrequently accessed data.

Nebulis01
Dec 30, 2003
Technical Support Ninny

EoRaptor posted:

Yeah, NDMP or other direct backup solution (as opposed to grabbing them from the share) is probably a stretch, but I still want to keep in mind that I have to back this up, and keeping my feature checklist to match that.

I could do Windows Storage Server and ExtremeZ-IP, I'd just need a chassis and controller setup to support it. What I have going now for network shares is a whitebox supermicro with Server 2008R2 on it. The OSX support is terrible, however, and fixing that is a requirement.

If WSS would work, look in to Server 2012? They added SMB 3.0 support and made some other changes that make it more appealing for storage usage. I don't know jack about OSX though, does it even support SMB?

Nebulis01
Dec 30, 2003
Technical Support Ninny

Docjowles posted:

Amazon S3?

or Amazon Glacier if your backups won't be deleted before 90days or so.

Nebulis01
Dec 30, 2003
Technical Support Ninny

Docjowles posted:

Or both! You can set up lifecycle policies on S3 buckets that automatically transition data into Glacier after N days

Totally didn't know you could do that automagically, that's pretty slick.

Nebulis01
Dec 30, 2003
Technical Support Ninny


Windows Server 2012 with Scale Out file services on a Clustered Shared Volume could be a possible solution for you. Every server can access the shared resource and you could add a number of hosts to the cluster

http://technet.microsoft.com/en-us/library/hh831349.aspx

Nebulis01
Dec 30, 2003
Technical Support Ninny

Wicaeed posted:

Probably more for the DBA thread, but correct me if I'm wrong, don't MSSQL HA Availability Groups use Cluster Shared Volumes for storage? Or can they use attached local storage as well?

The presentation I've been given says we are going to be using our SAN for write-intensive storage in a 2 node availability group, and then using another 2 nodes availability group for the read-intensive operations. Either way we are potentially wasting 50% of our resources since:

A) We have 4 nodes to use, and all 4 nodes have the same storage on them
B) Only two of those nodes (the read-intensive workloads) will be used in an availability group
C) Two nodes will have all of their local storage going to waste

CSVs are only supported in SQL2014 and above, also Availability groups are only an option if you're running Enterprise Edition of MSSQL. Standard only supports using a Failover Clustered Instance based on Windows Server Failover Clustering which requires shared storage.

It seems silly to that you're willing tos pend the $$ licensing enterprise but not drop $22k on the Nimble/Equallogic box.

Adbot
ADBOT LOVES YOU

Nebulis01
Dec 30, 2003
Technical Support Ninny

So we're looking for a new SAN our requirements are pretty minimal. This is interal Hyper-V cluster and a few SQL boxes.

Our existing stuff is about 4K IOPS, 250MB/sec and ~4TB used our existing infrastructure is 47% read, 53% write. We're looking for something that will handle 15K IOPS and give us 6-8TB usable with 10GigE and allow us to put a trial 5-10 users on to VDI with room to expand that to 60-70 down the road. Our budget is in the $40k range for this project. I'm looking at multiple vendors for this and so far we've narrowed it to Tegile, Nimble and NetApp.


NetApp wants us on an FAS2552A outfitted with 4x 200GB SSD and 20x 1TB 2.5" 7.2K (not willing to promise an IOPS benchmark for the array other than to say 'fits your needs')
Nimble presented a CS300 array with 4x 160GB SSD and 20x 1TB 2.5" 7.2k (rating 30K IOPS)
Tegile is quoting an HA2100 with 3x 200GB SDD and 13x 2TB 3.5" 7.2K (rating 10K IOPS, moving to an HA2300 with 1TB 2.5" 7.2K gets us in to the 30K range)

All of the quotes are in the same $40k+- range with 3 years of NBD support and a cold spare kit for HDD and SSD.

Are these prices too high (they seem a might high to me)?

I'm also really looking for feedback on the quality of Tegile, the platform seems like a nice front end to what is essentially a commercial version of ZFS with some bells and whistles. But I'm hesitant to go with such an unknown and young company for such a mission critical piece of infrastructure.

Are there other vendors we should consider?

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply