Enterprise Storage Megathread: Why is my NAS a SAN?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Enterprise Storage Megathread: Why is my NAS a SAN?

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Anyone making SANs with SFF SAS drives yet? We're trying to standardize our environment around 72/146 GB 2.5" SFF SAS drives (300 GB by end of year)

unknown posted:

What are people using for doing their I/O tests of boxes?

Also recommending IOMeter. But more importantly, try to simulate the actual workload that you expect to use. I recently had a vendor tell me "you should be getting at least 12,000 IOPS on that LUN, not sure why you're only seeing 8,000". As it turns out, their test was performed using 512B blocks, 100% sequential, 100% read. Well duh.

small blocks = higher IOPS
big blocks = higher throughput

To simulate our SQL workload, we use 8k blocks, 60% write, 40% read, 100% random.

brent78 fucked around with this message at 05:39 on Sep 3, 2008

# ¿ Sep 3, 2008 01:35

Adbot: ADBOT LOVES YOU

# ¿ Apr 27, 2024 06:48

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Anyone else going to be at the HP Storage Symposium in Colorado Springs next month? Send me a PM, I'm staying at the Broadmoor (as is everyone else I imagine)

# ¿ Sep 23, 2008 22:55

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

rage-saq posted:

Hmm, are you going through a distributor or through HP? One of my distributor guys whispered something about another storage event soon (last one was in SF).

Our HP VAR invited me.

# ¿ Sep 24, 2008 18:09

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Any thoughts on the Dell EqualLogic PS5000XV? I'm helping a friend who is an all Dell shop and it's difficult to look at the quotes because they don't break down the pricing at all. The quote is for:

Dell EqualLogic PS5000XV, High Performance,15K SAS Drives (223-6501)
2.3TB capacity, 16 X 146GB 15K SAS, Dual Controller (341-6484)
EqualLogic Complete Care Plus,Same Day 4 Hour Parts,PS5000XV,Full, Initial Year (987-6390)

for $41k, seems steep considering I could purchase a comparable HP MSA 2012i for much less. Dell also told him they don't do evals, which doesn't seem right. I think he needs to evaluate one against his workload to make sure it's adequately sized. I'd throw out that $31-34k is more reasonable, but then again, don't know a lot about Dell.

# ¿ Dec 30, 2008 21:22

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Just picked up an EqualLogic PS5000XV fully populated with 300 GB 15k SAS. Will report back when I have it hooked up.

# ¿ Jan 9, 2009 08:42

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Jadus posted:

If it's to tape, does LTO4 provide enough speed to complete a backup within a reasonable window?

We use the HP MSL tape libraries with LTO4 and it will saturate the gigabit network at a sustained 940 Mbps. I'd say that's pretty fast. It's usually the backup source disk i/o that's the bottleneck, not the tape.

# ¿ Jan 22, 2009 00:07

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

I just saw an article about Pillar Data laying off 30% of their workforce.. and here I am with 100k to spend on a SAN and can't even get them to return my phone call. Anyone using Lefthand VSA in production? It sounds very cool and scary at the same time.

brent78 fucked around with this message at 04:13 on Feb 4, 2009

# ¿ Feb 4, 2009 04:11

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Just wanted to post that I got a shelf of EqualLogic 5000VX setup in the lab and I'm very impressed with it's performance. It's configured with 16 x 300GB 15k disks, active/backup controller and all 6 gigE connected to a pair of 3750's. Using jumbo frames and receive flow control as well. I'm achieving 200 MB/s writes with ease and barely sweats with mixed random reads/writes. This shelf as configured was 40k, not the cheapest thing out there but on par with 15k SAS. The equivalent NetApp or EMC solution would have been double considering all their retarded licensing costs. Ohh you want iSCSI, caa-ching.

# ¿ Feb 11, 2009 02:40

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Intrepid00 posted:

Chiming in as well that you should give Lefthand a try. We just purchased it and haven't regerted it yet.

One step ahead... I have a call in now for a fully populated SAS shelf with dual controllers. My rep said I probably wouldn't be happy with it's remote management because it's based off a DL185. Nonetheless, working to get one in house in the next week or two. They keep trying to sell me on Lefthand's VSA (virtual storage adapter). Are they moving away from hardware based solutions?

Edit: Whats a ballpark figure for a fully populated SAS lefthand solution?

brent78 fucked around with this message at 00:27 on Feb 12, 2009

# ¿ Feb 11, 2009 18:52

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

rage-saq posted:

I'd advise against getting any kind of advice on important system design from what is essentially the best buy of product fulfillment centers.

This. Never buy hardware from CDW unless it's parts or software. Find a reseller that's certified in the solution you're interested in and has good pre-sales engineering support that can design solution. CDW is essentially NewEgg with account reps. CDW also won't be able to get you the lowest pricing. With that said, CDW does have a nice interface for managing software licensing like Adobe, Microsoft, etc..

# ¿ Mar 19, 2009 19:51

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

I have 4 Coraid SR-1521's with 15 x 500GB drives each (7.5 TB RAW each), willing to unload real cheap to the first people who PM me. I was using them for a media project and now no longer need them.

# ¿ Mar 27, 2009 04:11

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

bmoyles posted:

What's the scoop on Coraid, btw? I tried looking into them a few years back, but they didn't do demo units for some reason so I passed.

I have four Coraid SR1521's populated with 15 x 500GB SATA each. I was excited when I got them a couple years ago, basically 15 hot swap drive bays, N+2 power supply configuration and 2 x 1 GbE network connectivity all for $5k each (not including the drives, which was another 2k). I started off doing some xen virtulization since AOE wasn't (and still isn't) supported by VMware. I enabled jumbo frames, configured the drives as a single RAID-10 with one hot spare. Never was able to achieve anywhere near the published numbers. With a moderate amount of disk I/O the shelf would really start to lag bad. They fact that it has zero cache really hurts the performance. If they would slap in a 2GB cache and decent management and alerting tools it would be killer. I ended up buying an equallogic shelf and consolidated the VMs from all four CR1521's on to it, and still have IOPS to spare. Keep in mind though that the EQ box has 16 x 300GB 15k SCSI. I will sell the CR1521's to anyone who wants them for a song. Coraid sells an HBA that allows you to use them in VMware now.

brent78 fucked around with this message at 18:32 on Jun 16, 2009

# ¿ Jun 16, 2009 18:29

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

InferiorWang posted:

Has anyone noticed any price drops on Lefthand gear since HP swallowed them up;

Not really, expect to pay 30-35k per shelf.

# ¿ Jul 29, 2009 19:43

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

The 80GB boot drive is a waste, they should be booting from the network via PXE. I'm also curious what they do when a drive fails, since it's not hot swap. Again, seems like MogileFS would have a perfect fit and probably higher uptime.

Edit: If you look at the pics, the disk arrays sit on top a rail. I wouldn't want to be the one that has to pull one out for maintenance.

brent78 fucked around with this message at 09:03 on Sep 7, 2009

# ¿ Sep 7, 2009 09:00

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

EnergizerFellow posted:

I do wonder that as well, such as why they have a seemingly high-speed CPU. I wonder if they have single-thread performance issues.

They mention doing all their operations via HTTPS, so encryption/decryption is all done by the CPU.

H110Hawk posted:

Sometimes it's hard to get 200v power in datacenters.

If you mean mom and pop basements, then yes. I get 208V 60A 3-Phase from my local co-lo down the street. 208 is standard fare for a datacenter. They host all their gear at 365 Main, so not a problem there.

# ¿ Sep 7, 2009 20:54

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

My iSCSI / multipath notes for CentOS 5.4, using an EqualLogic PS5000XV and 6000XV. Hopefully they will be of some help to someone.

Configure 2 NICs on the iSCSI network, for me that was eth2 and eth3, then run a discovery

code:

iscsiadm -m iface -I iface0 --op=new
iscsiadm -m iface -I iface1 --op=new
grep -i hwaddr /etc/sysconfig/network-scripts/ifcfg-eth2
grep -i hwaddr /etc/sysconfig/network-scripts/ifcfg-eth3
iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress -v 00:15:17:29:C5:8E
iscsiadm -m iface -I iface1 --op=update -n iface.hwaddress -v 00:15:17:29:C5:8F
iscsiadm -m discovery -t st -p 172.16.1.240 --interface=iface0 --interface=iface1

After discovery, let's login..

code:

iscsiadm --mode node --targetname iqn.2001-05.com.equallogic:0-8a0906-c014ea803-495000004df498e6-lun01 --login

/etc/multipathd.conf - Get the wwid's by running 'multipath -ll'

code:

defaults {
        user_friendly_names yes
}

multipaths {

        multipath {
                wwid                    36090a03880ead4837699a44f00009042
                alias                   eqlogic-lun0
        }

        multipath {
                wwid                    36090a03880ea14c0e698f44d00005049
                alias                   eqlogic-lun1
        }

}

device {
        vendor "EQLOGIC"
        product "100E-00"
        path_grouping_policy multibus
        getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
        features "1 queue_if_no_path"
        path_checker readsector0
        failback immediate
        path_selector "round-robin 0"
        rr_min_io 128
        rr_weight priorities
}

/etc/sysctl.conf

code:

net.core.rmem_default = 65536
net.core.rmem_max = 2097152
net.core.wmem_default = 65536
net.core.wmem_max = 262144
net.ipv4.tcp_mem = 98304 131072 196608
net.ipv4.tcp_window_scaling = 1

Restart multipath...

code:

/etc/init.d/multipathd restart
multipath -v2
multipath -ll

eqlogic-lun0 (36090a03880ea94e8e698244e00009091) dm-6 EQLOGIC,100E-00
[size=185G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
 \_ 12:0:0:0 sdg 8:96  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 11:0:0:0 sdh 8:112 [active][ready]
eqlogic-lun1 (36090a03880ea14c0e698f44d00005049) dm-3 EQLOGIC,100E-00
[size=180G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:0  sde 8:64  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 10:0:0:0 sdf 8:80  [active][ready]
etc....

I may have missed something as I grabbed most of this from my .bash_history. This should get anyone 95% of the way there. I've had no problems achieving 200 MB/s+ with this setup (multiple target LUNs)

brent78 fucked around with this message at 23:49 on Mar 10, 2010

# ¿ Mar 10, 2010 23:41

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

Insane Clown Pussy posted:

At this rate we're either going to go for Equallogic or bite the bullet and get some more HP/Lefthand boxes.

Please explain. We are looking at picking up 6 shelves of Lefthand. I've used EqualLogic in the past and loved everything about them, except my boss is anti Dell these days. If LeftHand sucks, please tell before I get neck deep in it.

brent78 fucked around with this message at 15:59 on Mar 20, 2010

# ¿ Mar 20, 2010 15:56

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

EoRaptor posted:

Dell just pitched me an 8TB raw PS4000E Equalogic at around 25k Cdn.

16 x 500GB SATA? Put it in RAID-50 with 2 hot spares and you won't be disappointed. Make sure to download SAN HQ (it's free), really great software for monitoring performance of the group.

# ¿ Mar 31, 2010 04:20

Adbot: ADBOT LOVES YOU

# ¿ Apr 27, 2024 06:48

brent78: Jun 23, 2004; I killed your cat, you druggie bitch.

About to pick up 40 TB of Compellent storage. I liked their solution the best out of Lefthand, Equallogic and Netapp. Anything I should be aware of before dropping this PO?

# ¿ Jun 10, 2010 04:44

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Enterprise Storage Megathread: Why is my NAS a SAN?