|
NippleFloss posted:And I think Madsushi's point about QOS was that the aggregated IO from a big pile of SSD is so high that it would be very hard to legitimately oversubscribed it. A DB query from your average environment isn't going to drive the millions of IOPs required to crowd out other workloads (on the disk layer, controller or transport layer is another question). Sequential IO like DB logging activity could definitely do it though, and I've seen it happen. Yes, that was the gist of my post. Now, if we're talking multi-tenant, I do understand the requirement to limit one tenant from affecting another (I work with a hosted multi-tenant private cloud environment). I typically see VMware disk resource limitations or SIOC used to ensure that one tenant doesn't go out of control. I can see how moving those controls to the SAN could make sense.
|
#
?
Apr 13, 2013 20:03
|
|
|
|
| # ? May 6, 2024 13:06 |
|
|
So our storage/server refresh is now at the "Need a plan" stage and I need around 15-20tb usable of storage for our VMware cluster. Right now everything is iSCSI on P4000 which is clustered across two sites over dedicated 10GbE using their network RAID with a third site running their witness node. The VM's that are in VMDK format total around 1.5tb (40 or so VM's) with a round 10tb of file server and Exchange and SQL databases on iSCSI volumes that are mounted from in the guest. 95% IOPS is just under 2000 with a read/write ratio of 92/8. At the moment my plan is a pair of Dell R720XD's or HP DL380 G8's each stuffed with nearline SAS and a couple of SSDs with the respective CacheCade/SmartCache option enabled on the RAID controller. I then plan on running a couple of P4000 VSA's on each node along with the various other VM's. Everything is on the VMware HCL and I have a preference for "storage as software" because it means that if our needs change in eighteen months time I'm not left with $100k of proprietary SAN/NAS in a rack that doesn't do what we need - sure we've paid for some VSA licenses but it's a trivial amount. Any feedback would be welcome.
|
|
#
?
Apr 14, 2013 13:34
|
|
|
Moey posted:So we were gifted a single HP P4300 SAN. I brought it online today just to poke around the management of it and figured I would do some updates. If anyone cares about this, apparently the update did wipe out the admin login I created. Lousy HP.
|
|
#
?
Apr 15, 2013 15:39
|
|
|
Moey posted:If anyone cares about this, apparently the update did wipe out the admin login I created. Lousy HP. I actually care, I have p4300 g2 I maintain. Did it delete your default admin account or just an account you created to do this update? Also, how did you fix it? Ive got the latest update staged and ready to install but would rather not if its gonna nuke some stuff.
|
|
#
?
Apr 15, 2013 16:17
|
|
|
I did read of a P4000 issue which affected passwords containing certain characters like ~ so that might be worth a look.
|
|
#
?
Apr 15, 2013 18:24
|
|
|
I'm looking at implementing a two EMC VNXe3150's paired with EMC Data Domain DD620's. This is in a VMware 5.0 Windows environment split between our main office in Seattle and a smaller office in Portland. Our main office is running an IBM DS3300 SAN paired with one newer x-series server and 3 older IBM x-series servers and a Portland office with just one IBM x-series server. lovely Netgear switches, no VLAN'ing. Backups are currently at the file system level of the VM's, which makes me extremely nervous. On top of that, we're paying some local IT company over a grand a month to maintain the BDR's and to provide offsite backups. Not exactly an ideal set up. I'm looking at replacing basically our entire infrastructure (servers, SANs, switches, backup appliances) but my questions are with regards to the SAN/Backup pieces. I want to use each office as the primary off-site backup location and want to replicate backups of Seattle to Portland and vice versa. That way I can eliminate the monthly charge we're paying for our offsite backups as well as have backups at the VM level. Any thoughts on the hardware/implementation/pitfalls, etc would be appreciated. Also; anyone have a recommendation for a cheap cloud service for dumping the backups as a secondary off-site backup?
|
|
#
?
Apr 15, 2013 19:56
|
|
|
Moey posted:If anyone cares about this, apparently the update did wipe out the admin login I created. Lousy HP. I like to hear updates on it, my new job is a good deal of HP storage, so any comments or insight you have into it I would love to hear about.
|
|
#
?
Apr 15, 2013 19:58
|
|
|
goobernoodles posted:Also; anyone have a recommendation for a cheap cloud service for dumping the backups as a secondary off-site backup? Amazon S3?
|
|
#
?
Apr 15, 2013 20:09
|
|
|
Docjowles posted:Amazon S3? or Amazon Glacier if your backups won't be deleted before 90days or so.
|
|
#
?
Apr 15, 2013 21:30
|
|
|
Or both! You can set up lifecycle policies on S3 buckets that automatically transition data into Glacier after N days
|
|
#
?
Apr 15, 2013 21:33
|
|
|
At what level of queue depth or latency should I start to be alarmed? I'm seeing high read queue depth (in the 100s) on my 2-node P4300 system and read latency is hovering around 30ms per system.Moey posted:If anyone cares about this, apparently the update did wipe out the admin login I created. Lousy HP. I haven't had that happen to me yet, but last big LeftHand update left me with 2 VMs that couldn't be re-connected. Had to copy them to a new folder on the datastore and then it worked. Taught me to not do updates on friday afternoon.
|
|
#
?
Apr 15, 2013 22:00
|
|
|
Docjowles posted:Or both! You can set up lifecycle policies on S3 buckets that automatically transition data into Glacier after N days Totally didn't know you could do that automagically, that's pretty slick.
|
|
#
?
Apr 15, 2013 23:04
|
|
|
Syano posted:I actually care, I have p4300 g2 I maintain. Did it delete your default admin account or just an account you created to do this update? Also, how did you fix it? Ive got the latest update staged and ready to install but would rather not if its gonna nuke some stuff. I only had one admin account setup as I just brought this thing online on Friday. It ended up nuking that account and not being able to log in to compete the update. I was still able to log in with that admin account locally though. After that I just created a new admin account and was able to complete the update. If this thing was in production, it would have required an outage as it rebooted a few times during the update and is the only node. It was "gifted" to us from another department and will eventually be slated for a PHD Virtual backup repository. Also for some insight, this is my first tango with HP storage. Pretty different than what I am used to (Dell and Nimble). This thing really is meant to run in "network raid" providing redundancy as this is really just a server with some disks and a single raid card. Seems to run some *nix OS but the console is just about entirely locked down. Moey fucked around with this message at 23:14 on Apr 15, 2013 |
|
#
?
Apr 15, 2013 23:12
|
|
|
Misogynist posted:I'm curious, do any of you other guys in the tiering discussion work in an environment with petabytes-range of raw data? SSD caching is great for applications with repeatable hotspots, but it's going to be a long time before it can play with the HPC kids. SSD caching is at best a fad. SSD prices are quickly getting to price parity with spinning disk. In less than 3 years it will probably be just as cheap to outfit an array with solid SSDs and skip the slow tier all together. I can't wait for the day when IO sizing is essentially a thing of the past.
|
|
#
?
Apr 16, 2013 00:03
|
|
|
goobernoodles posted:I'm looking at implementing a two EMC VNXe3150's paired with EMC Data Domain DD620's. This is in a VMware 5.0 Windows environment split between our main office in Seattle and a smaller office in Portland. Hey GooberNoodles, VNXe developer here. Are you looking to setup replication sessions between both offices, share the destination volumes with your backup hosts/servers, and then backup at the respective destination sites? I would only suggest you be careful with what you use as your replication service. [Edit: Totally ignored the fact that you were using Data Domain here. Pretty fuzzy on the specifics of their product suite. If they can manage your replication sessions, you should be ok, especially with in-line dedupe] The VNXe supports iSCSI replication sessions, but requires a server-side application, like EMC Replication Manager, to manage these connections. However, Replication Manager (to my knowledge) will not be supported in upcoming releases, instead being replaced by it's successor AppSync. Another caveat (again, to my knowledge) is that AppSync will only support newer arrays in upcoming releases. So make sure you dont get yourself stuck with your replication session management preferences. Tyren fucked around with this message at 02:48 on Apr 17, 2013 |
|
#
?
Apr 17, 2013 02:43
|
|
|
Bit of a noob question here if you don't mind: Let's say I have two 1 Gbps interfaces that are bonded / teamed in active-active mode for 2 Gbps total throughput on a Linux box. Which of these can provide 2 Gbps of throughput (well not the whole 2 but >1 and actually take advantage of the bonding): NFS v4 TCP connection CIFS / SMB connection iSCSI with MPIO HTTP I know iSCSI with MPIO can, and I'm pretty sure HTTP can't but I'm not sure about the others.
|
|
#
?
Apr 17, 2013 02:57
|
|
|
hackedaccount posted:Bit of a noob question here if you don't mind: Let's say I have two 1 Gbps interfaces that are bonded / teamed in active-active mode for 2 Gbps total throughput on a Linux box. iSCSI with MPIO is the only one.
|
|
#
?
Apr 17, 2013 03:10
|
|
|
CIFS, NFS, and HTTP can do it I think if there are multiple connections, but iSCSI is the only one that can do it for a single connection.
|
|
#
?
Apr 17, 2013 03:57
|
|
|
FISHMANPET posted:CIFS, NFS, and HTTP can do it I think if there are multiple connections, but iSCSI is the only one that can do it for a single connection.
|
|
#
?
Apr 17, 2013 04:09
|
|
|
hackedaccount posted:Bit of a noob question here if you don't mind: Let's say I have two 1 Gbps interfaces that are bonded / teamed in active-active mode for 2 Gbps total throughput on a Linux box. You're technically asking about two different things here. SCSI MPIO is not the same as link aggregation and they work at different layers of OS. When you aggregate links at the network layer you take two or more physical ports and bond them into a single virtual interface that you can then slap an IP address on. Your two ports act like a single port for the purposes of configuration and the drivers at the network layer decide which link to push data across. This is done using a hashing algorithm and there are a lot of different possible values to hash including source and destination IPs, source and destination macs, source and destination IPs plus destination port, and a a few others. Link aggregation like this is protocol agnostic. You could send snmp, http, iscsi, NFSv3, NFSv4, HTTP, and whatever else across them and it won't matter at all for the way the traffic is distributed because it's only going to look at the values in the ethernet header it needs to compute the hash and decide which link to send a packet down. All of the common hashing mechanisms have the property that they always send packets for a single TCP connection over the same link. So if you have a single client connecting to a single storage server via NFS that client will only ever use one link to reach that server. However connections to other servers from that same virtual interface might traverse the other link. And depending on your hashing mechanism connections from that same client to that same storage server via HTTP or iSCSI would use the other link. From the perspective of a storage server where you generally have multiple clients connecting this means that both links tend to get used with some frequency. You can most certainly get close to the aggregates bandwidth of all of your links if you get creative with creating IP aliases to spread the load across all of the links, but you will rarely see a perfectly even usage split when you do network aggregation. The plus side of link aggregation is that it's pretty quick to detect failures, especially if using LACP, failover is pretty seamless, and ANY traffic you send across the aggregated interface gets to take advantage of that resiliency, no matter what it is. You can do HTTP, and iSCSI, and NFS, and SMB, and SNMP and SSH and anything else you want and they will all participate equally in the load balancing and fault tolerance. SCSI MPIO isn't link aggregation. It works at the SCSI layer and each separate port with connectivity to the storage is treated as a different path through the network. Each port maintains it's own IP address and still appears to the OS as a separate port, and the MPIO driver just treats them as multiple paths through the network to the same LUN. SCSI MPIO can get pretty granular because it establishes a TCP session for each path and splits individual SCSI commands up between all of the paths in a round robin fashion (if you choose, there are other load balance mechanisms). All traffic related to a single SCSI command stays on a single path, but multiple commands can be multiplexed across all available paths. But if you used on of those interfaces to also serve NFS, or CIFS, those protocols would not know or care about the MPIO because it's happening a few layers above the network stack. YOLOsubmarine fucked around with this message at 04:39 on Apr 17, 2013 |
|
#
?
Apr 17, 2013 04:34
|
|
|
FISHMANPET posted:CIFS, NFS, and HTTP can do it I think if there are multiple connections, but iSCSI is the only one that can do it for a single connection. Yeah my bad I should have said that: I was thinking a single TCP connection from a single client type thing. I assume if there are multiple connections that the OS is smart enough to spread those over the available links. NippleFloss posted:iSCSI MPIO That makes sense. Like FC it's working at the "storage driver" (or whatever) level and not on the network stack. With FC you would have 2 HBAs with unique WWPNs, with iSCSI you would have 2 NICs with unique IPs, and they would both aggregate the bandwidth and multipath in a similar fashion. NippleFloss posted:All of the common hashing mechanisms have the property that they always send packets for a single TCP connection over the same link. Perfect! Like Misogynist mentioned if the protocol (SMB 3.0) is smart enough to open multiple TCP streams it could use multiple links, but HTTP and NFS aren't designed that way so they're always gonna be a single link. Thanks for the info, I appreciate it.
|
|
#
?
Apr 17, 2013 06:18
|
|
|
I have a esxi 5.1 cluster with stores on a dell md3200i. I back up the VMs on this cluster with Veeam and I want to change over to using direct SAN backups. What I cant find is anywhere in the 3200i that I can assign the luns to the backup server as read only. It looks to me like I am going to have to add my backup server as a cluster member on the 3200i in order to give it access to the luns. I am paranoid about this. Is there any way a windows machine will try to mount or write to those luns if I just attach it to the iscsi initiator? Am I being overly paranoid?
|
|
#
?
Apr 18, 2013 13:39
|
|
|
Syano posted:I have a esxi 5.1 cluster with stores on a dell md3200i. I back up the VMs on this cluster with Veeam and I want to change over to using direct SAN backups. What I cant find is anywhere in the 3200i that I can assign the luns to the backup server as read only. It looks to me like I am going to have to add my backup server as a cluster member on the 3200i in order to give it access to the luns. I am paranoid about this. Is there any way a windows machine will try to mount or write to those luns if I just attach it to the iscsi initiator? Am I being overly paranoid? Can't do this with Equallogic either, so I'd be surprised if it could be done with the MD line. I don't think Windows will do it automatically, but if someone logs into your backup server and tries to put those disks online then it will cause problems, I believe.
|
|
#
?
Apr 18, 2013 15:00
|
|
|
That's funny, because in Dell's Compellent line you get a popup when doing the second assignment asking you override it if you DON'T want read only. One thing that vmware recommends is turning off automount -- start up diskpart and type automount disable.
|
|
#
?
Apr 18, 2013 15:48
|
|
|
Ah well I got it figured. If youre on Server 2008 R2 you just go to diskpart and set the san policy to offlineall. The disk will attach as offline read-only. Someone could still log in to the backup proxy and force the disk online but if youre careful about your access to your backup proxy I suppose it isn't an issue.
|
|
#
?
Apr 18, 2013 16:57
|
|
|
Misogynist posted:I'm curious, do any of you other guys in the tiering discussion work in an environment with petabytes-range of raw data? SSD caching is great for applications with repeatable hotspots, but it's going to be a long time before it can play with the HPC kids. I manage about 10PB of storage (4560 disks), if I had a ton of free SSDs lying around I'd use them as metadata-only disks. Currently the metadata is on the same LUNs as actual data, so under heavy loads the "interactive user experience" goes down. HPC storage is usually all about large-block sequential bandwidth, capacity, and saving as much money as possible so it can be spent on the computational aspects of the system.
|
|
#
?
Apr 18, 2013 19:56
|
|
|
The_Groove posted:I manage about 10PB of storage (4560 disks), if I had a ton of free SSDs lying around I'd use them as metadata-only disks. Currently the metadata is on the same LUNs as actual data, so under heavy loads the "interactive user experience" goes down. HPC storage is usually all about large-block sequential bandwidth, capacity, and saving as much money as possible so it can be spent on the computational aspects of the system.
|
|
#
?
Apr 18, 2013 20:04
|
|
|
The_Groove posted:I manage about 10PB of storage (4560 disks), if I had a ton of free SSDs lying around I'd use them as metadata-only disks. Currently the metadata is on the same LUNs as actual data, so under heavy loads the "interactive user experience" goes down. HPC storage is usually all about large-block sequential bandwidth, capacity, and saving as much money as possible so it can be spent on the computational aspects of the system. When you talk about metadata in this context what type of storage are you talking about? Object storage where you keep it's metadata on SSDs, or normal file system metadata where you somehow put the superblocks and inodes on the SSD and keep the actual data on slower drives?
|
|
#
?
Apr 18, 2013 20:22
|
|
|
It may be unique to the GPFS filesystem, but when defining a new "disk" you can specify if you want it to have data only, metadata only, or both. It's basically normal filesystem metadata (inodes) that GPFS keeps on whichever disks you allow it to. So if you scan/stat all your files it shouldn't touch the data-only disks at all unless you do a read/write. The latest version of GPFS can even store tiny files in the inode, which I'd assume would still reside on the metadata-only disks.
|
|
#
?
Apr 18, 2013 20:37
|
|
|
Yeah, I just did some Googling and it looks like ext4 and xfs support metadata / log devices. I remember them from back in the day when I worked with Veritas but had no idea the concept had made it into mainstream Linux file systems. I assume you just chop up an SSD, present a SSD partition and a LUN to a host, and create the file system that way. Learned somethin new today.
|
|
#
?
Apr 18, 2013 21:18
|
|
|
Tegile is a storage startup that has modified ZFS to allow this sort of metadata separation and their big thing is that their appliances include an SSD tier for metadata. It's obviously a big deal for ZFS because it's pretty metadata heavy due to the possibility of large numbers of indirect blocks needing to be read to get to a cluster of data blocks, and also because things like raid rebuilds need to read all of the filesystem metadata for anything living on the failed disk. Metadata storage is a pretty neat application of the high random IO capacity of SSD and something I hadn't really thought of until someone mentioned Tegile to me. Interesting to hear that it's also supported in GPFS and some other commercial filesystems.
|
|
#
?
Apr 18, 2013 22:23
|
|
|
I could use some information from any ZFS experts in this thread. I'm trying to put together a pair of ZFS boxes to export iSCSI volumes for Debian virtualization servers. Rather than spend 2-3x as much on some kind of ZFS appliance or other exotic HA solution, I figured I'd build out a pair of 12-bay chassis we already have lying around. My plan is to use zrep (http://www.bolthole.com/solaris/zrep/) to send snapshots to the secondary server every minute or so (I work at a private college that can handle small amounts of data loss, but we need to be back up and running quickly). My question is -- what free OS has the best ZFS implementation these days? I started with SmartOS and found a lot to like, except that it doesn't support certain features that zrep wants to use (recv doesn't support -o). Right now I'm waiting for Solaris 11 to finish installing so I can see how that feels, but some of the reading I've been doing suggests that the ZFS in Solaris is lagging behind Illumos, which is confusing to me because Wikipedia shows Solaris 11 as version 34 with SmartOS as version 28. I'm not even looking at Linux implementations of ZFS because it seems way too risky. If you have a ZFS setup that you're proud of I'd love to hear about it. Or explain to me why I will regret this and should ask my boss for twice as much money for an appliance.
|
|
#
?
Apr 19, 2013 00:23
|
|
|
Nomex posted:SSD caching is at best a fad. SSD prices are quickly getting to price parity with spinning disk. In less than 3 years it will probably be just as cheap to outfit an array with solid SSDs and skip the slow tier all together. I can't wait for the day when IO sizing is essentially a thing of the past. The fact that NAND loses both write endurance and speed as process nodes shirk makes me think that we'll never get to the stage where flash storage will be anywhere close to cost competitive with big dumb disks in terms of cost per capacity. And keep in mind we're running close to the end of our ability to continue shrinking transistors even under more ideal applications, so it might take a completely new technology (Phase Change RAM, MRAM, etc) before we finally get to the hallowed land if disk-less storage. I certainly believe NAND will still get a lot cheaper, but we're not going to have 4TB eMLC SSDs selling for $400 in three years like we have SATA disks now.
|
|
#
?
Apr 19, 2013 00:40
|
|
|
Novo posted:I could use some information from any ZFS experts in this thread. Between OpenSolaris and Solaris 11 Oracle closed up the ZFS codebase, so Solaris ZFS and open source ZFS are now two seperate things. Open Source ZFS now has feature flags that determine what features the code support rather than strict version numbers. So open source ZFS has features that don't exist in Oracle ZFS.
|
|
#
?
Apr 19, 2013 00:45
|
|
|
Novo posted:I could use some information from any ZFS experts in this thread. OmniOS; http://omnios.omniti.com/ ; OSS, and support contracts are v. reasonable. ZFS versions are meaningless since Oracle and Illumos maintain two different forks. Stay away from Solaris 11; there's still a hilarious data loss bug that still isn't fixed in their free release. I think FreeBSD's ZFS implementation si pretty stable, and the Linux ports are making surprisingly decent process.
|
|
#
?
Apr 19, 2013 00:51
|
|
|
PCjr sidecar posted:OmniOS; http://omnios.omniti.com/ ; OSS, and support contracts are v. reasonable. We've been using NexentaStor Community Edition in production for about 6 months now. I believe it's also built on illumos. Only problem I had was trying to start up a about 15 VMs at the same time off a single LUN backed by SATA disks, causing the box to lock up. We only do 1 VM up or down at a time now until we upgrade the storage to SSD.
|
|
#
?
Apr 19, 2013 17:51
|
|
|
Cpt.Wacky posted:problem I had was trying to start up a about 15 VMs at the same time off a single LUN backed by SATA disks, causing the box to lock up.
|
|
#
?
Apr 19, 2013 18:02
|
|
|
evil_bunnY posted:Enterprise loving storage, ladies and gents. I do what I can with the resources I'm given. Non-profit, rural area, constant budget cuts etc. It isn't the right way to do it and I know that. No need to be a dick about it.
|
|
#
?
Apr 19, 2013 18:34
|
|
|
Cpt.Wacky posted:I do what I can with the resources I'm given. Non-profit, rural area, constant budget cuts etc. It isn't the right way to do it and I know that. No need to be a dick about it. Sounded like he was more talking about the hardware/software itself.
|
|
#
?
Apr 19, 2013 20:49
|
|
|
|
| # ? May 6, 2024 13:06 |
|
|
Novo posted:I could use some information from any ZFS experts in this thread. I support 1.4PB of ZFS based storage across our two sites. In the field, we have over 4PB of ZFS storage. Our older systems are all based off NexantaStor and everything in the last year off OpenIndiana- both running Napp-it 0.8h. Half is linked via Infiniband to our cluster, the other half is archival/local storage. I just ordered another 100TB of disks/chassis, one will be for consolidating our VM's local storage and completely SSD based- the other two for local backup storage. I am very eager to see what 22x240gb Intel 520 series will do performance wise. The other two are 48 3TB 7200rpm spindles. We have suffered some pretty bad disk failures thanks to the dance club downstairs, but not once lost data. (24 failed drives in 12 months.) If you have any specifics, let me know. Everything is based on Supermicro dual quad xeons, 96gb+ of ram, LSI HBA's, and Seagate/Intel drives. the spyder fucked around with this message at 21:15 on Apr 19, 2013 |
|
#
?
Apr 19, 2013 21:12
|
|