|
So who does unified (NFS and iSCSI at a minimum) with synchronous replication i.e. P4000 but with NFS? Pipe dream? 
|
#
?
Mar 22, 2012 18:17
|
|
|
|
| # ? Apr 23, 2024 23:44 |
|
|
Bitch Stewie posted:So who does unified (NFS and iSCSI at a minimum) with synchronous replication i.e. P4000 but with NFS? Synchronous over what distance? And what is your RPO on the synchronous data? NetApp has a synchronous flavor of Snapmirror but it's obviously got some latency restrictions that you must be able to meet between the two sites for it to work. It gives you an RPO of within a few seconds, so it's not truly synchronous, though it is close enough to work for most applications. The other NetApp option would be a MetroCluster deployment, which is a true geographically dispersed cluster and provides absolutely synchronous replication, though again distance/latency limitations are a factor. I believe EMC also has some products that could provide it.
|
|
#
?
Mar 22, 2012 18:49
|
|
|
NippleFloss posted:Synchronous over what distance? And what is your RPO on the synchronous data? NetApp has a synchronous flavor of Snapmirror but it's obviously got some latency restrictions that you must be able to meet between the two sites for it to work. It gives you an RPO of within a few seconds, so it's not truly synchronous, though it is close enough to work for most applications. We have fibre on-site, distance would be around a mile, maybe a little less. P4000 is true synchronous and what we have now, it just doesn't do NFS (which we don't need but would be nice to have for VMware).
|
|
#
?
Mar 22, 2012 18:58
|
|
|
Bitch Stewie posted:We have fibre on-site, distance would be around a mile, maybe a little less. When I say truly synchronous with an RPO of zero I mean that when the link between the two sites drops all processing also stops completely. For a mirroring solution to be truly 100% synchronous data must be written and acknowledged at the secondary site before the client receives an acknowledgement that it's write request has completed. When your link is up between the two sites that works fine. However if the link goes down you CANNOT acknowledge the client write because there is no way to perform that write at the secondary site. So you either have to continue allowing write processing on the primary site and accept that the secondary will be out of sync for a period of time, or you must accept an outage while the link is down. The NetApp metrocluster solution will continue writes on the primary side if the link between the two sites is down. It prioritizes up-time over absolute equivalence between data at the two sites in this circumstance. The EMC solution I referenced works the opposite way. If the link between the two sites is lost then write processing stops at the primary site. As long as the link between your primary and secondary site is up and has suitable latency (and you certainly should given the distance you're talking about and fibre in between) then the NetApp metrocluster solution will be fully synchronous. The question is how you expect your primary site to behave when that link has issues.
|
|
#
?
Mar 22, 2012 19:20
|
|
|
With our P4000 we have a quorum "failover manager" at a third location - the SAN acts as a cluster so the links are redundant but if a site disappears the quorum manager gives quorum to the remaining site. I've not really started looking yet as the replacement's some time off, but I'd assumed all the alternatives would work in a similar manner. It's the only problem with the P4000 - it's cheap to the point where whilst it's just iSCSI, nothing else seems to come close at the price in redundancy terms.
|
|
#
?
Mar 22, 2012 19:31
|
|
|
NippleFloss posted:The CS240 is 24TB raw and 16TB usable. I believe you're thinking of the CS260, though you have the raw and usable numbers flipped. Those are reasonable prices, but they aren't knocking it out of the park on the GB/dollar axis. No, I meant CS240. Instead of usable I should have said "effective". http://www.nimblestorage.com/products/nimble-cs-series-family/ I have a CS240 and the compression on VMDKs exceeds the estimated 50% compression. your compression ratio may be different.
|
|
#
?
Mar 22, 2012 20:18
|
|
|
Bitch Stewie posted:With our P4000 we have a quorum "failover manager" at a third location - the SAN acts as a cluster so the links are redundant but if a site disappears the quorum manager gives quorum to the remaining site. This page should give you a rough idea of how some of the different SAN vendors handle this. http://dctools.blogspot.com/2011/06/long-distance-vmotion-part-2.html Based on your requirements I'd imagine MetroCluster would be a good fit but I'm not an expert on it. I'm not sure you'll get into any solution nearly as cheaply as you have with your Lefthand gear though. Like you said, it's very niche, but also very cheap. complex posted:No, I meant CS240. Instead of usable I should have said "effective". http://www.nimblestorage.com/produc...-series-family/ "Effective Storage" is marketing speak. There is no way to guarantee a compression ratio. The "usable" storage on an array is the amount available for filesystems, not the amount of data you can cram on those filesystems after running that data through compression or dedupe or whatnot. There's nothing wrong with Nimble pitching compression as a built in feature that will save you some amount of space, but including "Effective Storage" as an actual metric that they then compare to usable storage on other vendor hardware is goofy. YOLOsubmarine fucked around with this message at 20:53 on Mar 22, 2012 |
|
#
?
Mar 22, 2012 20:41
|
|
|
Beaten.
Nomex fucked around with this message at 21:07 on Mar 22, 2012 |
|
#
?
Mar 22, 2012 21:03
|
|
|
Nomex posted:Netapp does. Price will be the kicker. I was reading the link provider by NippleFloss and from the pricing I had on Netapp when we were looking prior to our current P4000 it's just way too expensive for us to justify. There are things I hate about our P4000 (mostly every time I have to deal with HP) but when you get down to the nitty gritty of being able to buy $6k's worth of SAN licenses, a pair of $2k servers (and of course switches and the trivial matter of fibre between locations), et voila you can have a storage cluster across two sites for $10k, bottom line is nothing's going to come close is it, I don't even know why I keep looking.
|
|
#
?
Mar 22, 2012 21:09
|
|
|
Bitch Stewie posted:Price will be the kicker. I was reading the link provider by NippleFloss and from the pricing I had on Netapp when we were looking prior to our current P4000 it's just way too expensive for us to justify. If you can automatically fail over your primary site using HP then there's no reason to switch. If you have to manually intervene to bring up your DR site, then the cost of that downtime might make the savings on the storage seem trivial.
|
|
#
?
Mar 22, 2012 21:12
|
|
|
Nomex posted:If you can automatically fail over your primary site using HP then there's no reason to switch. If you have to manually intervene to bring up your DR site, then the cost of that downtime might make the savings on the storage seem trivial. We can - it's pretty cool in that I can power down all the storage nodes (either intentionally or the time we had an extended power outage) in one of our computer rooms and nothing even notices. For us, sure I'd love to look at NFS, and Netapp make some cool software features (other probably do but they're the only vendor I really looked at that offers a software "suite") but bottom line is they're not things we could ever sell to the business, they'd make our lives a little more interesting as IT, but more at a geek level than anything sellable.
|
|
#
?
Mar 22, 2012 21:17
|
|
|
Sounds like a pretty sweet setup. Don't mess with it. NFS is cool but not worth compromising what you have.
|
|
#
?
Mar 22, 2012 21:59
|
|
|
complex posted:No, I meant CS240. Instead of usable I should have said "effective". http://www.nimblestorage.com/products/nimble-cs-series-family/ I have a CS220 and the compression on VMDK's far far far exceed 50% compression. EDIT: put succinctly when I was shopping for this thing: I sized out out pretty much to what we needed with about 2tb of "Growing space", with the assumption that compression may not work at all. Now I'm trying to find more data to huck on this thing because I've used about a quarter of what I was expecting to use. Rhymenoserous fucked around with this message at 22:05 on Mar 22, 2012 |
|
#
?
Mar 22, 2012 22:00
|
|
|
I just spent most of the day trying to setup a Microsoft Failover cluster using our Sun x4500 as the iSCSI initiator, only to discover that Solaris 10 (at least not update 8) doesn't support SCSI-3 commands over iSCSI, mainly persistent reservations. It looks like it's available in Solaris 11, but anyone know if it's in a subsequent version of Solaris 10?
|
|
#
?
Mar 22, 2012 22:04
|
|
|
FISHMANPET posted:I just spent most of the day trying to setup a Microsoft Failover cluster using our Sun x4500 as the iSCSI initiator, only to discover that Solaris 10 (at least not update 8) doesn't support SCSI-3 commands over iSCSI, mainly persistent reservations. I understand it doesn't like MPIO either (I could be wrong).
|
|
#
?
Mar 22, 2012 22:07
|
|
|
Internet Explorer posted:Sounds like a pretty sweet setup. Don't mess with it. NFS is cool but not worth compromising what you have. It is tbh. Right now we use the hardware nodes but we got an insane deal (around $23k) on a P4500 "Virtualisation Bundle" which also gives us 100tb of VSA licenses. When the lease comes up on the hardware I'm interested in buying it out and extending the support, and buying several 1U or 2U boxes, stuffing them full of drives and basically using the VSA's (fully supported on the VMware/Hyper-V HCL) on top of those servers. Up to 10 nodes/100tb of fully supported replicated highly available storage on HP/Dell commodity hardware, sounds great in theory, what could possibly go wrong.... 
|
|
#
?
Mar 22, 2012 22:11
|
|
|
You should see if leasing the equipment fits into your price range. Then you can toss the gear every 3 years and get new stuff, and all it ever costs is the same monthly fee. The best thing about the VSAs is you can install both the VSA and some of your servers on the same box.
|
|
#
?
Mar 22, 2012 23:06
|
|
|
FISHMANPET posted:I just spent most of the day trying to setup a Microsoft Failover cluster using our Sun x4500 as the iSCSI initiator, only to discover that Solaris 10 (at least not update 8) doesn't support SCSI-3 commands over iSCSI, mainly persistent reservations. Also, you appear to be confusing the nomenclature of "initiator" and "target."
|
|
#
?
Mar 22, 2012 23:34
|
|
|
Misogynist posted:SCSI-3 PGRs were introduced in the COMSTAR storage stack (which did not exist in Solaris 10) somewhere around OpenSolaris snv_118. Support will never be in Solaris 10, I'm sorry to say. Well good to know. I guess someday when we upgrade this thing to Solaris 11 I can move forward with this. Or we could just scrap it for a real SAN, we'll see. And why couldn't they just use client server naming? Initiator target is so confusing.
|
|
#
?
Mar 23, 2012 00:31
|
|
|
If you are in the DIY mood, buy one of these http://www.supermicro.com/products/nfo/sbb.cfm and slap nexenta/openfiler on it with some minor GUI changes. Tada you are now qualified to sell super low end NAS/SAN systems apparently.
|
|
#
?
Mar 23, 2012 03:10
|
|
|
FISHMANPET posted:Well good to know. I guess someday when we upgrade this thing to Solaris 11 I can move forward with this. Or we could just scrap it for a real SAN, we'll see.
|
|
#
?
Mar 23, 2012 03:28
|
|
|
ghostinmyshell posted:If you are in the DIY mood, buy one of these http://www.supermicro.com/products/nfo/sbb.cfm and slap nexenta/openfiler on it with some minor GUI changes. Tada you are now qualified to sell super low end NAS/SAN systems apparently. Those are the platform Nimble is built on. 
|
|
#
?
Mar 23, 2012 03:31
|
|
|
ghostinmyshell posted:If you are in the DIY mood, buy one of these http://www.supermicro.com/products/nfo/sbb.cfm and slap nexenta/openfiler on it with some minor GUI changes. Tada you are now qualified to sell super low end NAS/SAN systems apparently. I was just looking at that today (briefly). They're not exactly cheap though, and after you load them up with drives, you're possibly in the range for a low-end NetApp. Granted you could use a bunch of refurbished SAS drives or something and get it built for 4k, but stocking with new drives is basically going to double the cost of the unit (~6500) -- that's only for 4.8TB. I could use the interposer on my existing 320GB WD RE3 SATA drives, but they're still ~$50/ea. So for my budget it'd either be loading it with a bunch of refurb SAS drives or using existing SATA drives for the same cost. I'd go with the SAS and just order another dozen or so for spares haha.
|
|
#
?
Mar 23, 2012 05:48
|
|
|
Rolling your own SAN might become more viable after this release: ONTAP-V finally coming out! http://www.theregister.co.uk/2012/03/23/netapp_ontapv/
|
|
#
?
Mar 23, 2012 06:38
|
|
|
madsushi posted:Rolling your own SAN might become more viable after this release: ONTAP-V finally coming out! I was coming to post it.. How much is it going to be?
|
|
#
?
Mar 23, 2012 06:54
|
|
|
I was wondering when they were going to make the public knowledge. I'm quite looking forward to it, it's certainly going to be something I test thoroughly at home. Having said that, I've heard that it's fairly restrictive in what disk you can attach, it's mostly for converting ESXi hosts local disks into a "filer", for use in edge office scenarios. We'll see what happens I guess.
|
|
#
?
Mar 23, 2012 07:29
|
|
|
That does look interesting, but I can't help but think there will be some limitations that will prevent use as a primary SAN unless you're in quite a small environment. Presumably you'll still be having to license the add-on software, it won't be an "all-in" model?
|
|
#
?
Mar 23, 2012 17:51
|
|
|
There's no "all-in" physical filer.
|
|
#
?
Mar 23, 2012 17:57
|
|
|
True, but with my very limited dealings with Netapp the software add-ons were loving expensive - more than the hardware IIRC (which I might not). It seems a bit counter-intuitive to skew this even more, but I'm assuming a Netapp VSA would be significantly cheaper than a hardware Netapp, it may not be I guess.
|
|
#
?
Mar 23, 2012 18:13
|
|
|
Bitch Stewie posted:True, but with my very limited dealings with Netapp the software add-ons were loving expensive - more than the hardware IIRC (which I might not). No one knows what this is going to cost yet, or how they are going to bundle it. NetApp software does tend to be more expensive than the hardware, but that's because the software is really what provides value and requires the most development resources. The hardware is pretty much commodity stuff with a little special purpose fabrication. The current pricing for software is tied to the hardware that you will be running it on. So the same software license will be cheaper on a 2040 than it would be on a 6280. Since I expect that OnTAP-V will be considered a relatively low tier of "hardware" the software licenses should not be exorbitant. It's really meant to operate in edge offices or remote sites where a true SAN/NAS is overkill. Pushing into those areas opens up larger sales opportunities down the road once you've got the customer invested in and comfortable with the NetApp ecosystem. So while I don't expect OnTAP-V to be a loss leader I'd be surprised if it isn't priced fairly competitively.
|
|
#
?
Mar 23, 2012 18:24
|
|
|
NippleFloss, I can only hope that one day I'm half the articulate storage architect you are. But yeah, pretty much every NippleFloss said is correct. This is going to be for edge deployments on VMware servers, and it's a good move by NetApp. I say that because I, and likely NippleFloss and most other NetApp people, typically see companies happy to have largish central office NetApp deployments but balk at dropping 2xxx series out at edge offices, even though they desperately want to. This puts us in the retarded situation of dealing with cheap Dell, or Buffalo or Qnap lol, systems at edge sites. I just thank god that my company refuses to touch anything non-NetApp.
|
|
#
?
Mar 23, 2012 19:46
|
|
|
marketingman posted:NippleFloss, I can only hope that one day I'm half the articulate storage architect you are. This goes for a lot of people in the thread, honestly. I consider myself a decently smart guy who is pretty good at my job. Like most of you, I have a passion for it and I like doing it correctly. Storage and Virtualization are both incredibly complex topics, especially when you start talking about different vendors and different products. I wish I was half as knowledgeable as some of the people in this thread or the Virtualization thread. I would love to work on this stuff all day.
|
|
#
?
Mar 23, 2012 19:50
|
|
|
I emailed a NetApp engineer a couple questions about Ontap-V, from what I'm seeing here it doesn't sound like an option as a primary SAN, but it could potentially work well as a backup storage unit hosted at a colo. We need to eventually do both, so maybe this will make the second phase of our storage project cheaper.
|
|
#
?
Mar 23, 2012 20:57
|
|
|
Internet Explorer posted:Storage and Virtualization are both incredibly complex topics, especially when you start talking about different vendors and different products. I wish I was half as knowledgeable as some of the people in this thread or the Virtualization thread. I would love to work on this stuff all day. This is pretty much my goal. I'm on my third major virtualization project in the last 6 years. Every company I go to shall virtualize. Every.
|
|
#
?
Mar 23, 2012 21:03
|
|
|
Rhymenoserous posted:. Every company I go to shall virtualize. Every.
|
|
#
?
Mar 23, 2012 22:18
|
|
|
marketingman posted:I say that because I, and likely NippleFloss and most other NetApp people, typically see companies happy to have largish central office NetApp deployments but balk at dropping 2xxx series out at edge offices, even though they desperately want to. This is exactly the situation I'm in with my customer. They have around 60 data centers across the US all running NetApp hardware but they also have around 800 remote sites that are often nothing more than a trailer with a fractional T1 link and a Dell R710. Even a 2020 would be horribly impractical and expensive, but they currently have no reliable method to perform local backup nor to get those backups off-site. The hope is that we can throw ONTAP-V on them, virtualize the server, and use the VSC for backups and snapmirror with compression for off-site backup replication. That would get us closer to where we need to be, though available bandwidth will still be an issue. At least we would have a unified backup strategy instead of some places dumping data to a USB drive, and some dumping it to locally attached tape, and some trying to use OSSV, and some doing nothing at all about backup.
|
|
#
?
Mar 23, 2012 22:58
|
|
|
drat, and I thought I sat at the adult table, but that customer is twice the size of my largest. Regardless, this is exactly the same situation I'm facing in that customer as well. And it just repeats for almost every customer medium size and up. It's always been a traditional weakness that other vendors use as a foot in the door.
|
|
#
?
Mar 24, 2012 04:23
|
|
|
Has anyone here had much experience with Ceph, particularly on Ubuntu (look at a 12.04 x64 build specifically)? Beginning architecture design for a multi-node Ceph cluster with the aim of eventually slotting in to our OpenStack and would like to brain storm and share experiences with anyone else in regards to this.
|
|
#
?
Mar 24, 2012 07:01
|
|
|
Ugh, tonight I was doing some UPS maintenance and had one of the SAN slots down while I was moving in the new UPS, and the other slot decides to panic, taking down the whole thing. Ruined my night. It came back up fine, but more unexpected downtime isn't what I needed.
|
|
#
?
Mar 24, 2012 07:11
|
|
|
|
| # ? Apr 23, 2024 23:44 |
|
|
I talked to Nimble today. Their SAN is very interesting, though I'm not sure how much some of their performance and compression features are going to matter for our situation. I do like the ability to take 10,000+ snapshots and I like that when you update the controllers it will automatically take a controller off line apply the updates, then bring it back and repeat the process for the other controller. Their management software seems decent, it looks like they do all or most of the same things that NetApp does, albeit in what seemed to be a slightly clunkier way. As for cost, I think they said the street price for the 220 is $40k and they were very adament about being willing to do what it takes beat NetApp's price. They also offered to send out a SAN for us to try for 10 days, which we may do. I also talked to CDW, who we would be purchasing the NetApp equipment through. I gave them some more detailed information about what Nimble was willing to do with their price and also mentioned that we have some other equipment that we need and they're going to come back with a couple quotes and I'm curious to see what kind of discounts they will offer. I'm hoping that we'll get some pretty heavy discounts, which would make getting this project approved a lot easier. Aniki fucked around with this message at 01:22 on Mar 27, 2012 |
|
#
?
Mar 27, 2012 01:20
|
|