Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Boner Buffet
Feb 16, 2006


Great, thanks for the tips. Actually, I didn't even think of just using free ESXi at the DR site, which would free up a host if I ever want to add a third host to the 2 host production vmware cluster I'm planning. The essentials plus bundle is up to three hosts, two cpus per host, and six cores per CPU I believe, so burning that third host for a DR site seems like a waste to me.

Boner Buffet fucked around with this message at 12:41 on Jan 6, 2011

Adbot
ADBOT LOVES YOU

three
Aug 9, 2007

i fantasize about ndamukong suh licking my doodoo hole

1000101 posted:

I think you're walking down the right road for your budget. I assume the DR site is probably running free ESXi so you'll have to do a couple things:

1. Enable sshd/remote support shell on the ESXi server
2. Create yourself a handy recovery script to script bringing the volumes up! Now you have something you can type in, get a cup of coffee then come back.

Things you'll want to know how to use in the script:

Check with the VSA to figure out how to present your replicated LUN to your server. It probably entails breaking replication and making it writable. Ideally this can be scripted via the CLI.

Next some VMware specific stuff:
'esxcfg-volumes' -This command lets you tell VMware its okay to mount a replicated volume. You'll want to let it resignature the LUNs in question.

'esxcfg-rescan' -Use this to rescan the iSCSI initiator after you present the LUN and allow for a re-sign (don't recall if this is 100% required in 4.X anymore, the last script I wrote for this was for 3.5.)

Since its ESXi you're going to want to fiddle about in 'vmware-vim-cmd' (this is a way to get into the VMware management agent via the CLI) and feed it arguments via a find of .vmx files in your new datastore after the re-scan.

At this point you can actually use vmware-vim-cmd to power everything on for you and answer the question "hey did you move this or copy this?" (you probably just want to say you moved it.)

I had to build something like this for one of my customers who's outsourced IT is probably worse than herding cats. I use ssh key authentication for everything and all some guy has to do is run "StartDR.sh" and the script does everything he needs.

Did you charge him ~$5k per 25 VMs?

Intrepid00
Nov 10, 2003

I'm tired of the PMs asking if I actually poisoned kittens, instead look at these boobies.

what is this posted:

Yes and not to rehash this discussion for the 20th time but it's more like "only oracle hosts supports database filesystem writes over NFS." And even then it's only in their rigorously controlled environment where they validate all the parts of the NFS chain.

For almost every database you want block level storage, whether that be an iSCSI LUN presented to the OS, fibre channel, or direct attached storage.


Here's a good read on NFS VS iSCSI too.

http://lass.cs.umass.edu/papers/pdf/FAST04.pdf

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


adorai posted:

HA and DR are completely different concepts.

Exactly, that's why I said HA - apparently you got confused by the name "DR site"...

...or perhaps missed the most important part of his post, namely

The piece I really need to dig into is how I would then make that DR site live.

quote:

If you want an easy, drop-in, push one button for DR solution in a VMware environment with all VMDKs you would probably use SRM. It's like $3k a proc.

Did you even read his post? I doubt it - he has no money and he wants to make the DR site live, not recover from it...

...you know, HA vs DR.

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


three posted:

It's Per-VM now, I believe.

That's the killer part - he barely has money for one VSA ($3k or less.)

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


InferiorWang posted:

Thanks for the feedback. Manual Intervention is fine assuming it's something that can be properly documented and done by a trained , again, even if it takes an hour or two to get up and running.

There would have to be a real disaster like the building burning down or being pulled into the gamma quadrant through a wormhole for me to go to the redundant box, which is why I'm looking at VSA as a cheaper alternative over a second P4300 which may never have to be put into production.

That's why I am saying that your P4300 + VSA + (free) FOM gives you remote HA, without a push of a button. I don't run VMware so I cannot help you there - I rather spend my money on better/safer hardware, better backup etc things instead of giving it to EMC for things that are free in Hyper-V or (some) even in XenServer.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


szlevi posted:

That's why I am saying that your P4300 + VSA + (free) FOM gives you remote HA, without a push of a button. I don't run VMware so I cannot help you there - I rather spend my money on better/safer hardware, better backup etc things instead of giving it to EMC for things that are free in Hyper-V or (some) even in XenServer.
Sorry, but SRM features aren't free in any product, unless you think all SRM does is send a "power up" command to a pile of VMs at another site.

Syano
Jul 13, 2005


szlevi posted:

I don't run VMware so I cannot help you there - I rather spend my money on better/safer hardware, better backup etc things instead of giving it to EMC for things that are free in Hyper-V or (some) even in XenServer.

I dont even understand what this means?

adorai
Nov 2, 2002

10/27/04 Never forget

Grimey Drawer

szlevi posted:


Did you even read his post? I doubt it - he has no money and he wants to make the DR site live, not recover from it...

...you know, HA vs DR.
he wants to know how to initiate a dr failover, making his dr site live in the event of a disaster.

Mausi
Apr 11, 2006



szlevi posted:

I rather spend my money on better/safer hardware, better backup etc things instead of giving it to EMC for things that are free in Hyper-V or (some) even in XenServer.

Pray tell, sir goon, of these wondrous free availability features of Hyper-V which do not exist mostly in Xen and certainly in VMware?


1000101 is on the money; Once you've got either the live data or the image-level backups replicated to the DR site, you have to break the replication, bring them into a bootable configuration and power the drat things up.
The only real way to do this reliably is to directly reverse the process which got them there in the first place, but all too often budgets don't allow for that - best advice I can give you is EVERY time you deviate from your production kit at the DR site be drat sure you know the impact.
Also Cover Your rear end when it comes to your boss taking on the risk that it all doesn't work, be clear and concise with exactly how reliable this system they've paid for is.

Boner Buffet
Feb 16, 2006


Mausi, I'm not sure I'm following this:

quote:

EVERY time you deviate from your production kit at the DR site be drat sure you know the impact.

The production SAN wouldn't be at the DR site, unless that's a typo.

Overall, after reading a little more about VSA, it seems like it's not exactly what I was thinking it was. It seems to be geared for vmware, not just to duplicate the P4000 series SAN on cheaper hardware.

I'm not just using the production SAN for vmware data stores. I'm going to have iscsi luns on it that three or four cluster nodes will be talking to. Those cluster nodes will happen to be virtual machines, but the critical data they "serve up" will be located on basic iscsi accessible luns...nothing to do with vmware directly.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


InferiorWang posted:

The production SAN wouldn't be at the DR site, unless that's a typo.
No, it's the entire point of the post. You're not running your production SAN at your DR site. Fully and completely understand every single difference between the two SANs, and the consequences of those differences, if you want your DR plan to work well and not bite you with untimely little surprises like "oh, hey, turns out we're only licensed for 16 storage partitions instead of 64 at our DR site. Whoops!"

Boner Buffet
Feb 16, 2006


After reading it in that context, it makes sense now.

The local inside rep at HP refuses to call me back despite leaving multiple messages. I think it would be worthwhile looking to an outside source to help me out with this, especially if I can't get anything from HP.

three
Aug 9, 2007

i fantasize about ndamukong suh licking my doodoo hole

InferiorWang posted:

After reading it in that context, it makes sense now.

The local inside rep at HP refuses to call me back despite leaving multiple messages. I think it would be worthwhile looking to an outside source to help me out with this, especially if I can't get anything from HP.

Call his boss and complain.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


InferiorWang posted:

The local inside rep at HP refuses to call me back despite leaving multiple messages. I think it would be worthwhile looking to an outside source to help me out with this, especially if I can't get anything from HP.
HP is funny in that in lots of cases, you might get better pricing from a VAR anyway. Give CDW or someone else a call while you wait.

Boner Buffet
Feb 16, 2006


I dropped an email to a vmware rep who has been helpful in the past. I'll give my CDWG rep a shout too. It's funny you mention that because I've read people say to stay away from CDW in these cases. However, I'm not in the market for a huge rear end fibre channel san spending a quarter of a million dollars which might have been the context of those posts.

Thanks for suggestion.

Crowley
Mar 13, 2003


Pulled the trigger on a 162TB HP X9720. We're going to be mirroring 2x60GB blocks for online media streaming and taking regular backups of the rest. When WORM gets supported (around Q2 says HP) we'll take that over the mirror, but for now we'll make do. We made room for an extra rack next to it too, just in case we get exited and need another sooner than expected. :haw:

I'll post a trip report in a few months.

Edit: Those storage blocks are seriously compact!

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


InferiorWang posted:

I dropped an email to a vmware rep who has been helpful in the past. I'll give my CDWG rep a shout too. It's funny you mention that because I've read people say to stay away from CDW in these cases. However, I'm not in the market for a huge rear end fibre channel san spending a quarter of a million dollars which might have been the context of those posts.

Thanks for suggestion.
CDW, like most VARs, varies highly in price depending on the quality and aggressiveness of your rep. It never hurts to get a quote.

conntrack
Aug 8, 2003

by angerbeet


You need to be worthy (spend millions of dollars) before the the HP sales gods(in their own mind) deign to grant you an audience.

Boner Buffet
Feb 16, 2006


When I say introduce myself and say I'm from a K-12 school district the excitement of potentially large sales dissipates rapidly. Our CDW rep is good because he knows we're not spending any money on anything extra and doesn't bother pushing us. Some vendors and vars don't get that local school districts around me don't have the money to buy whatever they're trying to sell.

The decision making process is basically like this. How much? Just about nothing else matters.

Syano
Jul 13, 2005


For someone that works with HPs lefthand products could you answer a question for me: Since the kit, when using network raid, abstracts the fact that the data is being stored across multiple units, does the kit allow you to build mirrored or raid 5 luns or anything like that? Or do you basically just set up network raid and then carve out a lun and let the kit decide what to do with it? Or did I just completely not make any sense?

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH




Syano posted:

For someone that works with HPs lefthand products could you answer a question for me: Since the kit, when using network raid, abstracts the fact that the data is being stored across multiple units, does the kit allow you to build mirrored or raid 5 luns or anything like that? Or do you basically just set up network raid and then carve out a lun and let the kit decide what to do with it? Or did I just completely not make any sense?

You can set different Network Raid levels. You can set it up so it can be tolerant of multiple simultaneous node failures if you want to but putting a volume in Network Raid 5 mode. I believe there's Network Raid 6 as well (I'd have to go look in the management console.)

Different volumes can have different levels set on them, so you can prioritize your availability on a volume by volume basis.

Syano
Jul 13, 2005


Number19 posted:

Different volumes can have different levels set on them, so you can prioritize your availability on a volume by volume basis.

Oh thats pretty killer. Cool this is what I needed to know. Now to make a purchase!

wang souffle
Apr 26, 2002


Is there a recommendation on mixing SAS and SATA drives in a ZFS build? I assume highly unrecommended, but how else can you use SAS hard drives for primary storage and SATA SSD drives for ZIL/L2ARC? The SATA-->SAS interposer solution sounds pretty hacky.

Bluecobra
Sep 11, 2001

The Future's So Bright I Gotta Wear Shades

wang souffle posted:

Is there a recommendation on mixing SAS and SATA drives in a ZFS build? I assume highly unrecommended, but how else can you use SAS hard drives for primary storage and SATA SSD drives for ZIL/L2ARC? The SATA-->SAS interposer solution sounds pretty hacky.
AFAIK, SAS controllers are downwards comparable with SATA disks but not the other way around. At work I have a Franken-open-storage clone that consists of an X4100 server + an external J4400 SATA storage away. On the X4100 I have two SAS disks for the OS and two Intel SATA SSD drives for the L2ARC cache. The Intel SSDs share the same SAS controller with the SAS disks and I have no problems with this configuration.

edit: I should also add that the X4100 uses two external SAS controllers with multipathing enabled to talk to the J4400 array.

Bluecobra fucked around with this message at 00:11 on Jan 13, 2011

conntrack
Aug 8, 2003

by angerbeet


The issue is mixing spindles of different quality/speed. Having a pool with a vdev that is a lot slower than the others will kill the speed of the entire pool.

As you are not doing this you are in the clear.

Unless you put the ssd drives on some $10 sata controller you found in the garbage.

wang souffle
Apr 26, 2002


conntrack posted:

The issue is mixing spindles of different quality/speed. Having a pool with a vdev that is a lot slower than the others will kill the speed of the entire pool.

As you are not doing this you are in the clear.

Unless you put the ssd drives on some $10 sata controller you found in the garbage.

Nope, would be on some solid SAS controllers. I'm wondering why some setups (like this one) use SATA-->SAS interposers. Is it because of the multipathing?

Speaking of multipathing, why do I care?

skipdogg
Nov 29, 2004
Resident SRT-4 Expert





New toys just showed up. Too bad they're not for me :(

EoRaptor
Sep 13, 2003





wang souffle posted:

Speaking of multipathing, why do I care?

If a controller fails, those drives don't vanish, they are picked up by the other controller. Access is universally slower now, but you stay up and data should stay consistent.

There are several ways this can be set up (active/passive, active/active, smart backplane, etc) that each have performance and cost tradeoffs.

qutius
Apr 2, 2003
NO PARTIES


EoRaptor posted:

If a controller fails, those drives don't vanish, they are picked up by the other controller. Access is universally slower now, but you stay up and data should stay consistent.

There are several ways this can be set up (active/passive, active/active, smart backplane, etc) that each have performance and cost tradeoffs.

MPIO protects against more than just a controller failing and the partner taking over disk ownership, which is more of an high availability thing.

MPIO will protect against any failure on the fabric - so target/initiator port, cable, switch, switch port, etc. Some of these failures would be protected against HA too, but MPIO is needed at the driver level too.

But maybe I'm splitting hairs...

ragzilla
Sep 9, 2005
don't ask me, i only work here




skipdogg posted:




New toys just showed up. Too bad they're not for me :(

Make sure when they cable up the backend loops, you don't put a both ends of a backend loop on the same ASIC of a controller (adjacent ports), try to keep them on separate cards if you have multiple FC cards for backend.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


wang souffle posted:

Nope, would be on some solid SAS controllers. I'm wondering why some setups (like this one) use SATA-->SAS interposers. Is it because of the multipathing?

Speaking of multipathing, why do I care?
SAS supports multi-lane cables which are designed to hook up numerous drives to a single larger backplane, SATA does not, and this is true of connections to both internal backplanes and external enclosures. Most vendors have no intention of supporting systems which require 36/48/60 individual cables from the hard drives to the RAID controllers.

SATA does have port multipliers, but these are very rarely seen in enterprise-grade storage platforms because they're bottlenecked to the speed of a single SATA connection. I don't know how relevant this is now that we're at SATA-3, though.

Multipathing is definitely a big reason on larger enclosures, but if you're small enough to not be looking at SAN storage, there's usually not enough of an availability requirement to use SAS for that alone, though.

Vulture Culture fucked around with this message at 13:20 on Jan 14, 2011

Boner Buffet
Feb 16, 2006


I started to spec up a server to run the VSA software on. We're a Dell server shop, so I figured I'd stick with them. I've asked for a quote for an R710 with embedded ESXi 4.1, the free one. I'm looking at doing a simple RAID5 SATA 7.2k array. Since this would be DR and only the most critical services would go live, I'm guessing using SATA isn't a horrible choice in this case. No oracle or mssql except for a small financial package using mssql, which has 10 users at the most at any one time. GroupWise(no laughing) and all of our Novell file servers would be brought online too. 32GB of RAM. Anyone see anything completely wrong with that hardware setup?

Also, our Dell rep has a bad habit of ignoring what you write in an email. I gave him the equote# and asked to change the support options and to NOT give me the promotion pricing he mentioned. So, he gives me a quote with the wrong support option and with the promotion pricing.

Xenomorph
Jun 13, 2001


Pretty basic I guess, but I just ordered a Dell NX3000 NAS, loaded with 2TB drives.

I was looking at a storage option that offered Fibre Channel, since all of our old Apple Xraids used that. I've seen iSCSI recommended a lot. I'm not familiar with all that stuff.
I just know the NX3000 let me pick standard 7200 RPM SATA drives (so I can pick up an Enterprise WD or Segate off Newegg as replacements), and had it's own OS to share stuff over Ethernet, without requiring another computer to manage it.

The issues I've had with existing storage are the types of drives (Xraids use now-difficult to replace IDE drives), and the type of connection (we had to dedicate one system to fibre channel connections to get access to half a dozen storage units).

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.


Xenomorph posted:

The issues I've had with existing storage are the types of drives (Xraids use now-difficult to replace IDE drives), and the type of connection (we had to dedicate one system to fibre channel connections to get access to half a dozen storage units).
The issues you've had with existing storage were mostly due to you ignoring SH/SC telling you to replace them a year ago and finding bizarre justifications to keep trucking along on your broken and unsupportable kit, IIRC

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


Misogynist posted:

Sorry, but SRM features aren't free in any product, unless you think all SRM does is send a "power up" command to a pile of VMs at another site.

Err,

1. FOM is exactly for that, right, not to manage anything....
2. ...maybe you're confusing it with P4000 CMC?
3. Wait, that's free too...

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


adorai posted:

he wants to know how to initiate a dr failover, making his dr site live in the event of a disaster.

Which is exactly HA, with a remote node, right.
DR would be if he would recover from it, as in Disaster Recovery.

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


Mausi posted:

Pray tell, sir goon, of these wondrous free availability features of Hyper-V which do not exist mostly in Xen and certainly in VMware?

They exist but not for free. :)

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


InferiorWang posted:

After reading it in that context, it makes sense now.

The local inside rep at HP refuses to call me back despite leaving multiple messages. I think it would be worthwhile looking to an outside source to help me out with this, especially if I can't get anything from HP.

HP do this all the time, I now only work with channel - they better at literally everything, from pushing down prices to getting back to me in time and with what I wanted (instead of something else.)

Adbot
ADBOT LOVES YOU

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]


InferiorWang posted:

I dropped an email to a vmware rep who has been helpful in the past. I'll give my CDWG rep a shout too. It's funny you mention that because I've read people say to stay away from CDW in these cases. However, I'm not in the market for a huge rear end fibre channel san spending a quarter of a million dollars which might have been the context of those posts.

Thanks for suggestion.

Try Insight or PC Mall, they both work great for me.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply