Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Moey
Oct 22, 2010

I LIKE TO MOVE IT

Docjowles posted:

The HP MSA 2000 line is super entry level, comparable to Dell's md3200. I wouldn't be surprised if the SAS ports are for host connectivity (in addition to expansion shelves).

With the Dell units, the i after denotes iSCSI (MD3200i MD3220i). Their controllers have ethernet ports for iSCSI an one for management. Their SAS ports are used for adding on disk shelves (MD1200, MD1220). The units without the i (MD3200, MD3220) are DAS units. Instead of the iSCSI ports, they have SAS ports to connect directly to hosts.

I don't use HP storage really, but it looks like they have similar options.

Adbot
ADBOT LOVES YOU

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

They actually make SAS switches. Well, like one company does, anyway.

Wicaeed
Feb 8, 2005
:woop:

Finally got approval for our Nimble project (2x CS300 shelves). Can't wait to get them in house and start testing!

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

If anyone is in the Portland area and is NetApp savvy and wants a job working there, they will be looking to back fill my position. Shoot me a PM if interested.

Internet Explorer
Jun 1, 2005





In regards to SAS DASes -

But they do allow for multiple hosts to connect using Shared Storage. They are really nice for smaller environments, actually.

Internet Explorer fucked around with this message at 22:48 on Sep 11, 2014

Klenath
Aug 12, 2005

Shigata ga nai.

Docjowles posted:

The HP MSA 2000 line is super entry level, comparable to Dell's md3200. I wouldn't be surprised if the SAS ports are for host connectivity (in addition to expansion shelves).

Interesting. It looks like this thing allows you to switch host ports so it can be either direct connect SAS or more SAN-like via FC or iSCSI. There's a separate SAS port on the storage processors for shelf expansion. It appears you just order the appropriate module for the host connectivity you need. Dell's MD3000-series is SAS-only unless there's an "i" at the end indicating iSCSI host connectivity.


Moey posted:

With the Dell units, the i after denotes iSCSI (MD3200i MD3220i). Their controllers have ethernet ports for iSCSI an one for management. Their SAS ports are used for adding on disk shelves (MD1200, MD1220). The units without the i (MD3200, MD3220) are DAS units. Instead of the iSCSI ports, they have SAS ports to connect directly to hosts.

Quite correct. We have an old MD3000i for two file servers' user data storage and for one application server's user data storage. Expanded it a few years ago with a MD1000 via its SAS ports. Works pretty well and we've had no issues aside from an occasional failed disk, which happens. No major component failure and firmware updates went smoothly. We opted for Dell-assisted firmware updates to be safe, which was quite useful when they did a major version firmware update a few years ago. We have a number of MD1000-series units connected to various servers via RAID cards as well. None of the MD-series of shelves does FC - they have Compellent for FC.

The difference between the MD1000-series and MD3000-series is the MD1000-series are dumb shelves while the MD3000-series have RAID controllers in them. This offloads RAID processing from a host controller to the array and lets you put dumb SAS controllers in the servers instead of RAID controllers with external ports.


Internet Explorer posted:

In regards to SAS DASes -

But they do allow for multiple hosts to connect using Shared Storage. They are really nice for smaller environments, actually.

I know the Dell MD1000/3000-series can be put into "split" mode so you can hook up two hosts to one MD array. I don't know if the HP units can do this.

Internet Explorer
Jun 1, 2005





Klenath posted:

Interesting. It looks like this thing allows you to switch host ports so it can be either direct connect SAS or more SAN-like via FC or iSCSI. There's a separate SAS port on the storage processors for shelf expansion. It appears you just order the appropriate module for the host connectivity you need. Dell's MD3000-series is SAS-only unless there's an "i" at the end indicating iSCSI host connectivity.


Quite correct. We have an old MD3000i for two file servers' user data storage and for one application server's user data storage. Expanded it a few years ago with a MD1000 via its SAS ports. Works pretty well and we've had no issues aside from an occasional failed disk, which happens. No major component failure and firmware updates went smoothly. We opted for Dell-assisted firmware updates to be safe, which was quite useful when they did a major version firmware update a few years ago. We have a number of MD1000-series units connected to various servers via RAID cards as well. None of the MD-series of shelves does FC - they have Compellent for FC.

The difference between the MD1000-series and MD3000-series is the MD1000-series are dumb shelves while the MD3000-series have RAID controllers in them. This offloads RAID processing from a host controller to the array and lets you put dumb SAS controllers in the servers instead of RAID controllers with external ports.


I know the Dell MD1000/3000-series can be put into "split" mode so you can hook up two hosts to one MD array. I don't know if the HP units can do this.

On the Dell MD3200 series you can actually connect up to 4 hosts, shared storage, with active/active controllers. For someone just looking at a small VMware Essentials Plus network with no-frills needed, it's great.

Richard Noggin
Jun 6, 2005
Redneck By Default

Internet Explorer posted:

On the Dell MD3200 series you can actually connect up to 4 hosts, shared storage, with active/active controllers. For someone just looking at a small VMware Essentials Plus network with no-frills needed, it's great.

I'll offer a counter opinion: Dell md32xx series arrays suck. They're overpriced, offer a crappy management solution (really? a 1GB download just to get management software, PLUS having to upgrade said 1GB package with every firmware release?), and only offer block. For the same or less money you can get a VNXe3150 or 3200 with unified storage and built-in management. In casual observance, the EMCs perform better than the Dells too.

Oh - did I mention that Dell acknowledges there is a very real possibility of data corruption during firmware updates? Yeah...not impressed.

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Wicaeed posted:

:woop:

Finally got approval for our Nimble project (2x CS300 shelves). Can't wait to get them in house and start testing!
I also got approval for my Nimble purchase today. I am also getting 2x CS300 units.

Internet Explorer
Jun 1, 2005





Richard Noggin posted:

I'll offer a counter opinion: Dell md32xx series arrays suck. They're overpriced, offer a crappy management solution (really? a 1GB download just to get management software, PLUS having to upgrade said 1GB package with every firmware release?), and only offer block. For the same or less money you can get a VNXe3150 or 3200 with unified storage and built-in management. In casual observance, the EMCs perform better than the Dells too.

Oh - did I mention that Dell acknowledges there is a very real possibility of data corruption during firmware updates? Yeah...not impressed.

I hadn't heard about the data corruption issues. For that dumb of a device, it's usually update firmware before install and that's it. You save a bit on redundant switches and all that, and 6 Gb/s without having to do anything fancy. Plus it just works.

skooky
Oct 2, 2013

Richard Noggin posted:

I'll offer a counter opinion: Dell md32xx series arrays suck. They're overpriced, offer a crappy management solution (really? a 1GB download just to get management software, PLUS having to upgrade said 1GB package with every firmware release?), and only offer block. For the same or less money you can get a VNXe3150 or 3200 with unified storage and built-in management. In casual observance, the EMCs perform better than the Dells too.

Oh - did I mention that Dell acknowledges there is a very real possibility of data corruption during firmware updates? Yeah...not impressed.


Pretty much everything you posted just then is wrong.

Management software does not need to be updated with every firmware upgrade.

Having probably done 100s of MD3xxx firmware upgrades, I have not come across an issue once. The "possibility" is standard CYA stuff.

Klenath posted:

I know the Dell MD1000/3000-series can be put into "split" mode so you can hook up two hosts to one MD array. I don't know if the HP units can do this.

With the MD3000 series the split mode switch is ignored. It has 2 Active controllers so you can have up to 4 Hosts Single Path or 2 Hosts Redundant.

skooky fucked around with this message at 04:20 on Sep 12, 2014

JockstrapManthrust
Apr 30, 2013

NippleFloss posted:

If anyone is in the Portland area and is NetApp savvy and wants a job working there, they will be looking to back fill my position. Shoot me a PM if interested.

Where are you moving on to? Thank you for your candid views on netapps tech and direction, its been very useful.

Richard Noggin
Jun 6, 2005
Redneck By Default

skooky posted:

Pretty much everything you posted just then is wrong.

Management software does not need to be updated with every firmware upgrade.

Having probably done 100s of MD3xxx firmware upgrades, I have not come across an issue once. The "possibility" is standard CYA stuff.


Interesting. Every one I've done has resulted in having to upgrade the array manager after. I've had several Dell techs speak adamantly of the "possibility", and with enough conviction that we don't even think about doing the upgrade with any sort of IO to the SPs. EMC is plenty happy with active upgrades. But hey, it's really just Ford vs Chevy. :iiaca:

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

JockstrapManthrust posted:

Where are you moving on to? Thank you for your candid views on netapps tech and direction, its been very useful.

I'm going to a VAR to do pre and post-sales, and teach training classes. They're a NetApp partner, so I'll still be involved with NetApp, but I'll also get to sell and work with Nimble and Tintri, which should be fun. More VMware and Cisco hands on time, as well.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

So I promised a detailed post on how NetApp handles deletes and why it takes a while for free space to show up when processing large deletes, so here it is.

Because of the fairly high level of virtualization required for FlexVols there are a number of extra files that maintain information about the state storage allocation in FlexVols and containing aggregates. At the aggregate level you have one bitmap file that traps each block used in the active filesystem, one bitmap file that describes the active blocks held in any aggregate snapshot, and one bitmap file that tracks blocks used in either the active filesystem or any one snapshot, which is constructed by doing an logical OR for each bit in the active and snapshot bitmaps. These bitmaps are called the active map, the snapmaps, and the summary map. If a block is marked as used in the summary map then it is not available for allocating to a new write, and it shows up in the used space calculations for the aggregate. A FlexVol is really just a file in an aggregate on which a WAFL filesystem is placed, sort of like a loop device. So you've got a WAFL filesystem virtualized on a file living inside an aggregate that is also running WAFL. That means that there is a file, known as the container file, within the aggregate, that represents the Flexvol, and contains all of the blocks allocated to that FlexVol. Each FlexVol has a container file, so you can have many of these within an aggregate.

In addition to the container file, each FlexVol also contains it's own set of active, snap, and summary maps, since it needs to track block utilization within the FlexVol as well. A block within a volume is only loosely coupled to the back end storage, which is what allows blocks to change their physical location constantly via things like out of place overwrites, de-duplication, reallocation, and other processes. So FlexVol needs to know which of it's own block it is using (identified by the VVBN, or virtual volume block number) and the aggregate needs to know which PVBNs (physical volume block number) is associated with each VVBN so that it can mark that block as used and write it out to the proper location on disk. That's a lot of different files to maintain and keep relatively consistent, and because these files live within the aggregate and FlexVol respectively, they are self-referential. The blocks used to store the active map file within the aggregate are represented by bits in the active map, so whenever a block changes within the filesystem and the active map needs to be updated the changes to the active map file also need to be represented in the new active map. So if a large amount of data is deleted from a volume the active map file in the FlexVol needs to be updated to reflect that these blocks are now free. That update means overwriting blocks in the active map, and since WAFL never overwrites in place that means allocating new blocks for the active map and de-allocating the old ones, and representing all of this in the new active map, which may mean changing yet other blocks in the active map and rewriting those and so on...this can add up to a lot of extra read and write work on the map files for the volume. Then, once the volume has updated it's active maps the aggregate still needs to be made aware that those flexvol blocks are no longer in use so that it can a) update it's own active and summary map files and b) punch holes in the container file so that the the free blocks are no longer represented there and can be re-allocated elsewhere. Until all of the map file and container map updates have been completed that space can't be re-allocated to another write, so it shows as used.

Now we could just do all of this work synchronously: you delete a file and the filer updates the active map and summary map for the flexvol, then the aggregate updates it's map files and punches holes in the container file, all immediately when requested, and before returning the delete. But that could lead to hundreds of random writes (container and map access is always random) just to service deleting a file. It gets horribly inefficient when you're doing lots of delete work and the write work required to keep up can cause performance problems. So instead of freeing those blocks immediately (and really this is anything from a file truncate to a snapshot delete to an overwrite, since all of those things free blocks) we journal them in a log file, batch them up, and process them over time. There are some triggers to determine when we start processing them. When more than 1% of the blocks in the volume are delayed frees sitting in the delete log we sort them, determine how to most efficiently do the work with a relatively small amount of writes, and then clear them in batches. The goal is have the background work keep the amount of delayed free work between 1 and 2% of the total aggregate size.

Initially delete logging only happened on aggregates since the bulk of the work was container file updates. The volume active map was updated synchronously (at the next CP, really) and the aggregate work was delayed. But as volumes have grown the size of the block allocation map files got substantially larger and it became inefficient to update them synchronously, so there is a delete log at the volume level as well. So when you delete a huge amount of data in a volume (say, delete a large VMDK) it gets dumped into a volume delete log, which churns through it and updates the maps over time, and the free space comes in as that happens, and then those newly freed blocks are processed at the aggregate level means that the newly freed space starts to show up there as well.

This explanation ignore the refcount file used for dedupe work, which is another filesystem metadata file that needs to be updated during delete work, and adds another level of write overhead for deletes.

There are probably some particulars here that are out of date or slightly wrong, but this is the basic explanation for how free space is managed on NetApp, and why deletes of files or snapshots do not necessarily show up immediately in the free space output.

Syano
Jul 13, 2005
I have for my entire career used nothing but Clariion storage from EMC but I accepted a position earlier this year with an organization that uses a VMAX 40k. I have a couple folks in the storage group who know the VMAX well and they help out and explain things when they can but the architecture is different enough so that there is still a ton for me to learn. I would love to do some self study on these things and start flexing a bit but its like the training material on VMAX is non-existent, save for EMC authorized couses which cost a month's salary. Does anyone know of any other material I may be able to get my hands on?

the spyder
Feb 18, 2011
I figured I would toss this here: I know it's not EMC, Netapp, or Nimble, but I'm prepping 1.8PB of whitebox storage this week. Any testing you guys would like to see before it ships next week? They are very simple and designed to be nothing more then a rather large NAS. The only consumer part are the SSD's. This goes along side a whitebox processing array containing 240 cores, 2.5TB ram, and 20 Xeon Phi's. I'm going to be building another system in the near future, with E5 V3/SAS3/Intel SSD's, so I'm curious how they will compare.

Processing specs (dedicated system for running a custom application in):
2u Supermicro 24 2.5" chassis
Dual E5 V2 Xeon
512GB
LSI 9300-8e HBA's
Supermicro HBA's for the 24 2.5' bays
Mellanox Connectx3 40Gb's Infiniband
OS/ZIL/L2ARC SSD's
(20) 1TB Samsung 850 Pro SSD's
(90) 4TB WD RE4'S
(2) 847 45 bay JBOD's

Archival spec's (handles raw data/finished product) x2:
Same as above minus the (24) 1TB SSD's and with double the 9300-8e/JBOD's + 180 4TB RE4's each.

(Initially I did not want to use the RE4's, but you try buying 480 4TB HDD's with two weeks notice. Initial choice was a 4TB SAS drive. Same for the SSD's, Intel 3700's were the first choice, but unavailable.)

in a well actually
Jan 26, 2011

dude, you gotta end it on the rhyme

the spyder posted:

I figured I would toss this here: I know it's not EMC, Netapp, or Nimble, but I'm prepping 1.8PB of whitebox storage this week. Any testing you guys would like to see before it ships next week? They are very simple and designed to be nothing more then a rather large NAS. The only consumer part are the SSD's. This goes along side a whitebox processing array containing 240 cores, 2.5TB ram, and 20 Xeon Phi's. I'm going to be building another system in the near future, with E5 V3/SAS3/Intel SSD's, so I'm curious how they will compare.

Processing specs (dedicated system for running a custom application in):
2u Supermicro 24 2.5" chassis
Dual E5 V2 Xeon
512GB
LSI 9300-8e HBA's
Supermicro HBA's for the 24 2.5' bays
Mellanox Connectx3 40Gb's Infiniband
OS/ZIL/L2ARC SSD's
(20) 1TB Samsung 850 Pro SSD's
(90) 4TB WD RE4'S
(2) 847 45 bay JBOD's

Archival spec's (handles raw data/finished product) x2:
Same as above minus the (24) 1TB SSD's and with double the 9300-8e/JBOD's + 180 4TB RE4's each.

(Initially I did not want to use the RE4's, but you try buying 480 4TB HDD's with two weeks notice. Initial choice was a 4TB SAS drive. Same for the SSD's, Intel 3700's were the first choice, but unavailable.)

Linux, FreeBSD, or Illumos/Solaris?

Mr Shiny Pants
Nov 12, 2012
Some benchmarks would be cool, especially if you could compare Solaris ZFS based workloads to Linux ZFS based workloads.

I am wondering if the ZFS performance differences between Linux and Solaris are really large.

Aquila
Jan 24, 2003

If need basic HA NFS what are my options beyond Netapp? Also what's the current basic netapp model for this, FAS2500? I'm looking to get ~20TB usable in ~6U, 3.5" nearline sas is ok. No flash (besides nvram cache type stuff) or other fance high performance stuff.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Aquila posted:

If need basic HA NFS what are my options beyond Netapp? Also what's the current basic netapp model for this, FAS2500? I'm looking to get ~20TB usable in ~6U, 3.5" nearline sas is ok. No flash (besides nvram cache type stuff) or other fance high performance stuff.
In that range, you're probably looking at IBM V7000 Unified or Dell EqualLogic FS7600.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Aquila posted:

If need basic HA NFS what are my options beyond Netapp? Also what's the current basic netapp model for this, FAS2500? I'm looking to get ~20TB usable in ~6U, 3.5" nearline sas is ok. No flash (besides nvram cache type stuff) or other fance high performance stuff.

Yea, you'd be looking at 2552/2554 for NetApp. Oracle ZS3 boxes also do NFS and HA for relatively cheap.

adorai
Nov 2, 2002

10/27/04 Never forget
Grimey Drawer

Aquila posted:

If need basic HA NFS what are my options beyond Netapp? Also what's the current basic netapp model for this, FAS2500? I'm looking to get ~20TB usable in ~6U, 3.5" nearline sas is ok. No flash (besides nvram cache type stuff) or other fance high performance stuff.
We have invested about $75k into each of our Oracle ZFS appliances, roughly 22TB usable on each before compression. 6U, 2.5" SAS, lots of cache.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug
So a VNXe 3200 is basically an encapsulated vSAN that routes IP and FC into a centralized storage device?

I was more impressed with someone adhering to me saying that 15-10k disks had use behind Fast Cache, where you have to understand the data's cache life via the cold data IOPS, than EMC doing encapsulated vSAN/vVol.

chutwig
May 28, 2001

BURLAP SATCHEL OF CRACKERJACKS

Anyone here use Tegile? We are looking to consolidate our Pure Storage + NetApp installations, and on paper the T3400/T3800 have everything we're looking for in a single platform: inline dedupe+compression, replication, NFS. We'll likely arrange to get a test unit to evaluate, but does anyone have any good/bad things to say about Tegile in particular?

Docjowles
Apr 9, 2009

the spyder posted:

I figured I would toss this here: I know it's not EMC, Netapp, or Nimble, but I'm prepping 1.8PB of whitebox storage this week. Any testing you guys would like to see before it ships next week? They are very simple and designed to be nothing more then a rather large NAS. The only consumer part are the SSD's. This goes along side a whitebox processing array containing 240 cores, 2.5TB ram, and 20 Xeon Phi's. I'm going to be building another system in the near future, with E5 V3/SAS3/Intel SSD's, so I'm curious how they will compare.

Processing specs (dedicated system for running a custom application in):
2u Supermicro 24 2.5" chassis
Dual E5 V2 Xeon
512GB
LSI 9300-8e HBA's
Supermicro HBA's for the 24 2.5' bays
Mellanox Connectx3 40Gb's Infiniband
OS/ZIL/L2ARC SSD's
(20) 1TB Samsung 850 Pro SSD's
(90) 4TB WD RE4'S
(2) 847 45 bay JBOD's

Archival spec's (handles raw data/finished product) x2:
Same as above minus the (24) 1TB SSD's and with double the 9300-8e/JBOD's + 180 4TB RE4's each.

(Initially I did not want to use the RE4's, but you try buying 480 4TB HDD's with two weeks notice. Initial choice was a 4TB SAS drive. Same for the SSD's, Intel 3700's were the first choice, but unavailable.)

This is cool and my coworker would be :circlefap:ing so hard over this. He's really, really into Ceph and to build a multi-petabyte whitebox solution on top of it would be pretty much his dream project. We currently have many petabytes of <big vendor> storage so it's theoretically possible, but sadly not in the cards any time soon. Management still wants the proverbial throat to choke, and "the sysadmins" don't spit out enough dollars when you choke us ;)

edit: nm you obviously indicated ZFS

Docjowles fucked around with this message at 06:13 on Oct 1, 2014

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

chutwig posted:

Anyone here use Tegile? We are looking to consolidate our Pure Storage + NetApp installations, and on paper the T3400/T3800 have everything we're looking for in a single platform: inline dedupe+compression, replication, NFS. We'll likely arrange to get a test unit to evaluate, but does anyone have any good/bad things to say about Tegile in particular?

Supported companies with it, it's basically what you get with equallogic with some FC capabilities.

Honestly all flash arrays aren't mature yet, they have they IOPS and write/read that wow people; but the manipulation of that data isn't stable from the VMUGs and people I hear implementing them. For the next few years go with a hybrid array like Nimble, Compellent, EMC, and to some extent netApp.

Also, don't design a storage solution on the basis of dedupe/compression will hold all your data and expansions of that.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Dilbert As gently caress posted:

So a VNXe 3200 is basically an encapsulated vSAN that routes IP and FC into a centralized storage device?

I was more impressed with someone adhering to me saying that 15-10k disks had use behind Fast Cache, where you have to understand the data's cache life via the cold data IOPS, than EMC doing encapsulated vSAN/vVol.

VNXe isn't very architecturally similar to VSAN at all.

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

NippleFloss posted:

VNXe isn't very architecturally similar to VSAN at all.

The underlying software feels like it. But the poo poo up to the software layer, run som VIP's and conversions. WHOA it's a virtual san that is hardware agnostic. Doesn't UCS chassis encapsulate/emplement basically the same poo poo with IP traffic via FCoE? I imagine VNXe 3200 is doing similar, but only abstracting the HW of raid, protocol layers, and such to the software.

http://www.vhersey.com/2014/09/vnxe3200-first-customer-install-fc-and-iscsi-access-to-the-same-lun/

We both talked "yeah that's probably what it is doing under it's skin".


Honestly would love to hear how it is working underneath, if it isn't doing a vSAN(abstracting the HW and pushing protocols and such to the software to make a unified solution); but that is all I can figure.

Dilbert As FUCK fucked around with this message at 07:27 on Oct 1, 2014

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Dilbert As gently caress posted:

The underlying software feels like it. But the poo poo up to the software layer, run som VIP's and conversions. WHOA it's a virtual san that is hardware agnostic. Doesn't UCS chassis encapsulate/emplement basically the same poo poo with IP traffic via FCoE? I imagine VNXe 3200 is doing similar, but only abstracting the HW of raid, protocol layers, and such to the software.

http://www.vhersey.com/2014/09/vnxe3200-first-customer-install-fc-and-iscsi-access-to-the-same-lun/

We both talked "yeah that's probably what it is doing under it's skin".


Honestly would love to hear how it is working underneath, if it isn't doing a vSAN(abstracting the HW and pushing protocols and such to the software to make a unified solution); but that is all I can figure.

FC and iSCSI are just encapsulating SCSI commands so it's not particularly tricky to allow hosts to access the same data using both. The data access layer in.The array doesn't need to know or care how the SCSI command got there, it just needs to know to read or write whatever block.

Netapp has supported this for at least a decade. It's just what you get with a unified storage system. All IO, no matter what protocol it uses, is translated into a system call for the underlying filesystem that the array runs so protocol support really just means translating the protocol semantics into native system calls. In the case of iSCSI and FC that's trivial since both are SCSI underneath.

I still don't see what it has to do with VSAN though, other than that both them abstract away the physical disks underneath? But then, like, every SAN in the world does that.

1000101
May 14, 2003

BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY BIRTHDAY FRUITCAKE!

Dilbert As gently caress posted:

Doesn't UCS chassis encapsulate/emplement basically the same poo poo with IP traffic via FCoE? I imagine VNXe 3200 is doing similar, but only abstracting the HW of raid, protocol layers, and such to the software.

A UCS chassis provides power and plumbing to blades and not a whole lot else. The intelligence all happens in the fabric interconnect and no it doesn't do anything with IP traffic via FCoE. IP Traffic gets stuffed into ethernet frames just like every other server vendor on the planet. FCoE is just wrapping fibre channel frames with ethernet and putting it in a 'no drop' QoS class to guarantee delivery.

Once the FCoE frame reaches the FI it can (depending on if it's end host or in switched fabric mode):

- Be sent out a native FC uplink to a fibre channel switch to deliver to wherever (where it strips off the ethernet header since it isn't needed anymore)

- Be sent out of an ethernet uplink via FCoE

- Be handed off to the FCF services on the FI where it'll go through the usual process of FCP delivery


quote:

Honestly would love to hear how it is working underneath, if it isn't doing a vSAN(abstracting the HW and pushing protocols and such to the software to make a unified solution); but that is all I can figure.

How you get data to the controller doesn't have to have anything to do with how the controller is actually storing and retrieving the data. You're going to take a SCSI command and stick it in either an iSCSI packet or FC frame and send it out the most appropriate HBA. Your storage network will take said packet and deliver it to the controller who's going to read out that SCSI command and do something fancy with it. In most cases a LUN is probably just a file (or set of files) on a set of backend disks somewhere being shuffled around however the controller sees fit.


edit for clarity.

KS
Jun 10, 2003
Outrageous Lumpwad

I'm surprised this is even article worthy, because I thought this was just a given. Compellent has done this for years. So has Netapp. So has 3Par. There's no special sauce required here.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

KS posted:

I'm surprised this is even article worthy, because I thought this was just a given. Compellent has done this for years. So has Netapp. So has 3Par. There's no special sauce required here.
Pretty sure you could even do this in COMSTAR in an off-the-shelf OpenSolaris box since 2008 or so.

Wicaeed
Feb 8, 2005
Woo!

I have twins!!



edit: oh god tables

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Wicaeed posted:

Woo!

I have twins!!



edit: oh god tables

Nice! Now please order some faceplates for those Dell servers on the right.

Are you doing scale out or just separate arrays?

devmd01
Mar 7, 2006

Elektronik
Supersonik
Brocade 320s, zoning is hosed on both. Typical theoretical setup of dual switches, dual controllers on the compellent, etc.

I know what I need to do to remediate, but it would be easiest if I can build out new aliases/zones on a per server basis instead of server groups and activate those before I deactivate the current ones.

Can I do this? Everything attached is mpio aware.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

devmd01 posted:

Brocade 320s, zoning is hosed on both. Typical theoretical setup of dual switches, dual controllers on the compellent, etc.

I know what I need to do to remediate, but it would be easiest if I can build out new aliases/zones on a per server basis instead of server groups and activate those before I deactivate the current ones.

Can I do this? Everything attached is mpio aware.

Assuming everything is dual-pathed through both fabrics, and MPIO is configured properly on all of your servers, then sure, you can fix one half of the fabric and then the other without taking everything down. When you run cfgenable you may see IO disruption on that fabric though, so be sure you've actually got multiple paths to all servers and that you can fail non-disruptively between them.

KS
Jun 10, 2003
Outrageous Lumpwad
Good time to move to virtual ports mode on the Compellent if for some reason you haven't. Also remember to create two zones containing just the physical front end ports, and just the virtual front end ports if you haven't.

devmd01
Mar 7, 2006

Elektronik
Supersonik
That's the problem, neither switch is consistent. Ultimately only three servers out of 15 are correctly multi-pathed, the rest are single path due to the hosed up zoning or even fiber cables disconnected. I inherited this mess and I already want to stab anyone who ever touched these. Guess i'm gonna have to do the long haul of documenting everything before I start making changes.

I barely had any fiber switch experience before this and I apparently already know more about how to do things right than the idiots who set it/the servers attached up.

E: yeah, compellent is set up with virtual wwns, at least that part is right.

devmd01 fucked around with this message at 21:31 on Oct 3, 2014

Adbot
ADBOT LOVES YOU

Dilbert As FUCK
Sep 8, 2007

by Cowcaster
Pillbug

1000101 posted:

A UCS chassis provides power and plumbing to blades and not a whole lot else. The intelligence all happens in the fabric interconnect and no it doesn't do anything with IP traffic via FCoE. IP Traffic gets stuffed into ethernet frames just like every other server vendor on the planet. FCoE is just wrapping fibre channel frames with ethernet and putting it in a 'no drop' QoS class to guarantee delivery.

Once the FCoE frame reaches the FI it can (depending on if it's end host or in switched fabric mode):

- Be sent out a native FC uplink to a fibre channel switch to deliver to wherever (where it strips off the ethernet header since it isn't needed anymore)

- Be sent out of an ethernet uplink via FCoE

- Be handed off to the FCF services on the FI where it'll go through the usual process of FCP delivery
correct


quote:

How you get data to the controller doesn't have to have anything to do with how the controller is actually storing and retrieving the data. You're going to take a SCSI command and stick it in either an iSCSI packet or FC frame and send it out the most appropriate HBA. Your storage network will take said packet and deliver it to the controller who's going to read out that SCSI command and do something fancy with it. In most cases a LUN is probably just a file (or set of files) on a set of backend disks somewhere being shuffled around however the controller sees fit.


edit for clarity.

Correct again, My thought wasn't the "vSAN" we know of in vmware but the layer of abstracting the disk Writes/Reads into a file, pushing the I/O to the application layer then treating it all the same to the HW. Which in term could be seen as virtualization of the Storage, it just got a new label and finally others are promoting it.


The more I think about it, the more I realize "gently caress this ain't nothing new; it's just manipulating data to the HW layer".

Misogynist posted:

Pretty sure you could even do this in COMSTAR in an off-the-shelf OpenSolaris box since 2008 or so.

You can do this poo poo with Openfiler the more I think about it...

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply