Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Internet Explorer
Jun 1, 2005





madsushi posted:

How big of an Exchange environment? 72x 15k disks could handle about 3,000 IOPS.

That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks.

Adbot
ADBOT LOVES YOU

Powdered Toast Man
Jan 25, 2005

TOAST-A-RIFIC!!!
What if they were SSDs? Just curious, I don't know much about SSD vs 15k performance.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Internet Explorer posted:

That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks.

Yea, it's low. We have 60 disk aggregates using 7200 rpm disks that provide more than 3k exchange IOPs at under 20ms, which is the magic number for 2010.

15k disks are going to be 2 to 3 times faster in random workloads like exchange.

The benefits you see from SSD are very workload dependent. For WAFL, specifically, you benefit the most from SSD on random reads. When workloads are sequential the throughput differences between SSD and 15k SAS aren't significant. You end up at a controller throughput bottleneck before a disk bottleneck in most cases. Since WAFL converts all write workloads into sequential writes (in a best case scenario where you have sufficient free spacE) SSD doesn't add much benefit to workloads that are already write heavy, or to workloads that are largely streaming reads.

Since Exchange 2010 has a sizable random read component SSD is certainly beneficial, though also prohibitively expensive and wasteful. A better approach is platter disks for capacity with an appropriately sized flash based cache, SSD tier, or SSD cache. Log writes and flushes stream to the platter disk at a high throughput while actual user reads are most often read from the cache layer, giving the best of both worlds.

YOLOsubmarine fucked around with this message at 22:16 on Jun 25, 2012

spoon daddy
Aug 11, 2004
Who's your daddy?
College Slice

Internet Explorer posted:

That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks.

That 10,000 is the raw disk capacity(standard approximation is 150 IOPs per 15k drive). With NVRAM cache layer writes should be higher though workload will dicate that. For reads, Flex cards can raise the bar even higher.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

spoon daddy posted:

That 10,000 is the raw disk capacity(standard approximation is 150 IOPs per 15k drive). With NVRAM cache layer writes should be higher though workload will dicate that. For reads, Flex cards can raise the bar even higher.

The NVRAM cache layer doesn't change the sustained performance characteristics of the disks. Ultimately all writes must hit disk (well, very nearly all writes) so putting cache in the middle does not increase sustained performance, it merely provides some smoothing for bursty I/O and allows writes to be acknowledged without waiting for them to reach disk.

On NetApp filers the NVRAM isn't really even cache in the sense that you mean it. NVRAM is only read in the case of a failure or takeover to apply any non-committed changes. In normal operation incoming writes are written to both NVRAM and system RAM with disk writes being served from system RAM. NVRAM is really just a very fast journal.

spoon daddy
Aug 11, 2004
Who's your daddy?
College Slice

NippleFloss posted:

The NVRAM cache layer doesn't change the sustained performance characteristics of the disks. Ultimately all writes must hit disk (well, very nearly all writes) so putting cache in the middle does not increase sustained performance, it merely provides some smoothing for bursty I/O and allows writes to be acknowledged without waiting for them to reach disk.

On NetApp filers the NVRAM isn't really even cache in the sense that you mean it. NVRAM is only read in the case of a failure or takeover to apply any non-committed changes. In normal operation incoming writes are written to both NVRAM and system RAM with disk writes being served from system RAM. NVRAM is really just a very fast journal.

Yeah, you are correct. I completely bungled that one. You'll never exceed your physical IOPs with NVRAM but you really do limit the impact of disk writes especially when disks that are not at full capacity. The NVRAM indeed acts like a journal so that when a write comes in, it's acknowledged once its written to NVRAM(faster) rather than wait til its written to disk(slower).

On the other hand, the Flex cache for reads can really improve your IOPs. I've seen as much as 20k IOPs offloaded to Flex cache on a 6080.

madsushi
Apr 19, 2009

Baller.
#essereFerrari

Internet Explorer posted:

That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks.

My bad, missed a 1, was supposed to be 13,000 IOPS. I always figure about 175 IOPS max for a 15k RPM drive, and 72x175 = 12,600.

For drive IOPS, I am usually referencing what I learned from a NetApp performance guy:

7.2K - 50-100 IOPS
10K - 100-150 IOPS
15K - 150-200 IOPS

Where the first number is "good to go" and the second number is "peak, latency will go up"

Nomex
Jul 17, 2002

Flame retarded.
I go by 180 for a 15k disk. We were able to get about 10,300 out of them before they maxed out. Suffice it to say 72 drives was woefully inadequate. We had to salvage 6 x DS14 15k disk shelves from one of our old filers to get things running smoother. It's still sub-optimal, but they've started moving mailboxes to 2010 now, so things are getting better every day.

madsushi
Apr 19, 2009

Baller.
#essereFerrari

Nomex posted:

I go by 180 for a 15k disk. We were able to get about 10,300 out of them before they maxed out. Suffice it to say 72 drives was woefully inadequate. We had to salvage 6 x DS14 15k disk shelves from one of our old filers to get things running smoother. It's still sub-optimal, but they've started moving mailboxes to 2010 now, so things are getting better every day.

Factoring in spares (1-3) and parity disks (8-10), 10,300 IOPS is actually pretty reasonable. However, you must have a massive Exchange environment if 10k IOPS wasn't adequate.

skipdogg
Nov 29, 2004
Resident SRT-4 Expert

I'm really curious as to how big this Exchange environment is as well. We outsourced exchange though and it's been wonderful. It's someoneelse's problem now.

paperchaseguy
Feb 21, 2002

THEY'RE GONNA SAY NO

Powdered Toast Man posted:

What's that you say? ANOTHER hosed UP SHARE? :shepicide:

His boss is back in town tomorrow. I can only hope...

well? :munch:

Rhymenoserous
May 23, 2008

skipdogg posted:

I'm really curious as to how big this Exchange environment is as well. We outsourced exchange though and it's been wonderful. It's someoneelse's problem now.

I'm trying to get this done now.

Nomex
Jul 17, 2002

Flame retarded.
Sorry guys, I can't go into to many specifics. It's a pretty large environment though.

Does anyone have any opinions on metro cluster? We're thinking of trying it out, but I don't know anyone whose used it before. We'll be running the heads about 9 km apart over a dedicated dwdm channel.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Nomex posted:

Sorry guys, I can't go into to many specifics. It's a pretty large environment though.

Does anyone have any opinions on metro cluster? We're thinking of trying it out, but I don't know anyone whose used it before. We'll be running the heads about 9 km apart over a dedicated dwdm channel.

As long as you've got the bandwidth and latency to handle it, it should work fine. Writes must commit on both sides of the cluster before they will be acknowledged to the client so very low latency on the inter-cluster link is a must. 9km is certainly well within the recommended limits.

Ask your sales team about getting some customer references for MetroCluster implementations. They should be able to come up with a few.

skipdogg posted:

I'm really curious as to how big this Exchange environment is as well. We outsourced exchange though and it's been wonderful. It's someone else's problem now.

Exchange 2003 requires around .75 to 1 IOP per user. He's probably in the 15,000 to 20,000 user range based on the amount of additional disk they needed to add to get to comfortable latencies.

Exchange 2010 is MUCH more efficient in it's disk utilizaiton. We have around 50,000 users with 3 copies of each MB database and we hit around 20,000 IOPs aggregated across the whole Exchange environment during peak hours.

YOLOsubmarine fucked around with this message at 22:46 on Jun 26, 2012

Nebulis01
Dec 30, 2003
Technical Support Ninny

Nomex posted:

Sorry guys, I can't go into to many specifics. It's a pretty large environment though.

If you go by the sizing guide here: http://technet.microsoft.com/en-us/library/bb125019%28v=exchg.65%29.aspx

Assuming medium performance on all 336 15k disks (150 IOPS/disk) and average (0.75 IOPS/user) workload he'd have roughly ~37,800 users. That's a pretty big exchange 2003 environment :)

Nukelear v.2
Jun 25, 2004
My optional title text
Anyone have ideas on why an EQL 6110XV might go non-responsive when doing 8k read IO? Seems to occur across all volumes on the unit. Doing some bench-marking with SQLIO and all my tests are good except for 8k random read. At 16k random read it was just under 16.5k iops so I thought maybe the switches were flooding so I dialed SQLIO back a single thread and it still dies.

We had a similar situation occur when I tried to format the volume, the volume became completely non-responsive to the initiator. EQL said Windows was trying to format with an 8k cluster size and that caused the volume to go non-responsive (no idea why). Recreated the volume and formatted with 64k explicitly (given the size of the disk windows auto detect should been using 64k in the first place)

2 x Powerconnect 8024F switches
2 x Broadcom 10G NICS w/ EQL MPIO in Win2008R2

Internet Explorer
Jun 1, 2005





Is it still under support? If so, I wouldn't hesitate to contact Dell. Their Equallogic support is generally pretty good.

Nukelear v.2
Jun 25, 2004
My optional title text

Internet Explorer posted:

Is it still under support? If so, I wouldn't hesitate to contact Dell. Their Equallogic support is generally pretty good.

It is and I shot an email off to our support gal. Wanted to see if there was something obvious I may be missing though.

Internet Explorer
Jun 1, 2005





Nukelear v.2 posted:

It is and I shot an email off to our support gal. Wanted to see if there was something obvious I may be missing though.

Okay, sorry. Wasn't trying to be smart. Even when I was hitting our EQL units with way more load then they were designed, I never had a LUN, let alone the entire unit go unresponsive. We were doing load where the latency was 2000+ ms and it never got so bad that the LUN stopped responding. What does SanHQ say when this happens? Are you running the latest firmware? Do you have more than 5% free space on the SAN?

The only time I had an issue similar to what you are saying also caused the management interface to become unresponsive, but that was on an older firmware and it was a rare bug where the UI would show that old snapshots were being deleted, but the space was not being freed up. The disks got completely full and support had to go into the shell on the unit (not even just command line, the linux shell on it) to fix the problem.

Internet Explorer
Jun 1, 2005





Alright, so I am posting this here because someone must have come across this problem before in their SAN/NAS travels.

We have an old file server that is running Windows 2003 that was laid out in a completely disorganized fashion. There is no documentation. I am trying to lay things out properly while moving the files over to our new file server, which is being handled via CIFs on our VNX 5300.

What I am looking for is a tool that records all files accessed on the old file server. My goal is to record everything, then slowly move things over while updating the dozens of apps and hundreds of scripts that use these files. It is going to be a complete nightmare, and my boss thinks it will be so simple because his solution would be to take the server named basically "fileserver2000" and just move the whole thing over as is to the NAS head on the VNX.

Any advice? I am really dreading this.

[Edit: I realize Windows does this natively but looking through the hundreds of thousands of event log entries wouldn't be fun.]

Powdered Toast Man
Jan 25, 2005

TOAST-A-RIFIC!!!
Lingering permissions issues continue.

"Yeah, we don't actually support auditing, mmmkay..."

They patched the loving kernel in their customized Linux OS on the appliance with custom code written just for us. :stare:

Oh, and it's a live patch. If the appliance reboots for any reason we're hosed. Weirdness like losing permissions that were already set continues. Every time I talk to him about it he digs himself into a deeper hole and there's no sign of a replacement that will be more reliable. Uh, the NAS, that is, not him. I have no idea about his status although I've heard that management isn't exactly happy and some people caught on to his responsibility in the matter and have been complaining.

The company is undergoing a huge PR crisis right now so fortunately for him, that has overshadowed things a bit. Supposedly a more permanent patch is coming but they keep finding new issues to add to it. Oh, and the patching is pretty loving stupid considering that the product was advertised as capable of doing all the things that it can't do.

TL;DR gently caress RELDATA/STARBOARD DO NOT BUY THEIR lovely PRODUCTS GET A REAL NAS (NETAPP).

Mierdaan
Sep 14, 2004

Pillbug

Internet Explorer posted:

Alright, so I am posting this here because someone must have come across this problem before in their SAN/NAS travels.

Varonis?

Internet Explorer
Jun 1, 2005





Thanks for the reply. Any idea on the price? Couldn't find anything on their site and the only mention I've seen online is $25,000+ which is way outside the ballpark.

ghostinmyshell
Sep 17, 2004



I am very particular about biscuits, I'll have you know.

Powdered Toast Man posted:

Lingering permissions issues continue.

"Yeah, we don't actually support auditing, mmmkay..."

They patched the loving kernel in their customized Linux OS on the appliance with custom code written just for us. :stare:

Oh, and it's a live patch. If the appliance reboots for any reason we're hosed. Weirdness like losing permissions that were already set continues. Every time I talk to him about it he digs himself into a deeper hole and there's no sign of a replacement that will be more reliable. Uh, the NAS, that is, not him. I have no idea about his status although I've heard that management isn't exactly happy and some people caught on to his responsibility in the matter and have been complaining.

The company is undergoing a huge PR crisis right now so fortunately for him, that has overshadowed things a bit. Supposedly a more permanent patch is coming but they keep finding new issues to add to it. Oh, and the patching is pretty loving stupid considering that the product was advertised as capable of doing all the things that it can't do.

TL;DR gently caress RELDATA/STARBOARD DO NOT BUY THEIR lovely PRODUCTS GET A REAL NAS (NETAPP).

I will never forgive you if my promised OS upgrade/fix from them is delayed because of this. I too would like to restart my NAS and not lose half their fixes.

Mierdaan
Sep 14, 2004

Pillbug

Internet Explorer posted:

Thanks for the reply. Any idea on the price? Couldn't find anything on their site and the only mention I've seen online is $25,000+ which is way outside the ballpark.

I wanna say maybe half that, but you also might be able to do what you need to do with the 30-day demo.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

NippleFloss posted:

Exchange 2003 requires around .75 to 1 IOP per user.
Trying not to be that guy, but IOPS isn't plural.

szlevi
Sep 10, 2010

[[ POKE 65535,0 ]]

Misogynist posted:

Trying not to be that guy, but IOPS isn't plural.

:D ".75 to 1 IO per per user"

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

szlevi posted:

:D ".75 to 1 IO per per user"
What Exchange user reads or writes from a disk one time and then never touches it ever again?

Mierdaan
Sep 14, 2004

Pillbug

Misogynist posted:

What Exchange user reads or writes from a disk one time and then never touches it ever again?

Obviously, anyone using S4 for their Exchange backend storage.

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Misogynist posted:

Trying not to be that guy, but IOPS isn't plural.
It's okay, you can be that guy, it was a goofy mistake. My brain saw S and wanted to singularize it.

Intraveinous
Oct 2, 2001

Legion of Rainy-Day Buddhists

Mierdaan posted:

Obviously, anyone using S4 for their Exchange backend storage.

That is outstanding.

Sylink
Apr 17, 2004

I'm trying to learn more about enterprise storage and the difference between a NAS and a SAN is hurting my brain more than it really should.

I get that a SAN is block level and a NAS is not, that makes sense.

But they are used in difference situations right?

Like typically a SAN would be plugged directly into a server or a few and used directly by the processes on said server(s), whereas a NAS is more like general storage (that could be used by a server) but is simply made available to users and programs via shares over the network while being managed by a server. Is that right?

evil_bunnY
Apr 2, 2003

Sylink posted:

I'm trying to learn more about enterprise storage and the difference between a NAS and a SAN is hurting my brain more than it really should.
There's much less of a difference today than there used to be. SAN used to be a dedicated fiber channel network for storage, NAS were storage devices attached to a plain old network (Ethernet mostly).

These days you've got "SANs" that only talk Ethernet, "SANs" that'll happily serve files (SAN used to mean block devices), etc.

Most people I've met use NAS for something that serves files, SAN for anything that can do block storage, or storage used mostly server side.

Sylink posted:

Like typically a SAN would be plugged directly into a server or a few and used directly by the processes on said server(s), whereas a NAS is more like general storage (that could be used by a server) but is simply made available to users and programs via shares over the network while being managed by a server. Is that right?
My employer's current SAN is plugged into a datacenter 10GBE network that moves all client-server and all hosts-storage traffic. The controller pair serves NFS to the virtualization hosts, and NFS/SMB to the users. It could do iSCSI (not the best in our situation) and FC (I'm not buying into SMB FC in 2012).

evil_bunnY fucked around with this message at 19:54 on Jul 3, 2012

Internet Explorer
Jun 1, 2005





^^^ :argh: leaving this wall of text ^^^

Sylink posted:

I'm trying to learn more about enterprise storage and the difference between a NAS and a SAN is hurting my brain more than it really should.

I get that a SAN is block level and a NAS is not, that makes sense.

But they are used in difference situations right?

Like typically a SAN would be plugged directly into a server or a few and used directly by the processes on said server(s), whereas a NAS is more like general storage (that could be used by a server) but is simply made available to users and programs via shares over the network while being managed by a server. Is that right?

Somewhat.

Really, the biggest difference is what you said originally. One is block-level and one is file-level.

A SAN is almost always connected to more than one device, otherwise it would be a DAS. The SAN side allows you to give block-level access to multiple devices over iSCSI or Fiber Channel. Generally, this allows you to create a LUN (block-level) and present it to a server, then that server formats it with it's own file system. In the case of Windows, that would be NTFS. VMware would be VMFS. You could also do EXT3, etc. [Edit: As a general rule of thumb, 1 LUN is presented to 1 host. This is because most file systems are not cluster-aware, or multi-access. NTFS will break if more than one host accesses it. The exception to this is cluster-aware file systems like VMFS or LVM.]

A NAS, which almost always used to come separately, now often times comes with the SAN. You usually create a few LUNs on the block-side, then present it to the NAS side. The NAS side formats it with whatever file system it is going to use (I think usually EXT3/4), but then serves it up over CIFs (SMB) or NFS. CIFs would be just like a normal file share, similar to what you'd make on a Windows server. NFS is a Unix-style file share, which Windows servers can access if you install the proper clients, but is more for use with virtualization. I'm sure someone else can chime in as I'm new to NFS, but from what I understand, from a virtualization point of view, the reason to use NAS functions like NFS as opposed to SAN functions like an iSCSI LUN, is that NFS supports file locking and journaling.

Hope that helps. My knowledge is fairly limited, but for a simple summary, that is how I understand it.

Internet Explorer fucked around with this message at 19:59 on Jul 3, 2012

YOLOsubmarine
Oct 19, 2004

When asked which Pokemon he evolved into, Kamara pauses.

"Motherfucking, what's that big dragon shit? That orange motherfucker. Charizard."

Technically a SAN is the network infrastructure that connects clients and arrays to provide block level storage from the latter to the former.

Most people in IT will call a disk array a SAN (myself included sometimes) but strictly speaking that is incorrect. A NAS, on the other hand, is the actual storage device.

Sylink
Apr 17, 2004

Thanks for the explanations, very helpful guys.

bull3964
Nov 18, 2000

DO YOU HEAR THAT? THAT'S THE SOUND OF ME PATTING MYSELF ON THE BACK.


Internet Explorer posted:

NTFS will break if more than one host accesses it.

NTFS does have some limited cluster support since 2008 R2 via Cluster Shared Volumes.

http://en.wikipedia.org/wiki/Cluster_Shared_Volumes

Intraveinous
Oct 2, 2001

Legion of Rainy-Day Buddhists

NippleFloss posted:

Technically a SAN is the network infrastructure that connects clients and arrays to provide block level storage from the latter to the former.

Most people in IT will call a disk array a SAN (myself included sometimes) but strictly speaking that is incorrect. A NAS, on the other hand, is the actual storage device.

This bugs me more than it should, especially when I fall victim to doing it so other people know what I'm talking about. You wouldn't call a server or a switch a LAN. :)

I won't say much else because I'd just be echoing the people above. Block vs File is the main differentiation, as has been more eloquently explained already.

evil_bunnY
Apr 2, 2003

gently caress you system manager


(can't resize any column either, and multiple config screens are blank and unresponsive). e: trying it on FF14, it doesn't even pop the config fields, which I guess is better than locking up the whole interface, but now all storage screen fields are empty.

This is after the webUI on the filer itself refused to ever work.

evil_bunnY fucked around with this message at 14:32 on Jul 4, 2012

Adbot
ADBOT LOVES YOU

Bluecobra
Sep 11, 2001

The Future's So Bright I Gotta Wear Shades
I had a fun day yesterday. We hit some bug on our FAS3210 in where the controller battery discharged to the point where it decided to shutdown both controllers in the middle of the day. :v:

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply