|
madsushi posted:How big of an Exchange environment? 72x 15k disks could handle about 3,000 IOPS. That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks.
|
# ? Jun 25, 2012 21:40 |
|
|
# ? Mar 29, 2024 01:03 |
|
What if they were SSDs? Just curious, I don't know much about SSD vs 15k performance.
|
# ? Jun 25, 2012 22:03 |
|
Internet Explorer posted:That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks. Yea, it's low. We have 60 disk aggregates using 7200 rpm disks that provide more than 3k exchange IOPs at under 20ms, which is the magic number for 2010. 15k disks are going to be 2 to 3 times faster in random workloads like exchange. The benefits you see from SSD are very workload dependent. For WAFL, specifically, you benefit the most from SSD on random reads. When workloads are sequential the throughput differences between SSD and 15k SAS aren't significant. You end up at a controller throughput bottleneck before a disk bottleneck in most cases. Since WAFL converts all write workloads into sequential writes (in a best case scenario where you have sufficient free spacE) SSD doesn't add much benefit to workloads that are already write heavy, or to workloads that are largely streaming reads. Since Exchange 2010 has a sizable random read component SSD is certainly beneficial, though also prohibitively expensive and wasteful. A better approach is platter disks for capacity with an appropriately sized flash based cache, SSD tier, or SSD cache. Log writes and flushes stream to the platter disk at a high throughput while actual user reads are most often read from the cache layer, giving the best of both worlds. YOLOsubmarine fucked around with this message at 22:16 on Jun 25, 2012 |
# ? Jun 25, 2012 22:04 |
|
Internet Explorer posted:That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks. That 10,000 is the raw disk capacity(standard approximation is 150 IOPs per 15k drive). With NVRAM cache layer writes should be higher though workload will dicate that. For reads, Flex cards can raise the bar even higher.
|
# ? Jun 25, 2012 22:41 |
|
spoon daddy posted:That 10,000 is the raw disk capacity(standard approximation is 150 IOPs per 15k drive). With NVRAM cache layer writes should be higher though workload will dicate that. For reads, Flex cards can raise the bar even higher. The NVRAM cache layer doesn't change the sustained performance characteristics of the disks. Ultimately all writes must hit disk (well, very nearly all writes) so putting cache in the middle does not increase sustained performance, it merely provides some smoothing for bursty I/O and allows writes to be acknowledged without waiting for them to reach disk. On NetApp filers the NVRAM isn't really even cache in the sense that you mean it. NVRAM is only read in the case of a failure or takeover to apply any non-committed changes. In normal operation incoming writes are written to both NVRAM and system RAM with disk writes being served from system RAM. NVRAM is really just a very fast journal.
|
# ? Jun 25, 2012 22:52 |
|
NippleFloss posted:The NVRAM cache layer doesn't change the sustained performance characteristics of the disks. Ultimately all writes must hit disk (well, very nearly all writes) so putting cache in the middle does not increase sustained performance, it merely provides some smoothing for bursty I/O and allows writes to be acknowledged without waiting for them to reach disk. Yeah, you are correct. I completely bungled that one. You'll never exceed your physical IOPs with NVRAM but you really do limit the impact of disk writes especially when disks that are not at full capacity. The NVRAM indeed acts like a journal so that when a write comes in, it's acknowledged once its written to NVRAM(faster) rather than wait til its written to disk(slower). On the other hand, the Flex cache for reads can really improve your IOPs. I've seen as much as 20k IOPs offloaded to Flex cache on a 6080.
|
# ? Jun 25, 2012 23:20 |
|
Internet Explorer posted:That sounds very low. Should be closer to 10,000 IOPS for 72 15k disks. My bad, missed a 1, was supposed to be 13,000 IOPS. I always figure about 175 IOPS max for a 15k RPM drive, and 72x175 = 12,600. For drive IOPS, I am usually referencing what I learned from a NetApp performance guy: 7.2K - 50-100 IOPS 10K - 100-150 IOPS 15K - 150-200 IOPS Where the first number is "good to go" and the second number is "peak, latency will go up"
|
# ? Jun 26, 2012 00:16 |
|
I go by 180 for a 15k disk. We were able to get about 10,300 out of them before they maxed out. Suffice it to say 72 drives was woefully inadequate. We had to salvage 6 x DS14 15k disk shelves from one of our old filers to get things running smoother. It's still sub-optimal, but they've started moving mailboxes to 2010 now, so things are getting better every day.
|
# ? Jun 26, 2012 06:09 |
|
Nomex posted:I go by 180 for a 15k disk. We were able to get about 10,300 out of them before they maxed out. Suffice it to say 72 drives was woefully inadequate. We had to salvage 6 x DS14 15k disk shelves from one of our old filers to get things running smoother. It's still sub-optimal, but they've started moving mailboxes to 2010 now, so things are getting better every day. Factoring in spares (1-3) and parity disks (8-10), 10,300 IOPS is actually pretty reasonable. However, you must have a massive Exchange environment if 10k IOPS wasn't adequate.
|
# ? Jun 26, 2012 07:43 |
|
I'm really curious as to how big this Exchange environment is as well. We outsourced exchange though and it's been wonderful. It's someoneelse's problem now.
|
# ? Jun 26, 2012 17:43 |
|
Powdered Toast Man posted:What's that you say? ANOTHER hosed UP SHARE? well?
|
# ? Jun 26, 2012 19:23 |
|
skipdogg posted:I'm really curious as to how big this Exchange environment is as well. We outsourced exchange though and it's been wonderful. It's someoneelse's problem now. I'm trying to get this done now.
|
# ? Jun 26, 2012 20:55 |
|
Sorry guys, I can't go into to many specifics. It's a pretty large environment though. Does anyone have any opinions on metro cluster? We're thinking of trying it out, but I don't know anyone whose used it before. We'll be running the heads about 9 km apart over a dedicated dwdm channel.
|
# ? Jun 26, 2012 21:47 |
|
Nomex posted:Sorry guys, I can't go into to many specifics. It's a pretty large environment though. As long as you've got the bandwidth and latency to handle it, it should work fine. Writes must commit on both sides of the cluster before they will be acknowledged to the client so very low latency on the inter-cluster link is a must. 9km is certainly well within the recommended limits. Ask your sales team about getting some customer references for MetroCluster implementations. They should be able to come up with a few. skipdogg posted:I'm really curious as to how big this Exchange environment is as well. We outsourced exchange though and it's been wonderful. It's someone else's problem now. Exchange 2003 requires around .75 to 1 IOP per user. He's probably in the 15,000 to 20,000 user range based on the amount of additional disk they needed to add to get to comfortable latencies. Exchange 2010 is MUCH more efficient in it's disk utilizaiton. We have around 50,000 users with 3 copies of each MB database and we hit around 20,000 IOPs aggregated across the whole Exchange environment during peak hours. YOLOsubmarine fucked around with this message at 22:46 on Jun 26, 2012 |
# ? Jun 26, 2012 22:33 |
|
Nomex posted:Sorry guys, I can't go into to many specifics. It's a pretty large environment though. If you go by the sizing guide here: http://technet.microsoft.com/en-us/library/bb125019%28v=exchg.65%29.aspx Assuming medium performance on all 336 15k disks (150 IOPS/disk) and average (0.75 IOPS/user) workload he'd have roughly ~37,800 users. That's a pretty big exchange 2003 environment
|
# ? Jun 26, 2012 22:35 |
|
Anyone have ideas on why an EQL 6110XV might go non-responsive when doing 8k read IO? Seems to occur across all volumes on the unit. Doing some bench-marking with SQLIO and all my tests are good except for 8k random read. At 16k random read it was just under 16.5k iops so I thought maybe the switches were flooding so I dialed SQLIO back a single thread and it still dies. We had a similar situation occur when I tried to format the volume, the volume became completely non-responsive to the initiator. EQL said Windows was trying to format with an 8k cluster size and that caused the volume to go non-responsive (no idea why). Recreated the volume and formatted with 64k explicitly (given the size of the disk windows auto detect should been using 64k in the first place) 2 x Powerconnect 8024F switches 2 x Broadcom 10G NICS w/ EQL MPIO in Win2008R2
|
# ? Jun 26, 2012 23:19 |
|
Is it still under support? If so, I wouldn't hesitate to contact Dell. Their Equallogic support is generally pretty good.
|
# ? Jun 26, 2012 23:26 |
|
Internet Explorer posted:Is it still under support? If so, I wouldn't hesitate to contact Dell. Their Equallogic support is generally pretty good. It is and I shot an email off to our support gal. Wanted to see if there was something obvious I may be missing though.
|
# ? Jun 26, 2012 23:31 |
|
Nukelear v.2 posted:It is and I shot an email off to our support gal. Wanted to see if there was something obvious I may be missing though. Okay, sorry. Wasn't trying to be smart. Even when I was hitting our EQL units with way more load then they were designed, I never had a LUN, let alone the entire unit go unresponsive. We were doing load where the latency was 2000+ ms and it never got so bad that the LUN stopped responding. What does SanHQ say when this happens? Are you running the latest firmware? Do you have more than 5% free space on the SAN? The only time I had an issue similar to what you are saying also caused the management interface to become unresponsive, but that was on an older firmware and it was a rare bug where the UI would show that old snapshots were being deleted, but the space was not being freed up. The disks got completely full and support had to go into the shell on the unit (not even just command line, the linux shell on it) to fix the problem.
|
# ? Jun 26, 2012 23:37 |
|
Alright, so I am posting this here because someone must have come across this problem before in their SAN/NAS travels. We have an old file server that is running Windows 2003 that was laid out in a completely disorganized fashion. There is no documentation. I am trying to lay things out properly while moving the files over to our new file server, which is being handled via CIFs on our VNX 5300. What I am looking for is a tool that records all files accessed on the old file server. My goal is to record everything, then slowly move things over while updating the dozens of apps and hundreds of scripts that use these files. It is going to be a complete nightmare, and my boss thinks it will be so simple because his solution would be to take the server named basically "fileserver2000" and just move the whole thing over as is to the NAS head on the VNX. Any advice? I am really dreading this. [Edit: I realize Windows does this natively but looking through the hundreds of thousands of event log entries wouldn't be fun.]
|
# ? Jun 27, 2012 00:09 |
|
Lingering permissions issues continue. "Yeah, we don't actually support auditing, mmmkay..." They patched the loving kernel in their customized Linux OS on the appliance with custom code written just for us. Oh, and it's a live patch. If the appliance reboots for any reason we're hosed. Weirdness like losing permissions that were already set continues. Every time I talk to him about it he digs himself into a deeper hole and there's no sign of a replacement that will be more reliable. Uh, the NAS, that is, not him. I have no idea about his status although I've heard that management isn't exactly happy and some people caught on to his responsibility in the matter and have been complaining. The company is undergoing a huge PR crisis right now so fortunately for him, that has overshadowed things a bit. Supposedly a more permanent patch is coming but they keep finding new issues to add to it. Oh, and the patching is pretty loving stupid considering that the product was advertised as capable of doing all the things that it can't do. TL;DR gently caress RELDATA/STARBOARD DO NOT BUY THEIR lovely PRODUCTS GET A REAL NAS (NETAPP).
|
# ? Jun 27, 2012 02:32 |
|
Internet Explorer posted:Alright, so I am posting this here because someone must have come across this problem before in their SAN/NAS travels. Varonis?
|
# ? Jun 27, 2012 02:35 |
|
Thanks for the reply. Any idea on the price? Couldn't find anything on their site and the only mention I've seen online is $25,000+ which is way outside the ballpark.
|
# ? Jun 27, 2012 03:12 |
|
Powdered Toast Man posted:Lingering permissions issues continue. I will never forgive you if my promised OS upgrade/fix from them is delayed because of this. I too would like to restart my NAS and not lose half their fixes.
|
# ? Jun 27, 2012 08:00 |
|
Internet Explorer posted:Thanks for the reply. Any idea on the price? Couldn't find anything on their site and the only mention I've seen online is $25,000+ which is way outside the ballpark. I wanna say maybe half that, but you also might be able to do what you need to do with the 30-day demo.
|
# ? Jun 27, 2012 13:06 |
|
NippleFloss posted:Exchange 2003 requires around .75 to 1 IOP per user.
|
# ? Jun 27, 2012 15:42 |
|
Misogynist posted:Trying not to be that guy, but IOPS isn't plural. ".75 to 1 IO per per user"
|
# ? Jun 27, 2012 16:26 |
|
szlevi posted:".75 to 1 IO per per user"
|
# ? Jun 27, 2012 16:37 |
|
Misogynist posted:What Exchange user reads or writes from a disk one time and then never touches it ever again? Obviously, anyone using S4 for their Exchange backend storage.
|
# ? Jun 27, 2012 16:39 |
|
Misogynist posted:Trying not to be that guy, but IOPS isn't plural.
|
# ? Jun 27, 2012 17:06 |
|
Mierdaan posted:Obviously, anyone using S4 for their Exchange backend storage. That is outstanding.
|
# ? Jun 28, 2012 23:15 |
|
I'm trying to learn more about enterprise storage and the difference between a NAS and a SAN is hurting my brain more than it really should. I get that a SAN is block level and a NAS is not, that makes sense. But they are used in difference situations right? Like typically a SAN would be plugged directly into a server or a few and used directly by the processes on said server(s), whereas a NAS is more like general storage (that could be used by a server) but is simply made available to users and programs via shares over the network while being managed by a server. Is that right?
|
# ? Jul 3, 2012 19:37 |
|
Sylink posted:I'm trying to learn more about enterprise storage and the difference between a NAS and a SAN is hurting my brain more than it really should. These days you've got "SANs" that only talk Ethernet, "SANs" that'll happily serve files (SAN used to mean block devices), etc. Most people I've met use NAS for something that serves files, SAN for anything that can do block storage, or storage used mostly server side. Sylink posted:Like typically a SAN would be plugged directly into a server or a few and used directly by the processes on said server(s), whereas a NAS is more like general storage (that could be used by a server) but is simply made available to users and programs via shares over the network while being managed by a server. Is that right? evil_bunnY fucked around with this message at 19:54 on Jul 3, 2012 |
# ? Jul 3, 2012 19:49 |
|
^^^ leaving this wall of text ^^^Sylink posted:I'm trying to learn more about enterprise storage and the difference between a NAS and a SAN is hurting my brain more than it really should. Somewhat. Really, the biggest difference is what you said originally. One is block-level and one is file-level. A SAN is almost always connected to more than one device, otherwise it would be a DAS. The SAN side allows you to give block-level access to multiple devices over iSCSI or Fiber Channel. Generally, this allows you to create a LUN (block-level) and present it to a server, then that server formats it with it's own file system. In the case of Windows, that would be NTFS. VMware would be VMFS. You could also do EXT3, etc. [Edit: As a general rule of thumb, 1 LUN is presented to 1 host. This is because most file systems are not cluster-aware, or multi-access. NTFS will break if more than one host accesses it. The exception to this is cluster-aware file systems like VMFS or LVM.] A NAS, which almost always used to come separately, now often times comes with the SAN. You usually create a few LUNs on the block-side, then present it to the NAS side. The NAS side formats it with whatever file system it is going to use (I think usually EXT3/4), but then serves it up over CIFs (SMB) or NFS. CIFs would be just like a normal file share, similar to what you'd make on a Windows server. NFS is a Unix-style file share, which Windows servers can access if you install the proper clients, but is more for use with virtualization. I'm sure someone else can chime in as I'm new to NFS, but from what I understand, from a virtualization point of view, the reason to use NAS functions like NFS as opposed to SAN functions like an iSCSI LUN, is that NFS supports file locking and journaling. Hope that helps. My knowledge is fairly limited, but for a simple summary, that is how I understand it. Internet Explorer fucked around with this message at 19:59 on Jul 3, 2012 |
# ? Jul 3, 2012 19:56 |
|
Technically a SAN is the network infrastructure that connects clients and arrays to provide block level storage from the latter to the former. Most people in IT will call a disk array a SAN (myself included sometimes) but strictly speaking that is incorrect. A NAS, on the other hand, is the actual storage device.
|
# ? Jul 3, 2012 20:11 |
|
Thanks for the explanations, very helpful guys.
|
# ? Jul 3, 2012 20:54 |
|
Internet Explorer posted:NTFS will break if more than one host accesses it. NTFS does have some limited cluster support since 2008 R2 via Cluster Shared Volumes. http://en.wikipedia.org/wiki/Cluster_Shared_Volumes
|
# ? Jul 3, 2012 20:57 |
|
NippleFloss posted:Technically a SAN is the network infrastructure that connects clients and arrays to provide block level storage from the latter to the former. This bugs me more than it should, especially when I fall victim to doing it so other people know what I'm talking about. You wouldn't call a server or a switch a LAN. I won't say much else because I'd just be echoing the people above. Block vs File is the main differentiation, as has been more eloquently explained already.
|
# ? Jul 3, 2012 22:07 |
|
gently caress you system manager (can't resize any column either, and multiple config screens are blank and unresponsive). e: trying it on FF14, it doesn't even pop the config fields, which I guess is better than locking up the whole interface, but now all storage screen fields are empty. This is after the webUI on the filer itself refused to ever work. evil_bunnY fucked around with this message at 14:32 on Jul 4, 2012 |
# ? Jul 4, 2012 13:38 |
|
|
# ? Mar 29, 2024 01:03 |
|
I had a fun day yesterday. We hit some bug on our FAS3210 in where the controller battery discharged to the point where it decided to shutdown both controllers in the middle of the day.
|
# ? Jul 4, 2012 16:20 |