|
StabbinHobo posted:Yea thats almost exactly what I was saying. They're implementing it because it makes for good buzzphrase-packed marketing copy. Keep in mind, the design in question only reduces the risk by a massive degree, its never fully gone. Therefore they can straight-faced claim to be making something more reliable with raid6, even if so minutely that its almost being deceptive to act like it matters. Sorry to revisit this again, but I think the following, taken from http://www.thestoragearchitect.com/2008/10/29/understanding-eva-revisited/ explains what StabbinHobo said quite succinctly ( for EVA's anyway ). thestoragearchitect.com posted:* EVA disks are placed in groups – usually recommended to be one single group unless there’s a compelling reason not to (like different disk types e.g. FC/FATA). The v in vRAID should really be the tip off to understanding it.
|
#
?
Apr 29, 2010 05:50
|
|
|
|
| # ? Apr 25, 2024 09:02 |
|
|
Can anyone unfortunate enough to be managing an IBM SAN tell me if there's a way to get performance counters on a physical array, or am I limited to trying to aggregate my LUN statistics together using some kind of LUN-array mapping and a cobbled-together SMcli script?
|
|
#
?
Apr 30, 2010 16:50
|
|
|
Misogynist posted:Can anyone unfortunate enough to be managing an IBM SAN tell me if there's a way to get performance counters on a physical array, or am I limited to trying to aggregate my LUN statistics together using some kind of LUN-array mapping and a cobbled-together SMcli script? I just started working at IBM (XIV really). PM me or post your specific question and hardware and I'll look it up on Monday.
|
|
#
?
May 1, 2010 01:19
|
|
|
paperchaseguy posted:I just started working at IBM (XIV really). PM me or post your specific question and hardware and I'll look it up on Monday.  I'm going to be setting up a Vulture Culture fucked around with this message at 21:59 on May 13, 2010 |
|
#
?
May 3, 2010 02:28
|
|
|
Vanilla posted:So I have a ton of perfmon stats from a certain server. we use this: http://pal.codeplex.com/ its brilliant, you feed it the log files in either binary or csv format, it churns the logs and spits out a html report, makes pretty graphs, highlights any issues and has explanations for what each counter means.
|
|
#
?
May 3, 2010 12:33
|
|
|
Oh hay IBM people. I've got a DS4300 that I need to wipe the drives on so we can surplus it. Does IBM have a nice fancy utility to do that in bulk or anything? Didn't see anything in the storage manager client, but we are using a version that's roughly 400 years old.
|
|
#
?
May 13, 2010 21:53
|
|
|
This is a long shot, but what the hell, lemme run it by you guys. I've got a pair of Brocade 300 switches (rebranded as IBM SAN24B-4), and I'm trying to connect up each switch's Ethernet management port to a separate Cisco Nexus fabric. Each one is run with a 5020, and the switches link up to a 2048T FEX. Problem is, whenever I do this, there is no link. I can hook up the 300 to a laptop and get a link. I can hook up the 300s to each other and get a link on both. I can hook it up to some old-rear end Cisco 100-megabit 16-port switch and get a link. I can hook up other devices, like an IBM SAN and a Raritan KVM, and get links. But for some reason, the goddamn things just will not show a link when I hook them up to the gigabit ports on the 2048T. Any ideas? The only thing I can think is that the Nexus has issues with stuff below 1 gigabit, but if that's the case, that's some of the most braindead poo poo I've ever heard. Vulture Culture fucked around with this message at 21:12 on May 20, 2010 |
|
#
?
May 20, 2010 21:08
|
|
|
Misogynist posted:This is a long shot, but what the hell, lemme run it by you guys. From what I recall during discussions with our resellers when we were picking up our 5000s + fabric extenders, Nexus 2000 is all 1gig, no 100 meg. EDIT: Apparently the 2248s do 100/1000. Maneki Neko fucked around with this message at 21:19 on May 20, 2010 |
|
#
?
May 20, 2010 21:16
|
|
|
Okay, cool. I scrounged up a Cisco 2940. Let's see if that gets us through for the time being.
|
|
#
?
May 20, 2010 21:34
|
|
|
Maneki Neko posted:From what I recall during discussions with our resellers when we were picking up our 5000s + fabric extenders, Nexus 2000 is all 1gig, no 100 meg. We just built out a new datacenter extension with 2148Ts all over the place and people where wondering why the DRACs didn't work... Good times. At least the 2248TP is launching next month, which does support 100 Mbit.
|
|
#
?
May 20, 2010 22:01
|
|
|
Sooo, I'm in the enviable position of looking at SSD-based disk arrays right now. I'm looking at the Sun F5100, and it's pretty expensive but seems completely solid (and loving awesome). Anyone have experience with other SSD arrays? Should I even be thinking about filling a standard disk array with flash disks (probably X25-Es?)
|
|
#
?
May 21, 2010 02:23
|
|
|
lilbean posted:Sooo, I'm in the enviable position of looking at SSD-based disk arrays right now. I'm looking at the Sun F5100, and it's pretty expensive but seems completely solid (and loving awesome). Anyone have experience with other SSD arrays? Should I even be thinking about filling a standard disk array with flash disks (probably X25-Es?)
|
|
#
?
May 21, 2010 02:53
|
|
|
lilbean posted:Sooo, I'm in the enviable position of looking at SSD-based disk arrays right now. I'm looking at the Sun F5100, and it's pretty expensive but seems completely solid (and loving awesome). Anyone have experience with other SSD arrays? Should I even be thinking about filling a standard disk array with flash disks (probably X25-Es?) I believe the major concern with using SSD's in an otherwise 'off the shelf' array setup is that the disk controller probably won't understand trim, and therefor be unable to present trim capabilities to the o/s, nor pass them along to each drive correctly. Eventually, this will cause performance degradation of the array, though I can't honestly say how much. Lifespan should be unaffected. There is also the scenario that the disk controller can't keep up with the i/o abilities of the drives, and bottlenecks the whole setup. No idea if this is a realistic possibility. Dedicated units, particularly ones that present over a shared file system (nfs, cifs, etc) should be okay, though I don't have any direct knowledge of the sun/oracle stuff.
|
|
#
?
May 21, 2010 03:00
|
|
|
adorai posted:It would be useful to know the intended purpose and budget. The floated budget is around 50K. Currently we have a pair of Sun 2530s with fast disks and multipath I/O but hey, if that's not enough then that's not enough.  EoRaptor posted:There is also the scenario that the disk controller can't keep up with the i/o abilities of the drives, and bottlenecks the whole setup. No idea if this is a realistic possibility. Edit: Another option is adding a pair or more of SSDs to the Oracle host itself and carving it up between some of the high-activity datafiles, ZIL and maybe give the rest over for L2ARC. lilbean fucked around with this message at 03:04 on May 21, 2010 |
|
#
?
May 21, 2010 03:00
|
|
|
lilbean posted:(Oracle 10 on Solaris SPARC systems Psst, It's Solaris 10 on an Oracle SPARC system.
|
|
#
?
May 21, 2010 03:04
|
|
|
FISHMANPET posted:Psst, It's Solaris 10 on an Oracle SPARC system. Edit: Also, to be fair to the SPARCs the Intel/AMDs rock their loving world on the CPU-bound queries but for IO throughput the SPARCS have kept the lead. At least in our environment. And ZFS is awesome. lilbean fucked around with this message at 03:11 on May 21, 2010 |
|
#
?
May 21, 2010 03:08
|
|
|
lilbean posted:Sooo, I'm in the enviable position of looking at SSD-based disk arrays right now. I'm looking at the Sun F5100, and it's pretty expensive but seems completely solid (and loving awesome). Anyone have experience with other SSD arrays? Should I even be thinking about filling a standard disk array with flash disks (probably X25-Es?) The F5100 is a phenomenal bit of kit. I had one in a test centre not that long ago, I passed on it for the F20 cards due to our use case. Don't fill a standard disk array with flash disks. You will want to stab yourself in the face as they thrash the poo poo out of each other. If the controller of the SSD has any onboard TRIM style algorithm, they will go nuts in a RAID group. The controller will not be able to handle the burst I/O that the SSD can produce, so you end up speed limiting the entire thing.
|
|
#
?
May 21, 2010 04:45
|
|
|
lilbean posted:Sooo, I'm in the enviable position of looking at SSD-based disk arrays right now.  But in all seriousness, what part of your current storage isn't up to snuff? Latency on reads and/or writes? Lack of IOps in reads/writes? Throughput? Not enough of the right cache types? Of that 400GB, how much is the typical working set? quote:I'm looking at the Sun F5100, and it's pretty expensive but seems completely solid (and loving awesome). quote:Anyone have experience with other SSD arrays? TobyObi posted:Don't fill a standard disk array with flash disks. You will want to stab yourself in the face as they thrash the poo poo out of each other. If the controller of the SSD has any onboard TRIM style algorithm, they will go nuts in a RAID group. The controller will not be able to handle the burst I/O that the SSD can produce, so you end up speed limiting the entire thing. EnergizerFellow fucked around with this message at 05:05 on May 21, 2010 |
|
#
?
May 21, 2010 04:51
|
|
|
Can anyone recommend an open source distributed file system that can stretch across multiple jbod systems ala Google FS? I am looking for one that will stripe across different drives in this "storage cluster". I've looked at ParaScale, HDFS, OpenAFS, etc... HDFS seems the most promising out of the bunch, but target metrics of multitude huge files is not quite what I was looking for. Basically we got a potential project where we may need to store a whole bunch of archive/backup/tier3 and 4 data with a fairly small budget and I wanted to explore possibility of "rolling my own" "storage cloud".
|
|
#
?
May 21, 2010 05:06
|
|
|
EnergizerFellow posted:We don't have any investment into any SAN deployment currently. We use SAS arrays for everything full of 15K disks, and at most have two hosts connected to each array for a failover situation so there's not really a cost to migrate off of it. I have been seriously looking at using the SSDs as the front-end to the disk arrays, but the bitch is that only ZFS can really take advantage of that from what I know of. If we do end up pulling the trigger in January or February and moving to X86 RHEL systems then we'll have to figure out the deployment plan again for how to use them decently. If we got an SSD array, then Linux and Solaris can take advantage of it completely (in theory). I've looked at the Ramsan stuff a long time ago, but had totally forgotten about them so I'll check again and try to get pricing.
|
|
#
?
May 21, 2010 14:01
|
|
|
oblomov posted:Can anyone recommend an open source distributed file system that can stretch across multiple jbod systems ala Google FS? I am looking for one that will stripe across different drives in this "storage cluster". I've looked at ParaScale, HDFS, OpenAFS, etc... HDFS seems the most promising out of the bunch, but target metrics of multitude huge files is not quite what I was looking for. Edit: There's also MogileFS.
|
|
#
?
May 21, 2010 14:06
|
|
|
I am looking for something entry level. We have about 14 servers we want to virtualize and we need some flexibility in the device so we are able to grow with it and my vendors keep coming back with these out of the ballpark priced solutions. What do you guys suggest for a good entry level unit?
|
|
#
?
May 21, 2010 14:28
|
|
|
Syano posted:I am looking for something entry level. We have about 14 servers we want to virtualize and we need some flexibility in the device so we are able to grow with it and my vendors keep coming back with these out of the ballpark priced solutions. What do you guys suggest for a good entry level unit? The big cost savings of virtualization is simple consolidation, and you can't determine how to maximize your consolidation ratio if you don't have the right numbers. From a storage perspective, we need to know what kind of IOPS (I/O transfers per second) you're looking for, and what the basic performance profile is of your applications (random vs. sequential I/O, reads vs. writes, and where your bottlenecks are). We also need to know what kind of reliability you're looking for, what your plans are regarding replication and disaster recovery features, whether you need to boot from SAN, and how you plan to back all of this up. You need to think about these things from both a business and technical perspective. In terms of the raw numbers, here are the places you need to look (I'm assuming you're a Windows shop): \PhysicalDisk(_Total)\Disk Transfers/sec \PhysicalDisk(_Total)\Disk Bytes/sec \PhysicalDisk(_Total)\Current Disk Queue Length You should probably let those perfmon logs cook for about a week before you take a look at them and start figuring out specs. These numbers, by themselves, don't indicate what your real requirements are. You need to look at the bottlenecks in your application to make sure that your disk is really performing where it should be. For example, if your system is starved for memory, you may be paging, or your database servers may be reaching out to disk a lot more than they need to because they don't have the memory they need for a query cache. Your best bet is to post over in the Virtualization Megathread, and I'll help you out with getting all the other numbers you need. Vulture Culture fucked around with this message at 15:07 on May 21, 2010 |
|
#
?
May 21, 2010 15:03
|
|
|
Welp! I mainly wanted just some pointers to some entry level SAN units but since you are willing to help to THAT extent I will get my performance numbers together and move on over to the virtualization thread when I get a moment.
|
|
#
?
May 21, 2010 15:22
|
|
|
lilbean posted:Misogynist posted:Do you need to grow your capacity? Your speeds? For how long? Where is your capacity/performance now? Where do you anticipate it in five years? If you want a solution that still scales out half a decade from now, you're going to be paying for it. You need to look at what it is you really need, and what the real business case is for it.
|
|
#
?
May 21, 2010 20:14
|
|
|
EnergizerFellow posted:What's the budget we're working with? Also, pretty much this... Edit: Also, there's very little long-term about this. The idea is to temporarily fix the busted applications now with this to give us time to fix the applications, with the caveat that we may never have the time. I wish I was joking.
|
|
#
?
May 21, 2010 21:17
|
|
|
lilbean posted:The budget for the storage improvement is 50K. We already have a pair of Sun 2530s with 12x600GB 15K SAS disks, and three HBAs to put in each host (active-passive). edit: I see that the 2350s are apparently not using zfs. If you don't need HA you could try a 7200 series unit from sun.
|
|
#
?
May 21, 2010 23:02
|
|
|
adorai posted:Can you just get ahold of sun and pick up a few zeus ssds for your ZIL and l2arc on the existing 2350s? We have a second 2530 that is going to be used for secondary purposes - other hosts, backups from Oracle, and so on but none of those uses are necessary... So we have the option of growing the pool across that chassis as well in the same config to improve spindle count and get more cache. The 50K dream spend is to basically either improve on that current scenario or replace it entirely with something bonerific like the F5100. Edit: It's basically a pretty solid setup right now, and it's obviously throwing more money at apps with poor performance. lilbean fucked around with this message at 23:48 on May 21, 2010 |
|
#
?
May 21, 2010 23:45
|
|
|
lilbean posted:The budget for the storage improvement is 50K. lilbean posted:I did a bit of testing though for the ZIL and it seems the two controllers in the array soak up the log writes to their cache anyways, and the system didn't get a big boost from moving the ZIL to SSDs. quote:Right now the setup is ZFS on a single 2530. We do hardware RAID10 on two groups of six disks, assign one group to each controller and then use ZFS on the host to concatenate the pools. Additionally we have two SSDs (Sun's rebranded X25E drives) that we're going to use for the L2ARC, a few hot Oracle partitions and possibly the ZIL. EnergizerFellow fucked around with this message at 02:55 on May 22, 2010 |
|
#
?
May 22, 2010 01:58
|
|
|
EnergizerFellow posted:Why not striped for the host-level combination? You sound bang on about 50K being low for the expansion (which is why I'm leaning towards the flash array).
|
|
#
?
May 22, 2010 03:08
|
|
|
Do you have any performance profiling data available? Which queries take the longest to run, which tables have the highest lock contention? Even just basic iostat stuff, such as read and write queue depth, average io size, etc. How high is the cpu usage ont eh oracle machines themselves? I wondering if the database is even disk i/o bound. It might be getting data from disk just fine, but simply be starved of cpu throughput or local memory bandwidth. You've said that application improvements our a no-go, but how about better hardware in the oracle cluster itself?
|
|
#
?
May 22, 2010 03:31
|
|
|
Misogynist posted:Ceph and Gluster are the two big ones that I'm aware of. Cool, appreciate the info. GlusterFS seems promising. Going to check that out along with Hadoop's HDFS and OpenAFS. MogileFS seemed interesting but requires their libraries to write to it, which is not what I was quite looking for. Going to check out ParaScale again as well, even though it's commercial. Ceph looks cool but seems a bit too raw even for dev/test environment.
|
|
#
?
May 22, 2010 04:13
|
|
|
lilbean posted:When zpools are concatenated the writes are striped across devices, so with multipathing and the striping it's pretty damned fast. $50K can get you 2 x Equallogic PS6010XV (you would need to beat up your Dell rep) with each having 16x450GB SAS drives. Your writes will be striped across each unit, so in affect you can have 28 drives (2 are hot space per each unit) striped in a single storage pool with 2 active controllers. What's your OS server side? IMO, NetApp FS3100 with PAM card is gonna run you a lot more then $100K. Hell, a shelf of 450GB fiber drives is going to run over $30K. However, a 2150 (or whatever the hell is the equivalent right now) with a fiber shelf can probably be had for around $50K. NFS license is pretty cheap for 2000 series too.
|
|
#
?
May 22, 2010 04:17
|
|
|
oblomov posted:$50K can get you 2 x Equallogic PS6010XV (you would need to beat up your Dell rep) with each having 16x450GB SAS drives. Your writes will be striped across each unit, so in affect you can have 28 drives (2 are hot space per each unit) striped in a single storage pool with 2 active controllers. What's your OS server side? Note: the mutli device stripping works for any platform, but will only get full throughput where an optimized multipath driver is available. Currently, only windows has such a driver, though a vmware one is in development. Other platforms have to wait for a redirect from the device they are hitting if the data they are seeking is elsewhere, leading to a latency hit at best. Equallogic has some great ideas, but some really weird drawbacks. A single EQ device filled with ssd's might actually be a faster solution, though it depends on what you are bound by (i/o, throughput, latency, etc). We are back at our SSD device discussion, however, and there are betetr players than equallogic, I feel.
|
|
#
?
May 22, 2010 04:29
|
|
|
oblomov posted:IMO, NetApp FS3100 with PAM card is gonna run you a lot more then $100K. Hell, a shelf of 450GB fiber drives is going to run over $30K.  quote:However, a 2150 (or whatever the hell is the equivalent right now) with a fiber shelf can probably be had for around $50K. NFS license is pretty cheap for 2000 series too. The FAS2040 is the only model I'd touch in that whole lineup. The FAS2020 and FAS2050 are effectively a major generation behind and I'd expect to get an EOL notice on them any day now. I'd also expect a FAS2040 variant with 10G and more NVRAM to show up soonish as well.
|
|
#
?
May 22, 2010 05:24
|
|
|
EoRaptor posted:Note: the mutli device stripping works for any platform, but will only get full throughput where an optimized multipath driver is available. Currently, only windows has such a driver, though a vmware one is in development. Other platforms have to wait for a redirect from the device they are hitting if the data they are seeking is elsewhere, leading to a latency hit at best. Well, there are workarounds with vSphere without native driver (no idea why Dell delayed that till 4.1). I have bunch of Equallogics running on MPIO just fine (just need to follow Dell/VMware guidelines on how many iSCSI vnics to create for the physical ones and then do some command line config on ESX hosts, not particularly difficult stuff there). That said, I just am not sure of the state of Linux iSCSI initiators beyond simple failover stuff. We usually use NetApp NFS for Linux beyond a few boxes with not too intensive IO. 10GB helps with Equallogic, but does not help addressing MPIO volume addressing limitations. One can configure most of that manually (in Windows at least, so imagine Redhat/Suse have similar options). Redhat/SUSE drivers are supposed to be coming out Q3/Q4 along with vSphere 4.1 plugin, I believe.
|
|
#
?
May 22, 2010 07:12
|
|
|
EnergizerFellow posted:Yeah, more like ~$200K. Nothing like overkill. Oh yeah, forgot about 2040. We have a few of the older FAS2020 and FAS2050 boxes around, and they are pretty nice. However, we started buying Equallogic lately for the lower end tasks like this one, they are very price competitive (especially on Windows side). All of that said, I am exploring distributed/clustering file system options on home-built JBODs now for couple different thing. I can certainly see why Google/Amazon use that sort of thing (well, and they have a few hundred devs working on the problem too :P). If only ZFS was cluster aware...
|
|
#
?
May 22, 2010 07:16
|
|
|
Thanks, excellent suggestions all around. EoRaptor - I can't post anything, but there's a good mix of CPU-bound and IO-bound queries. There's dozens of apps, and some of them perform as expected and a lot don't. We're replacing the server potentially too as part of this exercise with a separate budget. It's currently a SPARC T2+ system (1200mhz). We've tested on Intel systems and get some boost, but the SPARCs memory speed seems to nullify most of the advantage. On Monday we're going to profile the apps on an i7 as we're hoping the DDR3 memory bus will be better. If the SPARC really does have an advantage then gently caress. We could get a faster T2+ system or possibly an M4000 :-( Who the gently caress buys an M4000 in 2010? Anyways, that's life.
|
|
#
?
May 22, 2010 17:14
|
|
|
lilbean posted:EoRaptor - I can't post anything, but there's a good mix of CPU-bound and IO-bound queries. There's dozens of apps, and some of them perform as expected and a lot don't. We're replacing the server potentially too as part of this exercise with a separate budget. It's currently a SPARC T2+ system (1200mhz). We've tested on Intel systems and get some boost, but the SPARCs memory speed seems to nullify most of the advantage. On Monday we're going to profile the apps on an i7 as we're hoping the DDR3 memory bus will be better. Yeah, I didn't expect hard numbers, just that you had looked at the database performance counters, and know that you are i/o bound (i/o wait) not database limited (lock contention/memory limited) For a server, look at a Xeon 56xx or 75xx series cpu, cram as much memory into the system as you can afford, and you should end up with a database monster. It's not going to be cheap, but the cpu and memory throughput is probably untouchable at the price point.
|
|
#
?
May 22, 2010 22:18
|
|
|
|
| # ? Apr 25, 2024 09:02 |
|
|
EoRaptor posted:Yeah, I didn't expect hard numbers, just that you had looked at the database performance counters, and know that you are i/o bound (i/o wait) not database limited (lock contention/memory limited) For memory throughput especially, the 75xx series is really really good.
|
|
#
?
May 23, 2010 00:57
|
|