Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Wicaeed
Feb 8, 2005
I mean Nimble does have their expandable hybrid-flash arrays that last I looked were somewhat affordable.

Adbot
ADBOT LOVES YOU

Happiness Commando
Feb 1, 2002
$$ joy at gunpoint $$

We use about 20 TB SSD and 50 TB HDD in our datacenters, block only. I have a Dell ME series unit. It, uh, gets the job done, I suppose. I'm not familiar with any other manufacturers units that offer a 10 GbE iSCSI chassis that will take disks and flash, and there's neither money nor desire to run all flash. We just don't need it.

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.
In our latest Netapp purchase we got 150TB of SSD and 6PB of NL-SAS. I don't think Pure is our choice in the near future. At least this tender round was won by Netapp and it fit easily within our existing infrastructure. Previous tender was won by EMC and we decided not to purchase anything, we had just gotten rid of our last piece of EMC.

Crunchy Black
Oct 24, 2017

by Athanatos
idk how the vendor agreements shake out at the single-array scale for Pure but +1 for them. Everything just worked.

Aware
Nov 18, 2003
I picked up a white labelled dell r740xd recently for a cheap price. It came with 256gb ssd for OS? And 1x 8tb SAS drive. I think its got a perc740 RAID card. Currently it's running ESXi.

I should be able to buy a bunch of cheap drive sleds off Amazon and load it up with cheap SATA drives right?

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.
What are the downsides of top-loading storage chassis vs front/rear in traditional chassis design? The top-loaders offer more disk bays per dollar, and take up less rack space per disk bay.

Specifically, I'm comparing things like 36-bay front/rear capacity storage-focused traditional chassis vs 45 or 60-bay top loaders, which seem to offer a better value.

Are disk temperatures going to be significantly higher on the top loaders?

Pile Of Garbage
May 28, 2007



Twerk from Home posted:

What are the downsides of top-loading storage chassis vs front/rear in traditional chassis design? The top-loaders offer more disk bays per dollar, and take up less rack space per disk bay.

Specifically, I'm comparing things like 36-bay front/rear capacity storage-focused traditional chassis vs 45 or 60-bay top loaders, which seem to offer a better value.

Are disk temperatures going to be significantly higher on the top loaders?

I'd say the biggest issue with top-load is that you have to pull the chassis out of the rack to replace a HDD. Yes I'm sure that if it is properly installed with the right rails and a suitable cable arm fitted you could do the procedure without powering-off the chassis. However compared to front-load the chance of something going wrong is much higher.

Regarding disk temperatures one would assume that a higher density would lead to higher temperatures however it's really hard to say for sure as it depends on the layout, air-flow and workload.

Happy Dolphin
Apr 12, 2007

:shepface::shepface::shepface:
I just received an IBM FlashSystem 5035. After configuring the IP in the network setup, I change the cable to Port 1 and change the IP on my device to be in the same range as the IP of the SAN - However, the NIC ports doesn't show any link or activity lights. I can see the NIC adapters through the Technician UI but no dice getting to the initial configuration one the non-Technician ports.

I got a service contract on it, but I figured I would see if any goons had any experience with this before going through the hassle.

Update: I figured it out. Apparently Node1 Technician port turns into the 'Tech port' and Technician port on Node2 turns into the Initial Configuration - Despite IBMs documentation saying that Port 1 on same node turns into the Initial Config port. Welp.

Happy Dolphin fucked around with this message at 14:06 on Oct 4, 2021

BlankSystemDaemon
Mar 13, 2009



Pile Of Garbage posted:

I'd say the biggest issue with top-load is that you have to pull the chassis out of the rack to replace a HDD. Yes I'm sure that if it is properly installed with the right rails and a suitable cable arm fitted you could do the procedure without powering-off the chassis. However compared to front-load the chance of something going wrong is much higher.

Regarding disk temperatures one would assume that a higher density would lead to higher temperatures however it's really hard to say for sure as it depends on the layout, air-flow and workload.
There's still the issue of gyroscopic precession causing wear and tear on the platters and motors when you're moving up to 90 disks at a time.

Pile Of Garbage
May 28, 2007



BlankSystemDaemon posted:

There's still the issue of gyroscopic precession causing wear and tear on the platters and motors when you're moving up to 90 disks at a time.

Yeah true, didn't even consider that. Definitely only an issue with top-load where you have to move the chassis to replace disks.

Mierdaan
Sep 14, 2004

Pillbug
Weeks of back-and-forth with Pure support before we got a tech who pointed out that after we create our VIFs and assign them IP addresses, we have to assign them IP addresses. Please tell me this isn't normal support, and that Purity//FA isn't always this dumb?

Internet Explorer
Jun 1, 2005





We're using FC, so I can't speak to that side of things, but overall I have been quite happy with our Pure arrays. My only real gripe is that you can't do updates yourself, which seems weird to me in 2022.

Mierdaan
Sep 14, 2004

Pillbug

Internet Explorer posted:

We're using FC, so I can't speak to that side of things, but overall I have been quite happy with our Pure arrays. My only real gripe is that you can't do updates yourself, which seems weird to me in 2022.

Our FA shipped with 6.1.13 on it - that explains why there's no way to flash it to something newer. Does Pure reach out to you to coordinate an update?

devmd01
Mar 7, 2006

Elektronik
Supersonik

Internet Explorer posted:

We're using FC, so I can't speak to that side of things, but overall I have been quite happy with our Pure arrays. My only real gripe is that you can't do updates yourself, which seems weird to me in 2022.

Hell no. I don’t manage our pures anymore, but when I did it pretty much was “call support, schedule the upgrade, put in CR, enable support access in the right window” and sleep like a baby.

When we did our evergreen controller upgrades, it was middle of the day Monday with zero issues.

Internet Explorer
Jun 1, 2005





Mierdaan posted:

Our FA shipped with 6.1.13 on it - that explains why there's no way to flash it to something newer. Does Pure reach out to you to coordinate an update?

You schedule in Pure1.

devmd01 posted:

Hell no. I don’t manage our pures anymore, but when I did it pretty much was “call support, schedule the upgrade, put in CR, enable support access in the right window” and sleep like a baby.

When we did our evergreen controller upgrades, it was middle of the day Monday with zero issues.

I dunno. Controller upgrade should be bulletproof. If it's bulletproof, let me just press a button or schedule it myself.

Moey
Oct 22, 2010

I LIKE TO MOVE IT

devmd01 posted:

Hell no. I don’t manage our pures anymore, but when I did it pretty much was “call support, schedule the upgrade, put in CR, enable support access in the right window” and sleep like a baby.

When we did our evergreen controller upgrades, it was middle of the day Monday with zero issues.

Nimble I used to rip software upgrades whenever (mid workday included).

I have a handful of Unity XT arrays now, they keep insisting that I have a support ticket open before doing anything.

Unsure how I feel about that, but the performance and capacity has been untouched in terms of value so far. Also zero issues (knocks on wood).

Internet Explorer
Jun 1, 2005





Apparently the Pure gods have heard my prayers. They have added self-service updates in Purity 6.3.

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug

Internet Explorer posted:

Apparently the Pure gods have heard my prayers. They have added self-service updates in Purity 6.3.

Just in time for Pure stuff to hit the hobbyist market :getin:

kzersatz
Oct 13, 2012

How's it the kiss of death, if I have no lips?
College Slice
I work with something like 12 pure boxes and one flashblade, I swear if I could set fire to the flashblade and not get sent straight to the poor house, I'd consider it.

SolusLunes
Oct 10, 2011

I now have several regrets.

:barf:

Internet Explorer posted:

Apparently the Pure gods have heard my prayers. They have added self-service updates in Purity 6.3.

I was gonna say, refusing self-service updates just sounds like a way to say screw you to the hobbyists when the kit leaves enterprise.

Thanks Ants
May 21, 2004

#essereFerrari


SolusLunes posted:

I was gonna say, refusing self-service updates just sounds like a way to say screw you to the hobbyists when the kit leaves enterprise.

They'll still do that by paywalling the files you need

BlankSystemDaemon
Mar 13, 2009



Thanks Ants posted:

They'll still do that by paywalling the files you need
Only company I know of that doesn't do that is Supermicro.

Langolas
Feb 12, 2011

My mustache makes me sexy, not the hat

kzersatz posted:

I work with something like 12 pure boxes and one flashblade, I swear if I could set fire to the flashblade and not get sent straight to the poor house, I'd consider it.

Ive had a number of friends go to work for pure. All the ones that went flashblade left the company already to a competitor. All the ones in the other product groups are still there.

SolusLunes
Oct 10, 2011

I now have several regrets.

:barf:

Thanks Ants posted:

They'll still do that by paywalling the files you need

Yeah, but those sometimes make their way out of the paywall. This is just a more robust version of a loving hobbyists solution.

SolusLunes
Oct 10, 2011

I now have several regrets.

:barf:

BlankSystemDaemon posted:

Only company I know of that doesn't do that is Supermicro.

Netapp seems to be hit or miss on this- but I also only use a handful of their disk shelves, nothing else.

evil_bunnY
Apr 2, 2003

Thanks Ants posted:

They'll still do that by paywalling the files you need

you can get the files with the work account t

kzersatz
Oct 13, 2012

How's it the kiss of death, if I have no lips?
College Slice

Langolas posted:

Ive had a number of friends go to work for pure. All the ones that went flashblade left the company already to a competitor. All the ones in the other product groups are still there.

They jump to VAST?

Honestly, I wouldn't wish Flashblade on my enemies.
It's trash, in one calendar year of use, we've easily replaced 3/4 of the blades from failures.
Their reasoning? Excessive overwrites.
We're using it as a backup target with object lock, if you can't handle the waves stay out of the backup space.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
Reposting from the storage thread:

I am currently messing about with AWS' FSx ONTAP file system and will be building an iSCSI filer to compare/contrast price and performance vs EBS for my specific workload. That said,

I'm considering three scenarios for a MSSQL Server machine:

1. one LUN on ONTAP serving one iSCSI disk serving one volume on Windows with a set of databsae files on it (mdf, ndf and log files)
2. one LUN on ONTAP serving one iSCSI disk serving multiple volumes on windows
3. multiple LUNs on ONTAP, each serving one iSCSI disk with a single windows volume on it

One would think it wouldn't make a difference because the ONTAP filer is still serving the same about of thoughput, the same amount of storage and the same amount of IOPS. Am I missing something?

Internet Explorer
Jun 1, 2005





Haven't really worked with any of that, but from a general storage perspective, here's some of my thoughts.

I suspect AWS/NetApp already has specific guidance on this. One of those things that's good to follow, as if there are issues down the road they may just tell you to switch to the correct way if it really gets down to it in troubleshooting.

If you ask most DBAs, they'll want things separated out as much as possible, down to the individual hard drives, but lol. What are you using to make the iSCSI connection, Windows iSCSI? I'm not super up to speed on best practices there, but I would check to see limits on the adapter side. More adapters may alleviate some bottlenecks. In my experience, it also depends on performance needs. Covering for the highest performance may add additional complexity that unnecessarily complicates future operations if it's not needed.

Kivi
Aug 1, 2006
I care
My company buys capacity from this 3rd party vendor and we're having issue where supposedly two equally specced perform dissimilar, other being almost one fourth slower. I'm going to guess it's due to disk perf (it's database server with Xeon 56c Plats and 128 GB of RAM on both) so what would be quick benchmark to run? Bonnie++ springs to mind but it takes hours to run, is there anything simple and fast that would provide disk metrics we could easily compare?

evil_bunnY
Apr 2, 2003

Kivi posted:

My company buys capacity from this 3rd party vendor and we're having issue where supposedly two equally specced perform dissimilar, other being almost one fourth slower. I'm going to guess it's due to disk perf (it's database server with Xeon 56c Plats and 128 GB of RAM on both) so what would be quick benchmark to run? Bonnie++ springs to mind but it takes hours to run, is there anything simple and fast that would provide disk metrics we could easily compare?

bonnie++ shouldn't take hours to run, or can be config'ed not to.
iometer is the standard storage perf characterization tool. You can make it test something very close to your average workload (provided you've characterized that beforehand).

r u ready to WALK
Sep 29, 2001

Have a look at FIO, that gave me pretty consistent results when I had to do a bunch of storage benchmarks.
https://fio.readthedocs.io/en/latest/fio_doc.html

The challenge with benchmarking modern storage is that it is really easy to bottleneck on something else since all-flash enterprise storage arrays have become insanely fast. Different tools and test methods give wildly different results which is frustrating.
You usually need a combination of large IOs, deep queues or many threads to get anywhere near the max performance of modern storage and there's a ton of badly written software out there that wants to do everything synchronously on a single thread with whatever is the OS defaults for file io.

ILikeVoltron
May 17, 2003

I <3 spyderbyte!

Kivi posted:

My company buys capacity from this 3rd party vendor and we're having issue where supposedly two equally specced perform dissimilar, other being almost one fourth slower. I'm going to guess it's due to disk perf (it's database server with Xeon 56c Plats and 128 GB of RAM on both) so what would be quick benchmark to run? Bonnie++ springs to mind but it takes hours to run, is there anything simple and fast that would provide disk metrics we could easily compare?



FIO is fine, iometer is fine too, your issue is going to be setting up a proper test to get accurate IO from the test. This means either you already have a profile for the DB (80% read / 20% writes / 8k blocks / etc) or you need to identify the profile you want to use. With that in mind, you also should use a disk twice the size of ram in the system, so if you've got 128 gigs of ram, you should allocate out a 256 gig disk to run the test on. Also, try and keep things the same, if you end up writing the test file on an existing filesystem, make sure it's the same filesystem as the one your db is running on. If it's oracle (and using it's own custom format disk) maybe test on xfs.

Anyway, there's lots of things you should be accounting for to run a good test, ram, cpu, and disk not-withstanding. Hope this helps

Kivi
Aug 1, 2006
I care
Thanks for the help and suggestions!

It's PgSQL on XFS, with separate wal disk and tuning my former co-worker suggested (agcount 56 to match the CPU count)

I'll look at bonnie++ again, it just took forever on my testing server but that has "tier-2" disks (tiers go to 0-3, 0 being fastest) so I just assumed it was hours long benchmark. FIO looks like a good tool too.

unknown
Nov 16, 2002
Ain't got no stinking title yet!


Anyone got any recommendations/horror stories on low-end SAN systems (~100TB/hybrid/rackmount) for data storage? Was looking at something like XiSystems/Truenas X-series kind of thing. A step above homebrew kind of thing.

Qwijib0
Apr 10, 2007

Who needs on-field skills when you can dance like this?

Fun Shoe
I've never had a supported iX box in my datacenter, but freenas has been reliable for me so it's probably a reasonable choice. The pricing doesn't look too bad over the cost of bare hardware to get support with it. My current benchmark price for enterprise storage is Qumulo, and truenas looks to be 1/3 or less per TB than that.

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug
TrueNAS is hard to beat.

Kaddish
Feb 7, 2002
I got some quotes to replace some old IBM Unified filers. Mixed use - some PACS cache, home directories, random apps, database backups, etc.

One of the quotes was for a NetApp C250 with about 200TB. (And another for replication)

I can find no reason to get anything else, the price/performance ratio on this thing seems too good to be true.

Edit - The other arrays were a NetApp FAS 8300, and EMC Unity 380. Both with mixed disk, ie. not all flash.

Edit 2 - I was leaning EMC Unity until this NetApp came along.

Kaddish fucked around with this message at 16:45 on Apr 18, 2023

Qwijib0
Apr 10, 2007

Who needs on-field skills when you can dance like this?

Fun Shoe
Have you gotten a quote on some qumulo archive tier storage? Even that'll do 1GB/sec per node. I'm on my second cluster and the product and support are stellar.

Adbot
ADBOT LOVES YOU

Kaddish
Feb 7, 2002

Qwijib0 posted:

Have you gotten a quote on some qumulo archive tier storage? Even that'll do 1GB/sec per node. I'm on my second cluster and the product and support are stellar.

Never even heard of it but I'll check it out!

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply