Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
I finally decided to ditch my tower full of drives for a Drobo 5N.

I wish I'd done it from the start. This thing loving rules. It's also got a great developer network, so things like sickbeard, rutorrent, etc can all run on the box as well, so I'm really not losing anything in the way of functionality.

Adbot
ADBOT LOVES YOU

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

accipter posted:

Edit: It seems like an LVM is the way to go rather than ZRAID.

There is a ton of fantastic reading material out there about ZFS. I'm going to resurrect an old desktop and use it as a ZFS playground.

Of particular interest to your question is this blog comparing RAIDZ, traditional RAID and mirroring. Here's some design considerations for speed, and here's some info on using SSDs to boost ZFS performance. And before deciding on anything, remember to look at the ZFS best practices guide.

My research was mainly geared towards ZFS in the context of VM virtual disk storage, but hopefully it's still useful to you (and others) as well.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Just test the setup by pulling a drive and make sure it behaves as you'd expect. I got into storage spaces early and got burned... I had a mirrored set that still poo poo the bed when one of the hard drives went offline.

\/\/ Ahh, that would be more comfortable. I just bought a drobo instead.

H2SO4 fucked around with this message at 16:31 on May 28, 2015

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

It's actually been pretty great, but I'm outgrowing it. Too underpowered to use iSCSI for any real work, but it's nice for a set it and forget it file dumping ground. Assuming I get more comfortable with ZFS i'm sure that's what I'll migrate to, and I can give the Drobo to my parents or something.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Does FreeNAS really do some weird ZFS voodoo that prevents pools from being recognized on other platforms? That's disappointing.

I just started dipping my toe into the ZFS pool (that was truly an unintentional pun, but I'm proud of it). I started with Nexenta which was terrible in both performance and management. I just swapped over to the latest FreeNAS build and while management has improved performance is still not where I'd hoped to see it. I'm currently setup with two mirrored vdevs of 2x2TB 7200rpm HGST SATA disks in one pool and my write rate seems to "breathe" between 60-40MBps when copying over network, but a single drive should be capable of nearly double that. It's running on a box with an i7-870 with 16G of memory so it shouldn't really be starved for resources. An iostat seems to suggest that it is disk activity that's holding things up and I'd just like to get a gut check on what I'm seeing. Compression is enabled, and I've tried with and without dedupe without much difference.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

JacksAngryBiome posted:

Sounds like your network card could be the bottleneck. Is it a marvel controller?

It's an Intel PRO/1000 adapter so I don't think that would be the issue. You may be onto something with the controllers though, they're just Vantec add-on cards that Fry's had in stock.

I do have the mirrors split between two controllers so if one blows up I'd still have a functioning set of disks from each vdev. I'm not above buying a decent controller card and testing.

Mr Shiny Pants posted:

Did you use ashift=12?

Looks like FreeNAS did that automatically. The thing is, 3 of the four drives are actually not advanced format, could that be a contributing factor? I also just realized that one of the drives is a Seagate that does use 4k sectors, so I can't imagine that would be a good thing when used in a mirror with a device that's 512b.

H2SO4 fucked around with this message at 08:00 on Jun 17, 2015

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Well, Monday is the day the rest of my toys get here and I go full-out ZFS. Been testing for a bit and FreeNAS seems to be the way to go. The Drobo dropped a drive yesterday... I'm pretty sure it knows its days are numbered.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
My HBA and Rackable JBOD enclosure came in, and it's almost 2/3 full with spindles. I'm down to 1TB of data left to migrate from the Drobo, once that's done there's three more going in.

I may have a problem.

H2SO4 fucked around with this message at 19:34 on Jul 31, 2015

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Man, it's real easy to upgrade the storage in a Drobo, and real loving tedious to whittle it back down. It's been in basically a constant state of rebuilding as I'm moving data off of it and onto the FreeNAS box.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
The drobo is officially on ebay. Long live FreeNAS!

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Yeah, it looks like the new drive is unhappy to me as well. You might try joining #freenas or #zfs on freenode to get another pair of eyes on it.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Anyone have a RAID card recommendation for 16 ports of 6Gb SAS? I'm building a couple virtualization hosts and have 3 SFF-8087 connections from a backplane that need a home. I haven't really kept up with who rebrands what in the OEM game.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

D. Ebdrup posted:

What disks are you connecting to those 16 ports? Because if it's just platter drives, you won't get anywhere close to the 6Gbps that SAS offers while using port multipliers on the two SAS connectors that the IBM ServeRAID M1015 has.

It'll be a mix of platters and flash. The backplane has 3 SFF-8087 ports (12 actual drives) but 4-port cards seem pretty expensive so I'll probably just grab 2 H700s and call it a day.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Anyone else using a Rackable SE3016 JBOD expander? Had a power outage today that gave me a brief heart attack. The UPS shut everything down and then brought everything back up when power came back, but the array itself doesn't auto power-on and I needed to manually hit the power button and bounce FreeNAS for my volumes to reappear.

Short of whipping out the soldering iron and going all mad scientist, I'd like to see if there's an easier way to make the thing auto-poweron when power is applied.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
I had a generic Marvell controller and followed this guide to enable it in ESXi 5.5, might help for that case as well.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

UndyingShadow posted:

That's what I'm looking at now. I was thinking about doing mirroring in the GUEST os's themselves (making 2 datastores available for each VM, each on a different physical drive.) But now I'm reading some scary things that suggest that I NEED a raid card with a huge cache and a BBU or my ESXi performance will be garbage. This is all very confusing.

Wait, you mean doing something akin to software raid between two vmdk files on different datastores? That sounds awful. You don't have to have a card with BBU or giant cache, for home use you're probably not going to notice much of a difference. Depending on RAID layout, VM count, etc.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
My brain was apparently not firing on all cylinders when I bought HP p410 cards for a non-HP board. I don't get the prompt to get into the configuration utility on boot, but I can control it via hpssacli from VMWare so it should be fine.

That said, this box has two adapters and both are reporting that they're in slot 3. They're showing their respective physical drives just fine, but that just makes me a bit apprehensive about moving forward.

Edit: Oh boy. SGPIO doesn't look like it's working right. When I enable the ID light on one physical disk, four light up.

H2SO4 fucked around with this message at 21:45 on Sep 2, 2015

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Goddammit. Now I'm not sure what to do. It looks like I can control the p410s from ESX just fine, but the LED thing has me a bit worried. However, I don't know if it's the card or the backplane being stupid. I'd rather not drop $300 on more RAID cards just to find out they have the same problem.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

Agrikk posted:

Can you tell me more about your setup?

I have two cards running a total of 16 drives from 4 fan out cables. How is your backplane cabled? A single cable from HBA to board or multiple cables (each representing a drive)?

Don't sweat the two cards in slot 3 thing. I've had them running like this for maybe five years with no problems. Drive failures, yes, but no problems.

Sorry to bring this back from the dead, but I just installed the smartarray tools on ESX and created the arrays that way. Supermicro 836 chassis, 12 bays, setup a big 10 array across eight drives with fanout cables. Typical four drives per port.

Good to hear you've not had any issues; I just need to work out how to monitor the health status. I don't know if ESX will know if there's a bad drive since the HP platform monitoring components aren't installed.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Again, personal anecdotes aren't worth a whole lot but that said I've had nothing but great success with Hitachi NAS drives. I've got three 2TBs from 2011 still rocking hard and 7 4TBs that are 2014-2016. They've all been run nonstop and have been migrated from a Drobo to a Rackable SE3016 hooked up to FreeNAS and now to a Lenovo SA120. The 2TBs have been migrated to another site and are being used as a replication target. When I start replacing the 4TBs with 6TB units then the 2TBs will be phased out.

All in all, very impressed.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
I had a storage spaces box that poo poo the bed after only one of the drives died too - with mirrored spaces, at that.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Yeah, there's legit LSI HBAs on Amazon for like $60 that ke all that mess outta the picture. I got a couple 9200-8e cards that have worked swimmingly with FreeNAS and my sa120s.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Holy poo poo. Well done.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

garfield hentai posted:

Are devices like the Seagate personal cloud any good?

I have had horrible experiences with the lower end Seagate Business NAS devices. Like it failed, RMA comes in, and RMA fails a few weeks later. I had great luck with a Drobo 5N before I took the FreeNAS plunge.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord


Shucked a few 8TB drives, doing some preflight checks. Only 16 hours to do a SMART long offline test...

I'm replacing a 4 wide pool of 4TB mirrored vdevs on my FreeNAS 11.1 box. What's the thread's consensus on burn-in testing? FreeNAS forum guys like to use badblocks from what I've seen.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

Oh shiiiiiit. I like that. Right now I'm running off of an old HP z400 and a Lenovo SA120; this might make me look at putting together a SFF FreeNAS build.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
"let other people store whatever data they want to on your storage what could possibly go wrong with this scenario"

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

priznat posted:

Are the poweredge towers as insanely loud as the rack mounted ones? I am guessing not because those 2U fans have got to really scream.

Not generally, but if the fans need to turn up then they can drat sure get that way.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

red19fire posted:

takes more than an hour to move 50 gb.

That's about right for a saturated 100Mbit link. Check to see that both client and server (and anything inbetween if applicable) are negotiating at gigabit speeds.

That's only 14 megabytes per second which I wouldn't expect to max out the drives. I don't have any experience with Synology boxes but I wouldn't expect that to be typical.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

Yep. Mine were all Barracuda Compute (ST8000DM004) drives. The serial numbers do show that they were built inside an enclosure though, so if you have to RMA one you'll likely need to retain the packaging and enclosure.

H2SO4 fucked around with this message at 16:21 on Jul 9, 2018

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
My understanding is that secure erase is great on SSDs because instead of writing data to every sector the controller instead just zaps all of the cells simultaneously. I've not used it on a regular HD though.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Few things are more fun than loading up an SA120 enclosure full of drives and DBANing em all. Lights that bad boy up like a christmas tree.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
I bought an 8 bay R720 to expand my lab, and was pleasantly surprised that when I picked it up they gave me the 16-bay version instead. I think I'll throw an LSI HBA in there and connect the extra 8 bays to it, pass the PCI card through to a FreeNAS VM and have an additional backup destination.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

Harik posted:

Quick followup - in trying to get the X8SI6-F running I hit a snag. The on-board SATA does not work in BIOS, at all. I had to turn on the BIOS option for the SAS and put my boot media on there in order to even boot.

It must have worked at some point, as it came with one of those 16gb high-endurance SATA-socket SSDs installed in it, but that wasn't recognized at boot either.

It's booted, but it takes forever to start running the SAS 2008 ROM, I'd prefer to disable that and let it come up as the server starts.

(Also, I'm out 6 SATA ports if I can't figure this out)

At least updating the IPMI and cross-flashing the SAS to IT mode went smoothly.

Have you tried booting with that SATADOM unplugged? I've had misbehaving drives make an HBA unhappy at boot before.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
Tigerdirect has shuckable 8TB Seagate externals back at $150. Mine were Barracuda Compute drives inside, ST8000DM004.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

100% Dundee posted:

I run PLEX server on my computer and it hosts/uses my media libraries that are on the NAS. Could that be the culprit?

Yes, absolutely. Go to the Library settings on the Plex app and you can control when library updates and other functions are done.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

D. Ebdrup posted:

Windows Defender doesn't normally scan UNC paths.

Maybe not for periodic full scans, but I've definitely had the realtime engine pick up on things I'm accessing via UNC and take action.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
My parents are moving to a place that has a garbage internet connection, so I'm thinking of setting them up with a NAS box that can run Plex and repurposing some old hardware to be an automated media ripping station to feed their physical media into it. The Synology DS918+ looks to be pretty solid and reportedly does well with 4k transcoding, and the Automatic Ripping Machine setup looks to make the conversion pretty painless. Anyone gone through a similar exercise lately to provide some feedback? Their physical media should be pretty skewed towards DVD movies, with some TV shows and lesser blu-rays.

They're a mostly Apple household, will be using Apple TVs or Roku units to consume the media.

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord
I just bought another seagate backup plus 8TB from Amazon to fully populate my DS1019+. I've ran FreeNAS forever but i have been impressed by the synology - it's like a grown-up Drobo.

Adbot
ADBOT LOVES YOU

H2SO4
Sep 11, 2001

put your money in a log cabin


Buglord

CommieGIR posted:

I virtualized FreeNAS on my Xen cluster. Installed MDAM on the Xenserver, RAID-5'ed the 8 1TB drives together, then broke those into 8 500GB clusters in Xen to the the FreeNAS VM, and it actually is getting BETTER throughput than it did on metal.

What kind of demon are you trying to summon? The first commandment of ZFS is "present your disks directly to ZFS" and you're sticking like three layers of abstraction in the middle.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply