Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
PerrineClostermann
Dec 15, 2012

by FactsAreUseless
If this is the wrong spot, sorry. This is the closest I've seen to a dedicated HDD thread, like how there's one for SSDs.

I've got a hodgepodge JBOD setup for home storage made up of several old 500GB and 1TB HDDs that I'm looking to consolidate into a single or pair of drive(s). What should I be looking for in a drive? The data would be updated a lot and read a lot, but I'm talking things like video. Most of the drives I have now were cheap WD Greens in an enclosure I found for dirt cheap, so I really don't have an idea of good HDDs or prices.

Adbot
ADBOT LOVES YOU

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Ashex posted:

What OS are you running? I would recommend stepping up to WD Reds, they're like WD Green but tuned specifically for NAS purposes. How much data are you looking at storing and how much growth do you expect?

Right now the whole setup is on my personal computer running Windows 8.1. If my WD External was functioning properly, I'd have 9TB of space in total, but the damned thing has been in a half-dead state for the last two years. Taking into account probable duplication across a few drives out of sheer laziness, I probably have a total of 4-5 TB of data, about half of which would be on the old drives I'd want to replace. Over the last year it seems like I've grown my data by 2TB-ish. I'm currently working with 750GB of free space across it all.

2x 500gb
2x 1TB
2x 2TB


That, combined with a pair of SSDs, means I'm stuck working with several drives permanently in enclosures. I'm not sure what the 500 GBs are, I used them in previous computers. The 1TB+ drives are WD Greens and seem to work fine for my purposes.

How fast is network storage compared to native? HDDs don't get absurdly fast so I imagine it wouldn't be too big of a difference.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I...don't know if I've got the money to go into a full, proper RAID setup for now (though I've always wanted to set up RAID5 or similar). In any case, what are some 4TB HDDs you'd recommend?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
Any good deals on things? I'm thinking maybe...

http://www.amazon.com/WD40EZRX-Western-Digital-Internal-IntelliPower/dp/B00EHBEUZO/

But the 3TB is much cheaper, and whatever's in this old WD 3TB external has already crapped on me once (probably a green)

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I'm kinda nervous about buying Seagate. Are their NAS drives any good?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Ashex posted:

I'm cycling out drives in my media server right now and I've got a couple 2TB seagate drives in there which have been around for at least 5 years (these are the ones being removed).

That said, there was a study published last year indicating we should avoid Seagate.

http://www.pcworld.com/article/2089464/three-year-27-000-drive-study-reveals-the-most-reliable-hard-drive-makers.html

Backblaze doesn't use NAS drives though

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
That 3TB Red is a lot better per TB than the 4TB...hmm...

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

priznat posted:

That'd be messy though.. Albeit satisfying.

I bet 5.56 would work beautifully

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
Buy Once, Cry Once.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
So, I'm exploring the possibility of setting up a NAS, with cost being a big factor in how it will be built, if it will be built at all.

I'm imagining that I'll repurpose old hardware. I've got my old C2D system laying around unused. It's an e6750 @ 3.5ghz with 6GB DDR2 RAM, if I'm recalling correctly. The NAS isn't expected to be high performance, so I feel like that'd be a reasonable host. The system would use 3x new WD Red 2TB HDDs, along with two WD Green 2TBs I already have. That gives me 5 near-identical drives to play with. I'd like to be able to expand the NAS' storage a drive or two at a time in the future if need be, so I'm not entirely sure I'd want to use ZFS from what I've read. I'm currently researching/learning about btrfs as an alternative. The underlying system would probably be set up with a parity disc, so...

e6750 @ 3.5ghz
6GB RAM
5x2TB HDDs
- 1 Parity Disk
- 4 Data Disks
btrfs(?)

Total storage should be 8TB. What do you think? It'd save me the cost of buying NAS hardware or a new computer setup, but it's been a while since I built/put together a system, so...I have no idea if it'd be viable to reuse that hardware.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Skandranon posted:

I ran my Unraid server with a very similar setup, and it worked great for a number of years. I recently switched to using SnapRaid, which has a similar idea, but more flexible. Either one will allow you to easily add new disks (assuming they are smaller than the largest you have), and even if you are upgrading your Parity drive(s), it can be done without destroying the array. Another nice thing about Unraid/SnapRaid is that, if your array DOES become broken, you only lose data on the drives that died. The remaining drives are still perfectly accessible drives all by themselves, and can be moved to another machine with no effort.

I'll have to take a look at them. Do they allow you to replace drives with higher capacity ones in the future?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Skandranon posted:

Yep, that's what replacing your parity drive is. Your parity has to be as large or larger than all other drives. So if you have all 2tb drives, you first need to upgrade your parity to 4tb, and then you can add data 4tb drives that run alongside your 2tb drives. My current setup is a mix of 4tb and 3tb drives with 2 parity drives.

Is it just a RAID setup that protects against drive failure? Or does it have data integrity features like ZFS?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Skandranon posted:

I can supply a quick summary.

RAID5/6 stripe all files saved at the block level to each disk in the array. This is why recovery is impossible if more disks than redundancy are removed, since every disk effectively has 1/(x-1) of the file. The parity for each 'split' in this manner is stored on the last disk. RAID4 has a dedicated parity, wheras RAID5/6 distribute the parity block evenly to all disks.

On the other hand, Unraid/Snapraid have a dedicated parity drive (similar to RAID4), but instead of the data being striped at the block level, all data disks effectively function as normal drives, that could even be removed from the array and read from individually. The parity is then computed vertically (basically, there is a parity bit which coresponds to every bit 0 from every disk, then bit 1, and so forth) and stored on the parity disk. So, if any data disk fails, the data can be computed from reading from all disks and using the parity. If more disks fail, then you are basically back to having a number of disks with data on them.

Do they have check summing?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
So it looks like we've got 3 2TB HDDs ordered. They'll supplement my old 2TBs and give us five drives to play with. I'm thinking ZFS with 6TB storage, in RAID6. I figure an old e6750 C2D would be enough to service just old data, but the system would only have 6GB RAM. Being DDR2, it's not like I can exactly go out and buy more. How insane an idea is this?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
So I'm giving thought to programs like Cryptolocker and how to defend a NAS against it while still being convenient to use. The idea floating in my head is to make it possible to read and write files, but not to update or delete them. Is that reasonably possible with something like FreeNAS? Is there a better way to handle it?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Desuwa posted:

Set up regular snapshots and have offline/offsite/cloud backups.

That doesn't involve configuring a NAS to be resistant to something like cryptolocker. Off-site backup is a trivial response and not feasible or warranted for all kinds of data.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
Thanks guys, that helps a lot. Hopefully I'll have this thing up next week.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
So I installed FreeNAS on my old Core 2 Duo machine, with 6gb RAM. my drives are plugged into SATA2 ports. The pool is configured in RAIDZ2. I'm getting 30MB/s transfer speeds, would that be pretty typical for this kind of setup?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
From what I can tell, they're Marvell controllers

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Skandranon posted:

Is that on read or write? For writing, that may just be how fast you're able to calculate parity. If it's on reads though, then it's probably a network controller issue.

This is on write. Could be, it is an old e6750, running at its stock speed of 2.66GHz. But then again, FreeNAS is reporting plenty of spare CPU power. Looks like 20% CPU usage. Individual disks are at 8-10MB/s, according to FreeNAS...

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Skandranon posted:

Maybe? CPU might be starved for data.

Then the CPU wouldn't be the bottleneck. Even Marvell controllers should do better than 30mb/s, and my router is pretty decent (Asus NT66 or whatever, the comically expensive one).

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Skandranon posted:

The CPU isn't, but some part of the parity calculation probably is. Again, if you are able to get significantly more during read operations, then the problem is something to do with how it's calculating/writing parity.

...This is weird. Now I'm getting speeds over 60MB/s. Nothing's really changed.

I'm also getting read speeds close to 100MB/s or thereabouts, my last transfer was yesterday.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
RAID5 is scary :ohdear:

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
So by doing some testing, it seems my box has...interesting read write speeds:

Read: 503376780 bytes transferred in 1.483086 secs (339411702 bytes/sec) [300MB/s]
Write: 503376780 bytes transferred in 1.102334 secs (456646423 bytes/sec) [450MB/s]

...Being able to write faster than I read is...strange, to say the least. The speeds were tested using DD to copy video files to and from itself, RAM, etc.

Anyway, this is in contrast to my speeds on CIFS, where I write at...90MB/s.

Is this really just CIFS' overhead? Is there a better way for my windows machine to access my NAS?

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Combat Pretzel posted:

Same here. 110MB/s even while switched over the lovely switch inside my DSL modem.

Well what the hell then. Cat 5e, gigabit ports, intel NIC/marvell NIC, asus nt66 router.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
Computers interpret "Giga" "Mega" and "Tera" bytes as powers of two, instead of powers of ten. And by computers, I mean dumb poo poo like windows. The proper amount of space is actually there, it's just displayed dumb.

What Windows is actually reporting is space in "Gibi" "Mibi" and "Tebi" bytes, or GiB, MiB, TiB.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Saukkis posted:

It might be useful for drives, but I'm not sure I would be completely happy when the OS tells me I have 34.3597 GB of RAM.

RAM is different; it's advertised as GB, but uses GiB for its real capacity.

Almighty Wikipedia posted:

The gigabyte (/ˈdʒɪɡəbaɪt/[1]) is a multiple of the unit byte for digital information. The prefix giga means 109 in the International System of Units (SI), therefore one gigabyte is 1000000000bytes. The unit symbol for the gigabyte is GB.

This definition is used in all contexts of science, engineering, business, and many areas of computing, including hard drive, solid state drive, and tape capacities, as well as data transmission speeds. However, the term is also used in some fields of computer science and information technology to denote 1073741824 (10243 or 230) bytes, particularly for sizes of RAM. The use of gigabyte is thus ambiguous. For semiconductor RAM, the gigabyte denotes 1073741824bytes. For hard drive capacities as described and marketed by the drive manufacturers, the gigabyte denotes 1000000000bytes, but when a 500-GB drive's capacity is displayed by, for example, Microsoft Windows, it is reported as 465 GB, where GB then means 1073741824bytes.

So it's dumb in the other way.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
No need to buy an entire new device for every viewer on your server if you can just transcode.

Plus plenty of media formats are not supported natively on most devices

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

DrDork posted:

Get a Smart TV with a Plex player front end. Transcoding problem solved!

All kidding aside, while the "I want to stream TV shows from home over a cell connection while I travel" bit makes sense to me, how many people otherwise actually watch plex stuff on random iPhones or other crappy devices which have crappy enough software ecosystems to not have decent native file support one way or another? I mean, one way or another you can shoe-horn just about any file type "natively" onto an Android device, same with anything Windows or Linux based, so what does that leave?

What devices have native 10bit h.264 support with .rear end/.ssa subtitles?

Plus I use mine to keep track of my progress, too

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
Plus plex still sends the native file if the device supports it.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I'd just like to plug FreeNAS and ZFS. My server has 6x2TB HDDs, three WD Reds and three old drives. Saw one was reporting badly in SMART data, so I bought an extra Red just in case. Was too lazy to get around to replacing, the drive died. Pulled it out, hit 'replace,' and poo poo went super smoothly.

It was so wonderful not caring about a drive failure, I'm so used to panicking over it.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I literally built one out of old hardware:

Core2Duo e6750
6GB DDR2 RAM
Asus P5Q Mobo

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

Coredump posted:

Any way you have can get the hardware to setup another host to do plex, etc. and then point it at the zfs pool? http://www.newegg.com/Product/ComboBundleDetails.aspx?ItemList=Combo.2922450

This is what I do.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

fletcher posted:

I thoroughly dust all my computers every few weeks with this thing, it kicks rear end. Just make sure your fans don't spin when you blow air on them.

That...doesn't seem ESD safe.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
ECC is a problem for all use of consumer systems, not just data storage.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
If you're worried about bit flipping, just put your NAS in a faraday cage!

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I run 6x 2TB drives with 7TiB of usable storage in the pool on an e6750 C2D and 6GB of DDR2 RAM, works pretty well.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless

8-bit Miniboss posted:

Remote access to media and family sharing are big features for some.

This. It's super easy to use anywhere, for multiple people, on any device, and also keeps track of my shows and progress.

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
I'm running out of space on the 6 drive RAIDZ2 NAS I set up, but upgrading means buying a ton of 4tb drives...

Adbot
ADBOT LOVES YOU

PerrineClostermann
Dec 15, 2012

by FactsAreUseless
How do those compare with their NAS offerings?

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply