NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›842 »

wolfbiker: Nov 6, 2009

DrDork posted:

Not sure about the 68U specifically, but many routers are really only set up to recognize a single drive connected via USB and get confused if you try to connect up something that presents more than one drive at a time. Might work, though! But performance, even if it works, is typically pretty bad compared to an actual NAS.

Yeah, I figured as much. But what about one of those things plugged directly into the USB 3 port of my Sheild?

# ? Jun 5, 2018 12:13

Adbot: ADBOT LOVES YOU

# ? Apr 28, 2024 15:13

Raldikuk: Apr 7, 2006; I'm bad with money and I want that meatball!

So I got a bug up my rear end to build a Ryzen based box and the most logical build was for a file server\plex server. Its functionality was previously the responsibility of my main desktop.

I initially was going to use FreeNAS but shakey Ryzen support (last I checked people were able to get things pretty stable with Cool'n'Quiet disabled, but I never bothered to try) and the limitations with expanding vdevs kept me away. I was able to get Windows Server 2016 for free through my school so I figured that'd be a decent route to go. I paired it up with SnapRaid and have 2 drive parity set up.

For disks I have 6 5TB Toshiba x300s; not NAS drives but 3 of them have served me well so far with my old plex server setup. I have another 1 TB WD drive that I salvaged from another box that is partitioned into OS and media. I will likely be replacing that with a cheapo 32ish gb M.2 sata drive. I will also be buying some additional case fans since the Antec case I bought (which is quite roomy and easily accommodated my drives) only had a single 120mm fan.

One of my main goals was to keep the build cost reasonable and I was able to keep the build cost sans drives to $400. I threw in a Ryzen 3 1200 (4 core 3ghz) combined with an ASRock A320M motherboard. For memory it has 8gb of unbuffered ECC memory. I was a bit worried about 8gb not being enough but RAM prices are insane (not much diff between ecc and non ecc tho); but it has turned out to be the perfect amount. If I had to do the build again the only change would be getting the AB350M Pro since it has more PCIE x16 slots. I used the one x16 on mine for my LSI 9211-8IT and used a riser for my graphics card. The box boots up headless but I like being able to easily have it output the display in case it errors out.

The biggest sticking point with Ryzen that I've heard is its unvalidated ECC support. I've also seen the reviews showing that Windows wouldn't confirm ECC. The former point remains and will always be the case since AMD won't spend the time and money validating the lower end of their chips; I can confirm that ECC is detected as enabled through multiple methods. ASRock publishes support for unbuffered ECC on their support page and manuals as well.

So far it has been running perfectly and has been up steadily since I got it all set up. Unless one is dead set on FreeNAS I think a Ryzen build is a fine solution even if one wants/needs ECC and allows decent homebrew on a budget. Here is a link to an album showing the ECC check and some pics along the way. Like a dummy I didn't take any internal pics but in my defense i was more focused on cable routing and bundling.

# ? Jun 6, 2018 04:49

KOTEX GOD OF BLOOD: Jul 7, 2012

H110Hawk posted:

What is erroring out so quickly? Ours is mounted video NFS for plex, afp on macs, and smb on windows all the time and nothing errors out. It just stalls while the disks spin up.

Infuse was having some issues but I think it was fixed in a newer update actually ��just stalls out for a bit like you said.

SamDabbers posted:

Having the drives spin down and up causes more wear and tear than spinning continuously. Unless the power use or heat is unacceptable you're better off disabling spindown.

Interesting, can you cite anything to support this? It's hard to believe that having no power or anything running to the drives for all but ~4 hours a week is worse than having them run 24/7.

# ? Jun 6, 2018 22:17

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

Niche question: I am having trouble installing GOG games off my repo on my server. They appear to install correctly but when you go to run them they're missing files. Works fine if I copy them over to the PC before installing, works fine if it's just a single EXE, so I suspect something about the bin files is wrong but I haven't pinned down the cause more accurately than that. Ideas on how to fix that?

Paul MaudDib fucked around with this message at 10:23 on Jun 7, 2018

# ? Jun 7, 2018 10:12

H110Hawk: Dec 28, 2006

Paul MaudDib posted:

Niche question: I am having trouble installing GOG games off my repo on my server. They appear to install correctly but when you go to run them they're missing files. Works fine if I copy them over to the PC before installing, works fine if it's just a single EXE, so I suspect something about the bin files is wrong but I haven't pinned down the cause more accurately than that. Ideas on how to fix that?

Map the drive to a letter and try again.

# ? Jun 7, 2018 15:25

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

If you're using Samba, you need to configure things in Windows, so that the mounted share persists when the app/game switches to elevated mode, for whatever reason, like updating or anticheat.

Can't recall the registry key. Phone posting. My issues went away after doing that.

# ? Jun 7, 2018 23:43

spog: Aug 7, 2004; It's your own bloody fault.

I just managed to recover an older version of a file that had gotten accidentally overwritten amd household harmony was restored.

The smug feelingwas slightly tainted as it took 30mins to work out how to do it with synology's odd software.

# ? Jun 8, 2018 01:32

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

Combat Pretzel posted:

If you're using Samba, you need to configure things in Windows, so that the mounted share persists when the app/game switches to elevated mode, for whatever reason, like updating or anticheat.

Can't recall the registry key. Phone posting. My issues went away after doing that.

Is this it?

# ? Jun 8, 2018 17:08

Mr Shiny Pants: Nov 12, 2012

Paul MaudDib posted:

Is this it?

Yep.

# ? Jun 8, 2018 18:30

nerox: May 20, 2001

I've been looking at doing a NAS for my home, which will also serve as an offsite backup for my office for some files (my primary backup for my office will continue being a commercial cloud product).

I am going back and forth between buying a synology device and building a freenas server.

I want the server to take over plex for my desktop. I also want it to use it for bitorrent, which I have not been able to figure out if synology devices even support.

I would like a constant secure link between my office and my home via SSH tunnel or something similar to transfer the files.

Price

A synology device looks like its going to run me $400.00 (DS718+) to $550.00 (DS918+) for 2 bay or 4 bay. I can later expand either to 5 more bays if needed for about $500.00.

I've pieced together 2 different NAS systems, one using a supermicro server mobo w/ ECC ram and an i3 6300 for about $700 or a Ryzen 3 2200G system for about $600.00. Both would initially have 16 gigs of ram. The i3 system would be better for data storage whereas the Ryzen system would be better for transcoding, I am leaning towards the i3 system at this point. Both these systems can handle 6+ hard drives without any further expansion.

Adding Storage

This is where I am confused on synology. Can you just add hard drives to the pool? If I had 2 eight TB drives in a synology SHR setup, which would hold 8TB, can I put in a 3rd drive and the device expand it to 16TB?

From reading about FreeNAS and ZFS, it looks like you can't really edit a pool once it is set. If I have 2 eight TB drives in a Z1 setup, can I add another 8TB drive and it convert to 16TB?

This is a big question to me as I would like to start with only 2 drives.

Features

I know with FreeNAS I can do a lot with jails and they have plugins for most everything I would want, but what can synology stuff really do outside of plex and storage?

edit: My intention is to start with the two eight TB drives shucked from WD Easystores from best buy with either solution and expand as needed.

nerox fucked around with this message at 20:06 on Jun 8, 2018

# ? Jun 8, 2018 19:52

Raldikuk: Apr 7, 2006; I'm bad with money and I want that meatball!

nerox posted:

I've been looking at doing a NAS for my home, which will also serve as an offsite backup for my office for some files (my primary backup for my office will continue being a commercial cloud product).

I am going back and forth between buying a synology device and building a freenas server.

I want the server to take over plex for my desktop. I also want it to use it for bitorrent, which I have not been able to figure out if synology devices even support.

I would like a constant secure link between my office and my home via SSH tunnel or something similar to transfer the files.

Price

A synology device looks like its going to run me $400.00 (DS718+) to $550.00 (DS918+) for 2 bay or 4 bay. I can later expand either to 5 more bays if needed for about $500.00.

I've pieced together 2 different NAS systems, one using a supermicro server mobo w/ ECC ram and an i3 6300 for about $700 or a Ryzen 3 2200G system for about $600.00. Both would initially have 16 gigs of ram. The i3 system would be better for data storage whereas the Ryzen system would be better for transcoding, I am leaning towards the i3 system at this point. Both these systems can handle 6+ hard drives without any further expansion.

Adding Storage

This is where I am confused on synology. Can you just add hard drives to the pool? If I had 2 eight TB drives in a synology SHR setup, which would hold 8TB, can I put in a 3rd drive and the device expand it to 16TB?

From reading about FreeNAS and ZFS, it looks like you can't really edit a pool once it is set. If I have 2 eight TB drives in a Z1 setup, can I add another 8TB drive and it convert to 16TB?

This is a big question to me as I would like to start with only 2 drives.

Features

I know with FreeNAS I can do a lot with jails and they have plugins for most everything I would want, but what can synology stuff really do outside of plex and storage?

edit: My intention is to start with the two eight TB drives shucked from WD Easystores from best buy with either solution and expand as needed.

Are you planning on doing ECC if you went with Ryzen? If so which motherboard are you looking at? I know for ASRock they don't support ECC with any Raven Ridge chips (such as the 2200g).

Also since you wish to start with only 2 drives I would seriously consider how you plan on expanding if you go with FreeNAS. If you start with a 2 drive vdev you are stuck with the ramifications of that until you can wipe your data and start over.

# ? Jun 8, 2018 20:20

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

nerox posted:

From reading about FreeNAS and ZFS, it looks like you can't really edit a pool once it is set. If I have 2 eight TB drives in a Z1 setup, can I add another 8TB drive and it convert to 16TB?

This is a big question to me as I would like to start with only 2 drives.

ZFS has two concepts here: zpools and vdevs. A zpool consists of one or more vdevs, each of which can have type "simple", "mirror", "raidz1", etc. You can add additional vdevs to a pool to increase its size, but you cannot grow a vdev (and old data will not be migrated onto the new vdev, except via attrition as data churns). If any vdev fails completely, it's game over for the pool, since ZFS will be spreading blocks over whatever space it has available to it. You also cannot remove vdevs from a pool for this reason.

The official answer is that you should move the data off to a separate drive, then kill the pool and rebuild it with the new configuration. You could also just add additional vdevs, or create a new pool and copy data over and then add the original drives as a new vdev (should be slightly higher performance at the cost of some copy time).

raidz1 with two drives is effectively a mirrored vdev. If you want to use raidz vdevs and you intend to grow the pool, I'd say you should be expanding the pool in chunks of 4 drives at a time, otherwise you are taking a huge hit to your capacity. 25% parity drives is about right, so raidz1 is good for a 4-drive vdev and raidz2 for an 8-drive vdev, although you're probably reasonably safe up to 8 drives on a raidz1 for home use.

Otherwise, I guess I'd say to use two pools, one main pool that you grow, and one separate pool for backups with just one disk in it (or, grow as necessary). You can create separate filesystems on the main pool (which will be like specific directories inside the file tree) which automatically get zfs-send'd to the backup pool on a regular basis (there's tools for this, or you can roll your own with a cronjob or something).

Growing a redundant pool in small increments is really the one use-case that ZFS doesn't do well. If that's an absolute requirement then Unraid would do better.

Paul MaudDib fucked around with this message at 21:26 on Jun 8, 2018

# ? Jun 8, 2018 20:43

BlankSystemDaemon: Mar 13, 2009

Paul MaudDib posted:

Growing a pool in small increments is really the one use-case that ZFS doesn't do well.

That sentence needs to end with a big yet. Matt Ahrens said today at the FreeBSD Devsummit/conference at BSDCan that he's already finished the initial work of making it possible to add devices to a vdev while the pool is offline, and the only remaining work is to make possible to add devices while the zpool is online and being used.

# ? Jun 8, 2018 21:01

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

D. Ebdrup posted:

That sentence needs to end with a big yet. Matt Ahrens said today at the FreeBSD Devsummit/conference at BSDCan that he's already finished the initial work of making it possible to add devices to a vdev while the pool is offline, and the only remaining work is to make possible to add devices while the zpool is online and being used.

Oh, that's great. I'd heard they were working on it but I was figuring that would be a 'years' kind of timeframe.

He should push that code out as-is. Taking the pool offline is no big deal for home users, and enterprise users don't care about having to add 4-8 drives at a time. Good enough, ship it.

edit: do you have a source for this, or are you at the conference?

Paul MaudDib fucked around with this message at 21:33 on Jun 8, 2018

# ? Jun 8, 2018 21:05

nerox: May 20, 2001

Raldikuk posted:

Are you planning on doing ECC if you went with Ryzen? If so which motherboard are you looking at? I know for ASRock they don't support ECC with any Raven Ridge chips (such as the 2200g).

Also since you wish to start with only 2 drives I would seriously consider how you plan on expanding if you go with FreeNAS. If you start with a 2 drive vdev you are stuck with the ramifications of that until you can wipe your data and start over.

Paul MaudDib posted:

ZFS has two concepts here: zpools and vdevs. A zpool consists of one or more vdevs, each of which can have type "simple", "mirror", "raidz1", etc. You can add additional vdevs to a pool to increase its size, but you cannot grow a vdev (and old data will not be migrated onto the new drive, except via attrition as data churns). If any vdev fails completely, it's game over for the pool, since ZFS will be spreading blocks over whatever space it has available to it.

raidz1 with two drives is effectively a mirrored vdev. If you want to use raidz vdevs and you intend to grow the pool, I'd say you should be expanding the pool in chunks of 4 drives at a time, otherwise you are taking a huge hit to your capacity. 25% parity drives is about right, so raidz1 is good for a 4-drive vdev and raidz2 for an 8-drive vdev, although you're probably reasonably safe up to 8 drives on a raidz1 for home use.

Otherwise, I guess I'd say to use two pools, one main pool that you grow, and one separate pool for backups with just one disk in it (or, grow as necessary). You can create separate filesystems on the main pool (which will be like specific directories inside the file tree) which automatically get zfs-send'd to the backup pool on a regular basis (there's tools for this, or you can roll your own with a cronjob or something).

Growing a pool in small increments is really the one use-case that ZFS doesn't do well. If that's an absolute requirement then Unraid would do better.

No I wasn't going to do ECC on the ryzen server, but I am pretty much set on the intel based build if I do FreeNAS.

Thanks for the explanation of how ZFS works, I was looking at some slideshow on the freenas forums trying to figure it out, and I am still sort of confused.

Lets say I have 4 drives, all the same size. I would make a vdev that is raidz1. Which would give me 75% of the storage capacity of the total storage. I would then put this vdev in a ZFS volume. If I wanted to expand my pool, I would have to buy a second set of four drives make them a vdev that is raidz1, then add it to the same ZFS volume. If I do that, then I can have 1 drive fail out of each vdev and recover, but if 2 drives fail out of either vdev, then I lose everything?

I am fine with just doing mirrored drives right now, I don't have 8TB of data to backup yet, and storage just gets cheaper over time. The best buy external drive trick is just so much cheaper than anything else is why I want to use 8tb drives now, since 2 of those are $300+tax pretty often and buying 2 4tb WD reds is about the same price.

I haven't looked at Unraid, but I ran FreeBSD servers back during college, so I am pretty comfortable with playing with FreeNAS.

# ? Jun 8, 2018 21:20

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

nerox posted:

Lets say I have 4 drives, all the same size. I would make a vdev that is raidz1. Which would give me 75% of the storage capacity of the total storage. I would then put this vdev in a ZFS volume. If I wanted to expand my pool, I would have to buy a second set of four drives make them a vdev that is raidz1, then add it to the same ZFS volume. If I do that, then I can have 1 drive fail out of each vdev and recover, but if 2 drives fail out of either vdev, then I lose everything?

Correct. Although what you put vdevs into is actually a zpool, which you then create datasets and zvols on (dataset being an independent filesystem, zvol being a block device).

A zpool is analogous to a lvm volume group, like a "virtual hard disk" that spans across multiple physical hard drives/RAID sets (vdevs), then you instantiate logical volumes (datasets/zvols) inside of it. The logical volumes just happen to be mapped into the filesystem rather than existing as metadata. The dataset or zvol is the quantum which allows you to create snapshots or tweak settings like dedup/compression on a local basis rather than pool-wide.

There's no need for the second vdev to necessarily be identical to the first - eg you could have a 2-drive mirror vdev and then later add a 4-drive raidz vdev if you wanted. But yes, if any vdev fails then the pool dies. And you can add additional vdevs to the pool, but they only take subsequent effect, not on the existing data.

A pair of 4-drive raidz1s is nominally faster than a single 8-drive raidz2, while offering slightly worse redundancy (since you can have cases where 2-drive failures can kill the array).

Paul MaudDib fucked around with this message at 10:01 on Jun 9, 2018

# ? Jun 8, 2018 21:33

BlankSystemDaemon: Mar 13, 2009

Paul MaudDib posted:

Oh, that's great. I'd heard they were working on it but I was figuring that would be a 'years' kind of timeframe.

He should push that code out as-is. Taking the pool offline is no big deal for home users, and enterprise users don't care about having to add 4-8 drives at a time. Good enough, ship it.

edit: do you have a source for this, or are you at the conference?

I watched the conference, while it was being streamed. He'll probably mention it again tomorrow, but the presentations should also be up on youtube in what is hopefully a not-too-unreasonable timeframe.

The code is available, but it intentionally changes some things about the disk structure which means that you'll never be able to use that pool going forward - specifically to discourage people from using this on production pools, as it's not considered production ready (you know, like with ZFSOnLinux changes where people need to restore from backup because they make the mistake of using it).

# ? Jun 8, 2018 22:10

Devian666: Aug 20, 2008; Take some advice Chris.; Fun Shoe

quote:

Adding Storage

This is where I am confused on synology. Can you just add hard drives to the pool? If I had 2 eight TB drives in a synology SHR setup, which would hold 8TB, can I put in a 3rd drive and the device expand it to 16TB?

From reading about FreeNAS and ZFS, it looks like you can't really edit a pool once it is set. If I have 2 eight TB drives in a Z1 setup, can I add another 8TB drive and it convert to 16TB?

This is a big question to me as I would like to start with only 2 drives.

If you want to check out how SHR works their raid calculator is good for testing incremental expansion.

https://www.synology.com/en-us/support/RAID_calculator?hdds=

# ? Jun 8, 2018 22:14

H110Hawk: Dec 28, 2006

Paul MaudDib posted:

Is this it?

TIL. Thanks.

# ? Jun 8, 2018 23:12

ufarn: May 30, 2009

Anyone know how this WD mess with borked 2017 drives shook out? Is WD Red past 2016 versions just no bueno anymore? Which leaves us with, what, IronWolf?

# ? Jun 9, 2018 00:09

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

ufarn posted:

Anyone know how this WD mess with borked 2017 drives shook out? Is WD Red past 2016 versions just no bueno anymore? Which leaves us with, what, IronWolf?

That says Red Pro's which are different from Reds I guess?

I've got ten 2017+ Reds with none of that mess.

# ? Jun 9, 2018 02:39

G-Prime: Apr 30, 2003; Baby, when it's love,
if it's not rough it isn't fun.

I've got 8 2017 Reds as well that are quiet unless seeking, with no performance or heat issues, and have heard next to nothing about others having issues with regular Reds. Does sound like something limited to the Pro line.

# ? Jun 9, 2018 02:50

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

H110Hawk posted:

TIL. Thanks.

Haha, I'm way past "have you mapped it to a drive letter" territory. I'm into "have you considered [dumb SMB edge case]" territory now.

iirc there is also a sharp edge here if you try to map a directory as both a NFS and a SMB share on ZFS

Paul MaudDib fucked around with this message at 10:03 on Jun 9, 2018

# ? Jun 9, 2018 09:59

BlankSystemDaemon: Mar 13, 2009

Matt Ahrens is covering Flexible Disk Use in OpenZFS at BSDCan, ie. device adding and removal.

# ? Jun 9, 2018 15:05

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

D. Ebdrup posted:

Matt Ahrens is covering Flexible Disk Use in OpenZFS at BSDCan, ie. device adding and removal.

Matt Ahrens can rewrite my block pointers anytime.

not really, unless he thinks it's production ready, in which case he totally can

edit: sounds like you can reflow data but you lose parity correction during the conversion, and the reflow is infeasible past raidz2

e2: thanks for linking this

Paul MaudDib fucked around with this message at 16:04 on Jun 9, 2018

# ? Jun 9, 2018 15:12

BlankSystemDaemon: Mar 13, 2009

Paul MaudDib posted:

Matt Ahrens can rewrite my block pointers anytime.

not really, unless he thinks it's production ready, in which case he totally can

edit: sounds like you can reflow data but you lose parity correction during the conversion, and the reflow is infeasible past raidz2

e2: thanks for linking this

No, you don't lose the distributed parity of your pool while the reflow is happening, and similarily he mentioned specifically that while raidz2 and z3 will take longer, it isn't doing all combinatorics, but rather it's being clever about the patterns of combinatorics to avoid it taking more than the MTBF of the disk.

And since you asked this earlier, if you do want to lose data, click here to make your dreams come true.

EDIT: Oh neat, someone from Isilon asked something that answered what I was wondering about - the reason the checksum isn't verified during the device removal or addition is that it's stored in the block pointer (rather than in a dedicated place as some other filesystems do), and since the removal and additions happen on a different and faster layer (in the layering violation that ZFS is), it can't actually verify the checksums without doing a lot more work and potentially slowing the process substancially.

BlankSystemDaemon fucked around with this message at 16:13 on Jun 9, 2018

# ? Jun 9, 2018 16:08

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

D. Ebdrup posted:

EDIT: Oh neat, someone from Isilon asked something that answered what I was wondering about - the reason the checksum isn't verified during the device removal or addition is that it's stored in the block pointer (rather than in a dedicated place as some other filesystems do), and since the removal and additions happen on a different and faster layer (in the layering violation that ZFS is), it can't actually verify the checksums without doing a lot more work and potentially slowing the process substancially.

You can just say it. The blockchain (merkle tree) isn't efficient enough to process these checksums.

# ? Jun 9, 2018 16:18

H110Hawk: Dec 28, 2006

Paul MaudDib posted:

Haha, I'm way past "have you mapped it to a drive letter" territory. I'm into "have you considered [dumb SMB edge case]" territory now.

iirc there is also a sharp edge here if you try to map a directory as both a NFS and a SMB share on ZFS

Yeah, trying the simplest thing first and all. Yesterday I was about to murder a vm because it turns out docker and selinux don't work well together out of the box on centos. `setenforce 0` it is!

# ? Jun 9, 2018 20:17

salted hash browns: Mar 26, 2007; ykrop

What is the best cloud backup solution for a synology that lets me encrypt and manage my own private keys?

I've got a synology ds218, it's great. I've got some files on there I would like to backup to a remote "cloud" type solution (photos, etc.), and would prefer to encrypt those files with my a private key that I manage. This way the storage provider is just handling encrypted blobs. Ideally the backup client would just run in a docker container and I can point it at the directories I would like backed up.

Any recommendations on a service? Obviously I'm willing to pay. I've looked at Crashplan (no consumer plan anymore) , Backblaze (no option to manage my own key), Carbonite (private key management windows only). I would strongly prefer not to roll my own service with S3 or other infra provider.

# ? Jun 9, 2018 21:34

Thanks Ants: May 21, 2004; #essereFerrari

https://c2.synology.com/en-us/backup ?

# ? Jun 9, 2018 21:48

H110Hawk: Dec 28, 2006

salted hash browns posted:

What is the best cloud backup solution for a synology that lets me encrypt and manage my own private keys?

Backblaze (no option to manage my own key)

Backblaze B2. I got a PEM file from it. It uses some form of PBKDF so you can either manage the PEM or manage the password.

# ? Jun 9, 2018 22:23

SamDabbers: May 26, 2003

KOTEX GOD OF BLOOD posted:

Interesting, can you cite anything to support this? It's hard to believe that having no power or anything running to the drives for all but ~4 hours a week is worse than having them run 24/7.

Thermal cycling is a known cause of wear and failure in electronic components, including DC motors such as those used to spin the platters in hard drives. Electric motors also have a higher inrush current when they're starting up than when continuously spinning, which contributes to thermal cycling and hence wear. That's why chassis that are designed to hold many drives have a feature to stagger the spinup of the drives so they don't all try to draw too much from the power supply at the same time. Some of the wear metrics tracked by SMART include start/stop count, power cycle count, load cycle count (head parking), and spin-up time.

As part of the manufacturing and design process, hard drives and other electronic/mechanical devices undergo accelerated life testing where they are intentionally "aged" by repeated temperature cycles, vibration, and power cycling to try to catch weaknesses or flaws that can be corrected before the drives are released into the market.

That said, modern drives can withstand a lot of abuse, and are typically rated for many thousands of cycles. Unfortunately, the only entities other than the manufacturers who have a sufficient sample size to properly test just how much repeated spinup/spindown affects drive reliability are those who run large datacenters, and they typically keep theirs spinning 24/7 at a regulated temperature. The manufacturers are extremely unlikely to release that kind of information since it might affect their bottom line.

As an example of the power costs, a 8TB WD Red (WD80EFZX) draws 5.2W spinning idle and 0.7W in standby according to the datasheet. Assuming a rate of $0.12/kWh (national average for the US in 2016 was $0.1027/kWh), each drive costs about 5.2W / 1000 * 24h * $0.12 = $0.015 per day to run continuously, or about $5.47 per year. If you have 8 drives in your NAS, the electricity cost per year is about $43.73.

Now let's compare that to a 3% duty cycle (about 5 hours per week) where your drives are configured to spin down for the rest of the time. The calculation would look like this: ((0.7W / 1000 * 0.97) + (5.2W / 1000 * 0.03)) * 24h * $0.12 = $0.0024 per day, or $0.88 per year per drive. Again, for 8 drives the electricity cost would be about $7.02 per year.

Setting your drives to spin down will only save you about $4.59 per drive per year, or a whopping $36.71 per year for an 8 drive array. Considering that spindown is likely to cause increased wear and tear, and the cost savings in electricity vs keeping them running continuously are so small, it doesn't seem like a good value proposition to me.

SamDabbers fucked around with this message at 17:27 on Jun 10, 2018

# ? Jun 10, 2018 16:56

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

You also get to enjoy possible time-outs and paused access operations while a required drive is spinning up, which to me is worth far more than the $1/yr price difference.

# ? Jun 10, 2018 17:44

spog: Aug 7, 2004; It's your own bloody fault.

Thanks Ants posted:

https://c2.synology.com/en-us/backup ?

It never even crossed my mind that Synology would offer such a service.

For EUR10/year I could back up my entire NAS as I keep my photos in Glacier and I don;t care about my iTunes library. Thanks

# ? Jun 10, 2018 18:06

salted hash browns: Mar 26, 2007; ykrop

Thanks Ants posted:

https://c2.synology.com/en-us/backup ?

H110Hawk posted:

Backblaze B2. I got a PEM file from it. It uses some form of PBKDF so you can either manage the PEM or manage the password.

Super helpful, thank you! I had no idea Synology offered their own backup service.

While C2 looks great it looks like they require that you keep port 443 open on your synology device, which is a bummer since I override port 443 so I can use 443 to redirect to multiple docker services I have running on the host. B2 looks slightly cheaper for my use case, but appears the integration has had some stability issues in the past (since it's a 3rd party provider).

I would love to get off this system that requires me to override 443 on the synology host, but the only solution I can think of would require me to add another middleman into my network which I would rather not. Here is the current setup:

code:

[Synology]
- service1:5000
- service2:5001
- service3:5002
- nginx-letsencrypt:443

(https://service1.myhost.com resolves to the synology box. nginx just listens on 443 and proxies secure requests to service1, service2, and service3)

And the only other alternative I can think of is:

code:

[Synology]
- service1:5000
- service2:5001
- service3:5002

[Other Host (rpi?)]
- nginx-letsentrypt:443

(https://service1.myhost.com resolves to the other host. nginx forwards requests across the network to synology)

I'm worried that adding another hop to all my requests is too inefficient and will be too slow. Plus now I'll be sending non-secure requests over my network. Any suggestions?

# ? Jun 10, 2018 21:22

Thanks Ants: May 21, 2004; #essereFerrari

Are you sure they aren�t talking about outbound ports?

# ? Jun 10, 2018 22:33

insularis: Sep 21, 2002; Donated $20. Get well, Lowtax.; Fun Shoe

salted hash browns posted:

Super helpful, thank you! I had no idea Synology offered their own backup service.

While C2 looks great it looks like they require that you keep port 443 open on your synology device, which is a bummer since I override port 443 so I can use 443 to redirect to multiple docker services I have running on the host. B2 looks slightly cheaper for my use case, but appears the integration has had some stability issues in the past (since it's a 3rd party provider).

I would love to get off this system that requires me to override 443 on the synology host, but the only solution I can think of would require me to add another middleman into my network which I would rather not. Here is the current setup:
code:
[Synology]
- service1:5000
- service2:5001
- service3:5002
- nginx-letsencrypt:443

([url]https://service1.myhost.com[/url] resolves to the synology box. nginx just listens on 443 and proxies secure requests to service1, service2, and service3)
And the only other alternative I can think of is:
code:
[Synology]
- service1:5000
- service2:5001
- service3:5002

[Other Host (rpi?)]
- nginx-letsentrypt:443

([url]https://service1.myhost.com[/url] resolves to the other host. nginx forwards requests across the network to synology)
I'm worried that adding another hop to all my requests is too inefficient and will be too slow. Plus now I'll be sending non-secure requests over my network. Any suggestions?

Just run some VM and map your data shares into it, run Duplicacy and push that to B2. No special requirements that way, and you get client side encryption and multithreaded uploads.

My last bill for 3TB stored was $2.82/mo.

# ? Jun 11, 2018 05:09

Avian Pneumonia: May 24, 2006; ASK ME ABOUT MY OPINIONS ON CANCEL CULTURE

Is there any advantage to getting an external NAS and NAS drives versus getting a few extra traditional internal hard drives and setting them up with a plex server on a raid array on my existing desktop computer?

# ? Jun 11, 2018 19:43

Paul MaudDib: May 3, 2006; TEAM NVIDIA:
FORUM POLICE

Avian Pneumonia posted:

Is there any advantage to getting an external NAS and NAS drives versus getting a few extra traditional internal hard drives and setting them up with a plex server on a raid array on my existing desktop computer?

Depends on how you intend to use it. If your desktop is always running anyway, and you can always spare the horsepower to run Plex, then not really, and in fact having the hard drives locally is faster (unless you're using exotic networking gear). The point of a NAS is it's a separate machine, and it doesn't need to be real beefy, so you could turn off your desktop and save power while still having everything available for your TV/etc.

# ? Jun 11, 2018 20:01

Adbot: ADBOT LOVES YOU

# ? Apr 28, 2024 15:13

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

You also get to completely separate it from your desktop, which can have several advantages: if you gently caress up your desktop it doesn't kill your media. You can have the NAS serving Plex/whatever out to the greater internet for remote access without exposing your desktop where you do poo poo like online banking or whatever. You can shove it in a basement/closet/whatever and never have to see it or hear it. You can also use NAS-specific OSes like FreeNAS to take advantage of more reliable (but typically slower) filesystems like ZFS, as opposed to NTFS.

But, yeah, if none of those things matter to you (and they might not, if all you're ever going to do with it is watch your stuff on that one desktop), there's not some magical blow-job giving benefit for a home user to go with a separate system.

e; For instance, I use a separate one because it allows me to shove it in the basement where I don't give a gently caress about it, and so I can turn my desktop off when I don't need it and reduce the heat it dumps into my living spaces. Meanwhile, I have a smart TV and a few tablets that I like streaming media to, so I don't want to be limited to only accessing it while my furnace of a desktop is running. And that means I can leave it torrenting stuff 24/7, again without having to worry about heat/noise/etc. It also means I can keep my desktop small, since I don't need 8+ drives worth of physical case space.

DrDork fucked around with this message at 20:44 on Jun 11, 2018

# ? Jun 11, 2018 20:40

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›842 »