Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
CopperHound
Feb 14, 2012

WWWWWWWWWWWWWWWWWW posted:

The 8TB WD Elements, EasyStore, and MyBooks are all shuckable alike, correct? I just noticed I have three type of drives.
MyBook has a terrible implementation of drive encryption. It is shuckable, but not if you need to retain the data on the drive.

E: I guess technically you can mount the encrypted drive with extra steps. https://github.com/themaddoctor/linux-mybook-tools

CopperHound fucked around with this message at 19:41 on Jan 4, 2020

Adbot
ADBOT LOVES YOU

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

CopperHound posted:

MyBook has a terrible implementation of drive encryption. It is shuckable, but not if you need to retain the data on the drive.

Oh wow, what does that mean exactly? As far as I know I don't have any encryption enabled. Is that only something I need to worry about if I use BitLocker or something?

CopperHound
Feb 14, 2012

WWWWWWWWWWWWWWWWWW posted:

Oh wow, what does that mean exactly? As far as I know I don't have any encryption enabled. Is that only something I need to worry about if I use BitLocker or something?
My book drives have hardware encryption by default. If you format the drive when you plug it in it will behave just like any other drive. Otherwise you will not be able to directly read the drive without the enclosure unless you do some of the stuff mentioned in the above link.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

If I understand correctly, the enclosure is encrypting data written to the disk.

I would assume everyone is buying these and immediately shucking them without even using them in their enclosure so in that case the encryption doesn't even matter...right?

BlankSystemDaemon
Mar 13, 2009



Thermopyle posted:

If I understand correctly, the enclosure is encrypting data written to the disk.

I would assume everyone is buying these and immediately shucking them without even using them in their enclosure so in that case the encryption doesn't even matter...right?
I don't think this applies, but there are schemes like OPAL where the disk itself AES-encrypts stuff written to it.

Corb3t
Jun 7, 2003

Regarding affordable NAS back-ups: Has anybody else looked into a Google Enterprise account with unlimited Google Drive storage? It's only $12 a month and lots of people over on Reddit have backed up their NASes using rclone. Hell, a lot of people even setup a Plex server with content hosted on Google Drive Enterprise (encrypted, of course). Supposedly Google won't start throttling your usage until you hit 150 TB of space used.

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug
Got the FreeNAS box moved onto the 10GB fiber, works great, other than the FreeNAS box started freaking out and rebooting, but I think it was due to a crappy PCIe USB 3.0 card.

Hughlander
May 11, 2005

Gay Retard posted:

Regarding affordable NAS back-ups: Has anybody else looked into a Google Enterprise account with unlimited Google Drive storage? It's only $12 a month and lots of people over on Reddit have backed up their NASes using rclone. Hell, a lot of people even setup a Plex server with content hosted on Google Drive Enterprise (encrypted, of course). Supposedly Google won't start throttling your usage until you hit 150 TB of space used.

Yes that's what I use for backup. duplicacy with local encryption and then distinct folders per machine. Then for some content a second copy via rclone but they throttle iops there so saving billions of small files won't work very well.

Atomizer
Jun 24, 2007



D. Ebdrup posted:

The operative word, because it's doing a lot of work, in that sentence being 'can' - it only happens under scenarios which require careful preparation, such as ZFS dRAID that I mentioned before where meta-data is stored on non-volatile flash and data is written in 10MB blocks to the disks to make use of sequential I/O.

And even then, a long-tail request that's perfectly allowed for in the 8ms average that disks have, can ruin the fast I/O very easily - and they happen more often than people think.

I was referring to the best-case scenario, sequential transfers with large files, because the guy we're advising is using his drives mainly for multimedia storage. He's going to, infrequently, transfer multi-GB files and take advantage of the fastest speeds his drives can offer (while hitting any relevant interface bottlenecks along the way,) but then most of the time not need much bandwidth to access individual files being streamed, hence why I said that any of the bottlenecks/limitations won't really be an issue for his use-case.

WWWWWWWWWWWWWWWWWW posted:

The 8TB WD Elements, EasyStore, and MyBooks are all shuckable alike, correct? I just noticed I have three type of drives.

I only have one 8TB external that's not Western Digital and it's a Seagate. Are those shuckable, too? It's definitely a single 3.5" drive but I don't know if Seagate does some fuckery like changing the SATA plug or linking the firmware serial number to the USB controller or something.

"Shuckable" means you can open an external enclosure and extract the drive within to repurpose it elsewhere, and most of the time it's possible. The only exceptions I know of are some "portable" HDDs (e.g. specific WD models) that contain a 2.5" drive that has a custom USB connector rather than a standard SATA one.

Thermopyle posted:

If I understand correctly, the enclosure is encrypting data written to the disk.

I would assume everyone is buying these and immediately shucking them without even using them in their enclosure so in that case the encryption doesn't even matter...right?

D. Ebdrup posted:

I don't think this applies, but there are schemes like OPAL where the disk itself AES-encrypts stuff written to it.

Yes to all of that; there are enclosures whose controllers automatically encrypt, but that generally doesn't matter because the people who just buy them for the internal drives are immediately shucking them.

BlankSystemDaemon
Mar 13, 2009



Atomizer posted:

I was referring to the best-case scenario, sequential transfers with large files, because the guy we're advising is using his drives mainly for multimedia storage. He's going to, infrequently, transfer multi-GB files and take advantage of the fastest speeds his drives can offer (while hitting any relevant interface bottlenecks along the way,) but then most of the time not need much bandwidth to access individual files being streamed, hence why I said that any of the bottlenecks/limitations won't really be an issue for his use-case.

Yes to all of that; there are enclosures whose controllers automatically encrypt, but that generally doesn't matter because the people who just buy them for the internal drives are immediately shucking them.
I don't trust best-case scenarios. :colbert:

OPAL is a harddrive-spec, has nothing to do with enclosures.
Generally speaking I find that USB to SATA controllers are built down to a price, which precludes doing any hardware encryption even by including ASIC circuitry for the purpose.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

CopperHound posted:

My book drives have hardware encryption by default. If you format the drive when you plug it in it will behave just like any other drive. Otherwise you will not be able to directly read the drive without the enclosure unless you do some of the stuff mentioned in the above link.

I doubt I formatted it when I first got it. So what do I need to do? Dump all the data off it to another 8TB drive, then shuck the drive, then install it in my desktop and transfer the files back?

IOwnCalculus
Apr 2, 2003





WWWWWWWWWWWWWWWWWW posted:

I doubt I formatted it when I first got it. So what do I need to do? Dump all the data off it to another 8TB drive, then shuck the drive, then install it in my desktop and transfer the files back?

That would be the easiest solution, though you'll need to add "reformat" between "install in desktop" and "transfer files back".

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

IOwnCalculus posted:

That would be the easiest solution, though you'll need to add "reformat" between "install in desktop" and "transfer files back".

Thank you! Just formatting it inside of Windows disk management should be enough?

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull

D. Ebdrup posted:

I don't trust best-case scenarios. :colbert:

OPAL is a harddrive-spec, has nothing to do with enclosures.
Generally speaking I find that USB to SATA controllers are built down to a price, which precludes doing any hardware encryption even by including ASIC circuitry for the purpose.

USB to SATA controllers can easily include a hardware encryption block. They're most likely pad limited (*), and even if they aren't, an AES block is approximately no area in sub-90nm nodes. Which these have to be built on, since the SERDES and other IP cores required for USB3 and SATA aren't likely to be available in any node older than 65nm.

* - Pad limited refers to a phenomenon where chips with relatively little functionality have the minimum size of the die set by the size and layout of the I/O pad structures rather than the amount of area required for the logic. You can add extra features into a pad limited chip without increasing the marginal cost of production so long as you don't fill up all the unused space.

The Diddler
Jun 22, 2006


Gay Retard posted:

Regarding affordable NAS back-ups: Has anybody else looked into a Google Enterprise account with unlimited Google Drive storage? It's only $12 a month and lots of people over on Reddit have backed up their NASes using rclone. Hell, a lot of people even setup a Plex server with content hosted on Google Drive Enterprise (encrypted, of course). Supposedly Google won't start throttling your usage until you hit 150 TB of space used.

I do this. I have it set up to rclone my monthly VM backups, some videos that I may want to rewatch in the future, and I have an encrypted repository for sensitive documents. It took a while to get set up properly, but it's nice to have offsite backups of important stuff.

fatman1683
Jan 8, 2004
.
Any opinions on whether a pair of E5-2603v4s are enough CPU for a medium (~20TB) FreeNAS box? I'm going to be building a new ESXi host and converting my old one into a standalone NAS. 12 cores at 1.7GHz has been marginal for virtualization, but I'm hoping it'll be enough grunt to manage storage.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

If that's not enough CPU, FreeNAS sucks.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
I mean, it's enough cores but even fileserver stuff responds well to better single-threaded performance. rsync, samba, or tar all have significant single-threaded portions and 1.7 GHz is not super speedy.

I was running rsync today moving a bunch of music files off my desktop SSD onto my NAS, and at 1.2 gbit/s I was running 65% utilization on one thread of rsync and 25% utilization on one thread of ssh on my i3-7100 at 3.9 GHz (40% total processor utilization). Scaling against clockrates and factoring a 6% IPC lead, my procesor is 243% as fast yours, so rsync (65% on mine) would require 158% utilization on one thread of your processor, or you would be bottlenecked at something like 760 mbit/s of transfer per thread of rsync.

zfs itself is fast, the things you do with it are not necessarily.

(you could of course do more things in parallel if you have 10gbit or teamed gigabit, you do have 12 cores there, but you will often be limited by single-thread speed even on fileserver stuff. also it could be worse, these are 20-30 mb flac files, small mp3s or photos would be more cpu-intensive for rsync to manage, and this was moving them onto a virgin zfs dataset so that's as good as it gets as far as rsync overhead or zfs overhead. Samba overhead is even higher.)

I am personally not a huge fan of some of the common homelab recommendations. They are loud, power-hungry, and limited in all the wrong ways. You can do very nice builds for home users on Ryzen or Skylake, or if you really want RAM capacity then do X99 with RDIMMs up to 768GB very cheaply.

Your home NAS or homelab is not sitting there serving 100 users 24/7, mostly it's sitting there idle for 99% of the day and then for that last 1% you want it to run 1-3 tasks as fast as humanly possible. So a multi-socket, high-core-count, low-clocked, super-high-RAM server is not the thing you need (you don't even have the dual-ported (SAS) drives to do that if you wanted). You want a low-power system with 4-8 cores running at as high a clock as possible when loaded and idling as low as possible when not. And it'll be much quieter to boot. You can get a fair amount of RAM if you need, X99 will do 768GB on a single socket on an ASUS board, but unless you're doing tons of VMs or something the 64GB you get on consumer platforms will probably be OK.

If you want to stay on that system I would think strongly about upgrading your CPUs.

Paul MaudDib fucked around with this message at 03:54 on Jan 8, 2020

phosdex
Dec 16, 2005

fatman1683 posted:

Any opinions on whether a pair of E5-2603v4s are enough CPU for a medium (~20TB) FreeNAS box? I'm going to be building a new ESXi host and converting my old one into a standalone NAS. 12 cores at 1.7GHz has been marginal for virtualization, but I'm hoping it'll be enough grunt to manage storage.

More than enough. My media esxi server that has a 48tb freenas on it runs off just a e3-1230l v3.

fatman1683
Jan 8, 2004
.

Paul MaudDib posted:

:words:
If you want to stay on that system I would think strongly about upgrading your CPUs.

Thanks, I'm aware of the single-threaded elements of SMB and other such filestreaming tools, but I don't have any experience working with them on such a significantly CPU-constrained system. Right now, a RAID1 of WD Reds is my main bottleneck so I haven't hit any point at which having more CPU would matter, but when I expand out to a large array I'm anticipating that I'll hit that point rather quickly. I'm probably going to hold off on the upgrade until the next generation of Xeon Scalables drops and hope that the price comes down on v4 E5s.

edit: Alternatively, I could try and pick up a used v2 system on the cheaps from one of the refurb houses and try and sell my v4 kit. Odds on getting a decent price for a pair of 2603v4s, an X10 motherboard and 64GB of DDR4?

fatman1683 fucked around with this message at 03:03 on Jan 8, 2020

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

fatman1683 posted:

Thanks, I'm aware of the single-threaded elements of SMB and other such filestreaming tools, but I don't have any experience working with them on such a significantly CPU-constrained system. Right now, a RAID1 of WD Reds is my main bottleneck so I haven't hit any point at which having more CPU would matter, but when I expand out to a large array I'm anticipating that I'll hit that point rather quickly. I'm probably going to hold off on the upgrade until the next generation of Xeon Scalables drops and hope that the price comes down on v4 E5s.

edit: Alternatively, I could try and pick up a used v2 system on the cheaps from one of the refurb houses and try and sell my v4 kit. Odds on getting a decent price for a pair of 2603v4s, an X10 motherboard and 64GB of DDR4?

what about swapping your processors out for 2640v3s? they're $75 a pop on ebay and that gets you 2.8 to 3.4 GHz on 8 cores. Or you can get 2637v3 for $65 and that gets you 3.6 to 3.7 (but only 4 cores). Max of 768GB on all these.

Broadwell doesn't seem like a super important generation to hold out for to me. Early 14nm was baaaadddd (this is not 14+++++++) and clocks on most of the higher core count parts are abysmal unless you go for the very tippy top SKUs which are probably never going to be cheap. Broadwell is a bit lower power than Haswell, it supports 1.5 TB vs 768GB on Haswell, and up to DDR4-2400 vs 2133 for Haswell, but it's way more expensive and for home users it doesn't seem like a big deal.

here's your lists:

https://en.wikipedia.org/wiki/List_of_Intel_Haswell-based_Xeon_microprocessors#Xeon_E5-26xx_v3_(dual-processor)

https://en.wikipedia.org/wiki/List_of_Intel_Broadwell-based_Xeon_microprocessors#Xeon_E5-26xx_v4_(dual-processor)

Paul MaudDib fucked around with this message at 03:24 on Jan 8, 2020

fatman1683
Jan 8, 2004
.

Paul MaudDib posted:

what about swapping your processors out for 2640v3s? they're $75 a pop on ebay and that gets you 2.8 to 3.4 GHz on 8 cores. Or you can get 2637v3 for $65 and that gets you 3.6 to 3.7 (but only 4 cores). Max of 768GB on all these.

It's been awhile since I've bought hardware on ebay, I'm not sure I'd want to roll those dice necessarily, but one of the refurb places might have some v3 CPUs with a warranty of sorts. I'll look into it, thanks.

e:

Paul MaudDib posted:

Broadwell doesn't seem like a super important generation to hold out for to me. Early 14nm was baaaadddd (this is not 14+++++++) and clocks on most of the higher core count parts are abysmal unless you go for the very tippy top SKUs which are probably never going to be cheap. It is a bit lower power than Haswell, it supports 1.5 TB vs 768GB on Haswell, but it's way more expensive and for home users it doesn't seem like a big deal.

Yeah, noted. At the time I wanted to get whatever was current in the hopes of keeping it for as long as possible, and prices on Haswell hadn't started to drop yet. It's worked well enough for my purposes so far, but I definitely need to revisit it for this next round of upgrades.

fatman1683 fucked around with this message at 03:24 on Jan 8, 2020

Raymond T. Racing
Jun 11, 2019

fatman1683 posted:

It's been awhile since I've bought hardware on ebay, I'm not sure I'd want to roll those dice necessarily, but one of the refurb places might have some v3 CPUs with a warranty of sorts. I'll look into it, thanks.

e:


Yeah, noted. At the time I wanted to get whatever was current in the hopes of keeping it for as long as possible, and prices on Haswell hadn't started to drop yet. It's worked well enough for my purposes so far, but I definitely need to revisit it for this next round of upgrades.

Other than issues endemic to the hardware across the entire fleet, I've purchased a LGA1366 NAS setup, a LGA2011 setup because I was chasing the hotness, and then a different LGA2011 board because of the mentioned endemic issues across I believe every single motherboard of that SKU, I've had zero issues whatsoever with any ebay hardware.

It's a dice roll yes, but I'd bet it goes bad far less than you think.

sharkytm
Oct 9, 2003

Ba

By

Sharkytm doot doo do doot do doo


Fallen Rib
And if it dies immediately, PayPal buyer protection is always there to gently caress the seller. I bought a server that ended up having 2 dead ram slots, sent photos to seller, they sent a replacement SERVER. No bullshit, no delay, didn't even need to get eBay or PayPal involved.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
I personally stick to US-based sellers when at all humanly possible, I have less confidence in China to not swap heatspreaders around (on consumer gear) and repolish and relabel engineering samples and other such shenanigans. but I've never had a problem buying parts off ebay. as mentioned the buyer protection can gently caress a seller hard, like you really have up to 6 months if something drastically shits the bed (which it probably won't unless you do something)

paradoxically I think server chips may actually be treated better than gamerzzz chips. like yeah sure it's probably run nonstop for 3 years but that's what CPUs are meant to do and there's a virtual certainty that it's never been overvolted/overclocked. I'd take a CPU that's been run hard in a server for 5 years over a CPU that someone has gamed on for 3 years any day.

I have an (8-core) e5-1660v3 here ready to go into one of my systems. The fun thing there is they're overclockable and possibly binned better than the gamerzzz cpus. I specifically held out for a very late production batch that are supposed to overclock pretty reliably.

right now I have an 8-bay NAS that I'm outgrowing, I'm kinda debating whether I upgrade drives or set up a rack and go for more drives. I got one of those $60 Netapp disk shelves (that guy got hosed so hard on the free shipping lol) or whatever it was from a year ago, I swapped out the IOMs so I could just toss them on a rando LSI SAS card and bring it online. I didn't have a basement when I built my NAS but I bought a house and now I do. In principle I am one SAS card and some cables away from getting that online, once I finish running ethernet.

I'm thinking I may look for one of the ASUS X99 boards and plan on swapping up to LRDIMMs someday... edit: nah I guess UDIMMs are down to $165 for 32GB sticks, probably should just suck it up and do TRX40 once IPMI boards come out

(really I just need to clean up my plex library, but, lazy)

Paul MaudDib fucked around with this message at 06:48 on Jan 8, 2020

Crunchy Black
Oct 24, 2017

by Athanatos

fatman1683 posted:

Any opinions on whether a pair of E5-2603v4s are enough CPU for a medium (~20TB) FreeNAS box?
Oh my loving god yes. Maybe upgrade ZIL / SLOG to SSDs if you need to, but just slam it as full as RAM as you can afford and go hog wild.

I run a 2620v3 and have given thought to downgrading to a 2603 (which I run on my virtualization box in dual-socket) just for power conservation purposes because I never ever stress the 2620.

Paul MaudDib posted:

suck it up and do TRX40 once IPMI boards come out

(really I just need to clean up my plex library, but, lazy)

yeah this too for me for a variety of reasons.

Crunchy Black fucked around with this message at 04:46 on Jan 8, 2020

H110Hawk
Dec 28, 2006

Crunchy Black posted:

I run a 2620v3 and have given thought to downgrading to a 2603 (which I run on my virtualization box in dual-socket) just for power conservation purposes because I never ever stress the 2620.

Their TDP is the same, though I assume you can hit lower speeds on the 2603.

https://ark.intel.com/content/www/us/en/ark/compare.html?productIds=83352,92993

That being said for just file serving basically anything will do. Sure my Synology with its atom cpus pegs out, but I don't really care. It streams movies and files just fine. Most single threaded tasks it just doesn't matter how long it takes, within reason, and you will hit other limits first.

Also for computer "refurb" resellers they get everything at basically by-the-ton pricing, I've been the one on the sell side liquidating a thousand+ servers out of a site at once. They have to guarantee they take all of it or they get none of it. Never feel bad asking for a replacement, the worst they can say is "I don't have any more of that exact one, how about one better? Or two? Or a refund? I paid $8 for that $200 server you bought!"

Hadlock
Nov 9, 2004

So uh, I ordered one of these $50 4xSSD docking stations, it's only JBOD but my wife is making me give up four of my old laptops and I'm not yet ready to part with the SSDs inside of them. It's already arrived but I won't have a chance to play with it until this weekend. It's about the size of a Mayo jar with the fan/lid, maybe a little bigger than the size of my fist with the lid off. The lid seems to just exist for asthetic reasons; technically it has a "cooling fan" but they're ssd so :shrug:



https://www.amazon.com/Sabrent-Drive-Docking-Station-DS-4SSD/dp/B0711L68MS/

Normally I would consider this pretty unremarkable, but the next cheapest device on the market is $75, and after that is $115, and take up considerably more space as they're designed to accept 3.5 traditional as well as 2.5 ssd. This is SSD only. And again, JBOD.

BlankSystemDaemon
Mar 13, 2009



NFS is multithreaded and supported in Windows, macOS, Linux, and basically everywhere else.

Volguus
Mar 3, 2009

Hadlock posted:

So uh, I ordered one of these $50 4xSSD docking stations, it's only JBOD but my wife is making me give up four of my old laptops and I'm not yet ready to part with the SSDs inside of them. It's already arrived but I won't have a chance to play with it until this weekend. It's about the size of a Mayo jar with the fan/lid, maybe a little bigger than the size of my fist with the lid off. The lid seems to just exist for asthetic reasons; technically it has a "cooling fan" but they're ssd so :shrug:



https://www.amazon.com/Sabrent-Drive-Docking-Station-DS-4SSD/dp/B0711L68MS/

Normally I would consider this pretty unremarkable, but the next cheapest device on the market is $75, and after that is $115, and take up considerably more space as they're designed to accept 3.5 traditional as well as 2.5 ssd. This is SSD only. And again, JBOD.

Oh loving hell, it's $118 CAD. That's a $40 CAD premium over the USD price + USD shipping to canada converted to CAD at today's rate (without whatever taxes the govt would put on it). Sigh.`

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.
I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it!

The only thing I am a bit confused about: so lets say my DrivePool drive is F:. I have drives D: and E: making the "F" pool. Shouldn't that get "rid" of the D: and E: drives? Meaning shouldn't it free up drive letters in Windows? Or is really the point of DrivePool to use up all the empty space across a bunch of drives as one drive?

I was kind of expecting to get a nice 40TB single drive out of this thing, with that single drive showing up in Windows as a single drive. It's not a big deal if it doesn't do that, I just wanted to make sure I was using the program properly. Would be nice if my TV show collection (which spans two drives) could be all on "one" F:\shows or something.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

WWWWWWWWWWWWWWWWWW posted:

I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it!

The only thing I am a bit confused about : so lets say my DrivePool drive is F:. I have drives D: and E: making the "F" pool. Shouldn't that get "rid" of the D: and E: drives? Meaning shouldn't it free up drive letters in Windows? Or is really the point of DrivePool to use up all the empty space across a bunch of drives as one drive?

I was kind of expecting to get a nice 40TB single drive out of this thing, with that single drive showing up in Windows as a single drive. It's not a big deal if it doesn't do that, I just wanted to make sure I was using the program properly. Would be nice if my TV show collection (which spans two drives) could be all on "one" F:\shows or something.

It's been many years since I last used DrivePool, but IIRC it doesn't do anything with your individual drives...they're all still there and usable.

I think what it does do is use a special hidden folder at the root of each drive and drops your files into them based upon however you configure it to use free space and whatnot and then groups all of the free space across all of your drives into a virtual drive. When you save to that virtual drive ("F" in your case), it drops files into that hidden folder on the root of your real drives.

In other words, your other drives are still usable because DrivePool doesn't take them over it's just saving files to those drives just like you can.

Again, this is years-old, barely-remembered knowledge here so take it with a large grain of salt.

Krailor
Nov 2, 2001
I'm only pretending to care
Taco Defender

WWWWWWWWWWWWWWWWWW posted:

I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it!

The only thing I am a bit confused about : so lets say my DrivePool drive is F:. I have drives D: and E: making the "F" pool. Shouldn't that get "rid" of the D: and E: drives? Meaning shouldn't it free up drive letters in Windows? Or is really the point of DrivePool to use up all the empty space across a bunch of drives as one drive?

I was kind of expecting to get a nice 40TB single drive out of this thing, with that single drive showing up in Windows as a single drive. It's not a big deal if it doesn't do that, I just wanted to make sure I was using the program properly. Would be nice if my TV show collection (which spans two drives) could be all on "one" F:\shows or something.

DrivePool doesn't make any config changes to the drives that make up the pool.

Remember, Windows supports mounting drives to folders so if you don't want to see your source drives in the My Computer view you can mount them in a sub-folder somewhere so you only see your DrivePool drive.

I mount all of my source drives under C:/PooledDrives/Storage1...StorageN folders.

BlankSystemDaemon
Mar 13, 2009



If one were to draw a Venn-diagram of the people who run SPAN arrays and the people who don't make (programmable and automated) backups with verification, I don't think there'd be much left on either side that wouldn't be overlapped.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

D. Ebdrup posted:

If one were to draw a Venn-diagram of the people who run SPAN arrays and the people who don't make (programmable and automated) backups with verification, I don't think there'd be much left on either side that wouldn't be overlapped.


Not that you're wrong, but it might not be as bad as you might at first think.

I don't really have any experience with any sort of spanning other than WHS and DrivePool and it's been years since then, but it's not exactly what I pre-suppose more conventional span arrays to be like.

It operates at a file level instead of something more low level.

One of the nice things about DrivePool is that you can select on a per-file, per-folder, or per-regex-matching-against-filepath (or something like that, i don't remember exactly) which files get duplicated across X number of drives.

I mean, it's not backup, but it's better in some ways than where I'm at with 80TB of ZFS pools without backups (lol at backing up 80TB).

redeyes
Sep 14, 2002

by Fluffdaddy

WWWWWWWWWWWWWWWWWW posted:

I bought Stablebit DrivePool at the recommendation of this thread. It seems like a fantastic program. Almost too good to be true. I love it!

The only thing I am a bit confused about : so lets say my DrivePool drive is F:. I have drives D: and E: making the "F" pool. Shouldn't that get "rid" of the D: and E: drives? Meaning shouldn't it free up drive letters in Windows? Or is really the point of DrivePool to use up all the empty space across a bunch of drives as one drive?

I was kind of expecting to get a nice 40TB single drive out of this thing, with that single drive showing up in Windows as a single drive. It's not a big deal if it doesn't do that, I just wanted to make sure I was using the program properly. Would be nice if my TV show collection (which spans two drives) could be all on "one" F:\shows or something.

You can turn off the drive letters for the pooled drives, no problem. I do this as well. Just use a drive letter for the Pool itself. Then you have what you wanted.

Hadlock
Nov 9, 2004

I had drive pool back in 2012/13 and then Microsoft released "storage spaces" with Windows 10 Pro (or was it windows 8?) which is basically the Microsoft supported version of drive pool. It's pretty reliable as it's the basis of Microsoft's Enterprise storage solution(s). As soon as Microsoft released it I switched over and never looked back, and ran it without issue until last fall when I finally became part of the Synology Master Race.

CommieGIR
Aug 22, 2006

The blue glow is a feature, not a bug


Pillbug
Really want to pick up a Powervault M4110 for my M1000e Bladecenter, wonder if itll take SATA.

bobfather
Sep 20, 2001

I will analyze your nervous system for beer money

Hadlock posted:

I had drive pool back in 2012/13 and then Microsoft released "storage spaces" with Windows 10 Pro (or was it windows 8?) which is basically the Microsoft supported version of drive pool. It's pretty reliable as it's the basis of Microsoft's Enterprise storage solution(s). As soon as Microsoft released it I switched over and never looked back, and ran it without issue until last fall when I finally became part of the Synology Master Race.

Storage Spaces is actually really bad though, while DrivePool is really good!

Adbot
ADBOT LOVES YOU

lurksion
Mar 21, 2013
Noticed some major latency on I/O operations recently.

Looks like it's time for my first hard drive replacement needed.

code:
$ sudo smartctl -a /dev/ada2
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, [url]www.smartmontools.org[/url]

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Desktop HDD.15
Device Model:     ST4000DM000-1F2168
Serial Number:    Z303JDG5
LU WWN Device Id: 5 000c50 07adef059
Firmware Version: CC54
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Jan  8 22:31:33 2020 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

....

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   100   084   006    Pre-fail  Always       -       131219775
  3 Spin_Up_Time            0x0003   093   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       55
  5 Reallocated_Sector_Ct   0x0033   099   099   010    Pre-fail  Always       -       696
  7 Seek_Error_Rate         0x000f   086   060   030    Pre-fail  Always       -       457898683
  9 Power_On_Hours          0x0032   058   058   000    Old_age   Always       -       36939
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       55
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   001   001   000    Old_age   Always       -       2823
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       0 0 4
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   075   058   045    Old_age   Always       -       25 (Min/Max 20/28)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       23
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       129
194 Temperature_Celsius     0x0022   025   042   000    Old_age   Always       -       25 (0 17 0 0 0)
197 Current_Pending_Sector  0x0012   001   001   000    Old_age   Always       -       35720
198 Offline_Uncorrectable   0x0010   001   001   000    Old_age   Offline      -       35720
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       36939h+18m+31.253s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       42562585465
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       3525649925049

...

Error 2882 occurred at disk power-on lifetime: 36933 hours (1538 days + 21 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 30 ff ff ff 4f 00  11d+10:05:53.576  READ FPDMA QUEUED
  ef 02 00 00 00 00 40 00  11d+10:05:53.576  SET FEATURES [Enable write cache]
  ef aa 00 00 00 00 40 00  11d+10:05:53.575  SET FEATURES [Enable read look-ahead]
  c6 00 10 00 00 00 40 00  11d+10:05:53.575  SET MULTIPLE MODE
  ef 10 02 00 00 00 40 00  11d+10:05:53.575  SET FEATURES [Enable SATA feature]

Error 2881 occurred at disk power-on lifetime: 36933 hours (1538 days + 21 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 30 ff ff ff 4f 00  11d+10:05:47.206  READ FPDMA QUEUED
  2f 00 01 10 00 00 00 00  11d+10:05:47.199  READ LOG EXT
  60 00 30 ff ff ff 4f 00  11d+10:05:43.264  READ FPDMA QUEUED
  2f 00 01 10 00 00 00 00  11d+10:05:43.105  READ LOG EXT
  61 00 08 ff ff ff 4f 00  11d+10:05:37.989  WRITE FPDMA QUEUED

....
a whole bunch more errors

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply