NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›840 »

bobfather: Sep 20, 2001; I will analyze your nervous system for beer money

Chumbawumba4ever97 posted:

I really appreciate the help!

I think I messed something up. Here's my settings:

The 8tb O: was completely empty. My 10TB Z: had all my TV shows. So I made a Drivepool with the letter X: combining both the O: and the Z:.

Then I cut and pasted all of the TV shows out of the Z: and into the Drivepool (X:). It took nearly 2 days! Anyway, when I went to just check right now, it filled up both the O: and the Z: equally, despite the fact that I would greatly prefer all the TV shows to only be on the Z: (my TV show collection is under 10TB). Now I have random episodes of 30 Rock across both drives which is making me nuts. Are my settings messed up somehow? I wanted everything to be on the Z:, and only wanted it to start using the O: when the Z: got full.

The purpose of DrivePool is to combine many physical drives into a single, larger, virtual drive. In this case it is functioning as intended by storing approximately half your files across each drive.

Why don�t you go into the Windows partition settings and disable the drive letters for your O and Z drives? Your X drive is the only one you need to use from here on out, and when you browse that drive it will of course appear as if all of your TV shows are on that drive, as you prefer it.

Edit: Also, your initial file move did not need to take 2 days. In the future, follow StableBit�s steps to seed a pool: https://wiki.covecube.com/StableBit_DrivePool_Q4142489

bobfather fucked around with this message at 21:57 on Oct 11, 2020

# ? Oct 11, 2020 21:55

Adbot: ADBOT LOVES YOU

# ? Apr 19, 2024 19:31

madsushi: Apr 19, 2009; Baller.
#essereFerrari

Deal today: WD 12TB Elements - $175
$175 from Amazon (Prime Day)
$175 from NewEgg
$175 from B&H

BestBuy has a new easystore 12TB SKU for $175, different from the old one.

# ? Oct 13, 2020 11:20

TraderStav: May 19, 2006; It feels like I was standing my entire life and I just sat down

madsushi posted:

Deal today: WD 12TB Elements - $175
$175 from Amazon (Prime Day)
$175 from NewEgg
$175 from B&H

BestBuy has a new easystore 12TB SKU for $175, different from the old one.

Nice, was considering holding out until Black Friday in hopes that it's a milestone where the 14TB move into the price range as the current 12TB but unsure if that's going to happen. <$15/TB is still damned good. Have an older drive I'd like to start moving data off of.

Thanks for posting.

# ? Oct 13, 2020 12:22

Rooted Vegetable: Jun 1, 2002

Can't see anything good on Canadian Amazon yet ... Anyone spotted anything?

# ? Oct 13, 2020 14:34

Less Fat Luke: May 23, 2003; Exciting Lemon

Heners_UK posted:

Can't see anything good on Canadian Amazon yet ... Anyone spotted anything?

16TBs are down to 400$.

# ? Oct 13, 2020 14:44

Rooted Vegetable: Jun 1, 2002

Less Fat Luke posted:

16TBs are down to 400$.

CAD25/TB
Was hoping for below CAD20/TB

Best I've had was CAD16.25/TB ($130 for an 8TB WD Elements last year)

Edit: Here's what Less Fat Luke found and refers to:
https://www.amazon.ca/dp/B088S9PWNM/ref=cm_sw_r_cp_apa_fab_MiBHFbF611MN6
Thanks to Less Fat Luke

Rooted Vegetable fucked around with this message at 14:58 on Oct 13, 2020

# ? Oct 13, 2020 14:55

Samadhi: May 13, 2001

madsushi posted:

Deal today: WD 12TB Elements - $175
$175 from Amazon (Prime Day)
$175 from NewEgg
$175 from B&H

BestBuy has a new easystore 12TB SKU for $175, different from the old one.

Are the drives inside the same as the standard 5400rpm desktop drives on these (shuckable)? I was looking at getting 2x 8TB Seagate drives for storage on a new desktop, but this would be much better

# ? Oct 13, 2020 14:59

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

bobfather posted:

Why don�t you go into the Windows partition settings and disable the drive letters for your O and Z drives?

Because I'm an old-school Windows user (started in the 3.1 days) and I'm worried about not being able to see the individual drives in case I need to access them for some reason. If I disable the drive letters, can I still individually see the contents of each drive?

Furthermore, I have a batch command that runs every Monday that looks like this:

code:

dir e:\ /b /s >c:\filez\filesonEdrive.txt

It runs this for each hard drive. It dumps a text document with a list of all my files on each drive (in the above instance, the E:). What would happen if I removed the drive letter? I am assuming that it would stop working? Is there a way to modify the command to work on drives with no letters?

# ? Oct 13, 2020 15:00

TraderStav: May 19, 2006; It feels like I was standing my entire life and I just sat down

Chumbawumba4ever97 posted:

Because I'm an old-school Windows user (started in the 3.1 days) and I'm worried about not being able to see the individual drives in case I need to access them for some reason. If I disable the drive letters, can I still individually see the contents of each drive?

Furthermore, I have a batch command that runs every Monday that looks like this:
code:
dir e:\ /b /s >c:\filez\filesonEdrive.txt
It runs this for each hard drive. It dumps a text document with a list of all my files on each drive (in the above instance, the E:). What would happen if I removed the drive letter? I am assuming that it would stop working? Is there a way to modify the command to work on drives with no letters?

I just did this command for one of my network drives on my NAS

dir \\Tower.local\Movies /b /s >c:\temp\Movies.txt

It created a text file with every Directory and then every file. If there was a way to modify this to exclude files and only show Directories I'd be happier.

Also, want to run this on a batch script every week or something!

Figured that part out! Now it's this:

dir \\Tower.local\Movies /b /s /A:d >D:\"Google Drive"\!Movies.txt

It also overwrites without prompting.

I'll create a batch file and then find a way to schedule it so I have in the cloud the list of my files. In the event of a catastrophic failure of some kind I'll at least know what I HAD....

TraderStav fucked around with this message at 15:30 on Oct 13, 2020

# ? Oct 13, 2020 15:23

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

TraderStav posted:

I just did this command for one of my network drives on my NAS

dir \\Tower.local\Movies /b /s >c:\temp\Movies.txt

It created a text file with every Directory and then every file. If there was a way to modify this to exclude files and only show Directories I'd be happier.

Also, want to run this on a batch script every week or something!

Hey text document buddy! :hfive:

I made an individual txt document for each hard drive, and one "master" bat file that runs all the other bat files for each drive letter. If you want my "code" so you can use it too, let me know!

Anyway what would I type in to do it for a specific drive? Lets say I remove the E: and F: what would I type? dir \\Tower.local\ sounds like your server's name is "Tower" I'm guessing. But what drive is it doing it for? How does your script know?

Sorry if this is a dumb question. I've been using Windows and drive letters since I was in middle school so this is all a bit new/scary to me! :shobon:

# ? Oct 13, 2020 15:29

redeyes: Sep 14, 2002; by Fluffdaddy

Chumbawumba4ever97 posted:

Because I'm an old-school Windows user (started in the 3.1 days) and I'm worried about not being able to see the individual drives in case I need to access them for some reason. If I disable the drive letters, can I still individually see the contents of each drive?

Furthermore, I have a batch command that runs every Monday that looks like this:
code:
dir e:\ /b /s >c:\filez\filesonEdrive.txt
It runs this for each hard drive. It dumps a text document with a list of all my files on each drive (in the above instance, the E:). What would happen if I removed the drive letter? I am assuming that it would stop working? Is there a way to modify the command to work on drives with no letters?

TURN THE DRIVE LETTERS OFF DUDE. LOL! The data will not vanish.

# ? Oct 13, 2020 15:29

TraderStav: May 19, 2006; It feels like I was standing my entire life and I just sat down

Chumbawumba4ever97 posted:

Hey text document buddy!

I made an individual txt document for each hard drive, and one "master" bat file that runs all the other bat files for each drive letter. If you want my "code" so you can use it too, let me know!

Anyway what would I type in to do it for a specific drive? Lets say I remove the E: and F: what would I type? dir \\Tower.local\ sounds like your server's name is "Tower" I'm guessing. But what drive is it doing it for? How does your script know?

Sorry if this is a dumb question. I've been using Windows and drive letters since I was in middle school so this is all a bit new/scary to me!

Right, my server name is Tower, but Windows resolves it as Tower.local. There is no drive, it's a network drive and putting it in just like that works

# ? Oct 13, 2020 15:31

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

redeyes posted:

TURN THE DRIVE LETTERS OFF DUDE. LOL! The data will not vanish.

No I know. I just don't know how I will be able to reference the local drives in stuff like my batch file?

TraderStav posted:

Right, my server name is Tower, but Windows resolves it as Tower.local. There is no drive, it's a network drive and putting it in just like that works

Ah, okay. All my drives are local. If they don't have a drive letter, how are they differentiated in Windows? If I remove the drive letter for my G:, but lets say I wanted to run a file on my drive previously named G: but through the command prompt, what would I be putting in instead of G:? I hope I am making sense.

# ? Oct 13, 2020 15:58

redeyes: Sep 14, 2002; by Fluffdaddy

Can't you just reference the drive pool letter with your batch file?

# ? Oct 13, 2020 16:00

TraderStav: May 19, 2006; It feels like I was standing my entire life and I just sat down

Chumbawumba4ever97 posted:

No I know. I just don't know how I will be able to reference the local drives in stuff like my batch file?

Ah, okay. All my drives are local. If they don't have a drive letter, how are they differentiated in Windows? If I remove the drive letter for my G:, but lets say I wanted to run a file on my drive previously named G: but through the command prompt, what would I be putting in instead of G:? I hope I am making sense.

You're making sense, I'm not sure there's any other designation that you would use. I think what redeyes was saying is don't assign a network drive a letter, but if these are local drives then it makes sense.

I just set up my batch file and this is working more brilliantly than I imagined. Get this bad boy on a cron job and I'll be able to (slowly) recreate my library in the event of a total and complete failure. Which would mean that my UnRaid array was unable to be saved.

Looking to put my irrecoverable stuff on Backblaze.

# ? Oct 13, 2020 16:02

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

redeyes posted:

Can't you just reference the drive pool letter with your batch file?

Well in my scenario I would need to know which drive has which files on it in case one dies. I was just wondering how I could get a weekly file list dump of a drive with no letter associated to it.

So if one of the drives in my TV show DrivePool shits the bed, I can easily figure out what went missing. I am really braindead when it comes to using Windows without drive letters, so I apologize if these are stupid questions.

# ? Oct 13, 2020 16:09

redeyes: Sep 14, 2002; by Fluffdaddy

Chumbawumba4ever97 posted:

Well in my scenario I would need to know which drive has which files on it in case one dies. I was just wondering how I could get a weekly file list dump of a drive with no letter associated to it.

So if one of the drives in my TV show DrivePool shits the bed, I can easily figure out what went missing. I am really braindead when it comes to using Windows without drive letters, so I apologize if these are stupid questions.

Oh ok. You can mount the drives in a subfolder but there is no point. Just do what you are doing.

# ? Oct 13, 2020 16:12

madsushi: Apr 19, 2009; Baller.
#essereFerrari

Samadhi posted:

Are the drives inside the same as the standard 5400rpm desktop drives on these (shuckable)? I was looking at getting 2x 8TB Seagate drives for storage on a new desktop, but this would be much better

Yes, these are the shuckable ones (albeit might need the voltage / SATA power fix, depending). WD Reds inside, big enough to not be SMR.

# ? Oct 13, 2020 16:26

pzy: Feb 20, 2004; Da Boom!

Anyone shucked one of the new-looking Easystores? I wonder if it's any different

# ? Oct 13, 2020 16:36

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

Okay so I removed the drive letter for the hard drive with most of my TV shows. To access the drive, from now on I have to go to the Run window, type this: "\\?\Volume{5d62ece9-892f-4342-ad6e-68bbb128c9ea}\ ", and then it will show me that drive's contents.

But is there any way to use that drive from a command prompt? For example, my script is:

code:

dir e:\ /b /s >c:\filez\filesonEdrive.txt

However obviously this no longer works. I've tried this:

code:

dir \\?\Volume{5d62ece9-892f-4342-ad6e-68bbb128c9ea}\ /b /s >c:\filez\fileson8TBdrive.txt

but it refuses to work. Is there anything I can do to make it work?

Chumbawumba4ever97 fucked around with this message at 17:18 on Oct 13, 2020

# ? Oct 13, 2020 16:57

Flipperwaldt: Nov 11, 2011; Won't somebody think of the starving hamsters in China?

Chumbawumba4ever97 posted:

code:
dir \\?\Volume{5d62ece9-892f-4342-ad6e-68bbb128c9ea}\ /b /s >c:\filez\fileson8TBdrive.txt
but it refuses to work. Is there anything I can do to make it work?

Here it says to double the trailing backslash. Seems to work on my end.

# ? Oct 13, 2020 18:11

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

Flipperwaldt posted:

Here it says to double the trailing backslash. Seems to work on my end.

Wow, simply adding a backslash to the end worked! Thanks a billion!

# ? Oct 13, 2020 19:07

BlankSystemDaemon: Mar 13, 2009

Character escaping is endless amounts of fun, right up until it loses you data - so be careful.

# ? Oct 13, 2020 19:21

The Electronaut: May 10, 2009

Had a HGST Deskstar NAS 4TB 7200rpm from 2015 fail over the weekend. Synology did its part and started repairing to the hotspare. I'm left with a conundrum: do I grab another 4TB equivalent from Seagate/WD or do something else? The storage group that drive was in is for my general share which is comprised of 4 4TB drives with 8TB usable and only 2TB is used currently. I'm apt find a equivalent drive instead of trying to do a big uplift to something like 12TB shucks.

# ? Oct 13, 2020 19:37

EVIL Gibson: Mar 23, 2001; Internet of Things is just someone else's computer that people can't help attaching cameras and door locks to!; Switchblade Switcharoo

one of my 10 terabyte drives just died on my ZFS system. thought I could replace one of them with another vDev out of temporary array of four terabyte drives (until the replacement comes in) but I found out you couldn't just add the pool as a drive.

Found a really dirty way to do it* which I actually did when first setting up the original pool and would be much better if zfs could see the pool as a drive by itself.

*used truncate to create a single 11tb file on the new zpool and then replaced the bad drive with it.

# ? Oct 13, 2020 20:35

IOwnCalculus: Apr 2, 2003

EVIL Gibson posted:

one of my 10 terabyte drives just died on my ZFS system. thought I could replace one of them with another vDev out of temporary array of four terabyte drives (until the replacement comes in) but I found out you couldn't just add the pool as a drive.

Found a really dirty way to do it* which I actually did when first setting up the original pool and would be much better if zfs could see the pool as a drive by itself.

*used truncate to create a single 11tb file on the new zpool and then replaced the bad drive with it.

As someone with a lot of drive bays, a lot of 3TB drives, and a habit of abusing the hell out of ZFS... I loving love this.

How hosed is performance?

# ? Oct 13, 2020 21:22

DrDork: Dec 29, 2003; commanding officer of the Army of Dorkness

EVIL Gibson posted:

*used truncate to create a single 11tb file on the new zpool and then replaced the bad drive with it.

<chanting mantra>In *nix, everything is a file</chanting>

# ? Oct 13, 2020 21:28

Tricky Ed: Aug 18, 2010; It is important to avoid confusion. This is the one that's okay to lick.

Synology DS920+ is $440 today on Amazon. I've been eying it for a home Plex server/NAS for a while, but I'd rather wait if deals like this come up regularly. Also if it's a bad idea to do this for other reasons. Any feelings one way or the other?

# ? Oct 14, 2020 00:45

Share Bear: Apr 27, 2004

I'm just saving up enough money in my bullshit budget to pick one up, a synology. I've grown to hate a lot of online streaming services and would rather just archive my own stuff locally.

# ? Oct 14, 2020 01:33

phosdex: Dec 16, 2005

Tricky Ed posted:

Synology DS920+ is $440 today on Amazon. I've been eying it for a home Plex server/NAS for a while, but I'd rather wait if deals like this come up regularly. Also if it's a bad idea to do this for other reasons. Any feelings one way or the other?

That is a new model, so not a lot of price history. You can see Amazon hasn't been lower than $550 before:
https://camelcamelcamel.com/product/B087Z34F3R

# ? Oct 14, 2020 01:47

The Diddler: Jun 22, 2006

I grabbed a few of those Easystores. They'll be shucked and put in my freenas server: a Supermicro case with a SAS2 backplane. Is there any way to know if I need to tape the pin before I slam the drives in?

# ? Oct 14, 2020 12:43

Raymond T. Racing: Jun 11, 2019

The Diddler posted:

I grabbed a few of those Easystores. They'll be shucked and put in my freenas server: a Supermicro case with a SAS2 backplane. Is there any way to know if I need to tape the pin before I slam the drives in?

Taping won't hurt anything so you might as well.

# ? Oct 14, 2020 15:34

Chumbawumba4ever97: Dec 31, 2000; by Fluffdaddy

The Diddler posted:

I grabbed a few of those Easystores. They'll be shucked and put in my freenas server: a Supermicro case with a SAS2 backplane. Is there any way to know if I need to tape the pin before I slam the drives in?

But some Y adapters because they never include that pin. I found placing the tape properly to be almost impossible.

# ? Oct 14, 2020 15:54

Enos Cabell: Nov 3, 2004

Or you can go chaotic neutral like me and just rip the pin off with an x-acto knife.

# ? Oct 14, 2020 16:25

IOwnCalculus: Apr 2, 2003

The Diddler posted:

a Supermicro case with a SAS2 backplane.

Would be very surprised if you needed to tape if you're using this.

# ? Oct 14, 2020 16:41

EVIL Gibson: Mar 23, 2001; Internet of Things is just someone else's computer that people can't help attaching cameras and door locks to!; Switchblade Switcharoo

IOwnCalculus posted:

As someone with a lot of drive bays, a lot of 3TB drives, and a habit of abusing the hell out of ZFS... I loving love this.

How hosed is performance?

code:

                              capacity     operations     bandwidth
pool                        alloc   free   read  write   read  write
--------------------------  -----  -----  -----  -----  -----  -----
neriak                      19.6T  25.8T  1.53K    640   124M  31.0M
  raidz2                    19.6T  25.8T  1.53K    640   124M  31.0M
    wwn-0x5000cca267ee4500      -      -    393      2  30.9M  20.3K
    wwn-0x5000cca273efb9ce      -      -    367      1  30.9M  20.3K
    wwn-0x5000cca26ae68644      -      -    401      1  30.9M  20.3K
    wwn-0x5000cca0b0c4fb7b      -      -    401      1  30.9M  20.3K
    /rottenhand/tmpdisk1        -      -      0    638  34.0K  31.2M
logs                            -      -      -      -      -      -
  sdi64                      260K   123G      0      0     11     11
cache                           -      -      -      -      -      -
  sdi1                      14.7G  85.3G     44      1   178K   118K
--------------------------  -----  -----  -----  -----  -----  -----
rottenhand                  4.00T  6.88T    263    382  32.8M  40.7M
  wwn-0x5000cca25cc2a022    1.16T  2.47T     76    113  9.50M  11.9M
  wwn-0x5000cca24cc140e4    1.15T  2.47T     75    112  9.44M  11.8M
  wwn-0x5000cca25cc2a01c    1.69T  1.94T    111    156  13.9M  16.9M
--------------------------  -----  -----  -----  -----  -----  -----

The three disk pool "rottenhand" (supposed to be "rotten dwarf flesh" from Crushbone in EQ

) is a raid0; like I said, the drive is coming in today and I just didn't want to risk losing a second drive and lose parity.

The resliver process for replacing the drive was super fast to the three drive array , but as you can see in the above, the data split turned out to be heavily favoring the third disk storing 1.6TB compared to the other disks. Not complaining or anything, just interesting.

# ? Oct 14, 2020 18:55

Warbird: May 23, 2012; America's Favorite Dumbass

Grabbed that Synology NAS that�s on sale for PD and a couple of 12TB Easystore drives to shuck. I�m more executed than I should be. Any recommendations on methods to import and manage these files? Best practices and so forth. I�m fine striping our existing permissions as I 777�d the entire drat thing.

Also how big of a m.2 should I look at for caching purposes?

# ? Oct 14, 2020 18:56

Kia Soul Enthusias: May 9, 2004; zoom-zoom; Toilet Rascal

Does anybody know an external USB enclosure for a 3.5" that synology will actually read? None of the poo poo on their compatible list seems to exist anymore. It's for a DS218+

# ? Oct 14, 2020 22:00

EVIL Gibson: Mar 23, 2001; Internet of Things is just someone else's computer that people can't help attaching cameras and door locks to!; Switchblade Switcharoo

Charles posted:

Does anybody know an external USB enclosure for a 3.5" that synology will actually read? None of the poo poo on their compatible list seems to exist anymore. It's for a DS218+

I didn't know there was a compatibility list. I think that list just might be a list of products they actually physically confirmed it worked.

My synology has a random external dock I use as the cache drive for sonarr/radarr and it is not slow at all.

# ? Oct 14, 2020 22:05

Adbot: ADBOT LOVES YOU

# ? Apr 19, 2024 19:31

Chilled Milk: Jun 22, 2003; No one here is alone,
satellites in every home

Taping's not so bad since you can cover up those last 3 pins, you don't have to just get the one.

# ? Oct 14, 2020 22:08

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›840 »