Packrats unite! The consumer NAS/storage megathread

The Something Awful Forums > Discussion > Serious Hardware / Software Crap > Packrats unite! The consumer NAS/storage megathread

Saukkis: May 16, 2003

teamdest posted:

technically correct, sorry. however you're a fool to use a 250 with 2 500's, due to space loss.

Not a big issue with software RAID. Create a RAID5 from three 250 GB partitions and RAID1 from the remaining two 250 GB. You lose 250 GB compared to three 500 GB drive, but it's better than using the 500 GB drives as RAID1 and keeping the 250 GB drive standalone.

# ¿ Mar 24, 2008 05:29

Adbot: ADBOT LOVES YOU

# ¿ May 23, 2013 18:23

Saukkis: May 16, 2003

teamdest posted:

here's an interesting question:

I used LVM under Debian to create a Volume group, then logical volume on that volume group. then some bad poo poo happened, and I had to reinstall. I had consigned myself to the loss of what data was on this Logical Volume, but on attempting to rebuild it, i was greeted with "Can't Initialize Physical Volume /dev/hda of volume group a3p without -ff".

It had this error for all 4 of the disks I had planned to reinitialize. is there any way (since the metadata seems to be intact) to recover the array? all four disks are fine, the information regarding them has just been destroyed.

What exactly have you done? Did you try commands like pvscan, vgscan, lvscan, vgimport?

# ¿ Apr 1, 2008 10:01

Saukkis: May 16, 2003

complex posted:

The original ATA spec just didn't support hot-plug ability.

Pretty much. The Molex power connector wasn't designed for hotplugging either. If you look at the SATA power connector you see several longer pins. These are the ground pins and they connect before the other pins.

That doesn't mean you can't hotplug IDE, I've read about some people doing it. It's just riskier and you need to be more careful. I believe you also need to do a scan in device manager to have it show up.

SATA hotplug works pretty much as it should. Couple weeks ago I needed to connect a SATA drive temporarily, so I just disconnected my SATA DVD drive and connected the HDD and it immediately showed up.

# ¿ Aug 2, 2008 16:57

Saukkis: May 16, 2003

kalibar posted:

The OS hard drive (an old IDE Maxtor 120GB) in my home filebox just died on me today. I don't have any unused SATA ports, and I don't really want to buy another IDE drive.

I do have an open PCI-e x16 slot, an open PCI slot, and a spare 100GB 2.5" laptop SATA hard drive laying around. Is there some kind of magic part I can buy that would let me get this drive into the computer and use it?

One option is to get a SATA to IDE adapter. This is the most simple solution as it avoids any controller issues, but it isn't as cost effective as getting a SATA controller. You may also need rails to attach it to 3.5" bay.

# ¿ Aug 18, 2008 16:54

Saukkis: May 16, 2003

Interlude posted:

Bleh. So what, does Seagate just make all their drives with a 7 second error recovery period? Why is WD the only mfg with this "tech" ?

I believe Seagate has exactly the same system. They have desktop drives and RAID edition drives (ES series?). They call the feature Error Recovery Correction (ERC).

# ¿ Jun 14, 2009 18:12

Saukkis: May 16, 2003

Doh004 posted:

(Double Post)

I lied, it is running the 8.9x version that was causing other people problems too. Now I can't find an exe of the 8.8 version, only a bootable version that I put on my bootable flash drive, but I don't know which file to run. There's no exe or bat file. Just sys, inf, cat and an OEM file.

Try going in the device manager, open storage controller properties, update drivers and search for the flash drive.

# ¿ Aug 13, 2009 17:49

Saukkis: May 16, 2003

Vinlaen posted:

Well, I've been doing some research and apparantly Server 2008 supports SMB2 which would be an advantage I suppose.

It also supports something called "DFS Namespaces" which will combine folders into one share (or something).

Why else would Server 2008 be a bad choice? (what disadvantages does it have?)

DFS isn't really what you want, it's more useful when you have several servers. If you have serverA with shareA, serverB with shareB and serverC with shareC you could have them show up as separate folder under one \\domain\dfs\ share.

# ¿ Aug 15, 2009 21:19

Saukkis: May 16, 2003

Wanderer89 posted:

Oh you guys... you can expand zfs.... you just add an array to the pool! ~looks over at 4tb raidz (1tb * 4) quickly filling up, and cheaper-by-the-day 1.5TB tri-platters~

Does anyone have a cheap solution between home-fab a box that can handle 10+ drives? Am I living in a dreamworld trying to avoid paying a couple hundred bucks for a nice rack/box that can support 10+ drives? Just talking physical space, can do pci-e SAS expansion for the sataII ports.

Is it possible to create ZFS arrays from partitions/slices? You could then divide the drives into several partitions and make separate arrays of them. When you want to add a disk drop the arrays from the pool and recreate them one by one. This is what I do with my MDRAID+LVM setup.

And here's the simple harddrive "rack" I made years ago.

Only registered members can see post attachments!

# ¿ Dec 3, 2009 18:40

Saukkis: May 16, 2003

ufarn posted:

I just can't see how it works out in the end, as the drive will eventually fill itself up with prior files and folders that no longer exist.

Hopefully the program would have the functionally to only keep periodic copies of the files or remove backups for deleted files after certain time. For example if you have a file that changes daily, then a good backup system could keep old versions of it for the past week, after that it could have weekly backups for the past month, monthlies for the past 6 months and after that yearly backups only.

Another approach could be a system where you set how much space the backups are allowed to use and then the backup software figures out a way to maintain maximum amount of different versions as far back as possible without going over the limit.

Of course I wouldn't be surprised if a free backup software that comes with an external drive is lacking such features. But I would be hard pressed to call a software that only does synchronization a backup software.

# ¿ Oct 30, 2010 22:40

Saukkis: May 16, 2003

DLCinferno posted:

The way to use all your disk space is to create separate arrays for each combination of drive sizes, but in order to support at least RAID 5 on all your data you'll need at least 3 drives of each size.

Another way to accomplish the same is to split all the drives into suitably sized partitions, create arrays from the partitions and then combine them with LVM. I'm using an extreme version of this scenario with all my drives split to 10+ partitions. I did it for flexibility when changing and adding drives before RAID expansion was a practical option in Linux.

# ¿ Nov 7, 2010 14:01

Saukkis: May 16, 2003

DLCinferno posted:

True, but I didn't recommend that because you need to be very clever about how you're choosing your RAID levels on the partition arrays and which ones are going into the same array, otherwise a single drive failing could end up wiping out the entire array.

In a simple example, assume two 500GB drives and one 1TB drive. Partition the TB in half and create a RAID5 array across the four partitions. Unfortunately, if that TB drive goes down, it will effectively kill two devices in the array and render it useless.

I'd be curious to see what your partition/array map looks like - it must have a taken awhile to setup properly if you have over ten partitions on some disks?

It really doesn't require much cleverness, simply remember to build the arrays from separate sdX devices.

Here's an example partition table from cfdisk. /dev/sdb/sdc/sdd are similar, just with slightly different amount of partitions. /dev/sda1 is housing the operating system currently. In the future I'll RAID1 it with another partition.

code:

   Name      Flags    Part TypeFS Type        [Label]     Size (MB)
 -----------------------------------------------------------------
   sda1      Boot      Primary Linux raid autodetect       15002.92
   sda2                Primary Linux raid autodetect       81923.79
   sda3                Primary Linux raid autodetect       81923.79
   sda5                Logical Linux raid autodetect       81923.79
   sda6                Logical Linux raid autodetect       81923.79
   sda7                Logical Linux raid autodetect       81923.79
   sda8                Logical Linux raid autodetect       81923.79
   sda9                Logical Linux raid autodetect       81923.79
   sda10               Logical Linux raid autodetect       81923.79
   sda11               Logical Linux raid autodetect       81923.79
   sda12               Logical Linux raid autodetect       81923.79
   sda13               Logical Linux raid autodetect       81923.79
   sda14               Logical Linux raid autodetect       81923.79

Here's a snippet from /proc/mdstat. /dev/md10-md18 are 245GB RAID5 arrays with 4 devices. /dev/md19-md21 are 163GB RAID1 arrays with 2 devices. They are made from the leftover partitions from the 2 1TB drives.

code:

md21 : active raid1 sdc14[0] sda14[1]
      80003584 blocks [2/2] [UU]

md10 : active raid5 sdd2[0] sdc2[1] sdb2[2] sda2[3]
      240010752 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

md11 : active raid5 sdd3[0] sdc3[1] sdb3[2] sda3[3]
      240010752 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

Then I've created two LVM volume groups of those arrays, sized 915GB and 1.34TB, but I'm considering combining them into one VG. The original reason for the two VGs was that I didn't have enough drives for my needs. vg_safehouse was made of RAID1 or RAID5 devices for storing more valuable files and vg_warehouse was a single large drive for the rest. Now that everything is RAIDed there's no need for warehouse anymore.

DLCinferno posted:

Sure are. Literally, unplug from one machine, plug into the new one, and run one mdadm --assemble command per array. As long as the computer can see the same physical drives/partitions, it doesn't matter what hardware it's running.

That's one of the main reasons I like ZFS/mdadm at home - no need to buy pricey hardware controllers, but you get most of the same benefits.

I think if you use partitions se to RAID autodetect type you don't even need the assemble command. During boot up Linux kernel will see a bunch of partitions that seem to belong to an array and then it figures out which of them go together.

# ¿ Nov 13, 2010 18:04

Saukkis: May 16, 2003

FooGoo posted:

I didn't see this question posed in the fact or forum so here it goes:

Is there any advantage to buying an external drive versus an internal drive and an enclosure provided it will only be used occasionally and won't be thrown across the room?

Whenever I've looked externals were available for slightly cheaper then enclosure and drive. Externals often also have nicer or more refined looks and may have some extra features.

Otherwise I would usually recommend enclosure and drive. You can upgrade the drive to bigger one later, you can choose what drive goes inside, instead of most likely the cheapest drive the manufacturer could find. If there's some kind of failure with the enclosure you can take the drive out and have better chances of recovery without voiding the warranty which would probably happen with external.

# ¿ Dec 28, 2010 22:33

Adbot: ADBOT LOVES YOU

# ¿ May 23, 2013 18:23

Saukkis: May 16, 2003

Is any of the current 2TB eco-harddrives suitable for use with Linux software RAID? I vaguely remember that earlier WD Green Powers had some behaviours that made them not as good for RAID. I doubt the performance would matter much since the rest of the harddrives would be older models and the price difference to non-eco models is upto 40�.

# ¿ Sep 1, 2011 11:03

The Something Awful Forums > Discussion > Serious Hardware / Software Crap > Packrats unite! The consumer NAS/storage megathread