|
Doesn't the URE work out to a rated workload of ~150-200TB / year for disks, which would be between 15-20 scrubs every year, ie. more than a scrub every 31st day would get you? I think I recall something about URE being when the manufacturer thinks the worst-case of an URE happening is. EDIT: Just because you have 1:1x10^15 chances of getting an URE doesn't mean that when you read that quadrillion bit you're guaranteed to get a read-error. BlankSystemDaemon fucked around with this message at 19:02 on Jan 1, 2020 |
#
?
Jan 1, 2020 18:57
|
|
|
|
| # ? Apr 26, 2024 17:29 |
|
|
D. Ebdrup posted:Doesn't the URE work out to a rated workload of ~150-200TB / year for disks, which would be between 15-20 scrubs every year, ie. more than a scrub every 31st day would get you? Stupidly, I was scrubbing that pool once a week for a couple of years. Speaking of that old raidz2 of mine.... It's a 3TB-disk-based raidz2 we'll call "tank1". I want to recover a handful of disk bays and ports by removing these 3TB disks.
I guess this is the best way to do this?
|
|
#
?
Jan 1, 2020 20:29
|
|
|
eames posted:Neat, that answers my question then. I wasn't aware that ZFS is the default standard for raid arrays now. I'm sure there are still people out there running normal RAID (I seem to remember Linus Tech Tips being taken down a few years ago because they were doing some idiotic RAID0 setup), but the majority of people in this thread are running something more advanced. ZFS is very popular, but Synology's SHR and some of the other software implementations also get solid play. The thing about manufacturer specs is they should be considered a worst case base line: if they produce drives that perform worse than the numbers on the box, they'll get sued. So chances are that those are very conservative numbers, particularly because they haven't really changed much in years, nor do they vary much between models or product lines. Consider SSD write endurances: the stated limits have been found to often be something like 1/10th or less of what the drive is actually capable of doing before it fails.
|
|
#
?
Jan 1, 2020 20:56
|
|
|
Next major question is what’s the best way to really backup my crap before I swap drives out? Backblaze seemed like a good option but I’m not sure I understand their pricing.
|
|
#
?
Jan 2, 2020 21:26
|
|
|
Do you not have enough open ports to plug the new and old drives in at the same time? Or have a friend with a spare drive you can life boat everything on to to facilitate the move? Otherwise, yeah, Backblaze isn't a terrible way to go, though I seem to remember there being some hoops if you wanted it to run natively on a NAS.
|
|
#
?
Jan 2, 2020 21:45
|
|
|
DrDork posted:Do you not have enough open ports to plug the new and old drives in at the same time? Or have a friend with a spare drive you can life boat everything on to to facilitate the move? I just dont have a spare external drive. I do have a really, and I mean REALLY stupid question. Right now I have 2 out of my 8 drives as parity/backup whatever ZFS calls them. If one was removed (say the one that had the most errors on it) and used as a temporary external via a caddie I have, how does that affect my pool? I am assuming that it would be totally and completely disastrous and should not be done ever at all.
|
|
#
?
Jan 2, 2020 22:19
|
|
|
The entire point of parity drives is that it isn't disastrous. What would happen is you'd get an angry message about the pool being degraded. Otherwise it would continue to work normally. In fact, you could do that with TWO drives and still have everything be fine (please don't do that). Once you hit a third drive, though, the pool is now in a non-functional state. To your point, though, yes you could theoretically pull one drive to use as a transfer disk, though you might need to take some steps to keep the system from rejoining it to the pool as soon as it's connected, or from banning it entirely because you told the system it was a bad drive. A better (if somewhat exploitative) option would be to buy an additional drive and return it when you're done with it. Or use Backblaze. I don't mean to be a dick, and I know this thread is happy to answer your questions, but I gotta ask: how did you end up with a ZFS array without knowing the very basics of what Raidz provides? I mean, most people don't just accidentally end up setting up a custom NAS.
|
|
#
?
Jan 2, 2020 22:30
|
|
|
KKKLIP ART posted:Next major question is what’s the best way to really backup my crap before I swap drives out? Backblaze seemed like a good option but I’m not sure I understand their pricing. https://www.backblaze.com/b2/cloud-storage-pricing.html#calculator Put it in there, see what you get. If you do a full upload and download in a month it's $15/TB.
|
|
#
?
Jan 2, 2020 22:34
|
|
|
DrDork posted:The entire point of parity drives is that it isn't disastrous. What would happen is you'd get an angry message about the pool being degraded. Otherwise it would continue to work normally. In fact, you could do that with TWO drives and still have everything be fine (please don't do that). Once you hit a third drive, though, the pool is now in a non-functional state. To your point, though, yes you could theoretically pull one drive to use as a transfer disk, though you might need to take some steps to keep the system from rejoining it to the pool as soon as it's connected, or from banning it entirely because you told the system it was a bad drive. I think KKKLIP is saying they want to pull a drive out and reconnect it - still as part of the pool but now via USB instead of the internal SATA/SAS ports. Which - if your pool is set up to use UUIDs or other static identifiers, will be about as fine as it can be.
|
|
#
?
Jan 2, 2020 22:40
|
|
|
DrDork posted:I seem to remember Linus Tech Tips being taken down a few years ago because they were doing some idiotic RAID0 setup Doing dumb poo poo and making videos about it is his whole shtick, though?
|
|
#
?
Jan 2, 2020 22:49
|
|
|
DrDork posted:I don't mean to be a dick, and I know this thread is happy to answer your questions, but I gotta ask: how did you end up with a ZFS array without knowing the very basics of what Raidz provides? I mean, most people don't just accidentally end up setting up a custom NAS. No it isn't being a dick and is a fair question. I know that a raidZ2 gives me 2 drives that could drop. I've never had a drive fail, so it was more of a "if I pull a drive, can I still access the data and work with it or will it make me swap in a drive before I can access the data again." From what it sounds like, I theoretically could pull the drive, have access to the data, toss the "spare" into a drive caddy, format it and copy the stuff, replace the drives in my NAS, and paste the stuff onto the new drives/pool in my NAS. I think Crashplan is a better option, to be honest and I am going to try their free trial to see how that works before I start yanking a drive out, because I'd rather not run into an issue where I pull 1 aging drive and all of a sudden I have a second failure because of course I would. I built a NAS because I had too much crap for externals to be really feasible and there isn't anything specifically mission critical other than some easily backed up documents and pictures, so I didn't mind a learn as I go. IOwnCalculus posted:I think KKKLIP is saying they want to pull a drive out and reconnect it - still as part of the pool but now via USB instead of the internal SATA/SAS ports. Yeah I would pull the drive, let the pool complain a bit, format it in WIndows using a drive caddy and just manually copy/paste.
|
|
#
?
Jan 2, 2020 22:54
|
|
|
Oh, nevermind, I thought you were just trying to free up internal slots for the new drives. DrDork is right then.
|
|
#
?
Jan 2, 2020 23:00
|
|
|
KKKLIP ART posted:No it isn't being a dick and is a fair question. I know that a raidZ2 gives me 2 drives that could drop. I've never had a drive fail, so it was more of a "if I pull a drive, can I still access the data and work with it or will it make me swap in a drive before I can access the data again." From what it sounds like, I theoretically could pull the drive, have access to the data, toss the "spare" into a drive caddy, format it and copy the stuff, replace the drives in my NAS, and paste the stuff onto the new drives/pool in my NAS. Yes, you would basically bring yourself down to raidz1 during the degraded period. Access would work fine. I would make a backup of your data before you start, as you have plans on doing, for the reasons you are mentioning. A free trial of crashplan is fine assuming you can actually get the data up to them - they used to restrict their upload speeds on consumer plans pretty horrifically. I don't know if their corporate plans are any better in that regard.
|
|
#
?
Jan 2, 2020 23:05
|
|
|
H110Hawk posted:Yes, you would basically bring yourself down to raidz1 during the degraded period. Access would work fine. I would make a backup of your data before you start, as you have plans on doing, for the reasons you are mentioning. A free trial of crashplan is fine assuming you can actually get the data up to them - they used to restrict their upload speeds on consumer plans pretty horrifically. I don't know if their corporate plans are any better in that regard. Let's not confuse Crashplan and Backblaze. AFAIK, Crashplan ended their consumer segment a while back, and did indeed cap transfer speeds. Backblaze on the other hand claims not to throttle at all, so it should be pretty fast if you've got a good connection. As IOwnCalculus noted, the problem that might crop up if you just pull and format a drive is the system may still be smart enough to recognize the drive as having been part of the array, and then it'll try to pull it back into the pool. The caddy may or may not prevent that, depending on how it presents itself. Also I believe FreeNAS/NAS4Free (which I assume is what you're running) was still lacking USB3.0 drivers last I checked, meaning transfer rates would not be particularly great if done directly on the box--if you have a GigE network with your NAS and Windows box plunking the life raft drive onto Windows and copying things there would be faster. Buuuuut if you do that I wouldn't format it as NTFS (use extFAT) because Windows ACLs on FreeBSD like to gently caress poo poo up if you're not prepared for it. And you're right to want to back everything up first: when you go being fancy is absolutely exactly when things that had otherwise been working fine for ages decide that right loving now is a great time to die. Also remember that you can always just plug your old array back in at any point and it should Just Work, should it come to that.
|
|
#
?
Jan 2, 2020 23:19
|
|
|
DrDork posted:Let's not confuse Crashplan and Backblaze. AFAIK, Crashplan ended their consumer segment a while back, and did indeed cap transfer speeds. Backblaze on the other hand claims not to throttle at all, so it should be pretty fast if you've got a good connection. Ah, yes. Correct. I was making the distinction in my head but for clarity. Backblaze: Upload speeds are indeed zippy, both on B2 (their "S3-compatible object store" aka pay-per-byte upload service) and their desktop consumer application. I use both. Crashplan Consumer: Terminated, slow, bloated client, etc. Crashplan Business (likely has another name): Same bloated awful client, I don't know about speeds. Google will know.
|
|
#
?
Jan 2, 2020 23:28
|
|
|
I kept Crashplan on after they forced the transition to business for a year. Speeds were still poo poo.
|
|
#
?
Jan 2, 2020 23:30
|
|
|
I'm also remaining on crashplan, I've got 4TB I don't want to loose but don't need back in a hurry.
|
|
#
?
Jan 2, 2020 23:51
|
|
|
KKKLIP ART posted:No it isn't being a dick and is a fair question. I know that a raidZ2 gives me 2 drives that could drop. I've never had a drive fail, so it was more of a "if I pull a drive, can I still access the data and work with it or will it make me swap in a drive before I can access the data again." From what it sounds like, I theoretically could pull the drive, have access to the data, toss the "spare" into a drive caddy, format it and copy the stuff, replace the drives in my NAS, and paste the stuff onto the new drives/pool in my NAS. Actually almost asked this when you first posted, but it looked like you had just one drive fail -- you know you can just replace it and resilver the array to restore redundancy? If you're looking to upgrade for more silence or whatever, that's great -- but it looks like you have about an $80 problem. I'm still on Crashplan and it's getting pretty dire. Speeds with 9TB were poo poo -- like capped <100kbps -- unless you manually turn off deduplication. The newest clients no longer let you manually turn off dedupe.
|
|
#
?
Jan 3, 2020 00:28
|
|
|
Thermopyle posted:Yeah, this has always puzzled me. basically the takeaway from the perpetual argument about UBE rates and RAID5 resilvers is that the UBE rates that manufacturers publish are quite pessimistic... real world error rates are probably 10-1000x better (lower) than the spec Paul MaudDib fucked around with this message at 05:33 on Jan 3, 2020 |
|
#
?
Jan 3, 2020 05:30
|
|
|
And since it's a new page, someone should probably mention that RAID is not backup? 
|
|
#
?
Jan 3, 2020 06:16
|
|
|
To go the opposite route, the other day, I got a ticket from a company that was storing data on a single harddrive, and then the harddrive went bad. The conversation included, "so was that quote of 5800 dollars to recover the drive legitmate" and "I really need a mirrored drive to help prevent this". Convinced him to keep what he had and add on to it, so he could do a Raid 1 backing up to his Single drive partition HDD, and then a copy in cloud storage. Next time he won't have to drop 6000 bucks on harddrive recovery I bet!
|
|
#
?
Jan 3, 2020 08:25
|
|
|
So a few months ago I posted about my desktop PC and what a mess it was with maxed out internal hard drive space being used (because I was stubborn and insisted on keeping in stuff like 2 and 4 TB hard drives) and 10 external 8TB drives. Anyway I took a lot of the advice here to heart and got rid of all the smaller internal drives, and shucked the externals and put them internally in the tower. It looks a lot better, it's nice having less drive letters, and overall I'm glad I did it. I want to go one step further though and get rid of the 5 externals I have. It looks stupid, takes up 5 outlets on my UPS, and I just know I am going to accidentally tip one over some day. So I have been looking at 5 bay external enclosures (non-raid ones) and there's two I am trying to decide between (obviously this means I am going to shuck the 5 remaining external WD drives): https://www.amazon.com/ORICO-DS500U3-US-BK/dp/B07VMK6ND7 if you click on that link, there's two options. One is $179 USD, and one is $139 USD. The only difference I can see between the two is that one is USB 3.0, and the more expensive one is USB 3.1. Would the 3.0 one do just as good as a job? I am asking because all these external drives were USB 3.0 anyway, and I don't exactly think they are slow. That being said, I have no idea how USB 3.0 handles file transfers. For example, if I am copying and pasting data from drive #1 to drive #3 in that enclosure in Windows, is that going to "choke" the USB 3.0 "bus" or whatever it's called? I am not talking about copying and pasting data from an internal SATA drive to one of the 5 drives in the enclosure. I mean one of the drives in the enclosure to another drive in the enclosure. Not that I will be copying and pasting data that much, but I am trying to figure out how USB 3.0 handles a device like this. It's kind of weird, in my brain, for two hard drives to be copying/pasting data from one another on the same exact USB port. As for the $179 version, that one is USB 3.1, and I'd appreciate the peace of mind that it's faster. The only issue is my motherboard only has one USB C port, and it's being used by something. However, I do have a few red USB 3.1 ports free. Can I use something like this?  Is that simply a "gender changer"? Or is there circuitry going on inside that thing? I am hoping it's the former, because if it's the latter I am imagining the thing burning up trying to support a 5 bay hard drive caddy. But if it's simply a gender-changer (no "conversion" or chips involved at all) then I would imagine the USB-C version of the enclosure would work just as well as if it were plugged into a "real" USB-C port. But then again, I can't imagine this adapter would support an e-GPU or something so I really have no idea. Anyway thanks in advance for any help with my questions!
|
|
#
?
Jan 3, 2020 12:59
|
|
|
WWWWWWWWWWWWWWWWWW posted:...USB stuff... To answer your question on USB-A to USB-C adapters, yeah, they're just dumb little wire converters. However, they don't all have the pins to support the highest data speeds. The one you linked notes a max of 480Mbps (which means it's probably really USB 2.0 internally and meant more to charge lovely cell phones with than anything else), so I'd skip that and go for something that explicitly supports 10Gbps to ensure you can get the most out of your system. For USB 3.0 vs USB 3.1, it's unlikely to make much of a difference in this regard. USB 3.0 should support 5Gbps, or ~625MBps, which is more than 5x HDDs are likely to be able to saturate (figure they might be able to do ~120MBps each max, so you're looking at ~600MBps tops). USB 3.1 might up that to 10Gbps, but it doesn't say if it's implementing the 10Gb or the 5Gb option, so I'd suspect it's probably the same 5Gbps just using the USB-C connector. The little blurb on Amazon claims it's "up to 20% faster" but who knows how that'd actually play out, since if it were USB 3.1 gen 2 you'd expect them to be crowing about it and claiming much higher top speeds.  As for how data gets transferred, unfortunately external dock/bays like that are dumb bunnies and are basically a USB switching chip connected to a bunch of SATA connectors. What that means is that they're not smart enough to keep data traffic internal to the dock: data going from dock drive A to dock drive B will take a trip from drive A down the cable to your PC and then back up the cable to drive B. This effectively means you'll get half the transfer speed you'd get going from PC drive X to dock drive B or whatnot, since you're moving the data across the wire twice.
|
|
#
?
Jan 3, 2020 15:48
|
|
|
DrDork posted:For USB 3.0 vs USB 3.1 You mean USB 3.2 Gen 1 vs USB 3.2 Gen 2 (gently caress the USB IF forever) vvv isn't it just the most confusing, ridiculous crap? HalloKitty fucked around with this message at 02:35 on Jan 4, 2020 |
|
#
?
Jan 3, 2020 15:51
|
|
|
Ugh, yeah, I keep forgetting that they decided to ret-con their naming scheme. Point still stands: shouldn't make a difference for this use case.
|
|
#
?
Jan 3, 2020 15:53
|
|
|
Working on getting the Force10 switches in my M1000e configured to split the QSFP 40GB to 4x10GB so I can direct attach the NAS for VM storage. Hopefully get that done in a couple days.
|
|
#
?
Jan 3, 2020 16:41
|
|
|
CommieGIR posted:Working on getting the Force10 switches in my M1000e configured to split the QSFP 40GB to 4x10GB so I can direct attach the NAS for VM storage. Hopefully get that done in a couple days. Please tell me you are running a M1000e at home.
|
|
#
?
Jan 3, 2020 17:10
|
|
|
Moey posted:Please tell me you are running a M1000e at home.  Yup! Just installed the M915 with 4 x 16 core AMD 6378s and 128GB of RAM, going to migrate and retire my old SuperMicro quad Opteron box and run exclusively off the M1000e  (older photo, the M620 in slot 1 got replaced with the M915) The FreeNAS on the R720 has 2 x 10GB Fiber, I'm linking that into the Force10 switch on the M1000e to provide NFS share for the VMs. CommieGIR fucked around with this message at 17:32 on Jan 3, 2020 |
|
#
?
Jan 3, 2020 17:29
|
|
|
I'm a little surprised to not see one of these on your wall:
|
|
#
?
Jan 3, 2020 18:55
|
|
|
CommieGIR posted:
I love everything about this.
|
|
#
?
Jan 3, 2020 19:01
|
|
|
CopperHound posted:I'm a little surprised to not see one of these on your wall: Oh man, I dont but our office does, as its an old IBM building.
|
|
#
?
Jan 3, 2020 19:03
|
|
|
DrDork posted:To answer your question on USB-A to USB-C adapters, yeah, they're just dumb little wire converters. However, they don't all have the pins to support the highest data speeds. The one you linked notes a max of 480Mbps (which means it's probably really USB 2.0 internally and meant more to charge lovely cell phones with than anything else), so I'd skip that and go for something that explicitly supports 10Gbps to ensure you can get the most out of your system. So while those C female to A male adapters do technically exist, I'd strongly advise against purchasing any. That specific adapter configuration is considered a SHALL NOT by the USB forum and while they're dumb when it comes to naming generations, they do have very good reasons as to calling that a SHALL NOT. USB-A ports have all been designed with the assumption that they're the host port, not the slave, and there could never be any power flowing towards the host from either another host or a power source. Using a C female to A male adapter lets you get rid of that physical lockout, and allows you to plug a host into a USB-A host (which at best will just not work), and more dangerously allows you to plug a crappy USB-C power adapter into the USB-A port of a computer, a port which was never designed to handle power flowing into it, and since it was a physical impossibility, over current/voltage protections are minimal going inbound.
|
|
#
?
Jan 3, 2020 19:36
|
|
|
DrDork posted:To answer your question on USB-A to USB-C adapters, yeah, they're just dumb little wire converters. However, they don't all have the pins to support the highest data speeds. The one you linked notes a max of 480Mbps (which means it's probably really USB 2.0 internally and meant more to charge lovely cell phones with than anything else), so I'd skip that and go for something that explicitly supports 10Gbps to ensure you can get the most out of your system. You rock! Thank you! I will just go with the normal USB 3.0 one.
|
|
#
?
Jan 3, 2020 19:52
|
|
|
Buff Hardback posted:Using a C female to A male adapter lets you get rid of that physical lockout, and allows you to plug a host into a USB-A host (which at best will just not work), and more dangerously allows you to plug a crappy USB-C power adapter into the USB-A port of a computer, a port which was never designed to handle power flowing into it, and since it was a physical impossibility, over current/voltage protections are minimal going inbound. While you are correct on this, there's also zero reason power should be flowing up the line from the USB split chip that's on the dock/expander in the first place. But, yeah, don't use it to plug a power brick / charger in with.
|
|
#
?
Jan 3, 2020 22:01
|
|
|
DrDork posted:While you are correct on this, there's also zero reason power should be flowing up the line from the USB split chip that's on the dock/expander in the first place. But, yeah, don't use it to plug a power brick / charger in with. Really more than anything, it breaks the USB-A host/slave assumption, and if used improperly could cause bad things. They're a thing that doesn't add any positives for me, and just adds scare factor.
|
|
#
?
Jan 3, 2020 22:35
|
|
|
DrDork posted:For USB 3.0 vs USB 3.1, it's unlikely to make much of a difference in this regard. USB 3.0 should support 5Gbps, or ~625MBps, which is more than 5x HDDs are likely to be able to saturate (figure they might be able to do ~120MBps each max, so you're looking at ~600MBps tops). USB 3.1 might up that to 10Gbps, but it doesn't say if it's implementing the 10Gb or the 5Gb option, so I'd suspect it's probably the same 5Gbps just using the USB-C connector. The little blurb on Amazon claims it's "up to 20% faster" but who knows how that'd actually play out, since if it were USB 3.1 gen 2 you'd expect them to be crowing about it and claiming much higher top speeds. Just FYI, modern HDDs can hit around 200 MB/s in sequential transfers, but to be fair mixed transfers are more common and in all likelihood he's probably not going to be accessing all 5 drives simultaneously. Also the real world USB throughput is less than nominal due to overhead, and then there are the limitations of USB MSC vs. UASP. All I'm really getting at is that the dock he's looking at should be fine and he shouldn't be worried about the numbers.
|
|
#
?
Jan 4, 2020 09:54
|
|
|
The operative word, because it's doing a lot of work, in that sentence being 'can' - it only happens under scenarios which require careful preparation, such as ZFS dRAID that I mentioned before where meta-data is stored on non-volatile flash and data is written in 10MB blocks to the disks to make use of sequential I/O. And even then, a long-tail request that's perfectly allowed for in the 8ms average that disks have, can ruin the fast I/O very easily - and they happen more often than people think.
|
|
#
?
Jan 4, 2020 13:17
|
|
|
The 8TB WD Elements, EasyStore, and MyBooks are all shuckable alike, correct? I just noticed I have three type of drives. I only have one 8TB external that's not Western Digital and it's a Seagate. Are those shuckable, too? It's definitely a single 3.5" drive but I don't know if Seagate does some fuckery like changing the SATA plug or linking the firmware serial number to the USB controller or something.
|
|
#
?
Jan 4, 2020 17:44
|
|
|
Thermopyle posted:Stupidly, I was scrubbing that pool once a week for a couple of years. Conversation moved on but step 3 above really should be: 3. syncoid / zfs send/receive to copy snapshots from tank1 to tank3. But other than that, that's how I plan on updating my 6x 4TB drives in another 2-3 years.
|
|
#
?
Jan 4, 2020 18:57
|
|
|
|
| # ? Apr 26, 2024 17:29 |
|
|
WWWWWWWWWWWWWWWWWW posted:The 8TB WD Elements, EasyStore, and MyBooks are all shuckable alike, correct? I just noticed I have three type of drives. Yes. The Elements and Easystore are pretty much identical to each other, the MyBook is only marginally different. The MyBook my friend just shucked the other day required a torx bit to remove the drive from the internal carrier.
|
|
#
?
Jan 4, 2020 19:07
|
|