With Google photos dropping their free uploading in a few months I’m starting to look at building a NAS. I have a bunch of shingled 8 TB hard drives from a dumb foray into crypto a few years ago. Obviously shingled hard drives have abysmal write speeds but have acceptable read speeds. If I were to shuck these drives and toss them in a freeNas set up using an old laptop of mine to drive it would it work? This would mostly be for archival of my photos and maybe to run a single 4K stream capable Plex server. Obviously it being capable of 4K would depend on the power of the laptop. But it is able to drive a 4K video locally right now in windows so I would imagine it might work. Alternatively, could I just connect the drives via USB 3 instead of shucking them? That would simplify the set up at least. It’s really not a huge deal for me if it’s not the quickest thing in the world. I am perfectly willing to accept a five second to 10 second buffering time to start a video or back up a picture. I would set up that stuff to run on a schedule when I’m not using my phone anyway so it’s not a big deal if it takes longer than ideal while it’s charging on my nightstand at night. Nitrousoxide fucked around with this message at 04:24 on Jan 3, 2021 |
|
# ¿ Jan 3, 2021 04:21 |
|
|
# ¿ May 7, 2024 03:55 |
movax posted:I would try to avoid USB at all, even if it's low stakes application such as the one you described. Does that old laptop have Thunderbolt? At least then you could run to some kind of PCIe controller externally and mostly use your existing hardware. Yeah, two thunderbolt ports, so I could power it with one and use the other to drive a PCIe enclosure. Maybe that's the way to do it. Maybe something like this? https://eshop.macsales.com/item/AKi...cB&gclsrc=aw.ds Warbird posted:Without getting too into the weeds, what sort of crypto scheme were you running that left you with umpteen multiTB drives lying around? GPUs I could understand. As someone said before, they were for Burstcoin. A proof of capacity crypto where you process the math only once and write it to a harddrive for later searching. It's much lower power usage (since you're not driving gpu's with tons of power calculating stuff over and over) was appealing to me at the time. It never really took off, and they changed the math about a year into my foray into it and it invalidated all the stuff written into my harddrives and would have forced me to recalculate everything again. Since it took my piddly GPU a month to do it last time I just said gently caress it and put them into storage. Takes No Damage posted:My understanding (which is admittedly limited) is that SMR drives should not be used for NAS at all. It isn't just the slower performance, they are actively less reliable in a RAID config. It's that the data shuffling that comes with shingling butts up directly against the distributed data writing of a RAID, especially in ZFS which you'd get in Free/TrueNAS. It basically causes the drives to be thrashing around all the time, shortening their lifetimes that much more (and these are already old/used drives). So one of them dies and you put in a new replacement drive, your pool will have to resliver the array to write all the necessary data to the new drive, which will thrash all your other drives as it reads through each one 100%. That's already a slow and vulnerable process, and because of shingling it can take days instead of hours. Mirroring should be okay then though right? That's just a straightforward mirrored backup of what's on another drive. Obviously mirroring doesn't scale well, but if I just use two of these then that might be okay? I've got no other use for them otherwise. https://www.amazon.com/gp/product/B01HAPGEIE/ref=ppx_yo_dt_b_asin_title_o06_s00?ie=UTF8&psc=1 8TB will probably be sufficient for my needs for the time being. lampey posted:SMR drives are fine for raid configs that do not use distributed parity, raid 0, 1, 10, snapraid, unraid, spanned. And they can be acceptable for raid 5, it depends more on the specific disks, how much you are writing, and if the sustained write speed is still fast enough. Most of the 8tb SMR drives have 20-25gb of non shingled data. It is used as a cache before data is written to the shingled portion. With four drives in a raid 10 or spanned etc, for writing files less than ~100gb you will get speeds in the 125mb a sec range and wouldn't notice any difference between conventional drives. If you use a type of raid with distributed parity you will hit the limit much faster, parity is going to be computed multiple times as data is shuffled around, and specifically ZFS will split things into a greater number of smaller writes, more likely to need to rewrite shingled areas. For 8tb SMR disks you still get ~30mb a sec per disk for sustained writes instead of 150-180 for a non SMR drive, with 4 disks you are still limited by gigabit networking and not the disks. Smaller and older disks are slower. The main time this is an issue is when you are rebuilding a failed disk, you are getting much faster speeds without the network. It will take around 48-60 hours hours to rebuild an SMR drive compared to around 16 hours to rebuild a regular 8tb sata drive. I'm not sure which drive these are (linked above) since they're still in their enclosure. My research says they are most likely seagate archive drives, but that they also used other drives in their manufacture as well. Nitrousoxide fucked around with this message at 16:12 on Jan 3, 2021 |
|
# ¿ Jan 3, 2021 14:10 |
redeyes posted:When google said they were stopping the free photo storage I decided to do Nextcloud on a linux server. Threw that through Cloudflare. Tossed apps on my phone. AWESOME!! Did you just use the free cloudflare set up? Maybe I should just go with a proper NAS set up Instead of trying to rig up something with these shingled drives. There are a few things that I would want to do on there including media hosting for local streaming through Plex, back up of pictures and video taken by my phone, and hosting a handful of servers or modules like home bridge that are currently being hosted on a few raspberry pi's. I suppose I could still use the shingled drives to provide the offsite back up for my NAS. Just periodically back it up to them In case my house catches fire or something. Or would people recommend I use a cloud provider to provide the offsite backup portion of a secure and properly backed up NAS solution?
|
|
# ¿ Jan 3, 2021 23:48 |
Well, I got my TrueNAS set up with my laptop in a couple of those shingled drives. Man is it slow. It’s going to take about six or seven hours to do an 80 GB time machine back up After I set up a pool for that. I think I’ll just use this for Time Machine back ups. It’s way too slow to be any sort of media server. Later in the year I’ll probably build a proper NAS and port all the data I have on this one over.
|
|
# ¿ Jan 5, 2021 03:20 |
The laptop itself isn't really under powered. It's barely even moving the needle on the CPU and it has plenty of RAM. The disc write speeds are just really dreadful. The network speeds aren't anything to write home about either. I don't think it should be saturating a 5 GB USB 3 port. So I'm not sure why it's running so incredibly slowly. The only thing I can think of is that the drive itself is the limiting factor here. My guess is it's just not playing very nice with either the file system on the drives or how the USB interface is working with TrueNAS Edit: I found a speed test plug-in and installed that. It's definitely not an issue with the network interface or usb. So it's got to be the drives Nitrousoxide fucked around with this message at 05:11 on Jan 5, 2021 |
|
# ¿ Jan 5, 2021 05:06 |
Aside from my drive speed issues, which I'm sure are hardware related and not the fault of the OS, I have been extremely happy with TrueNAS. It was quick and easy to set up initially, the user and group permissions are pretty self-explanatory and sane. The jail-based system works similarly to docker, and you can install docker if you want to use that instead. Setting up docker in TrueNAS core https://youtu.be/XBVjuwgz0Cg There's a bunch of built-in stuff that could be really useful depending on what you're deploying it for. Like you don't need a separate module to do active directory or snapshots or home directory (especially useful if you're deploying this for use as a virtual machine server. Home directory is really nice because it'll let it automatically create individual share directories for users you give access to a pool rather than having to manually create them one by one) For me and my relatively straightforward use case. I'm able to set up all of the servers that I would want to through their officially supported plugins and the community plugins. the official list of community plugins actually include a lot of things that might be useful for you if your purpose includes Oh, and it uses the ZFS file system which is extremely easy to expand with additional storage if you need it later. Edit: TrueNAS will also be releasing TrueNAS Scale which is linux-based rather than bsd-based and will have full docker support out of the box rather than having to run a VM that runs docker like you do on the current one. The release version of it should be out later this year. Nitrousoxide fucked around with this message at 22:30 on Jan 5, 2021 |
|
# ¿ Jan 5, 2021 22:13 |
fletcher posted:How is the speed test proof that it's not an issue with USB? Yesterday I SSH’d into the server and did a connection test between it and one of my devices on my local network and it was extremely slow. So it’s probably something to do with the the network interface actually. I'm using a USB Ethernet adapter. looking through some of the discussions about the model I chose seems like people who were using BSD have complained about it not supporting it at full speed. So that's probably my problem. Nitrousoxide fucked around with this message at 14:47 on Jan 6, 2021 |
|
# ¿ Jan 6, 2021 13:35 |
I ended up experimenting with openmediavault as it's a Linux-based system. My hope was that it would have better driver support for my ethernet USB dongle. Luckily it did and now I'm getting full transfer speeds that I would expect. It probably wisely doesn't let me set up a raid with my USB drives so I've just set up one drive as a media source and added a second drive there which does periodic back ups from the first to the second. It has full docker and comes with yacht and portainer which makes installing them pretty easy. Overall this seems like a pretty good set up for the hardware that I have in hand so I don't have to spend anything to get it up and running an experiment with it. Nitrousoxide fucked around with this message at 00:46 on Jan 9, 2021 |
|
# ¿ Jan 9, 2021 00:42 |
GreenBuckanneer posted:Well I mean the viable alternative is raid 10 but then that's half the storage space. I'd probably go for raid 5 because it's unlikely for more than 1 drive to fail at a time, although I have seen entire arrays go down at once, it's uncommon enough in raid 5 that businesses have sued the supplier over it. For a person like me it's fine, i'd be backing up anything anyways A good RAID array is not a backup, it's just a setup to reduce the risk of failure and increase read/write speeds. You still need at least one off-site backup. I'd personally have an on-site backup as well as that allows a much faster data recovery in the event of a catastrophic failure of the array (and insures against a loss of quick internet access preventing recovery) but your budget will determine feasibility.
|
|
# ¿ Jan 10, 2021 23:57 |
movax posted:Goddammit. Stumbled onto today, my X11SSL-CF has a C232 which stops me from IGP pass-thru of the Kaby Lake (just upgraded to an eBay'd Xeon) for HW transcoding and I don't want to stand up another box to just do Plex — whole point in virtualizing the drat thing was to run everything in one box. Vaguely considering getting some little NUC-like thing and just making that dedicated Plex / media serving and mounting the NAS over Ethernet, but I already have this perfectly good box w/ 64 GB ECC (not that it's needed for Plex) that was supposed to "do it all". Would you mind putting together a pcpartpicker list for what you made? A mATX formfactor is pretty much exactly what I'm looking at, and even if I don't get exactly what you made, it would be nice to have a good starting point.
|
|
# ¿ Jan 12, 2021 04:40 |
Saukkis posted:Synology or QNAP are the minimal effort solution to achieve your needs, with not too much of an extra expense. An old desktop with FreeNAS/Unraid is the minor expense solution. RaspPi4 is the minimal expense, lower performance solution, but at least you don't have to shuck the drives. Also recommend checking out Open Media Vault in the same vein as FreeNAS/UnRAID.
|
|
# ¿ Jan 12, 2021 16:28 |
In my time playing around with TrueNAS I found that ZFS was way more straightforward than ext4. There's no messing around with different share folders or having to create discrete pre-sized partitions for something that you might have to change down the line. You just have a pool and create your data sets in that pool and they grow or shrink as necessary. You can add or remove data sets as you need. It's easy to back up a whole pool at once rather than having to do multiple backups of each data set or each shared directory. I really like the snapshot feature as well as it's a really space efficient way to provide redundancy against accidental deletions. Although obviously it's not a backup of any sort. It's probably the thing I miss the most now that I'm running open media vault.
|
|
# ¿ Jan 14, 2021 04:38 |
Axe-man posted:DS1618+ doesn't have hardware transcoding so you are basing it totally on the cpu which honestly is not going to have the power that plex would like for it. I'm personally considering whether to get a little 9U rack that I can double up as a table since they're about 20" tall, or a pretty typical table height, for next to my desk. Then I can relocate my networking gear into there, and build a 4U rack mounted NAS/Server to handle my storage/Plex/Home Hosting needs. I guess it would depend on how quiet I can make a 4U mounted computer. Then I can put my existing mATX desktop on top of the rack (which is itself just on a side table at the moment). Nitrousoxide fucked around with this message at 15:23 on Jan 14, 2021 |
|
# ¿ Jan 14, 2021 15:21 |
Can you run a zpool status? You could also run a scrub to see if that drive has a lot of checksums that are off. This page has a rundown of the sort of workflow recommended for diagnosing and resolving drive failures. https://docs.joyent.com/private-cloud/troubleshooting/disk-replacement
|
|
# ¿ Jan 18, 2021 21:49 |
IMO, if you want to gently caress around with BSD or LINUX or whatever, setup a NAS, like FreeNAS and a seperate hypervisor like ProxMox, use the NAS as your network drive that your hypervisor references for its storage (preferably connected with 10 GB ethernet) and then you have the best of both worlds. A super reliable NAS with an easy to restore setup, and a hypervisor that you can nuke/restore vm's as necessary. Or you could do it all in FreeNAS and use its built in VM support to do both in one box.
|
|
# ¿ Jan 19, 2021 19:01 |
Rooted Vegetable posted:Obviously that price is easily beaten if you do wish to shuck (10TB Seagate Expansion External for CAD$219 or 14TB of the same for CAD$299[/url). I've personally been using these for a few years, unshucked, in the 8TB variety, with no failures. I ran them almost continuously for the better part of a year in my crypto mining stint with zero failures, and they'll be integrated (shucked) into my NAS when I build it in a few months. I suppose I'll know then if the S.M.A.R.T. diagnostics report any failures since I don't get access to that while unshucked.
|
|
# ¿ Jan 19, 2021 21:19 |
BlankSystemDaemon posted:BTRFS on top of MDADM RAID0. Wow, you must really hate your data. What's wrong with BTRFS? I just switched over to a Linux distro that uses that from Windows and from what I read it has most of the benefits of ZFS like snapshots and hashing basically everything to avoid bitrot but runs faster.
|
|
# ¿ Jan 24, 2021 17:17 |
8-bit Miniboss posted:Didn't realize TrueNAS SCALE existed until today and now I'm rethinking my life choices as I start eyeing external drives to copy my data over and do a clean install... It's not really ready for a real deployment yet. I'd wait another few months until its feature complete unless you're putting it on a "loving around on" box. Regular TrueNas core and TrueNas scale will both use the same file system so you can literally just overwrite the BSD based OS of TrueNAS core with Scale when it's ready, remount your zRaid arrays and be back up and running unless you're also running a bunch of jails or whatever which won't work in Scale I'd imagine. What would people recommend for a silent NAS build for a TrueNAS setup? An Atom based system? It would be doing Plex serving, and possibly some transcoding of 4k, as well as some light server work with stuff like Sonarr and Nextcloud.
|
|
# ¿ Feb 3, 2021 15:40 |
Wouldn't something like this work fine for the GPU for the transcoding? https://www.amazon.com/ZOTAC-GeForce-PCI-E2-0-Graphics-ZT-71302-20L/dp/B01AZ7W88O/?tag=akshatblog198-20 Does TrueNAS support GPU passthrough for Plex to actually use it?
|
|
# ¿ Feb 4, 2021 15:56 |
BlankSystemDaemon posted:Synology is an appliance, meaning it's (intended to be) a set-and-forget thing. They also offer pre-built systems if you want to use TrueNAS but also still want an appliance-like experience. https://www.truenas.com/truenas-mini/
|
|
# ¿ Feb 20, 2021 22:17 |
Keep in mind that 1U rack units will be very loud because their tiny fans have to spin very fast to push the same amount of air. A larger 3 or 4u unit with bigger fans can spin them much slower for a quieter operation
|
|
# ¿ Feb 21, 2021 15:58 |
IOwnCalculus posted:To be fair, transcoding 1080p streams (I have a lot of users who apparently have no ide that they can turn it up from "720p 4Mbps") is not a lot of work for a 2667/2670 V2 system. I've had a dozen of those transcodes running before with no issues. I mean, if you have a modernish laptop you no longer use you can just put the plex server on that, while mounting the network drive from your NAS and then do the actual transcoding on that laptop with its lower power but still transcode capable cpu/integrated gpu.
|
|
# ¿ Feb 22, 2021 18:20 |
Well, friends, that is why you don't use USB devices as your primary storage for your Plex server. Almost lost everything to it coming unmounted somehow. I should really get a proper setup.
|
|
# ¿ Feb 23, 2021 01:28 |
What is time intensive about maintaining the freenas setup?
|
|
# ¿ Mar 8, 2021 22:40 |
When I first read that I took hung pirates to mean something completely different.
|
|
# ¿ Mar 11, 2021 14:23 |
Hadlock posted:Synology DS418 are $250 refurbished and run some variant of Linux I wonder if a Synology is not locked down so hard that you could install trueNAS on it.
|
|
# ¿ Mar 12, 2021 13:26 |
Nulldevice posted:Probably not possible for a lot of reasons. You'd need to be able to hook up a keyboard and mouse to do the initial install of TrueNAS or at least have something like IPMI. Neither of which Syno units have. At least in the consumer space. Now if you wanted to build something in a similar form factor, this case is a great one - https://www.newegg.com/black-superm...cB&gclsrc=aw.ds - I built one for the SO using one. Four hot swap bays with backplane, built in 250W bronze PSU, mini ITX form factor, and pretty damned small in person. Makes for a great TrueNAS box. This is about $30 less than I paid for the case around Xmas time. It will also house 2x2.5" SSDs for TrueNAS's operating system on the top and right of the hot swap cage very easily. You can slide out the motherboard tray as well. So a simple mITX board and cheap CPU and some RAM will get you going. You'll also want a low profile HBA for the main drives and can get away with a 4 drive model flashed to IT mode. I had most of my parts on hand already, just needed an adapter from Supermicro to adapt their header to fit regular power switch etc connections on the motherboard, and a CPU. Tossed in four 4TB drives in RAIDZ1 and that was it. Great little system. Obviously not going to be as power efficient as a Syno but similar form factor and you get TrueNAS. Hmm, i'm thinking something like an atom 3000 series mobo/chip like this: https://www.amazon.com/Supermicro-M...077BHMN8X&psc=1 You want a lot of RAM with TrueNAS and most mITX boards only have two dimm slots available. It's also got a nice low 16w TDP. I doubt the number of drives you're fitting in a mITX case are going to strain that CPU My main concern would be whether it would be able to handle a plex server, or if that would have to be hosted off of it and the network drive mounted to it.
|
|
# ¿ Mar 13, 2021 01:19 |
Nulldevice posted:That CPU would probably not be terribly good if you're transcoding. Direct play should be fine however. Depending on your pool size you might not need a huge amount of RAM. I've got 92TB of space with 32GB of RAM and no performance issues. But then I'm not running anything other than rsync, samba, and nfs on it. The ARC doesn't really take a lot of memory in the process of daily operations. It really depends on what your read load looks like. The old 1GB per TB of storage thing really doesn't matter as much as it used to unless you're running enterprise workloads. I looked up the CPU passmark score and it was only 1650. The single thread performance was under 600. My recommendation is to run Plex on a separate device and use the truenas server strictly for storage. The CPU is fine for that function. If you're looking to keep things compact, a NUC might be the best option. Just toss in all flash storage and a decent amount of memory and your choice of operating system, and mount the shares from the truenas server onto it. That's what I've been doing for a while now and it works perfectly fine. I do have a spare laptop that I'm currently running my openmediavault on along with plex and a few docker containers. It seems to do well enough on that but its just got a couple of usb SMA drives connected to it one as the primary storage and one that I rsync the primary drive to as a backup. It's not an ideal setup but it is one that works using the hardware I had on hand. Since this laptop has proven to be capable of handling my media server and container needs, maybe I can either directly install a Linux server distro on it and mount the TrueNAS network drive to it and port my containers over, or perhaps proxmox with the ZFS drive mounted to it and then a Linux VM. I would think an SSD drive (maybe 250gb) as a cache would probably overcome the downsides of the SMA drives by providing a sufficient buffer for writes to fill up so the file system can take as long as it needs to write to the slow SMA drives?
|
|
# ¿ Mar 13, 2021 18:22 |
Be careful which direction you r-sync your stuff kiddos. Just realized my r-sync had broken due to a drive getting unmounted and went to go start them back up, and started up the wrong r-sync, syncing from the now out-of-date backup onto the live drive.
|
|
# ¿ Mar 16, 2021 13:40 |
What you might be able to do is run Proxmox on the bare metal for the system, then run TrueNas in a VM in it passing through the hard drives for it to manage. That allows you to use Proxmox to pass through things like video cards to a VM, something I don’t think free BSD supports.
|
|
# ¿ Apr 5, 2021 05:05 |
KozmoNaut posted:I wonder if that is overall more or less harmful than proof-of-work crypto. The energy intensive ask is just done once, on the initial compute of the hashes that get written to the storage medium. After that the only energy used is the the processing power to read the hashes (minimal) and to operate the drives (still pretty low) You can scan through orders of magnitude more pre-computed hashes for the same energy cost as computing them. It largely works like a proof of stake coin, where you're essentially dumping your investment into storage media and the one time cost and energy cost for computing the hashes to prove that you are an interested party to the cryptocoin.
|
|
# ¿ Apr 17, 2021 23:19 |
Tuxedo Gin posted:I'm not sure this is the place to ask this, but I didn't see a Plex thread in the first couple pages, and I'm doing this on an unraid server so it is kinda relevant: I'm going to put together a plex server, but I never realized how messy all my media is. Is there a tool that assists in the mass renaming of hundreds (or thousands?) of videos? I don't mind having to be present to help the tool pick the right thing - just anything faster than me manually renaming everything and moving them into properly named folders as appropriate. Sonarr and Radarr, while usable for also will allow you to reorganize all of your stuff into a organized structure and will automatically rename everything and tell Plex where they all are, if you link it to your plex server.
|
|
# ¿ Apr 25, 2021 11:53 |
The Milkman posted:I got an alert from my TrueNAS server: It could have been a cable got jostled loose. Double check them. That's the most likely cause of seemingly random widespread errors in one drive that resolve themselves completely with no follow up checks confirm.
|
|
# ¿ Apr 27, 2021 19:31 |
Biowarfare posted:Once the plots are generated it's read many. Doesn't it do all that in RAM though?
|
|
# ¿ May 10, 2021 13:14 |
I do use a reverse proxy for nextcloud since its purpose is to be a google drive replacement, which means it needs to be able to share, and be accessed by, people outside of my network.
|
|
# ¿ Jun 7, 2021 10:55 |
Wireguard was astoundingly easy to setup on docker. Like I should have done this months ago. It took less than 5 minutes from hitting deploy on that docker image to having all 3 of my devices setup and going on a vpn to my home.
|
|
# ¿ Jun 18, 2021 01:27 |
withoutclass posted:Does it set up DNS and forwarding of packets too? Thinking of moving my wire guard off of digital ocean and hosting it myself. Would be cool to just drop a docker image on the server instead of configuring by hand. You can set the DNS to whatever you want in the docker image. If you leave it as auto it'll default to whatever the docker host's DNS is set to.
|
|
# ¿ Jun 18, 2021 01:46 |
PRADA SLUT posted:I have a Synology NAS running Calibre in Docker. What's the best way to expose it to I can access my library outside my home network? VPN? A VPN is the most secure method with the least amount of exposure. I recently installed wire guard on my own device and it was dead simple to set up. On the other hand if you are looking to be able to access the service from more than a handful of devices or by more than one just the people in your immediate family. Or by people you don’t want access in your general home network, then you will probably need to set up a reverse proxy and get HTTPS working.
|
|
# ¿ Jun 22, 2021 09:00 |
Smashing Link posted:Very important question for the thread. I am thinking of organizing my "unsorted" movie folder (~1400 movie files) into folders A-Z (and a folder for numeric titles). How should I address movies beginning with "The"? Go with first letter of second word of title, put all in "T" folder, or separate "The" folder. Inclined toward first option. I use Sonarr, Radarr, and Lidarr, to organize my media. They may have other uses as well.
|
|
# ¿ Jun 24, 2021 13:01 |
|
|
# ¿ May 7, 2024 03:55 |
modeski posted:I am researching my next NAS. Currently I'm running Windows Home Server 2011 with Stablebit Drivepool on some 4Tb/8Tb drives on an Athlon FM2 with 8Gb of RAM. I built the server in 2014 and it's time for a new one (and new drives) I've never cared for RAID much as I prefer to maximize storage. Maybe only 500Gb-1Tb of user-created pics/vids/documents is important to me and I back that data up elsewhere. I can always redownload Linux ISOs and re-backup my media, but I do want at least 30Tb of usable space. There can be reasons to go with the proxmox into a NAS VM approach. But you probably need to have a very specific purpose in mind for that. If for instance you're trying to build a home lab or something which requires you to want to share the hardware on your storage computer between multiple VM's through passthrough. The vast majority of people will have a far smoother experience just installing the NAS OS on the bare metal and either using a separate server to host the applications you want running, or picking a NAS OS that supports virtualization and containers to do that on the NAS box itself.
|
|
# ¿ Jul 8, 2021 13:37 |