NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

Tamba: Apr 5, 2010

wang souffle posted:

Can anyone convince me what to do with 8x 2TB drives in a ZFS setup? I currently have 5 of the drives in a RAIDZ1 setup and need a migration plan for existing data once I get 3 more.

1) 2x 4-drive RAIDZ1 groups in one pool.
2) 8-drive RAIDZ2 zpool.

Exactly how much more reliable is option 2 here? I'm considering option 1 as it seems possible to migrate the existing data without needing more drives.

In the first case, you lose half your data if two drives in the same RAIDZ1 die, in the second case, you can lose any two drives and still be fine (but then you lose everything if a third drives dies before you can rebuild)

# ¿ Apr 15, 2012 21:21

Adbot: ADBOT LOVES YOU

# ¿ Apr 25, 2024 22:40

Tamba: Apr 5, 2010

What's the process for reinstalling FreeNAS? Can I just export the config, set up a new flash drive and load the config from there, or do I have to export the ZFS-pool first?
I'm currently in a weird situation where an upgrade failed, and now it's still working, but unable to be upgraded any further. I'm hoping a reinstall can fix that.

# ¿ Mar 4, 2014 21:27

Tamba: Apr 5, 2010

The people in the tread you posted say that you also need to reset all the settings, because that's the easiest way to completely get rid of the old volume name.
So export the volume again and just start over.

# ¿ Jul 24, 2017 16:34

Tamba: Apr 5, 2010

In FreeNAS 9 it's System-->Settings-->Factory restore.
No idea if it's still the same in version 11.
The other option would be to just flash the USB stick again.

# ¿ Jul 24, 2017 16:46

Tamba: Apr 5, 2010

Bittorrent Sync (now called Resilio Sync) should work on Synology as well.

e2: wait...its not free anymore? Nevermind then.

Tamba fucked around with this message at 16:40 on Aug 26, 2017

# ¿ Aug 26, 2017 16:34

Tamba: Apr 5, 2010

Rescue Toaster posted:

Does anyone have a decent notification system setup? Ideally something that would have android, linux, and windows clients and would have simple command line sendmail-replacement style utilities so I could set it up to send notifications from pfsense, FreeNAS, and network ups tools. All these servers/daemons expect you to have a smtp server to send emails to.

Even something that I could run a server locally and open a port and all the phone/pc clients would connect to the server would be nice. The only cloud one I've seen that seems really simple is pushover, but there must be more (and it's not free, not that I would expect anything using somebody else's servers to be free).

The reality is, I don't log into the thing often enough that I'd ever see an actual disk failure probably. So after upgrading to 11.1 I figured I'd finally better set this up properly.

How about using an existing messaging app?
Telegram has a bot API that you could use to create a bot to message you whenever anything happens.
Should be relatively simple to set up as long as you're only sending messages, and don't want the bot to react to replies.

code:

curl -i -X GET https://api.telegram.org/bot<apikey>/sendMessage?chat_id=<chatId>&text=<someText>

Tamba fucked around with this message at 17:42 on Dec 31, 2017

# ¿ Dec 31, 2017 17:27

Tamba: Apr 5, 2010

I have an old 160 GB SSD that I don't use anymore, and my FreeNAS server has an empty SATA port.
Can I add that as L2ARC, or is there any reason why I shouldn't do that?

# ¿ Sep 28, 2018 18:15

Tamba: Apr 5, 2010

Twerk from Home posted:

Is anybody using an HP Microserver Gen10? Are they an awful value for money? I've been using an i5-4250u NUC that I got on eBay used for $110 with a 5TB external shingled Seagate hooked up to it as a VM host and NAS for almost 5 years now, and I finally want redundancy and more storage.

I'm willing to have a somewhat weird low performance setup in exchange for better power efficiency than mainstream parts could get me, but I don't want to be like that dude a few posts up who spent $1400 for 16TB raw storage. My current plan is shucking 4x10TB Easystores into a Microserver Gen10, but I could just as easily put an i3 into a Fractal Design Node 304 or something.

I want more than 16GB RAM, low power usage, and more CPU performance than a 15W Haswell. That's about all my requirements.

Edit with one more question: I have found near zero info about the Operon X3421 online. My current understanding is that it's two Excavator modules on 28nm at 35W TDP, right? Looks to perform favorably compared to Atoms, and C3xxx atoms look like a miserable value for money overall.

No idea if this fix for FreeBSD has made its way to the most recent versions of FreeNAS yet, but until it does, you'll need set some boot parameter, or it won't boot.
Other than that it works perfectly fine for me, but I'm not doing anything that needs a lot of performance with it.

# ¿ May 14, 2019 11:42

Tamba: Apr 5, 2010

eames posted:

I can’t comment on FreeNAS but last time I checked it wasn’t the first choice for docker (there is no native BSD version of docker, containers run in a small Linux VM).

FreeNAS has an option to automatically set up a linux VM (RancherOS) that you can use for your docker containers.
Seems like they don't care about supporting that though, so it will be gone with the next version. (You can still use existing VMs or manually make your own)

# ¿ Dec 9, 2019 16:16

Tamba: Apr 5, 2010

D. Ebdrup posted:

I'm curious about this, why are you saying not to encrypt?

It's more "Be really careful with encryption on FreeNAS".
It will work perfectly, until you need to replace a failed disk, don't do the extra necessary steps in exactly the right order and lose all data forever (because Raid was your backup).

# ¿ Feb 11, 2020 12:54

Tamba: Apr 5, 2010

One of the disks in my ZFS array seems to be on its way out.
Is there anything I can do to prepare for replacing it while it is still readable, or should I just pretend it has failed already and do the usual steps to replace a failed disk?

# ¿ May 21, 2020 19:25

Tamba: Apr 5, 2010

I finally retired my old FreeNAS system and set up openmediavault instead.
Next step is setting up backups.
The data is in a zpool that I'll back up using BorgBackup, but what's the best way to back up the OS drive so I don't have to do all this setup again if the SSD dies?
Should I just dd the whole drive somewhere and back that up, or is there a more elegant way?

e: looks like fsarchiver might be what I want

Tamba fucked around with this message at 10:21 on Jul 27, 2021

# ¿ Jul 26, 2021 16:48

Tamba: Apr 5, 2010

salted hash browns posted:

Option B, Build a big TrueNAS box
Get an Intel CPU, a bunch of drives, put it in a box, and install TrueNAS. Is TrueNAS reliable enough to set and forget? Stable enough that updates won't randomly break poo poo that I'll need to debug? Are there other OSes here to consider?

I moved away from Free/TrueNAS because Docker containers were a pain there, and ended up with Openmediavault.
The web interface feels similar to FreeNAS (which shouldn't be a surprise, because it was made by the same guy), but it runs on Linux (Debian) instead of FreeBSD.
I haven't had any issues with updates breaking things so far, even the big version upgrade from 5 to 6 (which included an upgrade from Debian 10 to 11) worked flawlessly.

# ¿ Jun 7, 2022 08:35

Tamba: Apr 5, 2010

fletcher posted:

I forgot I had a question too, about Syncthing. I am interested in replacing my Google Drive with a Syncthing + NAS + b2 backups solution.

However, in setting up my sync over LAN I discovered this public relay crap in Syncthing which I'm really not a fan of, mainly due to the transfer speed degradation but also just not liking the idea of my sensitive files going over the public internet (despite being encrypted). I do not wish to expose my Syncthing ports to the public internet and was planning on using VPN in order to access Syncthing from my phone.

In order to avoid the public relay servers, am I required to run my own relay? I wasn't really sure if running a relay was required, given that I'm doing everything over LAN. Seems like kind of a pain to have to do that, if that's the case.

No, you can just turn it off if you don't want your clients to talk to relays at all:

Hosting your own relay lets you tell your clients to only talk to that specific relay instead of the public ones, but that's not necessary if you have a VPN

e: and the clients will only use a relay if they can't make a direct connection, so I'm not sure why you're concerned about transfer speed degradation (slow transfer is better than no transfer at all?)

Tamba fucked around with this message at 21:35 on Jun 30, 2022

# ¿ Jun 30, 2022 20:58

Tamba: Apr 5, 2010

Don Dongington posted:

Running the hell away from FreeBSD is the right move though, and if they get the container system working again it should be a hell of a lot nicer to use than the current experience on Core. I'm going to wait for the upcoming July 7 hotfix and decide whether to stick with it, or shift everything onto an Ubuntu LTS server, as I was finding myself using the VM on my core install to do practically everything. Docker is so much nicer to work with than iocage, but dealing with passing storage through via NFS made me want to kill myself.

Maybe give OpenMediaVault a try? I've been using it for a few years now and it works perfectly for me.

# ¿ Jul 5, 2022 21:53

Tamba: Apr 5, 2010

Scruff McGruff posted:

Setting up a Reverse Proxy is the first thing that comes to mind. Basically, set up a domain, either by buying one or using https://duckdns.org to create one for free, have subdomains for whatever service you need, like plex.mydomain.com and idrive.webdomain.com. Then for those services, instead of pointing them at your public IP you point them to those webdomains so all their traffic comes to your network over port 80/443. Then you have a tool like Nginx Proxy Manager route that traffic to the appropriate location/port inside your network based on what subdomain it came from. It can even handle creating SSL certs via LetsEncrypt right in the GUI, or you can generate a wildcard cert from Cloudflare and use that, no messing around in config files. The only firewall stuff you have to do is point ports 80/443 to your proxy.

Getting this set up for me was a godsend, no more messing with port-forwarding on the router and sending them an IP address whenever my friends want me to spin up a Minecraft server, now it's just "create minecraft.scruffmcgruff.com in Cloudflare, create a corresponding endpoint in NPM, send traffic to the appropriate container IP:Port" and then tell them to just put in minecraft.scruffmcgruff.com.

Ibracorp has a good guide on YouTube on how to set this up. It's for Unraid/Docker but the config of the domain stuff and NPM are the same regardless of how you implement it.

I do something like that, but instead of pointing the domain at my public IP, I use my local one (192.168....) and VPN home using Wireguard (running on my router).
Now I still get easy to remember domain names, but only I can access them and the only thing exposed to the internet is the Wireguard endpoint.

# ¿ Sep 24, 2022 11:51

Tamba: Apr 5, 2010

Nitrousoxide posted:

You do still kinda run a reverse proxy inside your network too so that you're resolving stuff as HTTPS, otherwise someone connected to your wifi (like a visitor or a compromised device that you've added to your network) could just watch all the packets going back and forth with those unencrypted connections.

It not nearly as huge a risk as leaving stuff exposed to the internet at large of course, but is still best practices.

I've not personally done it yet myself, though I'm planning on spinning up a pi-hole instance sometime over this next month to use as an internal DNS resolver so I can use wildcard certs to encrypt all my connections internally.

Yeah, I do use NPM as a reverse proxy with letsencrypt HTTPS certificates..

Tamba fucked around with this message at 14:45 on Sep 24, 2022

# ¿ Sep 24, 2022 14:42

Tamba: Apr 5, 2010

Computer viking posted:

No, it's internal only - though we have other internal services (homeassistant, specifically) available when outside the house. What we've done is to set up wireguard on the router. We can connect the laptops to it, but the main way we use it is to have configured as an always-on per-application VPN in Android, to let the homeassistant app (and only that app) permanently live on the IoT subnet at home.

I imagine you could so something similar? Set up wireguard, and connect your phone/laptop when you want to use Jellyfin. You'd have to use HDMI to get it onto a hotel TV, but that's probably doable. Just how hard or easy it would be to set up WG at home depends on your router, of course. Our Mikrotik has it built in (and it's neither more nor less arcane than anything else in that OS), while with other routers you may have to forward a port and run WG on a separate machine.

I bought a cheap android HDMI stick just for hotel TV stuff. It has Wireguard + a few media apps (including a Jellyfin client), so all I need to do is connect it to the TV and network to get access to my files at home without exposing everything to the internet.

# ¿ Nov 18, 2022 02:08

Tamba: Apr 5, 2010

Nystral posted:

My only issue with TrueNAS is they expect you to do things a very specific way. You want screen? Learn tmux instead, it’s already there and does what you want. Want to carve out space on your boot array to host docker containers? Not a good idea so not even offered as an option out of the box.

I nuked my TrueNAS install for a basic Ubuntu + cockpit + portainer solution, but I’m growing weary of stuff requiring more care and feeding then I want to give it.

Give OpenMediaVault a try, it's Debian with a very FreeNAS like webUI on top (because it was literally made by a former FreeNAS developer)

# ¿ Dec 15, 2022 22:18

Tamba: Apr 5, 2010

You could also get some cheap Android HDMI stick instead (or a Shield TV if you want to spend a bit more).
Connecting a NAS directly to the TV feels wrong

# ¿ Dec 28, 2022 18:17

Tamba: Apr 5, 2010

Ihmemies posted:

How do I backup Windows disks as disk images to a network drive? I looked at Veeam, but a backup software's download taking TEN GIGABYTES is way too loving much. Why it takes so loving much space for a simple backup software? Even Windows takes half the size of that, and it's a whole complete OS with bells and whistles!

Any alternatives?

https://learn.microsoft.com/en-us/sysinternals/downloads/disk2vhd

This is intended to make disk images for virtual machines, but I guess you could use it as a backup tool as well :shrug:

# ¿ Mar 8, 2023 15:57

Tamba: Apr 5, 2010

Mr. Crow posted:

Man look like google is finally cracking down on google workspace and they're gonna restrict my unlimited storage for $20/month to 5TB (for $20, 5 more for every $20 iirc)

Whats the next cheapest offsite storage option for dozen or two TB of storage? DropBox Advanced and try to share it woth some people? What are you people with large collections doing, just ZFS with a lot of disks and hoping for the best? A separate backup machine thats mostly offline?

I haven't used it myself, but ArsTechnica recently mentioned https://www.idrive.com/ which seems to be quite cheap ($12.5 per month for 20TB or $31.25 for 50TB, if you pay for 2 years at once).

https://arstechnica.com/information-technology/2023/02/ars-archivum-top-cloud-backup-services-worth-your-money/

# ¿ May 20, 2023 20:23

Tamba: Apr 5, 2010

Are you posting from the future?
I don't think there's a laptop that can fit 100TB, and 10TB SSDs aren't really available as consumer products yet, the biggest external SSDs you can get are 8 TB (and horribly expensive).

e: oh, I see, the data is not on the laptop, you're just using it to back the data up from somewhere else. Still, you can probably build five 100 TB servers for the price of 100 TB of SSD storage.

e2: For this use case, the real answer is tape drives.

Tamba fucked around with this message at 20:19 on Jul 1, 2023

# ¿ Jul 1, 2023 19:50

Tamba: Apr 5, 2010

Boner Wad posted:

I think I’m preferring regular Linux distros and don’t want to run TrueNAS or UnRAID while I’m building my NAS. I’m planning on doing zfs with RAIDZ2.

Any issues with running Ubuntu or Arch and running zfs on there?

Openmediavault is just Debian with a very FreeNAS like webUI on top (because it was made by a former FreeNAS dev). It can do zfs.

# ¿ Jul 2, 2023 06:16

Tamba: Apr 5, 2010

My NAS is still sending me emails, but I use https://gotify.net/ for notifications from Home Assistant.

# ¿ Sep 19, 2023 12:46

Tamba: Apr 5, 2010

https://www.kickstarter.com/projects/icewhaletech/zimacube-personal-cloud-re-invented
https://nascompares.com/2023/11/10/the-zimacube-nas-teardown-early-review/

https://www.youtube.com/watch?v=wRowwdfCJ3Y

This looks like an interesting piece of hardware.
A ~22cm cube with space for 6 HDDs and 4 M2 SSDs.

Tamba fucked around with this message at 23:41 on Nov 10, 2023

# ¿ Nov 10, 2023 23:37

Tamba: Apr 5, 2010

Shrimp or Shrimps posted:

However it seems that OMV6 doesn't have temperature monitoring for disks?? 1 of my disks was averaging 50c with spikes of up to 55c through summer with TrueNas. I'm hoping that by moving over to SnapRAID, not having the disks spinning constantly and only spinning up for access / scrubs / syncs will keep their average temperatures down.

OMV does have temperature monitoring, it's in the S.M.A.R.T settings.
You can alert when it's above a maximum value, or when it has increased by more than X degrees since the last check.

# ¿ Nov 11, 2023 21:09

Tamba: Apr 5, 2010

Shrimp or Shrimps posted:

Oh yes I've seen that now. Having looked through some videos of OMV, it seems that it's not really lacking for all that much when compared to Truenas. From the way I'd read people talking about it on Reddit, I was expecting it to seem unfinished and janky but honestly it's going to do everything I need it to and way more that I don't need it to. Have decided to forego MergerFS and instead just deal with 4 mounted disks, and manually put files where I want them, after having read that when accessing the shared mergerfs pool via NFS, that all drives were spinning up. I'm not sure if the same behavior will happen over SMB, but I don't want it to and then have to tear it all down and restart.

I switched from FreeNAS to OMV5 and have been using it since then, with ZFS. Can't really say that I've missed anything from FreeNAS, and the OMV5 --> OMV6 upgrade went through without any issues.
I especially like the recent-ish update to the OMV extras where docker compose is just in the OMV UI now (it used to just install and configure Portainer for you and made you use that).

# ¿ Nov 12, 2023 14:32

Tamba: Apr 5, 2010

Mantle posted:

Both myself and a friend of mine have Synology devices with current DSM 7 support. What's the best way for us to share access to each other's media with minimal exposure to the internet as a whole?

I was hoping for some sort of DSM-based solution to set up a tunnel between the two NASes but I couldn't find one.

Before you try to set up something on the NAS, maybe first check your routers to see if they have an easy way to set up a VPN between your two networks

# ¿ Dec 18, 2023 12:54

Tamba: Apr 5, 2010

Eletriarnation posted:

6 the drives are set to never spin down, and typical power draw is still well under 200W.

Drives use the most power when spinning up :eng101:

# ¿ Mar 1, 2024 16:21

Adbot: ADBOT LOVES YOU

# ¿ Apr 25, 2024 22:40

Tamba: Apr 5, 2010

You could also give Openmediavault a try. It's a Linux (Debian) based distro with a web-UI very much like FreeNAS (because it was made by a former FreeNAS dev) and it should be able to just mount the old raid.

https://docs.openmediavault.org/en/latest/administration/storage/raid.html
https://docs.openmediavault.org/en/latest/administration/storage/filesystems.html

(I've never had to do that, but at least the documentation makes it seem possible)

If you insist on using ZFS, it can do that too, but there's no way around copying the data off (having a backup is never a bad idea!) and wiping the original disks.

quote:

mdadm --zero-superblock

This seems like a very dangerous command, you probably want mdadm --assemble

Tamba fucked around with this message at 08:26 on Apr 13, 2024

# ¿ Apr 13, 2024 08:23

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of