NAS/Storage Megathread: What is this "File Deletion" You Speak of

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›842 »

BlankSystemDaemon: Mar 13, 2009

Combat Pretzel posted:

Hmm, looking this up, because it kept confusing me, UDP is supposed to sent an ICMP packet when the port is closed. So that's how it detects it. Seems like this is something that Wireguard ought to do on its own, too, for cloaking.

Luckily my router does filter all ICMPs and keeps them from leaving the local network, so things should be generally invisible. And I'm not portmapping Wireguard to its default port, anyway.

If it's filtering all ICMP, you're probably getting timed out quite often.

# ? Nov 19, 2022 23:29

Adbot: ADBOT LOVES YOU

# ? Apr 25, 2024 15:46

Computer viking: May 30, 2011; Now with less breakage.

BlankSystemDaemon posted:

By being a lying liar, apparently.

Ha, is that test literally "we were able to send the packets out"? That'd do it, I guess.

# ? Nov 19, 2022 23:31

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

--edit: Nevermind, wrong section of the docs.

BlankSystemDaemon posted:

If it's filtering all ICMP, you're probably getting timed out quite often.

In what way? The default configuration of Fritzboxes is to drop all ICMPs on the edge.

Combat Pretzel fucked around with this message at 23:37 on Nov 19, 2022

# ? Nov 19, 2022 23:34

BlankSystemDaemon: Mar 13, 2009

Computer viking posted:

Ha, is that test literally "we were able to send the packets out"? That'd do it, I guess.

Seems so? Teaching sand to do maths was a mistake.

Combat Pretzel posted:

--edit: Nevermind, wrong section of the docs.

In what way? The default configuration of Fritzboxes is to drop all ICMPs on the edge.

ICMP Type 11 is TTL and fragment expiry, so devices on your network won't know when to stop sending something.

# ? Nov 20, 2022 00:08

Saukkis: May 16, 2003; Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

Combat Pretzel posted:

Hmm, looking this up, because it kept confusing me, UDP is supposed to sent an ICMP packet when the port is closed. So that's how it detects it. Seems like this is something that Wireguard ought to do on its own, too, for cloaking.

I don't think we would want that. If it sends an ICMP reply then we can assume that the port is open in the firewall, and a likely reason for that is that a Wireguard is using that port. Based on the assumption that most firewalls will silently drop any packets sent to walled ports.

# ? Nov 20, 2022 04:40

wolrah: May 8, 2006; what?

BlankSystemDaemon posted:

I think my point got away from me, but part of it was that it's not like it's impossible to tell that Wireguard is running on the port - it's just that nmap (or at least the version on in FreeBSDs base system) isn't up-to-date enough to recognize Wireguard.

It is absolutely impossible for nmap to ever "recognize" Wireguard because it doesn't respond at all unless you've sent a valid handshake packet.

You can certainly infer that Wireguard is running on a system if you see an anomaly in the results on port 51820 since that's the standard port and nothing else would generally be listening there, but that anomaly would be caused by misconfiguration of the system's firewall and not Wireguard itself revealing anything. If the system was configured to just ignore traffic to other ports then the Wireguard port would blend in to the darkness.

# ? Nov 20, 2022 07:57

Incessant Excess: Aug 15, 2005; Cause of glitch:
Pretentiousness

Is anyone familiar with Asus NAS systems? The AS6706T offers some nice upgrades over my Syno DS918+ like six bays, a new Intel CPU and 2.5GBase-T but I'm wary of anything prebuilt by someone other than Synology or maybe Qnap.

# ? Nov 20, 2022 11:12

BlankSystemDaemon: Mar 13, 2009

wolrah posted:

It is absolutely impossible for nmap to ever "recognize" Wireguard because it doesn't respond at all unless you've sent a valid handshake packet.

You can certainly infer that Wireguard is running on a system if you see an anomaly in the results on port 51820 since that's the standard port and nothing else would generally be listening there, but that anomaly would be caused by misconfiguration of the system's firewall and not Wireguard itself revealing anything. If the system was configured to just ignore traffic to other ports then the Wireguard port would blend in to the darkness.

It's not just that there's something listening on that port that lets you guess at it being wireguard.
It's also that if it was something that wasn't wireguard, it'd probably respond if probed or be more easily identified by nmap - because there aren't that many things that go to the trouble of pretending security through obscurity works.

# ? Nov 20, 2022 15:59

Saukkis: May 16, 2003; Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

BlankSystemDaemon posted:

It's not just that there's something listening on that port that lets you guess at it being wireguard.
It's also that if it was something that wasn't wireguard, it'd probably respond if probed or be more easily identified by nmap - because there aren't that many things that go to the trouble of pretending security through obscurity works.

There is most likely nothing else than Wireguard or a firewall that is using that port, and those are indistinguishable from each other. If you don't have a firewall of any kind then WG can be identified, but who would be running a device without firewall.

# ? Nov 20, 2022 16:07

BlankSystemDaemon: Mar 13, 2009

Saukkis posted:

There is most likely nothing else than Wireguard or a firewall that is using that port, and those are indistinguishable from each other. If you don't have a firewall of any kind then WG can be identified, but who would be running a device without firewall.

How can a firewall be using a port (by which I take it you mean listen on a port)?

# ? Nov 20, 2022 16:47

Saukkis: May 16, 2003; Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

BlankSystemDaemon posted:

How can a firewall be using a port (by which I take it you mean listen on a port)?

In the sense that it's controlling it. You send an UDP packet to 51820 and Wireguard will silently drop it. You send an UDP packet to any other port at the 10000-65000 range and the firewall will silently drop them.

The only thing you can deduce is that the whole port range is protected by firewall, because the target's IP stack never replied that the port is closed, or the router didn't reply that you are trying to connect to non-existing IP.

# ? Nov 20, 2022 16:55

Hughlander: May 11, 2005

There was a post up stream about watching for Cyber Monday sales on hard drives. I just checked reddit's buildapcsale and they don't have a flare for hard drives just SSDs, and shucks.top are only for shuckable drives. Anyone know where to find possible sales on just regular 'ol 18TB NAS drives?

# ? Nov 20, 2022 17:46

BlankSystemDaemon: Mar 13, 2009

I just realized that I haven't mentioned that the OpenZFS DevSummit videos are online, but they are - and I even have a couple of recommendations.

Faster ZFS scrub and other improvements by Alexander Motin:
https://www.youtube.com/watch?v=Y_tPs3xsIaA

Block Cloning by Pawel Jakub Dawidek:
https://www.youtube.com/watch?v=qsE3R0Ysc8g

Refining OpenZFS Compression by Rich Ercolani:
https://www.youtube.com/watch?v=_5JeyCYV5nE

Here's the complete playlist.

# ? Nov 20, 2022 17:56

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

It's pretty hilarious to see the contrasts. Here they're innovating tons on a filesystem, and then you have something like Storage Spaces.

I'm still watching the block cloning stuff. I hope he'll mention a lower block size limit to this, to avoid cloning tiny blocks and clogging the BRT, but haul rear end on big things, say video files with 1M record size. --edit: Nevermind, 8MB of RAM per TB.

Combat Pretzel fucked around with this message at 20:03 on Nov 20, 2022

# ? Nov 20, 2022 20:01

BlankSystemDaemon: Mar 13, 2009

Combat Pretzel posted:

It's pretty hilarious to see the contrasts. Here they're innovating tons on a filesystem, and then you have something like Storage Spaces.

I'm still watching the block cloning stuff. I hope he'll mention a lower block size limit to this, to avoid cloning tiny blocks and clogging the BRT, but haul rear end on big things, say video files with 1M record size. --edit: Nevermind, 8MB of RAM per TB.

Even if we stay within the realm of opensource, the contracts between OpenZFS and BTRFS are pretty staggering.

Mind you, I'm not saying BTRFS doesn't have any innovation - but if it's gonna catch up like Linux folks like to imagine it will, it's gonna have to pick up the pace a fair bit.

# ? Nov 20, 2022 20:06

Keito: Jul 21, 2005; WHAT DO I CHOOSE ?

BlankSystemDaemon posted:

Mind you, I'm not saying BTRFS doesn't have any innovation - but if it's gonna catch up like Linux folks like to imagine it will, it's gonna have to pick up the pace a fair bit.

There's anyone left with hope for btrfs? I thought that ship had sailed by now. Maybe bcachefs will be good one day.

# ? Nov 20, 2022 21:01

Ihmemies: Oct 6, 2012

Especially when you can finally extend pools with new disks in the new zfs version. It wastes some space, but it's better than nothing.

# ? Nov 20, 2022 21:13

BlankSystemDaemon: Mar 13, 2009

Keito posted:

There's anyone left with hope for btrfs? I thought that ship had sailed by now. Maybe bcachefs will be good one day.

Well, I'm pretty sure my opinions on BTRFS are well-known ITT, and probably most other tech related threads on these gay forums - but I still see people talking about it as if it's a real alternative so I suspect there are some.

Ihmemies posted:

Especially when you can finally extend pools with new disks in the new zfs version. It wastes some space, but it's better than nothing.

You've always been able to expand pools that are either single-disk pools or mirrored vdevs, either by adding more devices to a vdev, or adding more vdevs.

The thing that's being added is raidz expansion.

# ? Nov 20, 2022 21:50

Klyith: Aug 3, 2007; GBS Pledge Week

I think that btrfs isn't trying to be ZFS. Like, why bother? ZFS is already really good at being ZFS!

Btrfs is trying to be a good OS host, ZFS is trying to be a data storage FS. There are things btrfs can do that ZFS can't or is bad at -- flexible resizing, adding/removing drives easily, not using a lot of overhead memory. The WIP stuff for raid with different-size drives is also about flexibility and being general-purpose rather than re-implementing ZFS.

This is the NAS thread and ZFS is definitely better for a serious NAS. (OTOH it's easy to see why synology is using btrfs, when selling NAS devices to non-technical people who might want to add drives over time.) That doesn't mean btrfs is bad or lovely, it means they're both good for different targets.

# ? Nov 21, 2022 01:34

priznat: Jul 7, 2009; Let's get drunk and kiss each other all night.

Hoping for some black friday HDD deals to move my setup to a new one - might go with TrueNAS and decommission my unraid, or move the unraid usb stick over, haven�t decided yet.

Any idea what the sweet spot for sale drives might be? 12TB?

# ? Nov 21, 2022 02:24

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

On a Supermicro board with IPMI, if you have a network cable plugged into LAN1, it's as if the IPMI LAN port also has a cable plugged into it, and they both appear on the network with their respective MAC address. What's going on there exactly, is there a name for this functionality? I didn't see any settings about it and I was curious when I saw the behavior.

# ? Nov 21, 2022 05:04

IOwnCalculus: Apr 2, 2003

Screenshot from a SM manual but it's under IPMI LAN Selection - "Shared" makes it use the same physical port as LAN1.

# ? Nov 21, 2022 05:19

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

IOwnCalculus posted:

Screenshot from a SM manual but it's under IPMI LAN Selection - "Shared" makes it use the same physical port as LAN1.

Ahhh! Thank you. The web interface labeled it as LAN Interface

# ? Nov 21, 2022 05:34

wolrah: May 8, 2006; what?

BlankSystemDaemon posted:

It's not just that there's something listening on that port that lets you guess at it being wireguard.
It's also that if it was something that wasn't wireguard, it'd probably respond if probed or be more easily identified by nmap

Here's a nmap scan of my Wireguard server, listening on the default port:

code:

&#10140;  ~ sudo nmap -sU -sV -p 51819-51821 wg.domain.tld
Starting Nmap 7.80 ( https://nmap.org ) at 2022-11-20 22:03 EST
Nmap scan report for wg.domain.tld (172.x.x.x)
Host is up (0.040s latency).
rDNS record for ---------

PORT      STATE         SERVICE VERSION
51819/udp open|filtered unknown
51820/udp open|filtered unknown
51821/udp open|filtered unknown

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 100.67 seconds

Here's the same scan done against one of my SBCs which definitely does not run Wireguard:

code:

&#10140;  ~ sudo nmap -sU -sV -p 51819-51821 notwg.domain.tld    
Starting Nmap 7.80 ( https://nmap.org ) at 2022-11-20 22:09 EST
Nmap scan report for notwg.domain.tld (96.x.x.x)
Host is up (0.045s latency).
rDNS record for ----------

PORT      STATE    SERVICE VERSION
51819/udp filtered unknown
51820/udp filtered unknown
51821/udp filtered unknown

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 1.50 seconds

By not responding, you give away nothing.

Here's another scan of that second box, but to a range including one of the ports it's actually listening on.

code:

&#10140;  ~ sudo nmap -sU -sV -p 5059-5061 notwg.domain.tld
Starting Nmap 7.80 ( https://nmap.org ) at 2022-11-20 22:09 EST
Nmap scan report for notwg.domain.tld (96.x.x.x)
Host is up (0.042s latency).
rDNS record for ---------

PORT     STATE    SERVICE VERSION
5059/udp filtered sds
5060/udp filtered sip
5061/udp filtered sip-tls

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 0.60 seconds

It's a stateless SBC, so just like Wireguard, no valid traffic sent to the SIP port = no response.

Sure, if we weren't running any kind of firewall and were blasting back ICMP Port Unreachable to everything except those ports it'd be pretty obvious there's something there, but why in the world would we do that?

quote:

because there aren't that many things that go to the trouble of pretending security through obscurity works.

Ignoring invalid traffic, including probes, is not security by obscurity. It's DoS protection, the system isn't wasting time trying to process or or memory trying to maintain state for anything other than valid traffic. It fails quickly as soon as the small handshake packet can't be decrypted with the private key.

# ? Nov 21, 2022 06:11

BlankSystemDaemon: Mar 13, 2009

Will it protect you against the botnets that scan /0 for open ports to try to find something to attack? Probably.
Will it protect you against a determined attacker who can look up the port in any number of databases and make a qualified guess? Probably not.

Were you to move it to another port where WireGuard was running on, that'd probably obscure it even more - but if, say, a very determined attacker wanted to get you, would they be able to find you talking about hosting a WireGuard instance?

Any Denial of Service prevention should happen at a much lower layer than the application level - which on FreeBSD can be done using ipfw and pfilctl(8).

# ? Nov 21, 2022 17:30

Scruff McGruff: Feb 13, 2007; Jesus, kid, you're almost a detective. All you need now is a gun, a gut, and three ex-wives.

At that point just set up your DNS in Cloudflare with their proxy and enable their Zero Trust auth for whatever the exposed service is.

# ? Nov 21, 2022 19:42

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Those of you using Tailscale, sounds like an update is in order! https://emily.id.au/tailscale

# ? Nov 21, 2022 22:08

Thanks Ants: May 21, 2004; #essereFerrari

That's such a good writeup, lightyears ahead of that crap you get when some security company tries to give their exploit a brand name.

# ? Nov 21, 2022 22:24

havelock: Jan 20, 2004; IGNORE ME; Soiled Meat

fletcher posted:

Those of you using Tailscale, sounds like an update is in order! https://emily.id.au/tailscale

FWIW I got an email from them telling me to update this morning, which I did. The email mentioned that I had a windows machine as part of one of my networks so I needed to do it. The update was very easy.

# ? Nov 22, 2022 01:37

Former Human: Oct 15, 2001

Hughlander posted:

There was a post up stream about watching for Cyber Monday sales on hard drives. I just checked reddit's buildapcsale and they don't have a flare for hard drives just SSDs, and shucks.top are only for shuckable drives. Anyone know where to find possible sales on just regular 'ol 18TB NAS drives?

I don't know how trustworthy Platinum Micro is but they have 18TB WD Ultrastar and Toshiba drives for as little as $280.

# ? Nov 23, 2022 07:24

Ihmemies: Oct 6, 2012

I have four Toshiba mg09 18tb drives. They vibrate a lot but idle noise in define case is pretty quiet. Random seek noises are clearly audible.

# ? Nov 23, 2022 08:47

Hughlander: May 11, 2005

Former Human posted:

I don't know how trustworthy Platinum Micro is but they have 18TB WD Ultrastar and Toshiba drives for as little as $280.

https://www.bestbuy.com/site/wd-easystore-18tb-external-usb-3-0-hard-drive-black/6427995.p?skuI=&skuId=6427995

I picked up a 18 TB easy store from bestbuy yesterday for $279. They're still that price right now.

# ? Nov 23, 2022 17:28

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Keito posted:

Maybe bcachefs will be good one day.

Totally forgot about that one. Is it still an one man show?

# ? Nov 23, 2022 18:13

tuyop: Sep 15, 2006; Every second that we're not growing BASIL is a second wasted; Fun Shoe

I'm using my Synology for backups and I'm running the backups through a specific user. I see that I can set a user quota for the share that my backups are on, and I can set a quota for the share itself. I just discovered that it goes with the lesser of the two, but is there any reason to go with one over the other? I'm inclined to turn off the user quota and just use the share volume quota since I'll probably never look at the user again.

# ? Nov 24, 2022 00:40

Incessant Excess: Aug 15, 2005; Cause of glitch:
Pretentiousness

If I have 3 drives in raid5, can I add a fourth that's smaller or nah?

# ? Nov 24, 2022 16:47

Wibla: Feb 16, 2011

Incessant Excess posted:

If I have 3 drives in raid5, can I add a fourth that's smaller or nah?

Nope.

# ? Nov 24, 2022 16:51

Incessant Excess: Aug 15, 2005; Cause of glitch:
Pretentiousness

Wibla posted:

Nope.

That's a bummer but how about one that's larger?

EDIT: Thanks, so I just need to go 10tb + 10tb + 16tb instead of 16tb + 10tb + 10tb.

Incessant Excess fucked around with this message at 17:06 on Nov 24, 2022

# ? Nov 24, 2022 16:57

Nulldevice: Jun 17, 2006; Toilet Rascal

Incessant Excess posted:

If I have 3 drives in raid5, can I add a fourth that's smaller or nah?

Gotta be equal size or larger.

# ? Nov 24, 2022 16:58

Klyith: Aug 3, 2007; GBS Pledge Week

Incessant Excess posted:

That's a bummer but how about one that's larger?

Yes, and the extra space is wasted.

tuyop posted:

I'm using my Synology for backups and I'm running the backups through a specific user. I see that I can set a user quota for the share that my backups are on, and I can set a quota for the share itself. I just discovered that it goes with the lesser of the two, but is there any reason to go with one over the other? I'm inclined to turn off the user quota and just use the share volume quota since I'll probably never look at the user again.

If that's the one thing you need quota management for I'd absolutely go with just the volume quota. The user quotas have better management UI if you had many of them to juggle.

# ? Nov 24, 2022 17:20

Adbot: ADBOT LOVES YOU

# ? Apr 25, 2024 15:46

Mofabio: May 15, 2003; (y - mx)*(1/(inf))*(PV/RT)*(2.718)*(V/I)

I'm thinking of getting rid of RAID. I've been managing mdadm and now ZFS clusters on DIY NASes and I'm starting to realize that the downsides (even drive wear so everything fails around the same time, the potential for loving a whole cluster at once by user error, can't physically just move a drive to another computer and use it there, can't just give a drive to a friend, difficult expandability, can't stagger backups across the drives meaning the least-important data is as protected as the most-important in the cluster) outweigh the upsides (automated redundancy vs an rsync tangle, JBOD).

I'm using an SBC that only has power for 2 SATA HDDs, so part of the reason I'm thinking of switching to single-drive ZFS is so only one drive has to spin up at a time, rather than slamming the rail accelerating four+ platters at once.

Anyone else "downgrade" to just a bunch of drives?

# ? Nov 24, 2022 17:26

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > NAS/Storage Megathread: What is this "File Deletion" You Speak of

«‹›842 »