|
DoomTrainPhD posted:Somebody has to care so you don’t have to. file systems have been a solved problem for decades
|
#
?
Nov 11, 2021 20:37
|
|
|
|
| # ? Apr 18, 2024 04:00 |
|
|
Tankakern posted:josef bacik writes about upcoming on-disk format changes for btrfs from the writeup, it's pretty clear that the codebase and future changes are going to reflect Facebook's requirements, which (according to the autor) is heavily container based. So I guess btrfs may actually have a narrower niche in server computing than was widely considered... instead of using chattr to address various workloads, it might be better to just apply btrrs to straightforward container and NAS solutions.
|
|
#
?
Nov 11, 2021 20:46
|
|
|
Tbh, having a file system where you can shotgun random garbage data across all the disks in an array while the system is serving hundreds of MB/s of data is pretty cool if your job includes storage janitoring. I've done that during tests with ZFS and the fs never once served a corrupted file.
|
|
#
?
Nov 11, 2021 20:47
|
|
|
akadajet posted:file systems have been a solved problem for decades maybe for you but pretty much 100% of major OS vendors have been releasing new filesystems in the past couple of years. Apple has their new apfs, Microsoft has ReFS, and various linux distros have been pushing btrfs, so feel free to stick with ext4 but even that will likely change if we have to switch to 64-bit timestamps or larger.
|
|
#
?
Nov 11, 2021 20:53
|
|
|
File systems can be exciting - like when that guy killed his wife.
|
|
#
?
Nov 11, 2021 21:00
|
|
|
murderfs I always associate filesystems with statistical mechanics... the whole "that way lies madness" kind of deal
|
|
#
?
Nov 11, 2021 21:02
|
|
|
Antigravitas posted:Tbh, having a file system where you can shotgun random garbage data across all the disks in an array while the system is serving hundreds of MB/s of data is pretty cool if your job includes storage janitoring. I've done that during tests with ZFS and the fs never once served a corrupted file. this is what i've never understood about brtfs and zfs etc, why not use an actual cluster filesystem like idk ceph or something at that point, a single massive multi-block-device filesystem managed by a single os instance seems rather fragile and bottleneck-y. if you have a 50tb collection of tentacle hentai on your home nas then i suppose btrfs might be a good fit for that particular use case
|
|
#
?
Nov 11, 2021 21:09
|
|
|
You should, under no circumstances, contemplate creating your own file system. A spherical file system in a vacuum is easy enough to write (it'll just perform like arse), but you quickly gets into "100 falsehoods programmers believe about block devices"-territory and you won't make it out of there with your sanity intact. The OpenZFS developers have my full emotional support, but i'll stand back here content to never touch that code ever.
|
|
#
?
Nov 11, 2021 21:10
|
|
|
sb hermit posted:maybe for you yeah, and all a waste of time since NTFS already exists
|
|
#
?
Nov 11, 2021 21:15
|
|
|
Sapozhnik posted:this is what i've never understood about brtfs and zfs etc, why not use an actual cluster filesystem like idk ceph or something at that point, a single massive multi-block-device filesystem managed by a single os instance seems rather fragile and bottleneck-y. BTRFS may look a lot like ZFS (including having lots of Oracle copyrights all over the codebase, lol), but I wouldn't trust it with my data - and neither does Facebook.
|
|
#
?
Nov 11, 2021 21:33
|
|
|
Sapozhnik posted:this is what i've never understood about brtfs and zfs etc, why not use an actual cluster filesystem like idk ceph or something at that point, a single massive multi-block-device filesystem managed by a single os instance seems rather fragile and bottleneck-y. Ceph/gluster are extremely complicated pieces of machinery with their own very pernicious behaviours. You can, right now, order a 288TB raw storage whitebox server with 24 bays, safe in the knowledge that it'll run really well serving multiple 10gbps connected clients with, say, content for video editing or high resolution ground penetrating radar images. You buy a second one and asynchronously replicate your stuff to it. It's really hard to beat that if you are working with a ton of data. ZFS also brings a good number of management features that make administration easier. Snapshots, instant quotas, reservations, extremely fast creation and destruction of file systems, unified interface, transparent compression (that one saves us about a third of space. With 100s of TB…)… I'm serious when I say that ZFS is genuinely a nice part of my job because of how admin friendly it is. May be stockholm syndrome, but normally, storage is pain.
|
|
#
?
Nov 11, 2021 21:37
|
|
|
Oh by the way, traditional backup tools take over 24h trying to do a daily backup of some of our file systems because they contain an ungodly amount of small files. It takes a ton of iops. ZFS send/receive takes seconds and serialises all those small files into one sequential stream at hundreds of MB/s. It's so fast we can do it hourly at no performance impact. That is what fully converted me to the church of ZFS.
|
|
#
?
Nov 11, 2021 21:42
|
|
|
but btrfs can do that too and it's upstream zfs is really a church
|
|
#
?
Nov 11, 2021 23:41
|
|
|
Not to pick on BlankSystemDaemon, but I try to avoid getting any of my views from facebook.
|
|
#
?
Nov 12, 2021 00:01
|
|
|
|
|
#
?
Nov 12, 2021 00:09
|
|
|
now wheres nbsd to say that zfs is wrong, that the use cases it handles well likeAntigravitas posted:Oh by the way, traditional backup tools take over 24h trying to do a daily backup of some of our file systems because they contain an ungodly amount of small files. It takes a ton of iops.
|
|
#
?
Nov 12, 2021 02:03
|
|
|
Lysidas posted:should be addressed by bailing wire and duct tape solutions like lvm snapshots rather than native filesystem support for snapshots and incremental send/receive Pfft a real hacker would deal with a cluttered file system with a dd bs=67108864 if=/dev/xxx | gzip > /dev/nst0
|
|
#
?
Nov 12, 2021 02:45
|
|
|
I can't be the only one whose ever planned to deal with a catastrophic failure by finding a new job.
|
|
#
?
Nov 12, 2021 02:50
|
|
|
BlankSystemDaemon posted:I feel ambivalent towards POSIX - it's one of the better examples of interoperability standards we have it’d be hilarious to watch the Linux crowd try to actually pass the conformance suite without the ability to “well, ackshually…” anything
|
|
#
?
Nov 12, 2021 03:33
|
|
|
BlankSystemDaemon posted:Have you read Steve Jobs & the NeXT Big Thing? Because you really should. none of the people I know who were there have anything good to say about that book quote:I know how teletypes communicated with minis - but I don't see how that affects the reality that multi-user environment still benefits Unix-likes to this day; if it wasn't a multi-user environment, there wouldn't be such a thing as dropping privileges and anything you might want to run on the machine (such as a web server) would run under the same user you're using. there absolutely would because the actual privilege mechanisms used in the modern era are almost entirely decoupled from the UNIX user and group mechanisms because it turns out they’re the wrong level of granularity for end-user software and modern threat models turns out, VMS got it right after all “anything that runs as your user can modify anything owned by your user” is a bad model that leads to real users’ personal data being stolen by malicious software
|
|
#
?
Nov 12, 2021 03:40
|
|
|
i am reading uh UNIX, POSIX, and Open Systems (Addison-Wesley, 1992) right now and its both the most boring but also most enthralling thing
|
|
#
?
Nov 12, 2021 03:46
|
|
|
I bought a synology NAS the other day and it uses btrfs
|
|
#
?
Nov 12, 2021 11:35
|
|
|
eschaton posted:it’d be hilarious to watch the Linux crowd try to actually pass the conformance suite without the ability to “well, ackshually…” anything eschaton posted:none of the people I know who were there have anything good to say about that book  I think I phrased myself poorly, my entire point was that not everything a modern user runs runs as their own user; before Firefox on FreeBSD got OSSv4 compatibility re-added (because Mozilla quietly broke and removed it), pulseaudio would run as the pulse user by default, and on the off-chance that I ever install a httpd on my laptop, it's configured to use the www user by default (and there's an entire list which maps processes to UIDs). On top of that, with ACLs you get just about as much granularity as you could possibly want. matti posted:i am reading uh
|
|
#
?
Nov 12, 2021 11:52
|
|
|
BlankSystemDaemon posted:But conformance suites don't test strict compliance, in that they don't care about what's outside of POSIX. I'm talking about only using these. it occurs to me that the POSIX utilities specification has a standard ISO C compiler frontend specified but no assembler or linker
|
|
#
?
Nov 12, 2021 12:08
|
|
|
Kazinsal posted:it occurs to me that the POSIX utilities specification has a standard ISO C compiler frontend specified but no assembler or linker
|
|
#
?
Nov 12, 2021 12:18
|
|
|
neat, airlied is working on getting vulkan-facilitated video hw decode working on linux https://airlied.blogspot.com/2021/11/h264-video-decoding-i-frames-strike-back.html
|
|
#
?
Nov 12, 2021 13:53
|
|
|
BlankSystemDaemon posted:I don't remember reading that, so I'm gonna pretend this is a bookclub and add it to my list of books to read, because a then-current account of ways to counteract the UNIX wars might be interesting. keep notes, its an abbreviation hell also ~~enterprise graphs~~
|
|
#
?
Nov 12, 2021 13:53
|
|
|
POSIX was such a lowest common denominator spec of what a UNIX implementation had to provide that IBM was able to build a POSIX compliant ( and UNIX branded maybe??? ) abi into MVS - os/390 - z/OS that should tell you all you need to know about the value of something being POSIX compliant.
|
|
#
?
Nov 12, 2021 14:14
|
|
|
fresh_cheese posted:POSIX was such a lowest common denominator spec of what a UNIX implementation had to provide that IBM was able to build a POSIX compliant ( and UNIX branded maybe??? ) abi into MVS - os/390 - z/OS idk that is the o.g. POSIX.1 but the current (or even the previous) issue is pretty dece and ofc no one should program against anything but SUS
|
|
#
?
Nov 12, 2021 14:25
|
|
|
fresh_cheese posted:POSIX was such a lowest common denominator spec of what a UNIX implementation had to provide that IBM was able to build a POSIX compliant ( and UNIX branded maybe??? ) abi into MVS - os/390 - z/OS So clearly POSIX wasn't everything to everyone, even then.
|
|
#
?
Nov 12, 2021 15:34
|
|
|
docs say its currently POSIX.2 compliant for shell, some utilities and kernel abi and i think thats just nasty
|
|
#
?
Nov 12, 2021 15:53
|
|
|
BlankSystemDaemon posted:
i simply cannot innovate when you make me comply with these silly standards
|
|
#
?
Nov 12, 2021 15:58
|
|
|
the solution is innovative standards
|
|
#
?
Nov 12, 2021 16:14
|
|
|
Probed via linus's tip. Makes that guy close to laport I guess. When I said multi user, I definitely meant multiple simultaneous human users. Desktop linux was built on a bunch of technology revolving around and configured for multiple simultaneous users by default. When the super computer lab I worked in (college) got rid of their x-terminals, the Linux club was able to hook them up to their dual pentium pro system. It was a pretty stock install of debian and all it took was changing some xdm configuration and a line in the default shell profile. They complemented the vt220s nicely. I spent about an hour playing with the windows terminal services hack on old windows (xp). The big thing I noticed was file handle leak when you closed the rdp session w/o logging out. The only thing I see it being useful for is allowing multiple people to use software that has a hardware DRM dongle. It's very much a hack, and you're going to run into problems without the rest of the application server toolkit. edit: Human, flesh and blood users. SYSV Fanfic fucked around with this message at 18:10 on Nov 12, 2021 |
|
#
?
Nov 12, 2021 18:06
|
|
|
wait is there a rule against posting linus tech tips I didn't know that e: does it count as anime
|
|
#
?
Nov 12, 2021 18:21
|
|
|
Private Speech posted:wait is there a rule against posting linus tech tips We live under the capricious whims of our lord mods here. Praise be to merciful graph.
|
|
#
?
Nov 12, 2021 22:48
|
|
|
Private Speech posted:wait is there a rule against posting linus tech tips there is seriously no reason to ever engage with ltt
|
|
#
?
Nov 12, 2021 22:58
|
|
|
In this house we stan Anthony tech tips
|
|
#
?
Nov 12, 2021 23:20
|
|
|
linus seems fine to me if a bit dystopian in his love of consume consume consume
|
|
#
?
Nov 13, 2021 00:55
|
|
|
|
| # ? Apr 18, 2024 04:00 |
|
|
also he pronounces his own name wrong and builds monster gaming rigs for himself even though he doesn’t play video games
|
|
#
?
Nov 13, 2021 00:56
|
|