Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us $3,400 per month for bandwidth bills alone, and since we don't believe in shoving popup ads to our registered users, we try to make the money back through forum registrations.
«787 »
  • Post
  • Reply
astral
Apr 26, 2004



RFC2324 posted:

ok, if this is the case it makes perfect sense and I was wrong.

sorry, missed a vital detail

Actually, I re-read it and have to revise my previous post: they meant being able to set things up so DNS would respond to anything.domain.com with a certain IP.

Adbot
ADBOT LOVES YOU

RFC2324
Jun 7, 2012

Http 418


astral posted:

Actually, I re-read it and have to revise my previous post: they meant being able to set things up so DNS would respond to anything.domain.com with a certain IP.

Yeah, if you buy a domain you can do that, the registrar has no say. The dns provider might not let you, but changing dns providers should be easy and fairly transparent if they don't provide the service you require for your money

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt



astral posted:

Actually, I re-read it and have to revise my previous post: they meant being able to set things up so DNS would respond to anything.domain.com with a certain IP.

Correct. The web UI of the registrar I personally use lets me use asterisks in DNS records, but the one my company uses doesn't, so in order to deploy a new app we need to have someone define records for app.company.com. It takes like a minute so I haven't pushed to change registrar over it, but it's still totally unnecessary.

Methanar
Sep 26, 2013


NihilCredo posted:

Correct. The web UI of the registrar I personally use lets me use asterisks in DNS records, but the one my company uses doesn't, so in order to deploy a new app we need to have someone define records for app.company.com. It takes like a minute so I haven't pushed to change registrar over it, but it's still totally unnecessary.

Use a different dns service. Or delegate a subdomain zone to a dns server you operate and do your app stuff on the new subdomain. (don't do this just get a better dns host)

RFC2324
Jun 7, 2012

Http 418


NihilCredo posted:

Correct. The web UI of the registrar I personally use lets me use asterisks in DNS records, but the one my company uses doesn't, so in order to deploy a new app we need to have someone define records for app.company.com. It takes like a minute so I haven't pushed to change registrar over it, but it's still totally unnecessary.

You seem to not understand the difference between the dns registrar and the dns provider? You can change providers easily, and should do so if they won't let you create subdomains for yourself.

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt



You don't need to tell me to get a better provider. I already have a good one for myself, the "no wildcards allowed" issue is only with the one my company uses (and it's nowhere close to worth making a fuss over, or even finding a dedicated provider separate from the domain registrar + web host), and in fact the whole reason we are having this discussion in the first place is because I mentioned to another goon that they should check if their provider supports wildcards - which I thought was a given until I discovered one that didn't.

Let's move on.

Zotix
Aug 14, 2011





What's the go-to method/ course/ etc to go from Linux zero to hero in 4-6 months? I know that's a short time frame for a new OS and it's a constant learning process but I'd like to learn as much as I can. I have Ubuntu and CentOS iso's downloaded. I've also set Ubuntu up on a separate partition and dabbles in it last night. Not sure if I should skip Ubuntu and go right to CentOS? I also downloaded VM Ware so I could run it through a VM. What courses are recommended to get started? Thanks.

Mr. Crow
May 22, 2008

Snap City mayor for life


For desktop use? Need a use case since so much of the distro choice depends on that. Assuming desktop :

I would recommend Fedora, they get a lot of the new features while maintaining a rock solid platform.

CentOS is great for servers, dunno if it would be a good desktop experience since it lags behind in updates.

Ubuntu is also a good intro distro but canonical has been doing some stupid poo poo more frequently then normal lately.

PopOS is a good Ubuntu derivative, if you want to game.

Matt Zerella
Oct 7, 2002


Zotix posted:

What's the go-to method/ course/ etc to go from Linux zero to hero in 4-6 months? I know that's a short time frame for a new OS and it's a constant learning process but I'd like to learn as much as I can. I have Ubuntu and CentOS iso's downloaded. I've also set Ubuntu up on a separate partition and dabbles in it last night. Not sure if I should skip Ubuntu and go right to CentOS? I also downloaded VM Ware so I could run it through a VM. What courses are recommended to get started? Thanks.

Set up a NAS with ZFS and sonarr/radarr/torrents/plex with no GUI. Ssh only. Then turn it into an Ansible routine.

xzzy
Mar 5, 2009

wakey wakey to
this bowl of tasty


Yams Fan

Matt Zerella posted:

Set up a NAS with ZFS and sonarr/radarr/torrents/plex with no GUI. Ssh only. Then turn it into an Ansible routine.

Times have changed! When I got my first linux job in 1998 all I could do was compile the kernel and get a ppp connection configured.

Matt Zerella
Oct 7, 2002


xzzy posted:

Times have changed! When I got my first linux job in 1998 all I could do was compile the kernel and get a ppp connection configured.

I do not miss those days. I remember getting a red hat CD with a magazine and only had man pages to keep me warm.

RFC2324
Jun 7, 2012

Http 418


Matt Zerella posted:

I do not miss those days. I remember getting a red hat CD with a magazine and only had man pages to keep me warm.

there was usually a README

xzzy
Mar 5, 2009

wakey wakey to
this bowl of tasty


Yams Fan

By the time I was on scene TDLP was a thing that existed and it was good enough, but I had to actually download it to a local copy so I could reference it while I was loving around with stuff.

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


Grimey Drawer

Mr. Crow posted:

For desktop use? Need a use case since so much of the distro choice depends on that. Assuming desktop :

I would recommend Fedora, they get a lot of the new features while maintaining a rock solid platform.

For a similar but different distro, openSUSE is sort of the Euro/German equivalent of Fedora/CentOS (it can fill both roles, depending on the install choices you make), just like SUSE proper is to German enterprise what Red Hat is in the US.

There's both a regular stable version called Leap, and a rolling release version called Tumbleweed, which gets the latest versions of everything.

I use Leap, because I'm old and I don't mind not using the absolute latest version of everything.

CaptainSarcastic
Jul 6, 2013

HAIL SATAN


KozmoNaut posted:

For a similar but different distro, openSUSE is sort of the Euro/German equivalent of Fedora/CentOS (it can fill both roles, depending on the install choices you make), just like SUSE proper is to German enterprise what Red Hat is in the US.

There's both a regular stable version called Leap, and a rolling release version called Tumbleweed, which gets the latest versions of everything.

I use Leap, because I'm old and I don't mind not using the absolute latest version of everything.

My old desktop has Leap and my new desktop has Tumbleweed, and they have both been great. My old desktop has had like 4 or 5 in-place upgrades to new point releases of Leap and it was never a problem.

The only problem I've had on Tumbleweed is the wifi getting flaky, and like I think I've said before I'm pretty sure that is Realtek as a problem more than OpenSUSE.

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


Grimey Drawer

Yup, I've been running it on my laptop (TP X220i) for a while, everything just *works*. That influenced my decision to also use it on my NAS/HTPC. My desktop is still running KDE Neon, but I'm getting fed up with the Ubuntu underpinnings, so that's getting openSUSE as well, when I get around to it.

Kassad
Nov 11, 2005

It's about time.


I'm seeing that Fedora might make Btrfs the default over ext4 in their next release. I've been thinking about reinstalling my desktop anyway so I might give it a try, but is there any reason to choose Btrfs over ext4 if you're not running a server?

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


Grimey Drawer

Kassad posted:

I'm seeing that Fedora might make Btrfs the default over ext4 in their next release. I've been thinking about reinstalling my desktop anyway so I might give it a try, but is there any reason to choose Btrfs over ext4 if you're not running a server?

Btrfs (and ZFS, which share a number of key features) is a different approach than a more traditional filesystem. The biggest difference is that it is a Copy-on-Write filesystem, which means that any changes are done on copies of the original data and only actually committed they are successful. Think of it as the next step up in data integrity over a journaled file system like ext4. The drawback is that this does trade some outright performance for reliability, but there are ways to mitigate that for most use cases, for instance using compression.

Another major feature is that the handling of multiple devices is in the filesystem, rather than in dedicated RAID hardware or software (mdadm/LVM in Linux). For instance in my NAS, I started my storage pool on a single Btrfs-formatted 2TB disk moved over from my desktop PC. Then I added two additional disks (2TB+3TB) to the storage pool using "btrfs device add /dev/sdX /mnt/Storage", creating one big 7TB contiguous volume. When that was done, I used "btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt/Storage" to convert the storage pool to a 3.5TB RAID1* for data redundancy, and balance the data across the devices.

* (Btrfs RAID1 is not the same as hardware RAID1. It takes a storage pool with an arbitrary number of disks and makes sure there are always two copies of any data, distributed over all the devices. That means you can lose one drive without losing data.)

Another very important feature is the ability to create snapshots. It's a copy of a specific state of the filesystem, and as long as nothing changes, it doesn't take up any additional space, it's merely a reference. Then as you change things, it keeps log of the changes, so you can always roll back to the state of the FS when you took the snapshot. This is extremely handy for backups and "system restore"-type functionality in case of botched upgrades, openSUSE does this automatically by default. You can also send and receive snapshots between networked systems with Btrfs filesystems, for handy remote backups.

There are also subvolumes, which in openSUSE take the place of the traditional UNIX partitions for /usr, /home, /var and so on. They are still part of the base volume and expand as needed, they can have quotas applied to them, plus they're excluded from default from root volume snapshots and can be snapshotted individually. That means you can snapshot your system files and roll them back completely separately from your config files in /etc or your home folder.

Coming back to performance and compression, modern SSDs and CPUs are so fast that a good compression algorithm can improve performance, especially if your SSD is bottlenecked by the interface it's connected to. A fast compression method can save ~30% of storage space in most cases, while improving performance because it reduces the effective amount of data that needs to be read from the drive.

For ordinary everyday desktop usage, the main differences will be improved data integrity and snapshots, especially if your distro uses them like openSUSE does to provide roll backs in case of failed upgrades or config changes.

mystes
May 31, 2006



Btrfs: I came for the improved data integrity

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


Grimey Drawer

Haters vacate the premises

D. Ebdrup
Mar 13, 2009



KozmoNaut posted:

Haters vacate the premises
You gotta admit, Kirk McKusick who's one of the creators of UFS and still one of its maintainers over 40 years after, is right in saying, paraphrased: "The trouble with being a filesystem developer, is that once you lose peoples data, they're not going to trust you again".

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


Grimey Drawer

People got all hyped and started using it in production way too early. It's the eternal curse of always wanting the newest and shiniest stuff as quickly as possible. Their unwitting data sacrifices helped iron out a lot of bugs, though.

The same thing happened with KDE4. Yeah it was kinda technically 4.0, the big new rewrite on Qt4 and all shiny and stuff, but the KDE team tried really hard to remind people that it was to be considered an early release and to please wait until it was considered stable before using it. Obviously people didn't listen and got all huffy when it turned out to not be stable yet, surprise.

It's also why I've switched back to a normal release cycle distro now instead of a rolling release cycle one. I can't be arsed to be a tester in my free time anymore.

KozmoNaut fucked around with this message at 17:08 on Aug 4, 2020

RFC2324
Jun 7, 2012

Http 418


KozmoNaut posted:

People got all hyped and started using it in production way too early. It's the eternal curse of always wanting the newest and shiniest stuff as quickly as possible. Their unwitting data sacrifices helped iron out a lot of bugs, though.

The same thing happened with KDE4. Yeah it was kinda technically 4.0, the big new rewrite on Qt4 and all shiny and stuff, but the KDE team tried really hard to remind people that it was to be considered an early release and to please wait until it was considered stable before using it. Obviously people didn't listen and got all huffy when it turned out to not be stable yet, surprise.

It's also why I've switched back to a normal release cycle distro now instead of a rolling release cycle one. I can't be arsed to be a tester in my free time anymore.

I love rolling release in concept but gently caress tracking down why something is misbehaving for the 3rd time today

D. Ebdrup
Mar 13, 2009



KozmoNaut posted:

People got all hyped and started using it in production way too early. It's the eternal curse of always wanting the newest and shiniest stuff as quickly as possible. Their unwitting data sacrifices helped iron out a lot of bugs, though.
Matt Ahrens and Jeff Bonwick were using ZFS after just a year of development for their home folders, and Sun started using it internally a year after that. It wasn't announced until 2 years after that and didn't ship until 2 years after that.

BTRFS developers, meanwhile, removed the alpha notice seemmingly without having tested it, as the first reports of corruption of actual honest-to-goodness production data came in weeks if not days afterwards.
Meanwhile, the available documentation suggests that everything is still not all sunshine and daisies.

Ironically, it looks as if they've successfully managed to scrub the evidence of the removal of the alpha notice from the web.

D. Ebdrup fucked around with this message at 17:44 on Aug 4, 2020

Kassad
Nov 11, 2005

It's about time.


KozmoNaut posted:

Btrfs (and ZFS, which share a number of key features) is a different approach than a more traditional filesystem. The biggest difference is that it is a Copy-on-Write filesystem, which means that any changes are done on copies of the original data and only actually committed they are successful. Think of it as the next step up in data integrity over a journaled file system like ext4. The drawback is that this does trade some outright performance for reliability, but there are ways to mitigate that for most use cases, for instance using compression.

Another major feature is that the handling of multiple devices is in the filesystem, rather than in dedicated RAID hardware or software (mdadm/LVM in Linux). For instance in my NAS, I started my storage pool on a single Btrfs-formatted 2TB disk moved over from my desktop PC. Then I added two additional disks (2TB+3TB) to the storage pool using "btrfs device add /dev/sdX /mnt/Storage", creating one big 7TB contiguous volume. When that was done, I used "btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt/Storage" to convert the storage pool to a 3.5TB RAID1* for data redundancy, and balance the data across the devices.

* (Btrfs RAID1 is not the same as hardware RAID1. It takes a storage pool with an arbitrary number of disks and makes sure there are always two copies of any data, distributed over all the devices. That means you can lose one drive without losing data.)

Another very important feature is the ability to create snapshots. It's a copy of a specific state of the filesystem, and as long as nothing changes, it doesn't take up any additional space, it's merely a reference. Then as you change things, it keeps log of the changes, so you can always roll back to the state of the FS when you took the snapshot. This is extremely handy for backups and "system restore"-type functionality in case of botched upgrades, openSUSE does this automatically by default. You can also send and receive snapshots between networked systems with Btrfs filesystems, for handy remote backups.

There are also subvolumes, which in openSUSE take the place of the traditional UNIX partitions for /usr, /home, /var and so on. They are still part of the base volume and expand as needed, they can have quotas applied to them, plus they're excluded from default from root volume snapshots and can be snapshotted individually. That means you can snapshot your system files and roll them back completely separately from your config files in /etc or your home folder.

Coming back to performance and compression, modern SSDs and CPUs are so fast that a good compression algorithm can improve performance, especially if your SSD is bottlenecked by the interface it's connected to. A fast compression method can save ~30% of storage space in most cases, while improving performance because it reduces the effective amount of data that needs to be read from the drive.

For ordinary everyday desktop usage, the main differences will be improved data integrity and snapshots, especially if your distro uses them like openSUSE does to provide roll backs in case of failed upgrades or config changes.

Thanks for the explanation. It does potentially seem like overkill for a desktop from what you say (keeping in my mind that I back my poo poo up regularly).

Mr. Crow
May 22, 2008

Snap City mayor for life


Ignoring the data integrity issue, snapshots are helpful for any kernel or dist upgrades. I just is lvm though because in lazy.

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


Grimey Drawer

D. Ebdrup posted:

Matt Ahrens and Jeff Bonwick were using ZFS after just a year of development for their home folders, and Sun started using it internally a year after that. It wasn't announced until 2 years after that and didn't ship until 2 years after that.

BTRFS developers, meanwhile, removed the alpha notice seemmingly without having tested it, as the first reports of corruption of actual honest-to-goodness production data came in weeks if not days afterwards.
Meanwhile, the available documentation suggests that everything is still not all sunshine and daisies.

Ironically, it looks as if they've successfully managed to scrub the evidence of the removal of the alpha notice from the web.

RAID5/6 simply shouldn't have been implemented in the first place. There is absolutely no need for them.

Regarding the alpha notice, anyone who rolls out a brand new filesystem to production machines before testing it internally first deserves every single data loss they experience. Refusing to to the proper research, preparation and testing before rolling out any new hardware or software is simply bad practice. Never play fast and loose with production systems.

E: I don't really care about the turbulent history and flamewars, I care about that it works and that SUSE trusts it enough to ship it as the default filesystem. It's the same story as with systemd, it got a rocky start and there are cadres of Linux geeks with seemingly too much free time who are still moaning about flamewars that happened years and years ago and issues that have been fixed for ages. The sheer amount of ignorance and misinformation about the current state of Btrfs is impressive and saddening.

E2: Here's a fair article about Btrfs, from the perspective of Facebook, who uses Btrfs extensively: https://lwn.net/Articles/824855/
And it's not like ZFS is without its share of serious bugs: https://github.com/openzfs/zfs/issues/7401

KozmoNaut fucked around with this message at 19:19 on Aug 4, 2020

Adbot
ADBOT LOVES YOU

D. Ebdrup
Mar 13, 2009



KozmoNaut posted:

RAID5/6 simply shouldn't have been implemented in the first place. There is absolutely no need for them.

Regarding the alpha notice, anyone who rolls out a brand new filesystem to production machines before testing it internally first deserves every single data loss they experience. Refusing to to the proper research, preparation and testing before rolling out any new hardware or software is simply bad practice. Never play fast and loose with production systems.

E: I don't really care about the turbulent history and flamewars, I care about that it works and that SUSE trusts it enough to ship it as the default filesystem. It's the same story as with systemd, it got a rocky start and there are cadres of Linux geeks with seemingly too much free time who are still moaning about flamewars that happened years and years ago and issues that have been fixed for ages. The sheer amount of ignorance and misinformation about the current state of Btrfs is impressive and saddening.

E2: Here's a fair article about Btrfs, from the perspective of Facebook, who uses Btrfs extensively: https://lwn.net/Articles/824855/
And it's not like ZFS is without its share of serious bugs: https://github.com/openzfs/zfs/issues/7401
RAID5/6 is not the same as variable-sized stripes with distributed parity on a CoW filesystem, and I don't know what makes you think it is. The simple fact that not even BTRFS is subject to write-holes, which RAID5 cannot deal with, should be a clue.

You seem to have missed my point, which was that the notice was quietly removed instead of the developers coming out and saying "we've done our best to test it, but we'd awfully like it if we could get much more broad-scale testing, on non-critical systems", or something to that effect.
Had that happened, I wouldn't have nearly the same problems with it that I do - I don't even expect them to do what Sun did and run it for over half a decade[/i] in production before releasing it, just that a little more forethought goes into things which affect production systems because the world is absolutely devops mad and everything has to go at the fastest rate possible.

It's opensource - if you know that the information on what amounts to an official source of documentation is outdated and likely to be a source of information, send a patch or at least file a bug report. Bringing up a completely separate issue is just a misdirection - but the lwn article you link also mentions BTRFS issues associated with striping with distributed parity.
At least they don't dismiss it because they don't use it.

What is now the OpenZFS repo isn't the codebase that started at Sun in 2001, it's a reimplementation based on code that originates at LLNL - and that particular commit has neither made it into the Illumos codebase, nor is it in FreeBSD, macOS or NetBSD. Case in point, I seem to recall that ZFS was used to recover the data, which was never really lost, as it was actually written to disk.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«787 »