The Linux Questions Thread: a bunch of pitfalls, but technically it's possible

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Linux Questions Thread: a bunch of pitfalls, but technically it's possible

«‹›1026 »

nem: Jan 4, 2003; panel.dev
apnscp: cPanel evolved

Sparse file?

# ? Jan 14, 2019 23:10

Adbot: ADBOT LOVES YOU

# ? Apr 26, 2024 14:05

Slanderer: May 6, 2007

nem posted:

Sparse file?

That's it, thanks!

# du core
33892 core
# fallocate -d core
# du core
347 core
#

# ? Jan 14, 2019 23:11

ToxicFrog: Apr 26, 2008

Slanderer posted:

I'm seeing something weird with the du utility (from BusyBox v1.23.2). I noticed that the size of an old core dump file changed after copying it from one directory to another, according to du. ls still returns the correct size.

# du /opt/util/bin/core
435 core
# cp /opt/util/bin/core /opt/core
# du /opt/core
33892 /opt/core

# ls -al /opt/util/bin/core
-rw------- 1 root root 34705408 Jan 14 20:57 /opt/util/bin/core
# ls -al /opt/core
-rw------- 1 root root 34705408 Jan 14 21:44 /opt/core

Anyone know what might be going on?

Sparse file, probably. Try comparing `du --apparent-size` (the default is to compare size on disk), and/or copying with `cp --sparse=always`.

# ? Jan 15, 2019 00:20

Slanderer: May 6, 2007

ToxicFrog posted:

Sparse file, probably. Try comparing `du --apparent-size` (the default is to compare size on disk), and/or copying with `cp --sparse=always`.

Yup, it was sparse. Unfortunately I don't have those options with the busybox utils on this platform, but my (dumb) solution for the future will be to use `ls -sl` and see if the allocation size * 1024 is >= the file size. If there's an actual way to do this with busybox that'd be cool, but it's not a big deal.

# ? Jan 15, 2019 00:25

G-Prime: Apr 30, 2003; Baby, when it's love,
if it's not rough it isn't fun.

Different filesystems with different block sizes?

Edit: gently caress. I missed a whole page. Leaving this here for people to laugh at me.

# ? Jan 15, 2019 02:06

Computer viking: May 30, 2011; Now with less breakage.

Deduplication and compression are the other things that can give similar results - du returning on-disk sizes keeps tripping me up on the ZFS file servers at work. It's admittedly right there in the name of the utility, but still.

# ? Jan 15, 2019 02:24

Methanar: Sep 26, 2013; by the sex ghost

quote:

# time sleep 1
^C

real 0m0.444s
user 0m0.000s
sys 0m0.000s

What can cause a system to think time is actually much, much slower than it really is? Around on the order of 1/130th real time.

Also for commands to take forever in real time to execute, but if you time them, linux appears to think it was tenths of seconds.
CPU load appears to be basically nothing

Methanar fucked around with this message at 03:45 on Jan 15, 2019

# ? Jan 15, 2019 03:43

RFC2324: Jun 7, 2012; http 418

Slanderer posted:

I'm seeing something weird with the du utility (from BusyBox v1.23.2). I noticed that the size of an old core dump file changed after copying it from one directory to another, according to du. ls still returns the correct size.

# du /opt/util/bin/core
435 core
# cp /opt/util/bin/core /opt/core
# du /opt/core
33892 /opt/core

# ls -al /opt/util/bin/core
-rw------- 1 root root 34705408 Jan 14 20:57 /opt/util/bin/core
# ls -al /opt/core
-rw------- 1 root root 34705408 Jan 14 21:44 /opt/core

Anyone know what might be going on?

do you have some kind of compression enabled in the location you copied from?

Edit:

G-Prime posted:

Different filesystems with different block sizes?

Edit: gently caress. I missed a whole page. Leaving this here for people to laugh at me.

# ? Jan 15, 2019 03:50

Mr. Fix It: Oct 26, 2000; 💀ayyy💀

Methanar posted:

What can cause a system to think time is actually much, much slower than it really is? Around on the order of 1/130th real time.

Also for commands to take forever in real time to execute, but if you time them, linux appears to think it was tenths of seconds.
CPU load appears to be basically nothing

Looks like you're interrupting that sleep before it finishes. What happens when you don't hit Ctrl+C? Does it run for multiple seconds?

# ? Jan 15, 2019 04:37

Methanar: Sep 26, 2013; by the sex ghost

Mr. Fix It posted:

Looks like you're interrupting that sleep before it finishes. What happens when you don't hit Ctrl+C? Does it run for multiple seconds?

It will run for approximately 130 seconds while reporting that it only took 1 second.

I should mention this is a bare metal machine with an NFS root for extra information.

# ? Jan 15, 2019 04:50

Mr. Fix It: Oct 26, 2000; 💀ayyy💀

Methanar posted:

It will run for approximately 130 seconds while reporting that it only took 1 second.

I should mention this is a bare metal machine with an NFS root for extra information.

Hmm, never ran into something like that. How are the system clock and hardware clock behaving?

# ? Jan 15, 2019 05:01

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

It could be that the program itself is genuinely taking 1 second to run, but forever for the system to load the binary or its shared-object dependencies. Try running an strace on it and see what that does.

# ? Jan 15, 2019 05:04

Methanar: Sep 26, 2013; by the sex ghost

minato posted:

It could be that the program itself is genuinely taking 1 second to run, but forever for the system to load the binary or its shared-object dependencies. Try running an strace on it and see what that does.

Doesn't seem like it.

It hangs right on that nanosleep

code:

strace sleep 0.1
execve("/bin/sleep", ["sleep", "0.1"], [/* 21 vars */]) = 0
brk(NULL)                               = 0xb72000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f4271414000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=102601, ...}) = 0
mmap(NULL, 102601, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f42713fa000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1868984, ...}) = 0
mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f4270e27000
mprotect(0x7f4270fe7000, 2097152, PROT_NONE) = 0
mmap(0x7f42711e7000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7f42711e7000
mmap(0x7f42711ed000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f42711ed000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f42713f9000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f42713f8000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f42713f7000
arch_prctl(ARCH_SET_FS, 0x7f42713f8700) = 0
mprotect(0x7f42711e7000, 16384, PROT_READ) = 0
mprotect(0x606000, 4096, PROT_READ)     = 0
mprotect(0x7f4271416000, 4096, PROT_READ) = 0
munmap(0x7f42713fa000, 102601)          = 0
brk(NULL)                               = 0xb72000
brk(0xb93000)                           = 0xb93000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2981280, ...}) = 0
mmap(NULL, 2981280, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f4270b4f000
close(3)                                = 0
nanosleep({0, 100000000}, NULL)         = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

If I do something similar with a ping command, it hangs on the part where ping holds for its interval before sending the next one.

code:

strace ping -i 0.1 google.ca
....
write(1, "64 bytes from iad30s21-in-f3.1e1"..., 8864 bytes from iad30s21-in-f3.1e100.net (172.217.15.99): icmp_seq=1 ttl=57 time=0.000 ms
) = 88
gettimeofday({1547519551, 364180}, NULL) = 0
recvmsg(3,

< waits here >

0x7fffcca90d30, 0)           = -1 EAGAIN (Resource temporarily unavailable)
gettimeofday({1547519551, 464187}, NULL) = 0
gettimeofday({1547519551, 464187}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("172.217.15.99")}, msg_iov(1)=[{"\10\0s\7\7l\0\2?F=\\\0\0\0\0;\25\7\0\0\0\0\0\20\21\22\23\24\25\26\27"..., 64}], msg_controllen=0, msg_flags=0}, MSG_CONFIRM) = 64
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("172.217.15.99")}, msg_iov(1)=[{"E\0\0T\0\0\0\0009\1\226}\254\331\17c\n\340$\20\0\0{\7\7l\0\2?F=\\"..., 192}], msg_controllen=32, [{cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */}], msg_flags=0}, 0) = 84
write(1, "64 bytes from iad30s21-in-f3.1e1"..., 8864 bytes from iad30s21-in-f3.1e100.net (172.217.15.99): icmp_seq=2 ttl=57 time=0.000 ms

And sure enough if I try strace htop which takes absolutely forever, it hangs on a nanosleep in there as well between refreshes.

Methanar fucked around with this message at 05:18 on Jan 15, 2019

# ? Jan 15, 2019 05:10

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

That's a toughie. 'time' gets its data from the kernel, which would imply that the kernel is locking up somewhere. For the timers to pause, it'd probably mean an interrupt handler is hanging somewhere, which suggests a buggy device driver or a mis-behaving device.

# ? Jan 15, 2019 09:12

Computer viking: May 30, 2011; Now with less breakage.

Alternatively, you could try a different clocksource (which is what generates the timer interrupts). I think the RHEL 7 guide here should work - just try all the available ones and see if it changes anything.

# ? Jan 15, 2019 17:05

Sheep: Jul 24, 2003

Funnily enough I just got done troubleshooting clocksource issues a couple of days ago, had an HP server where hpet was causing the kernel to crash under high CPU/IO load. Passed hpet=disable via grub and problem solved.

# ? Jan 16, 2019 13:42

Shaocaholica: Oct 29, 2002; Fig. 5E

Can't seem to find any static builds for MPlayer for RHEL 6 and RHEL 7. Is there a better tool to play back rando videos?

# ? Feb 4, 2019 22:28

Last Chance: Dec 31, 2004

VLC?

# ? Feb 4, 2019 22:33

homercles: Feb 14, 2010

mpv

# ? Feb 4, 2019 23:32

Buttcoin purse: Apr 24, 2014

Shaocaholica posted:

Can't seem to find any static builds for MPlayer for RHEL 6 and RHEL 7. Is there a better tool to play back rando videos?

My solution to this is to run Fedora in a chroot (on CentOS 7, not tried with 6) so I can install all the stuff like that which you can't otherwise get (but have to create a new chroot every year due to the Fedora lifecycle).

It's like a "this is why people don't use Linux on the desktop" project though. If you want I can write up some instructions. Basically it involves running rpm and yum with options to make them chroot and install the Fedora packages you need for a base install (straightforward, the challenge is just in knowing that you can do this), setting up schroot so you can get into the chroot to do other things (not too hard), configuring schroot to set up some additional things so that PulseAudio works inside the chroot (somewhat obscure), and at least in my case getting the right nvidia libraries in the chroot.

On the plus side, just about everything I've ever wanted to do in the chroot has worked, other than installing Steam. With various Fedora versions, I've used this to run kdenlive and other video-related tools, various games, Amarok (also available on CentOS, but not with MP3 support), and other things I've forgotten.

I should probably start using Docker though :v:

# ? Feb 5, 2019 08:33

nem: Jan 4, 2003; panel.dev
apnscp: cPanel evolved

Supposedly it�s in EPEL?

# ? Feb 5, 2019 09:06

Bullet Magnet: Sep 26, 2007; it's THAT GUY!

Shaocaholica posted:

Can't seem to find any static builds for MPlayer for RHEL 6 and RHEL 7. Is there a better tool to play back rando videos?

I think there might be something in the Red Hat Software Collections repo. I've been doing a LOT of backend editing to TheForeman/Red Hat Satellite 6.4, and I've basically had to keep that drat repo enabled. I've noticed it seems to be a RedHat groomed EPEL, and I imagine there's probably a whole lot more in here than just every-goddamn-single-version-of-python-to-ever-exist.

# ? Feb 5, 2019 12:37

kujeger: Feb 19, 2004; OH YES HA HA

Don't use rhel/centos on the desktop. Use fedora or ubuntu.

# ? Feb 5, 2019 15:36

Bullet Magnet: Sep 26, 2007; it's THAT GUY!

kujeger posted:

Don't use rhel/centos on the desktop. Use fedora or ubuntu.

Sure, if your environment allows you that degree of freedom. Government environments require that you use software which is STIG approved to support FIPS 140-2. Basically, only RHEL and MacOS fit that requirement :suicide:

If you can't use EPEL at all, you might have luck using rhscl. If you CAN use Fedora, Debian, FreeBSD, Gonkulator, TempleOS, whatever you so wish, it's probably better to use that than even try using Red Hat's Workstation or Developer builds.

# ? Feb 5, 2019 16:18

Truga: May 4, 2014; Lipstick Apathy

homercles posted:

mpv

# ? Feb 5, 2019 16:41

mystes: May 31, 2006

Definitely mpv.

# ? Feb 6, 2019 03:00

Buttcoin purse: Apr 24, 2014

nem posted:

Supposedly it�s in EPEL?

Actually the article says it's in the repo "nux-dextop" (which I've never heard of before).

e: While trying to install mpv, I noticed that mplayer itself is available from the rpmfusion-free-updates repo. I can't remember why I don't use that - I think it might be because not all of the codec packages are available, or it's too old. I guess I should try it again and report back.

Buttcoin purse fucked around with this message at 03:18 on Feb 6, 2019

# ? Feb 6, 2019 03:14

waffle iron: Jan 16, 2004

Install flatpak and then install install mpv or whatever from flathub?

# ? Feb 6, 2019 03:40

An Enormous Boner: Jul 12, 2009

Buttcoin purse posted:

My solution to this is to run Fedora in a chroot (on CentOS 7, not tried with 6) so I can install all the stuff like that which you can't otherwise get (but have to create a new chroot every year due to the Fedora lifecycle).

It's like a "this is why people don't use Linux on the desktop" project though. If you want I can write up some instructions. Basically it involves running rpm and yum with options to make them chroot and install the Fedora packages you need for a base install (straightforward, the challenge is just in knowing that you can do this), setting up schroot so you can get into the chroot to do other things (not too hard), configuring schroot to set up some additional things so that PulseAudio works inside the chroot (somewhat obscure), and at least in my case getting the right nvidia libraries in the chroot.

On the plus side, just about everything I've ever wanted to do in the chroot has worked, other than installing Steam. With various Fedora versions, I've used this to run kdenlive and other video-related tools, various games, Amarok (also available on CentOS, but not with MP3 support), and other things I've forgotten.

I should probably start using Docker though

What the gently caress are you doing?

# ? Feb 8, 2019 01:11

RFC2324: Jun 7, 2012; http 418

An Enormous Boner posted:

What the gently caress are you doing?

Demonstrating the kind of poo poo people do that makes them think that linux isn't ready for the desktop

# ? Feb 8, 2019 05:57

Buttcoin purse: Apr 24, 2014

Yeah, I know I'm not using a desktop distribution, and it was a dumb way to say that it was a lot of screwing around.

# ? Feb 8, 2019 09:32

nem: Jan 4, 2003; panel.dev
apnscp: cPanel evolved

Buttcoin purse posted:

Yeah, I know I'm not using a desktop distribution, and it was a dumb way to say that it was a lot of screwing around.

At least your av agrees

# ? Feb 8, 2019 10:43

Major Ryan: May 11, 2008; Completely blank

Buttcoin purse posted:

Actually the article says it's in the repo "nux-dextop" (which I've never heard of before).

Nux is a guy who's done a lot of stuff for CentOS with extra packages. The repo or some version of it has been around for years. It's definitely unofficial, but it's one of the good ones.

I've used it in production for packages where someone's been desperate for a GUI on a CentOS box; I can't remember it ever causing any problems.

# ? Feb 8, 2019 17:23

Buttcoin purse: Apr 24, 2014

Major Ryan posted:

Nux is a guy who's done a lot of stuff for CentOS with extra packages. The repo or some version of it has been around for years. It's definitely unofficial, but it's one of the good ones.

I've used it in production for packages where someone's been desperate for a GUI on a CentOS box; I can't remember it ever causing any problems.

Thanks, nice to know.

Looking forward to seeing whether any of these repos include KDE for RHEL 8 when the time comes :ohdear:

# ? Feb 9, 2019 06:06

Xik: Mar 10, 2011; Dinosaur Gum

Maybe someone can point me in the right direction before I take an axe to this server:

This is bizarre, but a few days ago our fibre went out (now repaired, cable that terminated at roadside had broken). They obviously can't be related events but since then, ssh and samba performance on my local file server has taken a massive nose dive. Large delays in ssh feedback, samba directory listing, opening large files has huge delays to stream content etc. Here is where I'm at:

Locally all disks seem to be healthy. Have run long smart scans and nothing suggests failing drives. (Have had many failed drives so am a little familiar with this). The performance hit is also with shares across multiple drives.
The link is reported at full speed and if I go out to the Internet (not local) I saturate the external connection:
code:
```
224.49 Mbps -  46.0%
Completed in 2.0 seconds
```
The server is under basically no load, there is multiple GB of mem free, IO is barely above 0.
The same delays and poor performance are from multiple clients, both various Linux flavors and Windows.
When monitoring the server when attempting to open a large file over smb, there is minimal load on server, smbd takes a few % at most, iotop is reporting 0% disk read for the period of the delay, and then it jumps up when the file can actually be accessed on the client. Delay corresponds to size of file, eg: small file starts instantly reading disk, large file (say 20Gb) delay can be over a minute before server shows any sign of disk reads.
smb config files haven't changed, used testparm, everything appears fine.
I've combed journalctl and there are no signs of issues.
Have been through multiple samba troubleshooting guides, including a chapter out of a book on troubleshooting and everything appears fine. Have checked smbd and nmbd log files and nothing except messages of them starting up successfully.
Comedy step, but I've restarted literally all devices on the network, including APs and router.

I don't really know where to look or troubleshoot next. I'm tempted to just reinstall the OS to attempt to confirm if it is hardware related at this point.

# ? Feb 19, 2019 03:50

Last Chance: Dec 31, 2004

can you boot it into another OS or something before nuking the install?

# ? Feb 19, 2019 04:40

Xik: Mar 10, 2011; Dinosaur Gum

I don't have a spare drive lying around to test out an installed os, but I could probably do a test with a live distro from USB stick or something right? I haven't really used one in ages, do they mount a tempfs or something to be able to install new packages?

I might do that if that's a thing.

# ? Feb 19, 2019 04:49

taqueso: Mar 8, 2004

Yeah, you can install packages.

# ? Feb 19, 2019 04:50

Mr Shiny Pants: Nov 12, 2012

Xik posted:

Maybe someone can point me in the right direction before I take an axe to this server:

This is bizarre, but a few days ago our fibre went out (now repaired, cable that terminated at roadside had broken). They obviously can't be related events but since then, ssh and samba performance on my local file server has taken a massive nose dive. Large delays in ssh feedback, samba directory listing, opening large files has huge delays to stream content etc. Here is where I'm at:
Locally all disks seem to be healthy. Have run long smart scans and nothing suggests failing drives. (Have had many failed drives so am a little familiar with this). The performance hit is also with shares across multiple drives.
The link is reported at full speed and if I go out to the Internet (not local) I saturate the external connection:
code:
224.49 Mbps -  46.0%
Completed in 2.0 seconds
The server is under basically no load, there is multiple GB of mem free, IO is barely above 0.

The same delays and poor performance are from multiple clients, both various Linux flavors and Windows.

When monitoring the server when attempting to open a large file over smb, there is minimal load on server, smbd takes a few % at most, iotop is reporting 0% disk read for the period of the delay, and then it jumps up when the file can actually be accessed on the client. Delay corresponds to size of file, eg: small file starts instantly reading disk, large file (say 20Gb) delay can be over a minute before server shows any sign of disk reads.

smb config files haven't changed, used testparm, everything appears fine.

I've combed journalctl and there are no signs of issues.

Have been through multiple samba troubleshooting guides, including a chapter out of a book on troubleshooting and everything appears fine. Have checked smbd and nmbd log files and nothing except messages of them starting up successfully.

Comedy step, but I've restarted literally all devices on the network, including APs and router.
I don't really know where to look or troubleshoot next. I'm tempted to just reinstall the OS to attempt to confirm if it is hardware related at this point.

Can you tcpdump the machine? Maybe it is doing something network related which went TITSUP when your fibre went down.

# ? Feb 19, 2019 11:02

Adbot: ADBOT LOVES YOU

# ? Apr 26, 2024 14:05

Horse Clocks: Dec 14, 2004

Is there a DBAN alternative that can boot from USB, and preferably does SSDs?

I�ve tried everything under the sun to get it to boot from USB with no luck.

I could probably do it manually, but I�ve got about 20 drives to wipe and can only do 4 disks at a time. A plug in and wait tool would save me so much time.

I�d even be happy to shell out a few quid if needs be.

# ? Feb 24, 2019 13:57

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Linux Questions Thread: a bunch of pitfalls, but technically it's possible

«‹›1026 »