Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!
Anyone got a suggestion for scheduled backing up of a full CentOS system (minus /proc, /sys etc.) into a file?

Tried bacula: overcomplicated and wouldn't backup /boot

Just spent about 3 hours on urbackup, which seemed promising but I just couldn't get it to backup my VM. I had the urbackup server on one VM and it could see the other VM and was outputting to an NFS share but the bastard thing would not backup.

There must be something that'll run as a client on the guest which uses a master Linux box to orchestrate.

I want to backup a running system/systems.

Edit: I didn't mean miss out /etc, obviously. I'm just tired and bad syntax.

apropos man fucked around with this message at 23:45 on Oct 11, 2018

Adbot
ADBOT LOVES YOU

Volguus
Mar 3, 2009

kujeger posted:

I mean, sure, I hate mac users like everybody else, but I don't think that's the reason

Exactly. It's not like you hate Mac and its users, it's more that they're irrelevant. All 5 of them. They and their special needs OS cry loudly all day to give you the impression that they're dozens. Meh, let 'em cry.

Methanar
Sep 26, 2013

by the sex ghost

Volguus posted:

Exactly. It's not like you hate Mac and its users, it's more that they're irrelevant. All 5 of them. They and their special needs OS cry loudly all day to give you the impression that they're dozens. Meh, let 'em cry.

:yeah:

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


apropos man posted:

Anyone got a suggestion for scheduled backing up of a full CentOS system (minus /proc, /sys etc.) into a file?

Tried bacula: overcomplicated and wouldn't backup /boot

Just spent about 3 hours on urbackup, which seemed promising but I just couldn't get it to backup my VM. I had the urbackup server on one VM and it could see the other VM and was outputting to an NFS share but the bastard thing would not backup.

There must be something that'll run as a client on the guest which uses a master Linux box to orchestrate.

I want to backup a running system/systems.

Edit: I didn't mean miss out /etc, obviously. I'm just tired and bad syntax.

For free?

ToxicFrog
Apr 26, 2008


apropos man posted:

Anyone got a suggestion for scheduled backing up of a full CentOS system (minus /proc, /sys etc.) into a file?

Tried bacula: overcomplicated and wouldn't backup /boot

Just spent about 3 hours on urbackup, which seemed promising but I just couldn't get it to backup my VM. I had the urbackup server on one VM and it could see the other VM and was outputting to an NFS share but the bastard thing would not backup.

There must be something that'll run as a client on the guest which uses a master Linux box to orchestrate.

I want to backup a running system/systems.

Edit: I didn't mean miss out /etc, obviously. I'm just tired and bad syntax.

Does it have to be a single file? If not, you could use bup. If the master can ssh into the VM, install it on both and then `bup on <VM> index <paths> && bup on <VM> save /' or similar from the master.

evol262
Nov 30, 2010
#!/usr/bin/perl

Vulture Culture posted:

#!/usr/bin/env bash unless you hate Mac users

But I do...

The right answer is clearly ksh anyway.

For backups, I'm a grognard who still swears by amanda

evol262 fucked around with this message at 03:51 on Oct 12, 2018

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

jaegerx posted:

For free?

Not necessarily. I'm trialling stuff at home that's possibly going to be used at work. We went with Acronis Backup briefly, but their RAID driver support for recovery on a Dell PowerEdge sucked. I know that sounds unlikely but, believe me, trying to roll your own drivers into a WinPE recovery disk and then having it not recognise the LTO7 we were storing backups on was frustrating. Our VM's are Linux but the Linux version of the recovery ISO wouldn't recognise the RAID. Then the WinPE version recognised the RAID but not the tape drive. Catch 22.


ToxicFrog posted:

Does it have to be a single file? If not, you could use bup. If the master can ssh into the VM, install it on both and then `bup on <VM> index <paths> && bup on <VM> save /' or similar from the master.

I'll have a look at that. Thanks.

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

apropos man posted:

Anyone got a suggestion for scheduled backing up of a full CentOS system (minus /proc, /sys etc.) into a file?

Tried bacula: overcomplicated and wouldn't backup /boot

The Ceph group at work chose Bareos, a Bacula fork. I think one of their reasons was that it was simpler.

CRAYON
Feb 13, 2006

In the year 3000..

Are there any ffmpeg wizards in here, or could you point me to the correct thread?

I'm running into an issue with trimmed video files being the correct length but showing incorrect duration in media players. I can go into more details if anyone knows ffmpeg and wants to skim some logs.

Volguus
Mar 3, 2009

CRAYON posted:

Are there any ffmpeg wizards in here, or could you point me to the correct thread?

I'm running into an issue with trimmed video files being the correct length but showing incorrect duration in media players. I can go into more details if anyone knows ffmpeg and wants to skim some logs.

That's a funny one: I have used the ffmpeg C API to encode/decode/merge/resample poo poo but I have zero knowledge and experience with the ffmpeg console program that pretty much does the same thing. However, from my experience, when the standard video players show the wrong time it just meant that I messed up the frames drainage. That is, the video has a certain metadata and because for whatever reason i didn't write all the required frames ("drain" the encoder) to the output the video players just wing it and show whatever strikes their fancy.

But that's not a problem applicable to the ffmpeg console program, which does everything in the correct way.

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


apropos man posted:

Not necessarily. I'm trialling stuff at home that's possibly going to be used at work. We went with Acronis Backup briefly, but their RAID driver support for recovery on a Dell PowerEdge sucked. I know that sounds unlikely but, believe me, trying to roll your own drivers into a WinPE recovery disk and then having it not recognise the LTO7 we were storing backups on was frustrating. Our VM's are Linux but the Linux version of the recovery ISO wouldn't recognise the RAID. Then the WinPE version recognised the RAID but not the tape drive. Catch 22.


I'll have a look at that. Thanks.

Oh you want a DR solution. That’s not backup. You want to build bootable isos of your install?

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!

jaegerx posted:

Oh you want a DR solution. That’s not backup. You want to build bootable isos of your install?

What we've been using for years is Lonetar (I know, it's ancient), which makes a daily .tar of the whole OS (you can exclude certain dirs) and you make an ISO rescue CD for each system. The ISO has a kernel and partition table on it which is enough to recover the system that was tarred on any particular day.

I suppose I'm looking for something similar, and Acronis is supposed to do something similar except it uses a generic recovery ISO. An ISO which is incompatible with our hardware, whether you use the WinPE ISO with Dell PERC drivers rolled into it or whether you use the Linux ISO, which doesn't have the PERC drivers but can read our Ouantum tape drives.


Saukkis posted:

The Ceph group at work chose Bareos, a Bacula fork. I think one of their reasons was that it was simpler.

Another one to look into, cheers.

Double Punctuation
Dec 30, 2009

Ships were made for sinking;
Whiskey made for drinking;
If we were made of cellophane
We'd all get stinking drunk much faster!

CRAYON posted:

Are there any ffmpeg wizards in here, or could you point me to the correct thread?

I'm running into an issue with trimmed video files being the correct length but showing incorrect duration in media players. I can go into more details if anyone knows ffmpeg and wants to skim some logs.

Sure. What command are you using, or is it a GUI?

One thing I can think of is the beginning of your segments might not be at keyframes. That would require you to re-encode the entire video, or else weird issues like what you’re describing can crop up. Starting cuts at keyframes avoids that issue and doesn’t result in quality loss.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Why is it that none of the operating systems can't deal with multiple disks correctly anymore? When I installed Windows back then, with two SSDs in the system, it tried to install the bootloader to the wrong SSD. Now I tried to install Ubuntu to my system, it attempted the same and luckily failed. It let me chose a different drive after that error, but it didn't do squat with that choice. I actually had to spin up a VM, pass through the SSD and install things in there.

redeyes
Sep 14, 2002

by Fluffdaddy
UEFI means the mobo decides which disk gets priority.

CRAYON
Feb 13, 2006

In the year 3000..

Volguus posted:

That's a funny one: I have used the ffmpeg C API to encode/decode/merge/resample poo poo but I have zero knowledge and experience with the ffmpeg console program that pretty much does the same thing. However, from my experience, when the standard video players show the wrong time it just meant that I messed up the frames drainage. That is, the video has a certain metadata and because for whatever reason i didn't write all the required frames ("drain" the encoder) to the output the video players just wing it and show whatever strikes their fancy.

But that's not a problem applicable to the ffmpeg console program, which does everything in the correct way.

Haha, you're on to something. I'm using the ffmpeg-python library. More details below.

Double Punctuation posted:

Sure. What command are you using, or is it a GUI?

One thing I can think of is the beginning of your segments might not be at keyframes. That would require you to re-encode the entire video, or else weird issues like what you’re describing can crop up. Starting cuts at keyframes avoids that issue and doesn’t result in quality loss.

Okay so I am doing what I thought was the exact same thing with two different tools, ffmpeg-python and the regular command line tool. The ffmpeg-python tool is producing files that report incorrect duration while the command line tool files are reporting the correct duration. I'm specifically changing the filetype with both tools because I want ffmpeg to re-encode the video so that I can clip down to the millisecond and not have to start at a keyframe.


ffmpeg cli (log):
code:
ffmpeg -ss 01:15:05.296 -i input.mkv -to 0:00:11.803 output.mp4
ffmpeg-python (log):
code:
ts1=01:15:05.296
ts2=01:15:17.099

(
    ffmpeg
    .input(input, ss=ts1, to=ts2)
    .output(output)
    .run()
)
I was hoping to make a python tool for my super basic clip making needs but this duration issue is holding me up. Thanks for any help.

nem
Jan 4, 2003

panel.dev
apnscp: cPanel evolved
Is there a command that creates a ring buffer of the last n seconds? I've got a tricky hangup that I can reproduce only so often and I'd like to record the last 60 seconds of an strace without growing the log infinitely large.

minato
Jun 7, 2004

cutty cain't hang, say 7-up.
Taco Defender
It depends on how the process that logs writes its output. If it's to a file that the logging process constantly keeps open, you're probably out of luck because you can't remove the head of the file while it's open. If it's to a file that the logging process opens/appends/closes for each line, then use logrotate. And if it's logging to stdout or something similar, you can probably pipe it to a bash script that uses a "while read" loop to log the lines to disk, and then calls logrotate every X lines.

E: this could probably be code-golf'd better, but you get the gist. This will pipe the output of a "strace ls -l" into a bash script, and there's a line where you have the chance to run logrotate (but I didn't implement it). Also, look into the "split" command as this can automatically distribute a single input stream over many files, but it might not be appropriate for a ring buffer.
code:
#!/bin/bash
strace -o >(
  declare -i num_lines=0
  while IFS='' read -r line ; do
    echo "${line}" >> log
    ((num_lines++))
    if [[ $num_lines > 20 ]]; then
          # Run logrotate here
          num_lines=0
    fi
  done
) ls -l

minato fucked around with this message at 21:48 on Oct 20, 2018

xzzy
Mar 5, 2009

If you have Apache installed, look at the included rotatelogs command.

If not, I guarantee you can find a zillion stdout rotating solutions with some google.

nem
Jan 4, 2003

panel.dev
apnscp: cPanel evolved

xzzy posted:

If you have Apache installed, look at the included rotatelogs command.

If not, I guarantee you can find a zillion stdout rotating solutions with some google.

Perfect, thanks! Ended up with this which works great:

code:
(strace -s 1024 -f -p 9379,9422,9438 2>&1) | rotatelogs -c -t /tmp/log 60

The XKCD Larper
Mar 1, 2009

by Lowtax
I want to make a Vim mapping to put stuff in the mac clipboard. I do this with w !pbcopy in normal mode. How can I make one that works with visual line selection too?

jaxercracks
Oct 12, 2012
I have an Ubuntu 18.04 install that is running out of space on /boot. Currently the drive partitioning scheme is:
code:

Model: ATA Samsung SSD 840 (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End    Size   Type      File system  Flags
 1      1049kB  256MB  255MB  primary   ext2         boot
 2      257MB   500GB  500GB  extended
 5      257MB   500GB  500GB  logical                lvm


Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/ubuntu--vg-swap_1: 17.1GB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags: 

Number  Start  End     Size    File system     Flags
 1      0.00B  17.1GB  17.1GB  linux-swap(v1)


Model: Linux device-mapper (linear) (dm)
Disk /dev/mapper/ubuntu--vg-root: 483GB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags: 

Number  Start  End    Size   File system  Flags
 1      0.00B  483GB  483GB  ext4
It seems I could either:

Resize the lvm partion down and increase the boot partion or I could try to move boot to the root lvm partition and do away with the separate mbr boot partition all together. Which of these is the best idea / least likely to eat my data and how would I do that?

Docjowles
Apr 9, 2009

You might have old kernel versions hanging out in /boot that you can just delete to reclaim some space.

jaxercracks
Oct 12, 2012

Docjowles posted:

You might have old kernel versions hanging out in /boot that you can just delete to reclaim some space.

Ah, yeah I should I have been more specific. I have already done that. Basically I am at the point where I have to manually remove the one old kernel before there is space for a new kernel. Also I tried to run the upgrade to 18.10 the other day and it complained of lack of space on boot. Just tired of the constant janitoring of /boot and trying to get it to where I can just sudo apt upgrade and have it work without complaining constantly.

Horse Clocks
Dec 14, 2004


The XKCD Larper posted:

I want to make a Vim mapping to put stuff in the mac clipboard. I do this with w !pbcopy in normal mode. How can I make one that works with visual line selection too?

The + register is also your system clipboard.

I’d give you an example, but I honestly can’t think of the actual button presses. It’s all muscle memory.

Basically instead of yanking to an alphanumeric register, use +. You can also paste from it.

SoftNum
Mar 31, 2011

Horse Clocks posted:

The + register is also your system clipboard.

I’d give you an example, but I honestly can’t think of the actual button presses. It’s all muscle memory.

Basically instead of yanking to an alphanumeric register, use +. You can also paste from it.

It's "+yy (yank-to-system) or "+p (paste from system)

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

jaxercracks posted:

Ah, yeah I should I have been more specific. I have already done that. Basically I am at the point where I have to manually remove the one old kernel before there is space for a new kernel. Also I tried to run the upgrade to 18.10 the other day and it complained of lack of space on boot. Just tired of the constant janitoring of /boot and trying to get it to where I can just sudo apt upgrade and have it work without complaining constantly.
How did you remove the old kernels or verify that they were deleted, and what's using space on /boot?

VikingofRock
Aug 24, 2008




The XKCD Larper posted:

I want to make a Vim mapping to put stuff in the mac clipboard. I do this with w !pbcopy in normal mode. How can I make one that works with visual line selection too?

The previously-posted solutions in this thread work great and you should definitely use them, but I've been reading up on vimscript a bit and I thought I'd give this a shot.

code:
nnoremap <silent> <unique> <leader>c :set opfunc=CopyToClipboard<cr>g@
vnoremap <silent> <unique> <leader>c :<c-u>call CopyToClipboard(visualmode())<cr>

function! CopyToClipboard(type) abort
    " Save state
    let sel_save = &selection
    let &selection = 'inclusive'
    let reg_save = @@

    " Yank text
    if a:type ==# 'line'
	silent execute "normal! '[V']y"
    elseif a:type ==# 'block'
	silent execute "normal! `[\<C-V>`]y"
    elseif a:type ==# 'V'
        silent execute 'normal! `<' . a:type . '`>y'
    else
	silent execute 'normal! `[v`]y'
    endif

    "Run command
    silent execute system('pbcopy', @@)

    " Restore state
    let &selection = sel_save
    let @@ = reg_save
endfunction
This is mostly taken from Learn Vimscript the Hard Way and :help map-operator, and it seems to work okay (I used this mapping to copy the code into the browser). Critiques welcome.

lordfrikk
Mar 11, 2010

Oh, say it ain't fuckin' so,
you stupid fuck!
Having a separate boot partition for just home installation is a huge hassle: 1. Size will probably be wrong unless much bigger than standards in installers. 2. Windows 10 is just poo poo in general but it creates the EFI without asking about what size you want it, and if you as much as touch it will completely poo poo the bed. 3. Special snowflakes like Solus which make it even more of a hassle to manage several distros.

lordfrikk fucked around with this message at 10:06 on Oct 23, 2018

jaxercracks
Oct 12, 2012

anthonypants posted:

How did you remove the old kernels or verify that they were deleted, and what's using space on /boot?

Removed with apt the standard apt remove linux-image-blah-blah-blah then periodically clean up orphaned stuff by hand. I don't remember at this point how I ended up with this size boot partition. But it really seems just maybe half as big as needed. Here is my current /boot:

code:

-rw-r--r-- 1 root root 1.5M Oct 10 05:20 abi-4.15.0-38-generic
-rw-r--r-- 1 root root 212K Oct 10 05:20 config-4.15.0-38-generic
drwxr-xr-x 5 root root 1.0K Oct 19 21:28 grub
-rw-r--r-- 1 root root  57M Oct 22 17:10 initrd.img-4.15.0-38-generic
drwx------ 2 root root  12K Apr 17  2014 lost+found
-rw-r--r-- 1 root root 179K Jan 28  2016 memtest86+.bin
-rw-r--r-- 1 root root 181K Jan 28  2016 memtest86+.elf
-rw-r--r-- 1 root root 181K Jan 28  2016 memtest86+_multiboot.bin
-rw-r--r-- 1 root root    0 Oct 10 05:20 retpoline-4.15.0-38-generic
-rw------- 1 root root 3.9M Oct 10 05:20 System.map-4.15.0-38-generic
-rw------- 1 root root 7.9M Oct 10 06:43 vmlinuz-4.15.0-38-generic



Obviously I have recently cleaned this out.

So you can see that the current kernel stuff takes up right at 70mb. Which gives you just barely enough space to have three kernels in there. The kernel autoremove script indicates that depending on how it is called you might end up with up to four kernels. So it seems like enough space for four kernels plus a bit of slack would be the correct amount.

VikingofRock
Aug 24, 2008




lordfrikk posted:

Having a separate boot partition for just home installation is a huge hassle: 1. Size will probably be wrong unless much bigger than standards in installers. 2. Windows 10 is just poo poo in general but it creates the EFI without asking about what size you want it, and if you as much as touch it will completely poo poo the bed. 3. Special snowflakes like Solus which make it even more of a hassle to manage several distros.

What makes Solus a special snowflake distro? I don't really know much about it other than the fact that it's rolling release and people tend to like it.

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

jaxercracks posted:

Removed with apt the standard apt remove linux-image-blah-blah-blah then periodically clean up orphaned stuff by hand. I don't remember at this point how I ended up with this size boot partition. But it really seems just maybe half as big as needed. Here is my current /boot:

code:

-rw-r--r-- 1 root root 1.5M Oct 10 05:20 abi-4.15.0-38-generic
-rw-r--r-- 1 root root 212K Oct 10 05:20 config-4.15.0-38-generic
drwxr-xr-x 5 root root 1.0K Oct 19 21:28 grub
-rw-r--r-- 1 root root  57M Oct 22 17:10 initrd.img-4.15.0-38-generic
drwx------ 2 root root  12K Apr 17  2014 lost+found
-rw-r--r-- 1 root root 179K Jan 28  2016 memtest86+.bin
-rw-r--r-- 1 root root 181K Jan 28  2016 memtest86+.elf
-rw-r--r-- 1 root root 181K Jan 28  2016 memtest86+_multiboot.bin
-rw-r--r-- 1 root root    0 Oct 10 05:20 retpoline-4.15.0-38-generic
-rw------- 1 root root 3.9M Oct 10 05:20 System.map-4.15.0-38-generic
-rw------- 1 root root 7.9M Oct 10 06:43 vmlinuz-4.15.0-38-generic



Obviously I have recently cleaned this out.

So you can see that the current kernel stuff takes up right at 70mb. Which gives you just barely enough space to have three kernels in there. The kernel autoremove script indicates that depending on how it is called you might end up with up to four kernels. So it seems like enough space for four kernels plus a bit of slack would be the correct amount.
Last I heard, Ubuntu's kernel autoremove wasn't reliable, but how big is your /boot partition? 100M?

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!
It's also probably possible to specify how many old kernels to keep whenever apt goes through the removal process.

I know it's possible in Fedora because I've got a small VM set up, to use the current working kernel and keep only the previous one spare.

I'm on my phone right now but the method for Ubuntu should be just a Google away.

lordfrikk
Mar 11, 2010

Oh, say it ain't fuckin' so,
you stupid fuck!

VikingofRock posted:

What makes Solus a special snowflake distro? I don't really know much about it other than the fact that it's rolling release and people tend to like it.

Hey, I like it too! They have some good, fresh ideas but that doesn't make them super easy to use with other OSes. See the bootloader section here:

https://getsol.us/2018/02/15/exploring-solus-architecture/

The XKCD Larper
Mar 1, 2009

by Lowtax

Horse Clocks posted:

The + register is also your system clipboard.

:worship:

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
I installed vim 8.1 from Homebrew and it yanks to/pastes from the system clipboard by default and I can't figure out how to turn it off

Truga
May 4, 2014
Lipstick Apathy
touch ~/.vimrc

Saukkis
May 16, 2003

Unless I'm on the inside curve pointing straight at oncoming traffic the high beams stay on and I laugh at your puny protest flashes.
I am Most Important Man. Most Important Man in the World.

jaxercracks posted:

So you can see that the current kernel stuff takes up right at 70mb. Which gives you just barely enough space to have three kernels in there. The kernel autoremove script indicates that depending on how it is called you might end up with up to four kernels. So it seems like enough space for four kernels plus a bit of slack would be the correct amount.

You might be able to reduce the size of the initrd.img by editing /etc/initramfs-tools/initramfs.conf . Set MODULES to "dep" and COMPRESS to something other than "gzip". Admittedly I haven't tried these myself. 'man initramfs.conf'

post hole digger
Mar 21, 2011

For how long has Centos had security errata in yum (ie yum --security update)? I thought that was a RHEL only thing and was surprised to see it work the other day.

Adbot
ADBOT LOVES YOU

anthonypants
May 6, 2007

by Nyc_Tattoo
Dinosaur Gum

my bitter bi rival posted:

For how long has Centos had security errata in yum (ie yum --security update)? I thought that was a RHEL only thing and was surprised to see it work the other day.
CentOS (and Fedora) are kind of upstream of RHEL, so I'm gonna say almost exactly as long as RHEL

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply