|
I swear I had a list somewhere!  ninja-edit: great post, great snipe
|
#
?
Oct 28, 2020 22:07
|
|
|
|
| # ? Apr 26, 2024 16:37 |
|
|
Bob Morales posted:Random-rear end question, somewhat related to Linux: I bought keyring tags. Printed labels are also nice solution, but I usually don't have a label printer nearby.
|
|
#
?
Oct 28, 2020 23:14
|
|
|
Bob Morales posted:Random-rear end question, somewhat related to Linux: I PXE boot if I have to boot an OS, which I usually don't need to. MS MDT boots via wimboot from iPXE, the other Linux crap just boots directly from iPXE. I admit not everyone has the time to set this kind of thing up, but it's amazingly convenient. (The boot menu is also behind a login that authenticates against MS AD, everything is TLS encrypted after the initial bootstrap of the iPXE binary) I have a USB drive. It has iPXE on it, to run on systems that can't PXE for some reason.
|
|
#
?
Oct 28, 2020 23:45
|
|
|
Antigravitas posted:I PXE boot if I have to boot an OS, which I usually don't need to. How much time are we talking? Like without the whole AD part? That sounds like Nirvana
|
|
#
?
Oct 29, 2020 00:09
|
|
|
xzzy posted:I gave up years ago and keep a single usb key, and dd over the top of it with whatever image solves my current problem. The correct answer. Also, replace said usb key when it starts throwing weird errors. Got a job few months ago at a company as a C++ developer. So far I've been writing bash, ash (busybox shell) and python scripts (with a smattering of C++), for their in-house debian-based distribution. I've been flashing a USB drive that I had for 5+ years at least 20 times a day to install and reinstall the distribution (working on installer and early-boot guts). After a while I saw reading errors being spit on the terminal and the write speed slowed down quite a bit. gently caress it, buy a new one and expense it.
|
|
#
?
Oct 29, 2020 01:38
|
|
|
How often do you people boot from usb drives that the images aren't already out of date when you need them?
|
|
#
?
Oct 29, 2020 01:47
|
|
|
mystes posted:How often do you people boot from usb drives that the images aren't already out of date when you need them? For me it isn't just boot drives, but drives I've used to transfer files from one machine to another for safe-keeping but I can't remember exactly what drive on which computer, so I'm loath to blank them if I don't have to. Or drives that are kind of archival, like old school papers and such. Or drives that I can't remember the capacity until I plug them in (hello 1GB drive from yesteryear!). Stuff like that.
|
|
#
?
Oct 29, 2020 02:41
|
|
|
What's the correct protocol for dealing with a flash drive that is possibly infected with malware? Erasing it, formatting it. I'm using Ubuntu 20.04; assume I'm new to it. I guess the hardware could have bad extra stuff designed into it that lingers even after a format, but I'll think about that next.
|
|
#
?
Oct 29, 2020 09:17
|
|
|
namlosh posted:How much time are we talking? Like without the whole AD part? That sounds like Nirvana Most of the time is spent trying to get at the exact magic kernel parameters required to initiate a PXE boot, because the kernel/initrd needs to know that it has to fetch its squashfs first and that's kind of standardised but not really. The AD thing is just Apache configured with basic auth via LDAP against a domain controller. That is the easy part. The process is: * Client requests DHCP * DHCP server answers with the normal DHCP offer and PXE file location * Client fetches file via tftp and executes it * iPXE (in our case) executes an embedded script that runs DHCP again, then fetches a script via https that contains the login screen * You enter your creds, iPXE then requests https://username:password@yourserver.fqdn/menu.ipxe and executes it * You select the entry from the menu, iPXE fetches kernel+initrd via https and boots the kernel * The initrd does DHCP again, then fetches a squashfs image and mounts it. Boot continues as normal There are some annoying caveats: * You probably need to use a self-signed cert for that connection because large cert chains can not be embedded into the iPXE executable. If you're at home, I wouldn't bother, but you probably don't want to transmit your AD creds over plain http. * Your DHCP server should be able to reply differently to bios and efi clients so that bios clients get their bios binary and efi clients get their efi binary * You probably need to compile iPXE yourself to embed an initial script that does dhcp and chainloads the login screen script * EFI PXE firmware is often horribly broken, even more so than BIOS PXE ever was Happy Thread posted:What's the correct protocol for dealing with a flash drive that is possibly infected with malware? Erasing it, formatting it. I'm using Ubuntu 20.04; assume I'm new to it. Just blkdiscard it. If it doesn't support discard, blkdiscard -z zeroes the entire thing. If you don't want to touch the commandline, use gparted and write a new partition table.
|
|
#
?
Oct 29, 2020 09:52
|
|
|
Bob Morales posted:Random-rear end question, somewhat related to Linux: The image is a custom-made FreeBSD image via poudriere-image(8), using the iso/usb+zmfs image type. BlankSystemDaemon fucked around with this message at 12:35 on Oct 29, 2020 |
|
#
?
Oct 29, 2020 12:28
|
|
|
Bob Morales posted:Random-rear end question, somewhat related to Linux: stash all the isos on the same USB, make a grub menu to select which iso to boot at runtime vvvv: this is even better holy moley Truga fucked around with this message at 14:31 on Oct 29, 2020 |
|
#
?
Oct 29, 2020 12:37
|
|
|
Truga posted:stash all the isos on the same USB, make a grub menu to select which iso to boot at runtime If you have a rooted android phone, or an old android phone you want to root, you can install an app that will make connected computers boot from an ISO downloaded to the phone
|
|
#
?
Oct 29, 2020 14:20
|
|
|
Antigravitas posted:Most of the time is spent trying to get at the exact magic kernel parameters required to initiate a PXE boot, because the kernel/initrd needs to know that it has to fetch its squashfs first and that's kind of standardised but not really. Awesome right up... thanks very much for this. I need to start simple, but like I said... you’re living in the end game my friend
|
|
#
?
Oct 29, 2020 15:39
|
|
|
Happy Thread posted:What's the correct protocol for dealing with a flash drive that is possibly infected with malware? Erasing it, formatting it. I'm using Ubuntu 20.04; assume I'm new to it. Toss it. Malware infecting the actual hardware of a usb drive is trivial, and once the actual hardware gets infected it will reinfect every time you reinsert it. Flash drives are too cheap to try and clean up
|
|
#
?
Oct 29, 2020 16:06
|
|
|
I've been meaning to migrate my environment to ipxe for a while now, but normal pxeboot works just well enough that I haven't actually forced myself to do it. But ipxe has so many neat features that would make my installs work smoother, handing out vmlinuz/initrd over http would let me stop relying on UDP to ship those images across my network (it works most of the time, but breaks just enough to be irritating). Obviously with physical hosts I couldn't completely eliminate tftp, but the ipxe image is a lot smaller.
|
|
#
?
Oct 29, 2020 16:56
|
|
|
namlosh posted:Awesome right up... thanks very much for this. I need to start simple, but like I said... you’re living in the end game my friend I can post curated config examples tomorrow after I get my work done. There are some gotchas that are hard to debug if one doesn't know the protocols involved well.
|
|
#
?
Oct 29, 2020 19:09
|
|
|
RFC2324 posted:Toss it. Malware infecting the actual hardware of a usb drive is trivial, and once the actual hardware gets infected it will reinfect every time you reinsert it. Buying a cheap pile of drives ""new"" is what I did, but even then I still distrust them quite a bit. With so many knockoffs it seems hard to prove the manufacturer, even independent of the price. Isn't there a way to stop Ubuntu from being affected by any executable code on a USB drive upon insertion? Such that no matter what signals it gets back from the drive, it's not going to run anything? Is it enough to just disable the most obvious "autorun" type actions from OS options? It seems like I'd want to be sure about the moment of insertion before I think about running the blkdiscard command. If there's as you say still easily malware in the hardware, I guess at that point I'd only be vulnerable to MITM style stuff, as in files I put on there get messed with on the way back to me. I guess that still makes it very hard to turn them into bootable OS drives that are trustworthy...
|
|
#
?
Oct 29, 2020 20:28
|
|
|
Happy Thread posted:Buying a cheap pile of drives ""new"" is what I did, but even then I still distrust them quite a bit. With so many knockoffs it seems hard to prove the manufacturer, even independent of the price. Malicious HID devices are the most common target of experimentation, but someone who knew what they were doing could probably find actual usb driver exploits that would be much more effective. mystes fucked around with this message at 20:38 on Oct 29, 2020 |
|
#
?
Oct 29, 2020 20:35
|
|
|
I originally read about an exploit that allowed the microcontroller to be infected with a worm, and be an unblockage vector for infection that way, but can't seem to find it now.Happy Thread posted:Buying a cheap pile of drives ""new"" is what I did, but even then I still distrust them quite a bit. With so many knockoffs it seems hard to prove the manufacturer, even independent of the price. https://www.wired.com/2014/10/code-published-for-unfixable-usb-attack/ This has nothing to do with autorun type features E: More thorough writeup: https://www.howtogeek.com/203061/don%E2%80%99t-panic-but-all-usb-devices-have-a-massive-security-problem/ RFC2324 fucked around with this message at 20:55 on Oct 29, 2020 |
|
#
?
Oct 29, 2020 20:52
|
|
|
I need a little ffmpeg wizardry. I'm trying to split a video ripped from a DVD that has chapters into individual videos of each chapter. I leveraged some Python code online for this that I don't think matters. It's eventually just invoking ffmpeg with something like:code:
|
|
#
?
Oct 30, 2020 05:44
|
|
|
I'd go with it being a key-frame issue; if your _START_ doesn't align to a keyframe, copy on the video will have to wait until the next keyframe while audio behaves differently. Personally, I'd install a copy of avidemux and use that; you can still do a copy-copy without re-encoding but you can select the start/end times based on keyframes and get visual confirmation of what you're selecting. If you need to slice more time-specific than keyframes allow then re-encoding will be the only solution.
|
|
#
?
Oct 30, 2020 08:19
|
|
|
I ended up adjusting the script I was using so it went a tenth of a second earlier in the start time. Somehow that worked. I suspect it was starting right after that frame but I didn't really expect that hack to work lol. Regarding avidemux, I tried it out because I somehow had installed it in the past before on a Windows setup. I couldn't crop unless I specified a codec other than the copy codec. I'm guessing that means it would have ended up transcoding the result.
|
|
#
?
Oct 31, 2020 07:57
|
|
|
Rocko Bonaparte posted:Regarding avidemux, I tried it out because I somehow had installed it in the past before on a Windows setup. I couldn't crop unless I specified a codec other than the copy codec. I'm guessing that means it would have ended up transcoding the result. I'm assuming your tweaked ffmpeg start time works because it ended up on a keyframe (or close enough for you not to notice any audio sync change).
|
|
#
?
Oct 31, 2020 11:49
|
|
|
Antigravitas posted:I can post curated config examples tomorrow after I get my work done. There are some gotchas that are hard to debug if one doesn't know the protocols involved well. Yes please
|
|
#
?
Oct 31, 2020 14:44
|
|
|
Alright, I finally found the time to do condense this down. I'm going to drop the auth and tls stuff because it's complicated. Ingredients: * a tftp server (tftp-hpa) * a DHCP server (isc-dhcp-server here, but use whichever) * optionally, a http server (apache here) DHCP You'll want next-server to point at our PXE boot server where tftp and httpd are running. There is an interesting bit of configuration here: BIOS and EFI clients require different binaries and luckily most implementations send their arch in the DHCP DISCOVER packet. So we can tell dhcpd what that is: code:code:TFTP We're going to use /srv/tftp as our base of operations with tftp-hpa running tftp. We can use http to serve everything but the iPXE binary, but I prefer having one big tree reachable via both tftp and http instead of separating them. Folder structure below /srv/tftp code:I think there's a web buildbot thing available, but compiling iPXE is easy. We'll need git and build-essential, then: code:This is the script: code:code:So when a client starts executing our iPXE binary it will use dhcp to configure its interfaces. Immediately afterwards it will pull a script file from our server and execute it. Note that a bin-x86_64-efi/ipxe.usb EMBED=builtin.ipxe produces an image you can dd onto a usb drive so you can skip the entire PXE malarkey for non-cooperating computers. Let's write that second script file: code:Get GRML (or another Linux distro, I'm just using Grml as an example) Go to https://grml.org/download/, download the large amd64 iso. IGNORE THE NETBOOT PACKAGE, it doesn't do what we want right now. Unpack/Mount that iso and copy the files below boot/grml64full/ (initrd.img, vmlinuz) and live/grml64-full/grml64-full.squashfs to /srv/tftp/boot/grml/ Make sure tftp can read them, then try to PXE-boot a client. HTTP You can basically just put code:Getting the right kernel boot parameters for PXE boot This is tricky because distros have different scripts in their initrd, but the process is usually the same: * The kernel starts, along with a ramdisk (initrd) * Scripts in the initrd use dhcp and then try to fetch a squashfs containing the real root file system * squashfs gets mounted and the live image boots Note that this boot process isn't actually all that dissimilar from the normal boot process of a modern distro. That's why we took the normal Grml iso, most distros have the process built into their initrd even in their desktop images. Here's the kernel command line for clonezilla: code:code:
|
|
#
?
Nov 2, 2020 18:00
|
|
|
A note on ^^^^: I heavily, heavily redacted internal documentation there. There may be typos hiding, so you'll have to do the homework. It's not the step-by-step guide that it is internally, sadly. (We compile iPXE via CI, for example) Oh, and this basically replicates a bunch of stuff https://fogproject.org does as well… A few things of note: Since iPXE can request scripts from a server via http you can have a server respond dynamically to boot requests. It's really easy to do so, in fact. We have a few emergency janitoring utilities in our menu (gparted, clonezilla, grml), but most of the time it is used to run MDT (keyword: wimboot), or automated Debian installations that come with our ssh keys and initial config for automatic provisioning. Helldesk uses it as well for workstation janitoring (wimboot again).
|
|
#
?
Nov 2, 2020 18:10
|
|
|
Have you ever played with or looked at puppet razor? I love the basic idea, system boots ipxe and a bunch of info about the hardware get saved to a database and you can DO STUFF based on it. But the project doesn't seem to be moving very fast, and writing templates to install an OS really bugs me (I had to use cobbler for a while and hated it).
|
|
#
?
Nov 2, 2020 18:16
|
|
|
xzzy posted:Have you ever played with or looked at puppet razor? I have no experience with this, but this just reminded me of "serveradmin" which InnoGames use to deploy/administrate their infrastructure. There was a neat talk about it during FOSDEM19: https://archive.fosdem.org/2019/schedule/event/keeping_track_stateful_infrastructure/ Code: https://github.com/innogames/serveradmin & https://github.com/innogames/igvm Side note: this makes me sad, because I presume FOSDEM won't happen in 2021, seeing how Covid is going more generally across Europe, and in Belgium in particular. 
|
|
#
?
Nov 2, 2020 18:28
|
|
|
xzzy posted:Have you ever played with or looked at puppet razor? It does sound interesting, but in our case we don't operate at a scale where provisioning of hardware has to be centrally managed in this way. Ansible does all the Linux stuff except installing on bare metal and we don't need to provision new physical hosts that often. MDT does all the Windows bare metal crap, so we are covered.
|
|
#
?
Nov 2, 2020 18:55
|
|
|
Darn, my hunt for someone to chat about hardware provisioning continues.  Basically all I do is bare metal and I really want to overhaul our system but I'm unsure what the state of the art is (the path of least resistance is redhat satellite, a product I've evaluated many times over the years and discarded because it does nothing better than what I already have and comes with a huge price tag).
|
|
#
?
Nov 2, 2020 19:00
|
|
|
Computer viking posted:Also, nothing spends all its time doing parallel work - it may just be in a single-threaded phase right now. RFC2324 posted:Might check docker config. Following up, because I hate it when other people ask for help, then don't say 'thank you' when you fix the problem. After I fixed my first problem, I had a second problem, using up all the system memory and the system killing the process, that was fixed by messing with the docker configuration. I ended up running the process inside a slice that limited the usable memory to only 55 gb out of 64 gb.  So, thread, thank you! Bobulus fucked around with this message at 03:04 on Nov 3, 2020 |
|
#
?
Nov 3, 2020 02:41
|
|
|
Antigravitas posted:Tons of amazing stuff Thanks for taking the time to type that out. I’ll have to take a look later, but it looks like a really impressive setup man
|
|
#
?
Nov 3, 2020 14:51
|
|
|
Antigravitas posted:Ansible does all the Linux stuff except installing on bare metal and we don't need to provision new physical hosts that often. This is our situation. Outside of large hardware refreshes (every 3-5 years in the datacenter) we do manual USB installs of minimal CentOS 8 and then run an Ansible playbook to install and configure everything as necessary. For hardware refreshes we have a PXE kickstart server we enable on the IPMI network that is broadly analogous to what you provided. Foreman/Katello would be nice but would get used so infrequently that it's difficult to justify the overhead of administration/training/etc. Spacewalk did the trick before it got discontinued, but it was pretty janky in some places. Sheep fucked around with this message at 22:09 on Nov 3, 2020 |
|
#
?
Nov 3, 2020 22:02
|
|
|
Today's lesson: when using LUKS encryption, make a backup of the master key or header. I have a LUKS disk that suddenly won't decrypt. A dump of the header suggests all the keys have disappeared. I wasn't messing around with it so somehow it must have become corrupted; perhaps a failing disk? If I made a backup of the master key at the time of creating it, I've no idea where I stored it! Fortunately I can live with the stuff lost since my last backup.
|
|
#
?
Nov 6, 2020 12:56
|
|
|
I ended up in your situation once before. No idea what happened but none of the normal keys (user, admin, oh crap rescue) worked. I don't recall doing any deep dive, I think we just wrote it off as a failing disk. Because I didn't want to end up in that position again, one of the first things I did when building our clevis/tang setup was setup our deployment pipeline to automatically dump and backup the LUKS header once all of the keys were set. Edit: This is fine quote:load average: 8452.56, 10437.97, 10815.54 Sheep fucked around with this message at 15:39 on Nov 6, 2020 |
|
#
?
Nov 6, 2020 14:37
|
|
|
Sheep posted:Edit:
|
|
#
?
Nov 6, 2020 17:43
|
|
|
It's just processes waiting on RPC replies, but they do contribute to load average. https://access.redhat.com/solutions/23338 quote:NFS Client mounts the NFS export with the hard NFS mount option by default. This results in NFS RPC Requests being retried indefinitely. A side effect of hard-mounted NFS file system is that processes block (or "hang") in a high-priority disk with waiting I/O state until their NFS RPC Reply is received from the NFS Server. Hence, when an NFS Server goes down, the NFS mounts on the NFS Client will hang until the NFS Server recovers. Sheep fucked around with this message at 20:30 on Nov 6, 2020 |
|
#
?
Nov 6, 2020 20:27
|
|
|
Never do hard NFS mounts unless you WANT an issue with the NFS server to kill everything else.
|
|
#
?
Nov 6, 2020 20:35
|
|
|
All my users demand hard mounts because they're worried about data integrity. 
|
|
#
?
Nov 6, 2020 20:39
|
|
|
|
| # ? Apr 26, 2024 16:37 |
|
|
Pablo Bluth posted:Today's lesson: when using LUKS encryption, make a backup of the master key or header. cryptsetup can store the header in a different file, so it can be a bit easier to make a backup of it.
|
|
#
?
Nov 6, 2020 22:40
|
|