|
CRIP EATIN BREAD posted:i only used yocto on one project, and it was to build an embedded version of linux. yeah both buildroot and yocto are pretty good at doing this although yocto takes a more... proactive approach with a bunch of runtime checks that a license is present (whereas with buildroot, since it's more centralized, it's more of a "nobody's going to apply your patch into trunk if it doesn't have a license with it")
|
#
¿
Nov 1, 2021 22:34
|
|
|
|
| # ¿ Apr 28, 2024 12:21 |
|
|
here's my current (least) favorite embedded linux thing problem: let's say that you want an embedded linux device that has a predefined system on it so you want to use something like buildroot or yocto. let's also say that one thing you want running on it is a modern interpreted-language app. maybe it's javascript, maybe it's python, whatever (i know, i know, just come on this voyage of discovery with me). what do such "modern apps" all have in common? a shitload, an absolute shitload, of dependencies, often for a bad reason, sometimes for a good reason. here's the thing with buildroot and yocto: they weren't written for that at all. they're written for applications that don't have that many deps, and also don't have that many unique deps. and the expectation is that those deps aren't really unified by anything so they'll all need their own build handling. so the expectation is that any shared library or header file or python module or node module has a recipe that's in your system's recipe layer, and all your recipes mark their deps, and then you enable your end application and write down the deps and it activates all the transitive ones and you've got nice hash locked versioned dependencies. cool the thing is that python and js (again, work with me here) both already have that in terms of their language dependency ecosystem: npm and pypi. those places already have all those libraries with their dependencies marked. they have tooling bundled with the standard library or standard project tools to pull them and change them and add them and remove them, and all those deps are stored in some language specific metadata. so yocto and buildroot all expect recipes for every individual project dependency and for all those deps to be linked, and you've got a nodejs app with a package.json and yarn.lock or a python app with a setup.py or pyproject.toml or Pipfile or whatever and they're just completely different and so you get to pick one of: - manually maintain your recipe store and dependency links to correspond to the app's normal dependency management. just keep an eye on prs that change deps and make sure you do it on the target config too. simple as. i recommend this only if you bill by the hour. - autogenerate one from the other but then do the above - sidestep the language ecosystem's dependency management somehow. all the app devs will now commence ripping your guts out - sidestep the entire, like, point of buildroot or yocto and just hack some poo poo up to have the right cross flags set in some build environment and just call pipenv or yarn or whatever and then copy poo poo from the right magic dir to the right magic dir. you've just broken the smart is-a-build-necessary logic in some way or another, the library gathering logic, probably any build hermeticity you had up to this point seriously if one or another of this group would make it possible and ideally easy to add a python or js project or whatever and not have to write a recipe for every dep, while still being able to fetch all licenses and do smart rebuilds, so many nightmares would end
|
|
#
¿
Nov 1, 2021 22:53
|
|
|
right, both those things are various workaround options, right. like the node package in buildroot's workaround is the opting-out-of-buildroot thing. scanpypi is the recipe generation thing where you still have to manually sync there's just a tool to do it (yocto has something similar at least for python). poopernickel your way sounds like the best way. what i ended up doing was using a weird little python library to parse a pipenv lockfile and generate requirements and then install them with pip (didn't want to just edit SRC_URI because i also wanted some different install opts to basically install virtualenv blobs) for python, and just call yarn and say fuckit for the js thing. neither of these are terribly good though. it's kind of a fundamental problem: yocto/buildroot and yarn/npm/poetry/pypi/pip overlap in the domain of dependency management, and they just straight up don't nest. the only answer is integration on a deep level and nobody wants to deal with that poo poo particularly for js
|
|
#
¿
Nov 1, 2021 23:23
|
|
|
like IMO the actual answer here is that yocto at least, because it has a one-phase rather than two-phase model, should probably be able to let you specify a dep spec like a yarn.lock or pipfile.lock or whatever in your SRC_URI and generate (in yocto terms) RDEPENDS and DEPENDS from it. ideally it should also generate recipe content using the (de-facto standardized) install tools. maybe there's some carveouts, that's fine. this is of course a little harder with buildroot because it's two-phase and it's just kconfig so you'd have to do a loooot more work
|
|
#
¿
Nov 1, 2021 23:27
|
|
|
Sapozhnik posted:i hate to say it but use docker. or podman or w/e. qemu-build your p-lang crap into an oci container and then deploy it on fedora iot or similar. this doesn't help as much as you might think because you still have to janitor compiling things and you'll probably get a lot of duplication and updating is a pain (these devices aren't guaranteed to be network connected and need to be sneakernet updateable) CRIP EATIN BREAD posted:yeah, the problem is that yocto is designed into being very explicit about dependencies and their licenses. bringing in the free-for-all nightmare that is python and node dependencies is a nightmare. no, that's not the problem. you see, there's a lot of modern dependency management tools for plangs that in fact have the same goal, and let you decide every so often to update and when you do the update command it does dependency resolution, locks it down, and then whenever you're actually setting up an environment it downloads identical-by-hash locked versions of the deps that you specified. the problem is this overlaps with the stuff yocto and buildroot does. it could actually go in pretty seamlessly i think but nobody wants to deal with doing it (including me lol jesus christ imagining doing big internals work on either buildroot or yocto gives me insane heebie jeebies)
|
|
#
¿
Nov 3, 2021 04:25
|
|
|
also it's really funny that both peoples concept of what should be done involves having text patches to core functionality either applied or upstreamed via email. kill us all
|
|
#
¿
Nov 3, 2021 04:28
|
|
|
eschaton posted:
it's less that we overlook it and more that we're fundamentally at the whims of what happens to be supported and has drivers and so on and so forth. if soc/som vendors had good netbsd bsps and no or bad linux bsps we'd use them probably
|
|
#
¿
Nov 7, 2021 18:40
|
|
|
like honestly i think the thing that separates embedded dev isn't language or domain but a much tighter environment of constraints in what tooling you can use, what choices you get to make, what choices you can make, and also the business processes. embedded devs are a department of engineering in the same way that electrical and mechanical are. we are not in charge. we don't rule anything. we get input in what to do as much as the electrical folks get input in what to do. a lot of our poo poo is backwards and manual and some of that is our fault but not all of it
|
|
#
¿
Nov 7, 2021 19:28
|
|
|
i have a "homebrew" method which is to set up two root partitions that are mounted read only and then a leftover that's writable, and software updates are partition images of one of the root partitions and you just write to the partition blkdev after you hash/sigcheck the image. then once you're done writing, you hit a bootloader arg to flip the boot target and reboot. it's homerolled, which is bad, but it's dead simple since you're producing those disk images anyway in buildroot or yocto or whatever. it could be more efficient but you can usually keep those images small. you mount them readonly to prevent changes getting made that get wiped out during update. there's a certain amount of bind-mount fuckery for files that want to be in /etc but need to be configurable like a hostname cache or whatever but it's not that bad. fake e: the whole separated write-then-switch-bootarg thing also really minimizes the vulnerability to loss of power events bricking systems, and the a/b thing lets you recover from failed updates if you want if you can correctly identify when the update has failed, which systemd can help you do
|
|
#
¿
Nov 7, 2021 19:40
|
|
|
Poopernickel posted:I do something similar usually. But how do you package the firmware for deployment? Just download a .img.gz? Do you ever have to deal with bootloader updates too? oh, i take the image and zip up image, hashfile, hashfile sig and distribute that. on the shipping device it's a 384 meg image that zips down to about 100 which is basically the same size as the electron app lol. bootloader updates we Try Very Hard To Avoid, so far successfully aside from some random dtos that we just kind of grit our teeth and mount the requisite fat32 partition and gently caress around in
|
|
#
¿
Nov 7, 2021 19:50
|
|
|
i mean it sounds like you probably know this already but the single most important thing about an embedded device update mechanism is that it's reliable and bug free. possibly the only important thing. it can be inefficient. it can have a gross mechanism. it just has to never ever ever ever in all your testing fail in an unexpected way because that's a bricked device that you're shipping back. you know you have a good update system when you have not touched the code in it for years. you keep it as separate as you can and you keep it as dog loving simple and ideally dependency free as you can and you beat the poo poo out of it in testing and then you don't ever touch it lol
|
|
#
¿
Nov 7, 2021 19:56
|
|
|
does mender function with devices that aren't internet connected in a way that actually retains any of its advantages
|
|
#
¿
Nov 7, 2021 20:43
|
|
|
that's kinda cool, i'll look into it. edit: probably gonna prefer swupdate because it's open source but w/e
|
|
#
¿
Nov 7, 2021 20:45
|
|
|
that's nice. what about it is better than swupdate?
|
|
#
¿
Nov 7, 2021 20:51
|
|
|
uhh if it made the new version in a new recipe you’d need to set preferred version yeah. I think the place you want to do it is your distro.conf which is in a conf/ dir of one of your layers, somewhere when doing yocto poo poo it is critically important to figure out and distinguish between: - the base dir. this is what has the build/ dir. it might be called poky - layer dir. typically a lot of layers live in here - the layer thAt owns stuff like distros. this is probably called meta-yourcompany or if you’re lazy meta-yoursocorsomorsbcvendor. i hope you own one of the layers otherwise you have to do very annoying bullshit (see below) - if you own a layer, that layer probably has a conf/distro dir that includes yourcompanyname.conf. this is the “distro config” which is for a set of working software that you might want to install on one or more machines. a combination of distro and machine is an image. if you don’t have this, go to your image recipe and find out what distro you’re using, and then go make a distro conf that inherits from that - in here, you probably want to set your preferred version. if you decide that the distros version is fine in most cases but this device should really have a different one, put it in a machine conf - if none of the above works because for some reason you’re using a vendors entire tree, you can also put this in build/local.conf. please don’t. that poo poo shouldn’t exist. at most it should have some paths uh ask followups in here or pm me if you need more information I’m like harshly mediocre at this but I’ll try
|
|
#
¿
Nov 12, 2022 03:21
|
|
|
if you’re on a nice machine you can just blow away the entire sstate cache
|
|
#
¿
Nov 12, 2022 03:57
|
|
|
it interesting but it’s also true that you get to pick between 1) being able to do any of Shakespeare plays but only Shakespeares plays (buildroot) and being able to write whatever you want but only by picking letters from an Olympic sized pool of alphabet soup with tweezers (yocto)
|
|
#
¿
Nov 12, 2022 06:10
|
|
|
hobbesmaster posted:to link against or to qemu with
|
|
#
¿
Nov 13, 2022 01:09
|
|
|
i'm confused, what's the output artifact of the yocto build? a container image? half a container image? a disk image that gets slapped in a container and loopback mounted?
|
|
#
¿
Nov 13, 2022 03:26
|
|
|
I got electron running ok on a yocto build. the key is kind of bypassing yocto for the electron build part (although someone with more time and brains than me could do it). Run electron builder in your recipe for the linux-arm64 arch and it'll pick up the right prebuilds, and it's version controlled up to hash locking via language package control files like yarn.lock or whatever. The pain was getting it to play nicely with appropriate mesa stuff but you can now get it to run on wayland and so on with appropriate flags
|
|
#
¿
Mar 27, 2023 00:40
|
|
|
IMO there's in general tension between modern language setups that do their own dependency management and system builders like yocto or buildroot. i think i might have talked about it before in here. like whatever your opinions on the js ecosystem or the python ecosystem, there are language-native tools that your application developers expect to use that do things like separated "what version will i allow automated updates for and expect a minimal level of oversight to tolerate" (roughly the same as whether you'd be comfortable pulling upstream buildroot or some yocto layer) and "in a given build, use exactly these package versions by hash" (what you get in a given buildroot or yocto build). the only solution is for yocto or buildroot to ingest those tools' specification files but I think nobody wants to deal with that because it's still not _that_ common
|
|
#
¿
Mar 27, 2023 00:43
|
|
|
ye it is
|
|
#
¿
Mar 27, 2023 00:44
|
|
|
from now on you can imagine a thousand word disclaimer on each of my posts acknowledging that running electron on an embedded device will not optimize the function (user experience * 1/(bom cost)). I don’t feel like copy pasting it so simply rotate it in your mind. there are companies out there, including mine, who are making their first embedded device that has a screen, having already shipped an electron app for a while. there’s a bunch of middleware implemented in browserjs using recent ish standards; there’s a bunch of ui flows (model? I don’t know the formalism) that are implemented in react or whatever that are separable from the view that you need to change to make it work on a small touchscreen. the business is neither interested in rewriting all that - either in another language or in a flavor of browser js that something like qt webview can do - and they are not interested in then maintaining an app that is that different. it’s sort of hard to blame them. under that framework, it’s sometimes legitimately a good idea if you can afford it to get a processor that can run electron acceptably. the ideal case for this situation, and what will happen eventually almost certainly, is that someone makes a project somewhat like electron (browser engine paired with either a very limited message passing interface to a node process or a very contained api for outside-the-browser-api interactions) that doesn’t involve vendoring all of loving chrome. someday, oh someday. in the meantime, you can get this poo poo to work, and the trade off you get for all that loving disk space is that it works surprisingly well on less resources than you’d think given that chrome wasn’t made for embedded devices
|
|
#
¿
Mar 27, 2023 13:38
|
|
|
ultimately, I guess my point is: embedded development is a very big tent. it is “any programmable part that is not presented to the user as a general-purpose computer”. this often but not always intersects with cost or energy usage constraints. it always intersects with supply chain constraints both on the hardware side and the software side. sometimes your device is a dog collar with a gps antenna and a cell modem that needs to run weeks without recharging and have a $10 bom; sometimes it is a robot that in past decades would have an OEM windows pc with an fpga pcie card, has a bunch of $1000/unit actuators and runs on a bus bar and the cost of any kind of silicon falls right out. like most other kinds of non software engineering, embedded engineering is about optimizing an over constrained system in which some of the constraints are economic, and some of the constraints are social, and this does not always lead to implementations that are technically clean and minimalistic. anyway someone make me a good electron bbclass my giant monolithic do_compile is killing me
|
|
#
¿
Mar 27, 2023 13:46
|
|
|
|
| # ¿ Apr 28, 2024 12:21 |
|
|
sb hermit posted:Pretty sure that other companies just use a hodgepodge of unreadable shell scripts with inscrutable timeout values and mainly written by one person who left six years ago and future changes were made by junior engineers in the interim. yeah op i think this is your competition. especially if you're not gonna have the juice to be able to have a specific design implemented like a product. if you do have the juice for this, the right answer is definitely something like an SBC and a semicustom device tree that can force specific userspace bindings for things and then just write the software, a server/client or frontend/daemon architecture of some kind so you can manually control stuff if the ui doesn't present what you need. that's probably the right answer if you don't have the juice too you'll just have to abandon yourself to lots of config files keyed by machine type or some poo poo
|
|
#
¿
Jan 2, 2024 13:47
|
|