|
Nomnom Cookie posted:requiring retesting every time master moves is maybe fine if master doesn’t move often. if you’ve got a monorepo and a decent number of devs it’s not feasible. on the other hand, the doomsday scenario of conflicting PRs both passing CI and breaking master after merging is rare in practice. the only time I’ve ever observed it, it was my fault, actually, landing two PRs back to back We're in a mono repo situation where out full CI integration tests take about 8 or 9 hours so you only get to find out the next day if you've broken trunk. So we risk assess our branches and prompt Jenkins to run a CI on anything we think might be bad before merging. Anything else run a cut down set of tests to make sure you've not broken the REST API before merging.
|
#
?
May 31, 2020 10:10
|
|
|
|
| # ? Oct 4, 2024 13:30 |
|
|
animist posted:any bored yosposters wanna help somebody build an image metadata scrubber? that's a pretty nice and simple web app, just 1 html and 2 js files. i will definitely at least look at it later. it shouldn't be too difficult to add a service worker to this to make it into an offline web app that can be installed
|
|
#
?
May 31, 2020 10:40
|
|
|
Private Speech posted:Anyway while I think PRs are a good thing on the whole it's not always so cut-and-dry as Doom Mathematic makes it out. For say embedded development you can't really use such a tightly-integrated, daily-iterative, automated-testing approach. With some effort it's possible to do automated testing to a limited extent, but not really in the way you can with something like web stuff, which it seems like is where he's coming from (and I'll cop to looking at his post history to confirm because it felt like a very web-centric opinion when I read it). There are also various degrees of embedded, consider standing up new hardware, vs writing specific sw for existing hardware. The latter can often be automatically tested reasonably well -- at his previous job, my coworker setup an automated testing runner by plugging their target board into a dedicated machine with the CI runner. Because the embedded dev culture is terrible, he was the only one who used it in the 6 months before he quit 
|
|
#
?
May 31, 2020 13:54
|
|
|
Notorious b.s.d. posted:not one of you even bothered to read the fowler article 
|
|
#
?
May 31, 2020 13:55
|
|
|
Xarn posted:There are also various degrees of embedded, consider standing up new hardware, vs writing specific sw for existing hardware. The latter can often be automatically tested reasonably well -- at his previous job, my coworker setup an automated testing runner by plugging their target board into a dedicated machine with the CI runner. Because the embedded dev culture is terrible, he was the only one who used it in the 6 months before he quit I mean yeah that's why I said with "some effort". e: technically even that's doable in an automated fashion I guess, ultimately you always need dedicated testing setups that simulate the environment which store test data somewhere, they're just usually manual to some degree, which I guess is what you meant Private Speech fucked around with this message at 14:16 on May 31, 2020 |
|
#
?
May 31, 2020 14:03
|
|
|
and there’s many different levels of software even within an embedded context. it’s certainly possible to write automated software only tests for some parts of the code - and the effort to set that up can be great for forcing you to separate out your code into layers that talk through consistent defined apis, even for really tiny devices. you’re right though, it’s harder.
|
|
#
?
May 31, 2020 14:30
|
|
|
Doom Mathematic posted:If the master branch changes while the PR is open - for example, if a different PR is merged first - the CI pass is invalidated and has to be rerun.) If the reviewer requests changes, the PR cannot be merged. If the CI build fails, the PR cannot be merged. The temptation in this thought experiment here is to make one of the implicit promises of a PR-based workflow -- that master is never broken -- actually real. But it's not practical at anything but the smallest scale. You're giving up all horizontal scalability in your build process, and the smaller and more frequent your PRs are, the worse it becomes.
|
|
#
?
May 31, 2020 18:12
|
|
|
i love when embedded and kernel guys spend their whole career bragging about how their jobs are just plain harder than other kinds of engineering, webdev scrubs will never get it, you just have to hire a better class of programmer--until it comes to testing, then suddenly everything is impossible and not worth the effort
|
|
#
?
May 31, 2020 18:14
|
|
|
cool av posted:The temptation in this thought experiment here is to make one of the implicit promises of a PR-based workflow -- that master is never broken -- actually real. But it's not practical at anything but the smallest scale. suggesting that the possibility of mainline being broken somehow invalidates the enforced discipline of a pr-based workflow is idiotic oh no, my comprehensive test suite takes a long time to run and it means i can't land my pr for a while, guess that means prs are worthless
|
|
#
?
May 31, 2020 18:54
|
|
|
Spazmo posted:i love when embedded and kernel guys spend their whole career bragging about how their jobs are just plain harder than other kinds of engineering, webdev scrubs will never get it, you just have to hire a better class of programmer--until it comes to testing, then suddenly everything is impossible and not worth the effort harsh but true anyway I don't claim it's actually harder or anything, but web dev is pretty insane in a fair few ways too like npm or php or a million other things
|
|
#
?
May 31, 2020 19:34
|
|
|
web dev is harder cause you have to deal with other web devs
|
|
#
?
May 31, 2020 20:21
|
|
|
Bored Online posted:web dev is harder cause you have to deal with other web devs tell me about it neighboring team insist dates can just be hard coded for certain data loads because "surely it'll never change", and sure enough the dates loaded somewhere else changed and brought down everything
|
|
#
?
May 31, 2020 21:33
|
|
|
One of the major points in Fowler's article is that the CI build+test time should be kept as fast as possible, somewhere in the vicinity of ten minutes. Not everything is tested as part of that CI build, there are downstream builds which perform additional testing but do not block development.
|
|
#
?
May 31, 2020 22:34
|
|
|
where do I get one of these jobs where code gets tested before or after it’s committed?
|
|
#
?
May 31, 2020 23:56
|
|
|
Aramoro posted:We're in a mono repo situation where out full CI integration tests take about 8 or 9 hours so you only get to find out the next day if you've broken trunk. So we risk assess our branches and prompt Jenkins to run a CI on anything we think might be bad before merging. Anything else run a cut down set of tests to make sure you've not broken the REST API before merging. Woah Ever looked into parallelizing it or something like that? Why does it take so long?
|
|
#
?
Jun 1, 2020 00:07
|
|
|
Spazmo posted:i love when embedded and kernel guys spend their whole career bragging about how their jobs are just plain harder than other kinds of engineering, webdev scrubs will never get it, you just have to hire a better class of programmer--until it comes to testing, then suddenly everything is impossible and not worth the effort i will probably not unit test until forced to do so
|
|
#
?
Jun 1, 2020 00:57
|
|
|
DuckConference posted:i will probably not unit test until forced to do so literally just had a chat about unit testing during an interview for an embedded position at a multinational last week it went: inter/wvr: So did you do unit testing as well? me: Welll, it would have been possible but inter/wvr: Yeah it's not really practical, we don't do it either me: well, it's not easy in embedded inter/wvr: It could be done, but and yes I did get an offer from them
|
|
#
?
Jun 1, 2020 01:35
|
|
|
and the actual hardware peeps have to spend millions a turnaround so they have thousands of test machines and formal methods and poo poo
|
|
#
?
Jun 1, 2020 01:37
|
|
|
e: nm, something something PCBs cheap something, was a bit too specific
Private Speech fucked around with this message at 02:05 on Jun 1, 2020 |
|
#
?
Jun 1, 2020 01:39
|
|
|
yeah PCBs have lots of design tools to help but once poo poo gets real with emissions compliance/aggressors it quickly devolves into "bob soldered on some random caps and bits of copper tape and we improved by about 2.8dB"
|
|
#
?
Jun 1, 2020 01:54
|
|
|
DuckConference posted:yeah PCBs have lots of design tools to help but once poo poo gets real with emissions compliance/aggressors it quickly devolves into "bob soldered on some random caps and bits of copper tape and we improved by about 2.8dB" okay now I feel kinda bad for editing it out but I said something along those lines, plus mentioned design verification tools for ASICs and FPGAs which are kinda unit testing
|
|
#
?
Jun 1, 2020 01:57
|
|
|
man, ive wanted to transition into embedded dev so bad, but i just have no idea how to go from java/C# to "guys i really do know rust/c++ i promise" 
|
|
#
?
Jun 1, 2020 02:01
|
|
|
I wanna go back do embdev. Kinda wish I'd done an EE degree instead of CS/Math as well but hey. Pay is pretty bad compared to doing webshit though.
|
|
#
?
Jun 1, 2020 02:27
|
|
|
DuckConference posted:i will probably not unit test until forced to do so I do think automated, unit, etc. testing is legitimately harder in embedded or kernel programming, both because of the unusual nature of the programming environment but also because way more often than not, you're talking about bringing modern testing practices to vast codebases with decades of history. it's way loving harder to bring 10 existing mloc of bitbanging bullshit up to a modern standard of test coverage than it is to start from scratch and do good testing from day 1, just because the code almost certainly wasn't written to allow testing and there may be no way to run it outside of the real hardware environment it was intended for. so the sad fact is that even if--to pick an example--apple decided they wanted to hit a nice 80% unit test coverage on xnu, it would be impossible for them to do so unless they straight up stopped any feature or enhancement work for months or maybe even a year while the kernel teams refactor the whole world so they can rig up reasonable test harnesses to e.g. test mach_vm behavior in a lightweight userspace program that simulates the physical memory being managed, with all the debuggability and observability you would get in such a context, instead of needing to run a real kernel in a VM and observing second or third order effects of the behavior you care about. and beyond the cost of not delivering new shinies for six months, you're also facing a mountain of risk of regression in areas that your test suite doesn't cover so yeah, they'd have to be forced to do it, but of course the people who actually have that power (SVPs of this-and-that) will _never_ do so because they can't afford to stop adding new features or hardware configs or whatever inane poo poo is going to move more phones next year, and apple will continue to do boneheaded poo poo like accidentally reintroduce kernel bugs they'd previously fixed, allowing their latest and greatest OS to be jailbroken on release day that was almost an effortpost. i might be angry and anxious about other stuff and projecting onto the software industry
|
|
#
?
Jun 1, 2020 04:35
|
|
|
A lot of the most effective embedded testing I've seen involved a combination of unit testing and hardware testing. For the software, mocking system APIs takes some love and crosscompiling, but doing this in C is simultaneously sort of cool because you just preload libraries which slip in front of the real ones due to the trivial namespacing and lovely because there's not a lot of great infrastructure available. The hardware needs something that programmatically generates real signals and reads back real output. The company I work for engineered something from scratch which does this for canbus and it owns
|
|
#
?
Jun 1, 2020 11:37
|
|
|
tak posted:Woah Various reasons really. We have a lot of integration tests for various things which are hard to guarantee the execution if you run them in parallel, lots of Selenium tests which just take ages to run. The whole suite also runs on Oracle and SQL Server and I don't know why they run sequentially but they do, I think it's because our release manager is bad.
|
|
#
?
Jun 1, 2020 13:14
|
|
|
Beamed posted:man, ive wanted to transition into embedded dev so bad, but i just have no idea how to go from java/C# to "guys i really do know rust/c++ i promise" embedded dev can be really rewarding because the stuff you do makes physical changes in the world, which does rule. if you want to do embedded though, the first thing to do is decide what you mean by that. embedded development is a lot more like lob app development: there is no such thing as "an embedded development company", there's hardware companies that make products that require code, and products can have different needs. embedded dev is always the tail, never the dog. you'll also need to decide what you mean by "working on embedded stuff", and this is where you'll find useful entrypoints to the space. if you define "embedded dev" as "stuff that ships in the box", then you can be writing c or assembly but you could also be writing python or javascript or c++ (you're not gonna be writing rust tho lol). starting somewhere that makes a product that ships with linux running some process written in js/python/cpp is a good way to get your foot in the door and maybe try and go over to i guess whatever microcontroller tasks people want if you really really want to do microcontroller stuff, you don't necessarily need an EE degree but you _do_ need to be able to speak the ee language. this is because, again, embedded dev is always the tail and never the dog. you're programming a microcontroller that exists for a purpose that's integrated into the system, so you need to understand and be able to discuss the system as a whole on at least a basic level. on the theoretical level, you'll need to know about passive networks (ohm's law; basic passive filter design); at least some frequency domain analysis with enough information theory to figure out how to configure an adc properly; and enough semiconductor theory to understand how things like gpios and switches work. some of this you can learn on the job but like i said it's really about speaking the language and it'll be somewhat frustrating for all involved at the beginning if you don't (trust me on this, i've been involved in training people on basic embedded tasks coming from higher level backgrounds). so i guess my advice is: find somewhere that makes something kind of large and complex, because it'll probably run linux and use some hll that you can use to get your foot in the door; in the meantime do moocs or something about electrical design; and once you're there you can maybe move over to the level of your choice. or maybe you'll figure out the secret truth, which is that it's satisfying but it's also incredibly loving annoying sometimes both in detail (what the gently caress is this api, why am i hitting compiler and library bugs) and in process (i really should have pressed harder for a nicer micro but some ee had a bug up their rear end about doing something with a support ic instead of code and now we're hosed and everybody wants me to fix it but i'm not sure if it's physically possible). anyway there's no point learning rust lol
|
|
#
?
Jun 1, 2020 13:19
|
|
|
For learning microcontrollers I'd recommend getting an STM/TI dev kit (you'll probably need an swd/jtag programmer to use it, STM makes okay ones for cheap). It's effectively very similar to an Arduino but it's far closer in terms of API and bare metal. Also EEs don't always like to hear about arduinos, some do but it's not always considered professional. For the higher level stuff I'd go with something like an RPi (maybe a chinese clone because see above about arduino) and try to interface the dev kit with it, then get a sensor and electronics kit from amazon. All told it shouldn't be very expensive, plus you'll get to see what it's like. I'm not saying that will be enough to get you a job, but it would probably help in getting a foot in the door. e: Some links (UK but same stuff is available from US suppliers): stm32f4 dev kit, they also sell a variety of addon boards for LCDs, cameras, etc, has an inbuilt SWD debugger, about 80USD RPi clone with some extra features, 70 USD 37 sensor kit from amazon for 50 USD and some electronics starter kit (this is a bit harder to pick, but honestly doesn't matter that much). Having an oscilloscope and a soldering iron helps but isn't essential. Obviously if you already have some or all of that then all the better, and probably don't need any of this. Also the advice about getting a job writing hll on a linux embedded platform (so probably c++ or maybe C# .net core or something) is effectively how I got started and it's probably the best way. Especially helpful if it's a regulated industry, but they tend to be more closed up. e2: Also the discovery kit is a very arduino-like, have a look at the STM webpage for eval boards if you want something with fewer pins taken up by inbuilt peripherals. Private Speech fucked around with this message at 15:41 on Jun 1, 2020 |
|
#
?
Jun 1, 2020 13:46
|
|
|
Private Speech posted:Also EEs don't always like to hear about arduinos, some do but it's not always considered professional. is this a pride thing or does learning this stuff on Arduinos teach bad habits
|
|
#
?
Jun 1, 2020 15:01
|
|
|
Ciaphas posted:is this a pride thing or does learning this stuff on Arduinos teach bad habits mostly pride I think, but arduino has an interface/OS layer which both severely limits what you can do and isolates you a bit from the actual chip I think you can use it without that, but it's a consideration e: also there's such a variety of eval boards out there, looking at the farnell/STM listings is like being in a sweets shop or something, there's a real I want this feeling to it like this is pretty cool, maybe add a Xilinx FPGA to it in practice you're usually fairly limited by cost and features in terms of what chip you get to use on an actual device Private Speech fucked around with this message at 16:03 on Jun 1, 2020 |
|
#
?
Jun 1, 2020 15:06
|
|
|
neat to know. by coincidence I'm in that foot-in-the-door position Phobeste describes; I do software running on a custom Linux box (and sometimes a WinCE box, god help me), working directly with/adjacent to the hardware folks wiring CANbus controllers, EEPROM boards and SCSI terminators or whatever other stuff. Gotta figure it's in my best interest to learn more of it while I'm working here (adn ti's interesting, besides)
|
|
#
?
Jun 1, 2020 16:15
|
|
|
there are problems with arduinos from both the embedded-use-case perspective and interestingly from the professional software developer perspective, as well as from the perspective of usability and "intro to embedded development": from the embedded use case perspective: - arduinos are not cost competitive, at all - especially the "classic" arduinos are old, bad chips running on an essentially deprecated architecture (avr) that don't have the upsides that are otherwise the reasons people use old chips - cost, availability, size, capability relative to the above - arduinos are hard to integrate in systems because they come on boards, so either you have to buy that board and integrate it into your device (which is a) a waste in most cases and b) really looks unprofessional) or you have to develop your own board support for it, in which case why are you using arduino - it is hard to square with a normal supply chain - (again, especially "classic") arduinos tend to have some capabilities obfuscated or unavailable in odd ways, and their api presents equivalents to those capabilities that may not actually exist, like "analogWrite" support on pins without dacs where they recommend you bodge your own dac in by slapping a capacitor inline. this is Insane in a professional environment from the software perspective, the custom arduino toolchain is a nightmare. it's hard to fold into automated tooling; it's hard to impossible to have separate build profiles for tests. the way it manages its board support packages is awful. the custom editor is awful. the way it implements "libraries" is terrible. if you spend the effort to figure out how to tell it to build headless it still pops a splash page. it is every negative stereotype of a java based ui program. from a usability perspective, the api that it presents in software and the ui that it presents to the user are just not quite what is needed. the apis are too high level to be generalizable to other parts of embedded development. it tries to walk this line between c++ and c that doesn't really exist and requires odd circumlocutions like having its own almost-but-not-quite-like-c++ string type and having its own custom preprocessor. it completely hides the main() function for some reason and requires you to have a separate setup() (which runs once) and loop() (which loops) function, which is completely mystifying and always has been, and prevents you from using local variables at program scope because they're separate function invocations! any variables you want to persist between loop() invocations have to be static! i need to stop typing because i'm getting mad so, it doesn't prevent new capabilities; it doesn't elide complexity in a way that's actually conducive to learning more about embedded dev long term; and it presents several specific actively-harmful concepts. there's one use case where arduino is useful and where it gets used constantly and why it shows up in professional contexts: somebody who wasn't an embedded dev made a startup and used an arduino in that product because it could get something more or less working quickly, and it's in there forever after that, and it's awful. please don't use arduinos
|
|
#
?
Jun 1, 2020 16:58
|
|
|
"no that's wrong, you should be using this other service. every dev working this project knows about it. just use it" of course no one i talk to knows about the service and duder won't respond
|
|
#
?
Jun 1, 2020 17:03
|
|
|
a lot of the above actually applies to raspberry pis, too (certainly the 3/b/+, which i'm most familiar with). they have the same (again, from a professional perspective) problems with coming with their own board which you then have to integrate - and i suspect you can't really make your own because broadcom gives the raspberry pi foundation a lot of support that you might not get, and it's a really old chip i think they don't even want to deal with anymore, though this might not be true of the rpi 4 - and the price is not really right. there's not the same concern with apis and usability because it's a more general-use-case platform, but you do have some other problems, like - for some reason, the raspberry pi foundation maintains their own kernel tree that's often but not always upstreamed and/or updated, even though the mainline kernel has extremely good arm support - they recommend this debian based image that is again probably fine for "oh god i just want to get this working" but really not fine for an actual shipping product because it's hard to atomically update - they tend to recommend it as essentially a gpio platform that talks over networks, and it works ok for that, but when you start getting past that stage there's not much to recommend it (and it's probably overkill for that task anyway). - the actual architecture of the supporting board for the raspberry pi is really weird because of the chip they used. like, the ethernet port is internally on a usb bus for some reason. i've also personally had a _lot_ of issues with the rpi usb subsystem - some of their support packages are... suspect. like the recommended gpio support lib twiddles memory mapped io bits from userspace by mapping /dev/mem, which will work but like... why. put the effort in to get that subsystem fully featured in the kernel - it's slow as poo poo - sd card you plug in lol now, i'm more accepting of using an rpi in professional contexts because spinning your own motherboard for a soc like that is a lot harder than spinning one for a micro, and there's a lot more complexity in bringup for some of the same reasons. especially for use cases like "i need something to talk on a couple network interfaces, drive a display, and push data over some bus to a micro that does the timing-sensitive work" there's not really anything inherently bad about an rpi like there is with an arduino. but again you're going to be shipping a professional product with an rpi just screwed in there, which is kinda gross, and your supply chain might end up pretty weird, and you've still gotta spin some software environment like a buildroot setup because you really don't want debian, and the capabilities of the system might get in your way, and and and and and... basically if your company has the expertise to spin your own soc (or if you are ok with getting a whole som which is a LOT easier) you probably shouldn't be using an rpi, and if you're going to be making embedded linux products you probably should be looking into getting that capability if you don't already have it.
|
|
#
?
Jun 1, 2020 17:13
|
|
|
Phobeste gave a far more detailed and eloquent answer, I haven't even used Arduino personally outside of helping someone with their code. Arduino reminds me a bit of PHP, it's not a terrible programming analogy there are similar views around raspberry pi but I would say it's not quite as bad from a user perspective both dev boards and premade linux boards, like the ones I linked above, are very similar to arduino and rpi though, and I wouldn't be scared of them at all; in terms of ease of use the difference is far less than the difference between, say, PHP and C#, and they don't have many of the downsides e: beaten on the rpi, this was before I read the previous post Private Speech fucked around with this message at 17:24 on Jun 1, 2020 |
|
#
?
Jun 1, 2020 17:19
|
|
|
how do real embedded devs pick their platform on a greenfield project? i was an embedded dev for a while (at a baby company that didn't have any best practices) and i just developed on whatever they told me to develop on, and the boss or lead EEng seemed to choose pretty arbitrarily is there an quick way to choose something cost effective and easy* to develop on or is it just a matter of trying lots of samples until a workflow feels okay? * by easy I mostly mean predictable/documented. I did a little bit of dev on dpspics and the datasheets and examples were often just plain wrong for the chip i was working with
|
|
#
?
Jun 1, 2020 17:24
|
|
|
Easy: i.MX6/7/8 processors if you need Linux TI if you are automotive and need an RTOS. And even then, i.MX processors.
|
|
#
?
Jun 1, 2020 17:26
|
|
|
If you're doing super low volume stuff then tbf Raspberry Pi stuff is very cost competitive. At the previous trash fire where I worked there was some insane setup of a box containing a custom microcontroller PCB and a mounted RPi connected by internal cabling. SD card held /var, OS image was built using openembedded and lived entirely in initramfs, OTA updates downloaded a zip with an updated kernel and initramfs, then committed the update by updating a symlink; U-Boot was chain-loaded by the firmware to load the currently selected kernel and initramfs from ext4 /var. I spent a long time agitating for at least a RPi Compute Module 3 based solution and I finally managed to secure myself some involvement in the hardware design for that process. In the sense that I provided the hardware guy with a list of requirements for the PCB he designed. The end result was ... honestly pretty nice I think, we had a single board with no high-speed buses (well, no awkward high-speed buses) that a compute module would slot into, application microcontroller on a UART, RTC on I2C, the consumer RPi's SMSC NIC chip on the board's USB wires, and the whole thing took power from PoE. We made three prototypes of this new board and then the company folded lol.
|
|
#
?
Jun 1, 2020 17:27
|
|
|
Corla Plankun posted:how do real embedded devs pick their platform on a greenfield project? I like using STM or TI stuff and it's common in the industry, but even then you sometimes get a bit of, well, I don't want to say elitism but it's not always the right choice. As for how I went about choosing the few times I did - you pick the platform (so, say, STM, good to keep the same one for tooling), look at what you need in terms of IO (both STM and TI are very good at maintaining datasheets if nothing else), consider what you need in terms of speed, voltage, draw and ram, with the possibilty of any future scope expansion, then pick the cheapest option that meets your needs. Sometimes you pick something a little more powerful if it allows you to reuse existing code and design, but that's only really valid for small-scale production. That's about it. Also I guess STM might be a bit more common in Europe. You still get TI stuff but that's just a guess. I see it more than TI here. e: I thought about mentioning i.mx/NXP but, humm. It's the high-performance choice. As in GHz vs. 10s to low-100s MHz. Also FPGAs are Xilinx or Altera or I think the place I'll be working at in a few months uses Intel for some reason, but I have no idea WTF is that about, probably another performance or price thing. e2: sometimes availability and how long the chip is going to be in production is also a consideration Private Speech fucked around with this message at 17:55 on Jun 1, 2020 |
|
#
?
Jun 1, 2020 17:29
|
|
|
|
| # ? Oct 4, 2024 13:30 |
|
|
https://www.chromium.org/Home/chromium-security/memory-safetyquote:Memory safety quote:What we’re trying
|
|
#
?
Jun 1, 2020 17:33
|
|