|
gnatalie posted:ahhhh we give the auditors read access to everything in octopus deploy and let them have fun. I guarantee you they do not know what that is or what a deploy tool is either. if they did they could have said "ok deploy the latest prod artefact build to the test env and also refresh the database back from production" which is actually dead simple to do but I'm not gonna do it unless asked and we have other stuff in that env anyway last week they took screenshots of some code in gitlab as evidence of ??? and it was literally a wait loop that is used to just go "nothing to do, sleeping" also these *are* the sox auditors lmao
|
#
?
Mar 17, 2023 09:53
|
|
|
|
| # ? Apr 19, 2024 12:30 |
|
|
|
|
#
?
Mar 17, 2023 10:01
|
|
|
the middle should just say "satan"
|
|
#
?
Mar 17, 2023 10:12
|
|
|
The only workable auditing is “prove you are following the procedures you say you are.”
|
|
#
?
Mar 17, 2023 14:58
|
|
|
makes me lose my dang mind when an engineering team writes its own processes and then doesn’t follow them. just write what you do!! who are you trying to impress with all of this poo poo you write and not one person has ever done!
|
|
#
?
Mar 17, 2023 15:08
|
|
|
on today's misadventures: 1. We have a Github PR label called CI_SKIP that prevents automatic builds, intended for use in draft PRs or documentation updates. 2. Somebody wants to add a new label that they can use in draft PRs for continue-on-error behavior when running tests. All good so far 3. What name did they choose? CI_NO_SKIP. 4. When asked to change it to IGNORE_ERRORS, their response was "no, we called it CI_NO_SKIP in our old project from two years ago" gently caress I hate this job
|
|
#
?
Mar 17, 2023 16:30
|
|
|
Poopernickel posted:on today's misadventures: Time to start applying for new gigs!
|
|
#
?
Mar 17, 2023 16:39
|
|
|
should be called CI_ON_SKIP, i would argue
|
|
#
?
Mar 17, 2023 16:40
|
|
|
Bloody posted:makes me lose my dang mind when an engineering team writes its own processes and then doesn’t follow them. just write what you do!! who are you trying to impress with all of this poo poo you write and not one person has ever done! customers and or regulators, who often ask for copies of this poo poo
|
|
#
?
Mar 17, 2023 17:10
|
|
|
Bloody posted:makes me lose my dang mind when an engineering team writes its own processes and then doesn’t follow them. just write what you do!! who are you trying to impress with all of this poo poo you write and not one person has ever done! I had to read and then say I'd read like 15 different policies about poo poo like "how changes are deployed to production" and I don't think a single one of them was followed except maybe the one saying "this is the policy that defines how we define the policies" anyway I didn't read any of them and just kept clicking ok until I was done, gently caress all that
|
|
#
?
Mar 17, 2023 17:20
|
|
|
the only thing I’ve seen be effective is when every outage has to be tied back to a causing change, and if it turns out there was a change but it wasn’t recorded, heads roll.
|
|
#
?
Mar 17, 2023 17:49
|
|
|
Poopernickel posted:on today's misadventures: ok so CI_SKIP means skip CI builds CI_NO_SKIP means dont skip CI builds not sure i see the problem. unless i misread something, continue-on-error is ~orthogonal~ to that
|
|
#
?
Mar 17, 2023 18:01
|
|
|
Carthag Tuek posted:ok so Developer in question decided that CI_NO_SKIP is unrelated to whether CI runs or doesn't run, and should instead select continue-on-error behavior. His reasoning: "that's the label we're used to from our previous project". Reading his pipeline changes in the PR, that is indeed the behavior that he wants. Poopernickel fucked around with this message at 18:21 on Mar 17, 2023 |
|
#
?
Mar 17, 2023 18:16
|
|
|
so CI_SKIP skips all tests, and CI_NO_SKIP says don't skip any tests, even on test failure, easy
|
|
#
?
Mar 17, 2023 18:56
|
|
|
tef posted:so CI_SKIP skips all tests, and CI_NO_SKIP says don't skip any tests, even on test failure, easy A default run is a 90-minute C++ build followed by another 2 hours of automated test. a CI_SKIP PR skips all CI actions, period. Compile, test, publish, everything. A PR with CI_SKIP on it is ignored by every automated thing we have. a CI_NO_SKIP PR acts as if no label was set, except that all failures are treated as continue-on-error. thanks, $employer 
Poopernickel fucked around with this message at 19:11 on Mar 17, 2023 |
|
#
?
Mar 17, 2023 19:07
|
|
|
https://github.com/ijl/orjson huh
|
|
#
?
Mar 17, 2023 19:08
|
|
|
and a CI_10_SKIP coupon means you can fail up to 10% of your tests and we’ll still deploy it
|
|
#
?
Mar 17, 2023 19:09
|
|
|
Poopernickel posted:A default run is a 90-minute C++ build followed by another 2 hours of automated test. welp thats really dumb lol
|
|
#
?
Mar 17, 2023 19:09
|
|
|
people fuckin love making fast json parsers wheres that one that uses the GPU to parallelize it
|
|
#
?
Mar 17, 2023 19:10
|
|
|
boy howdy, I sure do love introducing Rust as a dependency because I'm too cool for the JSON parser that's literally built into the language I'm using - Github user ijil
|
|
#
?
Mar 17, 2023 19:12
|
|
|
orjson and the other fast jsons are good because python is the slowest language anyone would use on purpose the other popular one is ujson/ultrajson
|
|
#
?
Mar 17, 2023 19:14
|
|
|
lifg posted:and a CI_10_SKIP coupon means you can fail up to 10% of your tests and we’ll still deploy it 
|
|
#
?
Mar 17, 2023 19:15
|
|
|
cpython is getting faster as of late supposedly. if that mypy type checker is sufficiently expressive and well-supported then python might be worth another look these days.
|
|
#
?
Mar 17, 2023 19:17
|
|
|
if your application cares about how fast it's parsing JSON, then Python probably isn't the language you should be using.
|
|
#
?
Mar 17, 2023 19:18
|
|
|
mypy is pretty good but pylance is about the same good and is easier to use imo. microsoft makes good developer tooling newer versions of python also have protocols which puts the type system in "about as good as typescript, which is much better than golang" tier as far as modern dynamic languages go Poopernickel posted:if your application cares about how fast it's parsing JSON, then Python probably isn't the language you should be using. as other people have noted cpython is also slowly exiting the "we made the language slow on purpose as a joke" zip code. if you look up all the non-python python libraries and combine them in your project, or use a framework that does it for you, you can run applications at a significant fraction of golang speed. still slower than javascript. but you get dynamic/strong/gradual instead of whatever javascript has
|
|
#
?
Mar 17, 2023 19:31
|
|
|
12 rats tied together posted:orjson and the other fast jsons are good because python is the slowest language anyone would use on purpose yeah, we use ujson by default, but i want to put this through paces now, as it has some nice extras in addition to more performance Poopernickel posted:if your application cares about how fast it's parsing JSON, then Python probably isn't the language you should be using. in the end of the day, customer facing networked poo poo is customer-facing networked poo poo. it's undeniably true that any non-comedy example will have network/io wait time take the vast majority of the profile trace, but cpu time isn't free either. especially when your server is literally dying because it has to do bunch of expensive ml poo poo
|
|
#
?
Mar 17, 2023 19:31
|
|
|
especially since iirc even asyncio releases the gil on network io. of course you should profile your poo poo if it actually matters but it is my experience that limiting the amount of time the cpu is chewing on python bytecode is pretty high leverage when it comes to overall perf it's undoubtedly true that the majority of python web apps are just waiting on stripe though and it doesn't matter for them
|
|
#
?
Mar 17, 2023 19:42
|
|
|
yeah just profile the garbage app and its usually disk or io. maybe a bad loop. its generally not the parser unless youre that GTA port that got a 90% speedburst after some nerd disassembled it and discovered their json parser weas absolute garbagecinci zoo sniper posted:in the end of the day, customer facing networked poo poo is customer-facing networked poo poo. it's undeniably true that any non-comedy example will have network/io wait time take the vast majority of the profile trace, but cpu time isn't free either. especially when your server is literally dying because it has to do bunch of expensive ml poo poo wait how does json parsing impact machine learning at all like i can think up some pathological example that parses json in a loop on purpose but its not really ml-related
|
|
#
?
Mar 17, 2023 19:50
|
|
|
Carthag Tuek posted:wait how does json parsing impact machine learning at all Perhaps someone made the decision to put ingestion/etl on the same box as the ML code?
|
|
#
?
Mar 17, 2023 19:52
|
|
|
e: it was 70% but still lmao https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ Carthag Tuek fucked around with this message at 19:55 on Mar 17, 2023 |
|
#
?
Mar 17, 2023 19:52
|
|
|
MrQueasy posted:Perhaps someone made the decision to put ingestion/etl on the same box as the ML code? i guess? but that still implies a looot of json parsing that is likely redundant
|
|
#
?
Mar 17, 2023 19:54
|
|
|
MrQueasy posted:Perhaps someone made the decision to put ingestion/etl on the same box as the ML code? Carthag Tuek posted:i guess? but that still implies a looot of json parsing that is likely redundant hot json at high velocity.avi only 2 json calls per task - deserialize json input and serialize the eventual output to json. just a lot of tasks with big jsons on both ends, on the order of magnitude of megabytes per second per instance of throughput with local performance target below 20 ms per job, ideally below 10 ms per job (and ml poo poo taking several ms per call)
|
|
#
?
Mar 17, 2023 20:04
|
|
|
okay yeah thats getting pretty tight if json is huge
|
|
#
?
Mar 17, 2023 20:08
|
|
|
Hugh Jason
|
|
#
?
Mar 17, 2023 20:09
|
|
|
Carthag Tuek posted:Hugh Jason killed by autocorrect
|
|
#
?
Mar 17, 2023 20:10
|
|
|
cinci zoo sniper posted:killed by autocorrect killed by  i wrote that intentionally to pun on my previous post
|
|
#
?
Mar 17, 2023 20:19
|
|
|
webshit that parses a few mb of json, does approximately zero work, then serializes a few mb of json is not exactly rare. the ideal solution is generally to just stop doing that, but that involves rewriting a bunch of code in a bunch of lovely microservices, while rolling out a faster json parser can make all of those microservices faster at once.
|
|
#
?
Mar 17, 2023 20:32
|
|
|
I just removed a bunch of dead code two days ago and a few tickets trickled in, so I deployed the previous tag and am now applying each commit one-by-one to see where it broke. I found the commit. It still looks like dead code. Guess I'll go function-by-function!
|
|
#
?
Mar 17, 2023 20:41
|
|
|
CPColin posted:I just removed a bunch of dead code two days ago and a few tickets trickled in, so I deployed the previous tag and am now applying each commit one-by-one to see where it broke.  i had something similar today, which is taking a project that has had 2 unfinished refactors applied to it, and doing a third refactor on top that. random undocumented variables etc, very great just commenting like 3 functions out of class because they're not used anywhere and then discovering empirically if it was or wasn't important
|
|
#
?
Mar 17, 2023 20:44
|
|
|
|
| # ? Apr 19, 2024 12:30 |
|
|
Plorkyeran posted:webshit that parses a few mb of json, does approximately zero work, then serializes a few mb of json is not exactly rare. the ideal solution is generally to just stop doing that, but that involves rewriting a bunch of code in a bunch of lovely microservices, while rolling out a faster json parser can make all of those microservices faster at once. stripped out protobufs from python code and made them go faster and smaller with gzipped json (very string heavy data)
|
|
#
?
Mar 17, 2023 20:46
|
|