MononcQc posted:

my fav bit of the kafka architecture is whenever someone notices that oh yeah topic compaction is not enough to guarantee reliable long term storage (i.e. re-partitioning fucks with all the keys and therefore linear history of entries) so you need another canonical data source to act as a kind of backup, and so what you do is put a consumer that materializes the views in a DB.

But that's nice because you can use the DB for some direct querying. Except for some stateful component doing stream analysis over historical data; every time that component restarts, you need to sync the whole state to build the thing afresh, but doing this from a DB is not super simple so you do it from Kafka, but since Kafka can't necessarily tell you it has all the data and the DB is the one that's canonically right, you end up building ad-hoc diffs between a DB and a live stream for every restart

And there's like no good solution, you just cover your ears and hope you never make it to that point because you know you'll be hosed janitoring and reconciliating two data sources that don't necessarily have a good way to talk to each other aside from some small component/microservice written in a language only 1 person knew and they left 3 months ago

Maximum Leader posted:

i have a basic parent/child relationship in a database where i need to perform a basic join, problem is that the database is turbofucked by horrible decisions. the parent has fields parent/child relationship 1 through 12 containing a reference to the child instead of just putting the reference to the parent on the child. im at a loss with this, any advice guys?

floatman posted:

Tell them that they should get rid of all staging servers, push everything to production because what profit do you get from testing???

gonadic io posted:

In $oldjob I led the charge to sanitise data and overwrite the staging db with it (preferably regularly) because otherwise staging was basically unusable for e2e testing since the data there either failed constraints or took like 30 mins to correctly set up a realistic user across a half dozen different systems are their dbs

Ciaphas posted:

i'm only just starting to get caught up on this whole container fad, so I wanna check some clarification here

am I right in thinking that kubernetes is, roughly, a replacement for the part of docker that's about creating and managing containers (but still uses docker's containers, just not it's... er, factories, i suppose)?

Powerful Two-Hander posted:

*this is my favourite: performed every single logical test by returning a magic string from the db and checking its value, not even just "true" or whatever but poo poo like if(caniDoIt = "yes you can go ahead")

ratbert90 posted:

ratbert90 posted:

Jesus.

It's worse than that. They are using the branches as tags.
But also also, there is no develop or staging branch.

Edit*

I wrote a script that I periodically run when I think we have to many branches.
It merges with the develop branch, and then diffs the branch with develop. If there aren't any differences the branch is marked for deletion in an autogenerated script.

Branches with merge conflicts and differences are dumped to a seperate text file for review. It's a pretty handy script!

edit edit*

The Indian team's project is so convoluted that I had to log into a LIVE PRODUCTION server and dump the SQL database to figure out what version was live because they don't print the version name on the website and there's no information as to what branch is currently live.

bob dobbs is dead posted:

"sfdc apex code" is the point when your life went terribly wrong, the contractor doesnt have poo poo to do with it

ThePeavstenator posted:

One of my coworkers merged a branch of his into master for one of our web services today with about a month's worth of commits. It built but didn't work once deployed. Instead of swapping the slots back or just deploying the previous working build, he tried to undo the merge. He did this by using rebase on his commits instead of revert. He did this multiple times. He did this in master and pushed directly to master.

Our git history now looks like a piece of modern art, with things like commits dated 9/17/18, 4 Hours Ago, and 9/18/18 in that order all over the drat place.

Oh and the web service is still sitting there deployed in a broken state.

ThePeavstenator posted:

We do that, but the team I'm on manages our system and devops infrastructure so he's one of the 3 people other than me with the elevated privileges to do what he did.

DaTroof posted:

i don't wanna call anyone out because maybe what i inferred is not what they meant, but in a very general sense, some people seem to think type safety precludes testing, and HOLY poo poo are they wrong

floatman posted:

I'm doing forced refactoring now I.e. I don't want to do it but I have to because there's a guy writing code.
Imagine a class that's a library to perform http calls to a API gateway.
Okay now imagine this library is having lots of problems i.e. calls dying mysteriously and not having correct responses returned. According to the guy writing the library, it's due to the "complexities of caching".
So what caching is he taking about? Let's say I'm going to call the library to get foo: $service->getFoo($baz). Internally, it does this:

code:

public function getFoo($baz)
{
    $response = httpClient->post("foo", $baz);
    $this->cachedResponses[__FUNCTION__] = $response;
    return (object) ["success" => true, "response" => $response]
}

So apparently the "caching" is to store the last returned response in an array in the class itself. Note that the response is cached via function name as a key, but no poo poo is given to function arguments. This means I can get 2 different responses from calling getFoo("A") and getFoo("B") but it'll be all cached in the same slot regardless with the latter call trashing the earlier call.
So how is this "cache" used?

PHP code:

$result = $service->getFoo($baz);
if ($result->success != true) {
    $response = $service->getCachedResponse("getFoo");
    ...

What the gently caress.
Bonus: the $result variable already contains the loving $response object that you need so what the gently caress?
At that point I died.

Fatty Crabcakes posted:

People complained about typos in my commit messages but then stopped when I started doing commit messages almost entirely in emoji

DaTroof posted:

i used git for years before squash or rebase seemed like poo poo i might ever need. merge was plenty adequate

even now, i occasionally run into situations where merge is preferable because we don't want to lose the branch's history

floatman posted:

I just got asked by someone on if it was possible to access the underlying file stream descriptor of a monolog logger (PHP).

Apparently this is necessary because the "design" of the application was to:
1) create a monolog application logger with a temporary file description stream as the logging target.
2) pass this logger object down every object that's being instantiated for the lifetime of this PHP request
3) log html output to this logger object
4) once the task is completed, access and rewind the temporary file description then
5) echo the html it contains out back to a web page

It has filled me with a murderous rage.
Bonus: person claims this is a better design that previous legacy code.

gonadic io posted:

check ur coverage

(no really, it points out when you miss branches)

floatman posted:

The company has a "pull request" policy where in theory, code changes are reviewed by "development peers" and approved before being commited to the codebase. But what I'm starting to feel is that these PR reviews are ultimately very shallow and just end up being white space and punctuation changes, or at most superficial changes to one to three lines at most.
Either that process is flawed and fails to reveal these kind of "HURRR DURRR let's statefully hold generated HTML content in a logging framework" constructs or apparently I'm the first dev to have taken an issue to this.
I dunno man are PRs useful or are they only useful when the general quality of Devs are on average higher than circus clown

Arcsech posted:

unless it’s really ridiculous, just leave it for the pr and when you merge do git merge --squash

GitHub and gitlab have an option to make the merge button in the UI do this by default

DaTroof posted:

and we can't wait to get those apps started after we're done gluing together a usable framework from 300 disparate node dependencies

AggressivelyStupid posted:

I love having a 350mb node_modules folder

Private Speech posted:

what uhh, what does it .... do

at least in general terms

Boiled Water posted:

today in “the state is terrible at programming”: we have a db in which each field can be at the most 4kb (which is fine) but it also has a field pointing to an xml file which when empty is 2gb. yes this file is always present for each row even when empty

the talent deficit posted:

redshift is really good at joins and aggregates and absolutely nothing else

MononcQc posted:

npm certainly allows it and has done so for a long long time if not forever. For example, npm3 even advertised examples where version clashes were the thing: http://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html

The tool didn't even get actual lock files until version 5, and it has taken tools like yarn to actually prevent the flat installing of duplicated deps because that one tries harder. But even today, it doesn't work naturally. If you look at the current yarn lock file for React, you can see some libraries are duplicated in the locks:

https://github.com/facebook/react/blob/master/yarn.lock#L3113-L3121 shows 'bytes' at both 3.0.0 and 3.1.0
https://github.com/facebook/react/blob/master/yarn.lock#L3179-L3187 shows 'callsites' at both 0.2.0 and 3.0.0
https://github.com/facebook/react/blob/master/yarn.lock#L3197-L3210 shows camelcase with 3 different locked versions

But it gets worse:

pre:

$ cat yarn.lock | awk -F@ '/^[^ ]/ {print $1}' | sort | uniq -c | sort | tail -n5
      4 source-map-support
      4 strip-ansi
      5 async
      5 yargs
      6 kind-of
$ cat yarn.lock | awk -F@ '/^[^ ]/ {print $1}' | sort | uniq -c | grep ' 1 ' | wc -l
1056
$ cat yarn.lock | awk -F@ '/^[^ ]/ {print $1}' | sort | uniq -c | grep -v ' 1 ' | wc -l
256
$ cat yarn.lock | awk -F@ '/^[^ ]/ {print $1}' | sort | uniq -c | grep -v ' 1 ' | awk '{i+=$1} END {print i}'
749

In today's React, there are currently 6 versions of the 'kind-of' library locked and presumably fetched by default (that's before your own dependencies come in play). In total there are 1056 libraries installed just once, and at least 256 libraries installed more than once (for a total of 749 instances).

E: I may have accidentally dropped all the packages with names starting with @ so it's possibly worse than it looks.

abigserve posted:

In 2019 if you are editing config files on the server you have made some bad mistakes

Doom Mathematic posted:

It would be amazing if programming jobs were like Advent of Code. Tiny, fixed-sized, rigorously specified, closed problems with singular correct answers.

DaTroof posted:

what's the pospinion on elixir? i might get assigned to a phoenix project and if so i have about a week to learn it

Arcsech posted:

elixir is good if you are writing a thing that is in its wheelhouse. that is, high concurrency, not too much in the way of heavy number crunching, not too much in the way of giant data structures you’re passing around. basically it’s good at all the junk people claim server side javascript is good at. if you try to use elixir for something it’s not good at, you’ll know, because it will be incredibly painful

it’s superficially like ruby, but that’s just syntax-deep, it’s super different otherwise. spend some time learning the BEAM primitives and OTP and the whole Erlang philosophy, because that will be valuable even if you don’t end up using elixir professionally.

redleader posted:

ruby 2.0, but if no one used it

toiletbrush posted:

dogmatically writing unit tests is good to learn because unit-testing multiplies the pain of bad code x100 so it teaches you to architect things better, but once you've learned your lessons it's less valuable. You should deffo aim for 100% automated coverage but higher level testing is what actually proves your app works.

The exception is any code tainted with algorithms involving dates, strings, money or floating point which should be unit-tested to uncurse it

Soricidus posted:

we're moving to a nanoservice architecture. each time a new user registers, we spin up a container to hold their data

Xarn posted:

I am going to blow your mind.

You can set up your CI so it runs tests on a virtual merge commit

Jabor posted:

presumably you are submitting some kind of change where the description is "update X dependency to version Y", whether that's a change to your copy of the dependency's code or to your lockfile or whatever.

nobody's going to look at the contents of that change after-the-fact, who cares how many bytes it is?

Shinku ABOOKEN posted:

DO_NOT_USE_OR_YOU_WILL_BE_FIRED

CPColin posted:

I like that every time JIRA releases some bullshit UI change that's even slower, they respond to all my angry tweets with "plesse DM me so I can better understand the issue " and I just reply "it's poo poo"