Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
floatman
Mar 17, 2009
I am a programmer who's just good enough to know he's terrible and not good enough to ever improve, life is suffering and despair namaste.

Adbot
ADBOT LOVES YOU

floatman
Mar 17, 2009
I have tests for a codebase with no abstractions, no dependency injections. Imagine MVC, but no models so controllers and views are directly touching database tables.
"Tests" are set up by loading XML files which contain database state before the tests are run. These XML files are loaded into a database which is complicated and many foreign keys, therefore if I "test" a feature that touches table A that has a dependency on table B, my XML file needs to have table A and table B state/data in it. If table B has dependency on table C, then XML file has A,B and C in it. And so on so on.

All these XML files are produced by hand. There is no way to use a tool.

The XML files load directly into the database. This means that when I load a state on which I perform my tests, there is no guarantee that this database state is actually legitimate in the application itself. There are no factories or repositories where I can use the same logic that production code uses to create conceptual data entries in a test environment.

As expected, this leads to developers hand crafting application state XML documents that serve as basis for tests cases that are impossible to ever happen in production, but these tests are apparently what is used to show that the feature works.
The sheer expense involved in writing these XML files also mean at most, only the happy path test is written and edge cases are manually tested by some QA on staging environments.

floatman
Mar 17, 2009

CRIP EATIN BREAD posted:

at least you could generate the code to form the XML then instead of hand writing it

Here's the thing: I did write a tool to generate database XML fixtures instead of writing it by hand.
So the idea was to go to staging, say we want table A, rows X, then the tool would dump that, and then you would repeat for dependency table B, dependency table C, D etc.
All my tests to generate XML files for tests ended up being almost full dumps of the entire database structure. It was at this point I learnt that the database was a piece of highly dependent hell. The reason why developers started hand writing XML files was so that they could cheat and leave out references to reduce the XML file requirements, even though this basically resulted in a test where the database would be set up into an invalid state. Bad design is one of the things that you can't go back and fix with tools.
At this point I resigned myself to bloodshed.

floatman
Mar 17, 2009
Need help figuring out references in the terrible programmer thread.

floatman
Mar 17, 2009
How can I apply that to my terrible app which is achieve some bespoke artisan handcrafted events system via database triggers

floatman
Mar 17, 2009
Just fake your Belize and move to death then.

floatman
Mar 17, 2009
You could also join a lovely company and work on their legacy app, giving you a treasure trove of study material on how NOT to write and structure software, with the benefit of this aforementioned trove growing everyday

floatman
Mar 17, 2009
I'm amazed at the other pl thread that actually has the embarrassment of riches to actually argue about what type of unit tests is good, while here I am struggling for scraps with this pos test suite that takes 9 hours to run, but don't worry guys we got, "ideas to fix them!"

I literally just saw a test that performed a bunch of loving asserts, then the last line of the test was markTestSkipped(). When I comment out the last line it's actually a passing test. What the gently caress was that Dev on when it was marked as skipped I want some of that too to ease my pain

floatman
Mar 17, 2009
Look at you guys with your fancy rest APIs returning different http codes.
Here at SHITEJOB, we just use one endpoint that accepts XML which contains comma delimited strings and a HTTP header called ACTION.
Based on ACTION, it enters this singular PHP function with a giant switch case.
After each case statement, XML data is deserialised.
There are some case statements that fall through to the next. This is by design. XML data is then parsed twice. Apparently this is by design. Pro code. No REST for the wicked!

floatman
Mar 17, 2009
They call it go because when you use it you want to go away

floatman
Mar 17, 2009
MongoDB

floatman
Mar 17, 2009
I have a database that stores json as strings. These jsons are taken out of the database and wrapped in an XML format to be sent over http. Bonus: the json values are CSV strings.

Anyway, how do you tell people you work with that just because they can't solve the problem doesn't mean the problem isn't simple? It's literally a mocking expectation that is not being hit with the correct expected arguments.
Me: have you broken the unit down expectation to not use the convenience method but instead use a closure so that you can inspect that the objects are the same and the object comparison algorithms are sane?
Them: no that can't be the problem the problem should be this obscure stack overflow page from 2002.

floatman
Mar 17, 2009
Imagine building an app, where the final build of the app depends on the shittest compiler, also known as the SELECT statement.
Imagine building an app where extensions to features do not need to be done by writing any code at all. Imagine instead of writing code, you can generate and janitor 6 pages of jank SQL statements. Imagine having to keep these 6 pages of jank statements in your cvs repository.
Now imagine someone changes the schema, making your 6 pages of jank poo poo now a ticking time bomb that doesn't work because nobody realised that that is actually the source code that builds your software.

Imagine a world with no programming,
Just endless rerunning of poo poo SQL statements, trying to fix everything that breaks withing the ecosystem of modern programming languages and with the need to keep some lovely obscure db model in your head

floatman
Mar 17, 2009

prisoner of waffles posted:

hmmm, "SELECT statement is the shittiest compiler" is just misleading. query optimizer is your friend and what's more, ORMs have to generate select statements too, bud.

How is query optimiser my friend in this case when it's literally affecting the application features being generated? It's selecting software, not data.

We had an issue where the application crashed because a guy removed a file containing a class that was thought to be unused. Turns out the application selects records from a table. These records contain a varchar field, which is the class name, and another varchar field which contains an XML string which would be deserialised to form the constructor args for the class, which would be dynamically loaded via the language version of "eval" in the code base.
This class was never mentioned in the code base at all outside of its class definition; the only place where it was used was in a db trigger that would insert a row into aforementioned table.

Db trigger was never in the SQL migrations repository. Only in production.

floatman
Mar 17, 2009
The question becomes, "why are you trying to tightly couple databases and database implementation across microservices?"

Like, what are you trying to do that needs "secret" implementation details that are not publicly visible from a microservice?

What if the microservice is not even storing it's data in a format that has primary key IDs? (I.e. flat XML file stored in MongoDB)

It's kinda a design smell I guess.

floatman
Mar 17, 2009
"Designed by idiot hellfucker" has a nice ring to it.
Look in my dead souless eyes; I'm beyond caring about ORMs vs SPROCs or whatever.
I just want databases to store data, not serve as some bespoke handcrafted artisan way of extending a programming language.

floatman
Mar 17, 2009

gonadic io posted:

hi thanks for your urgent hotfix pr, however sorry for being pedantic but can you remove these newlines and add some here please :) thanks

Whenever an application breaks, the users can 100% tell that the reason for it breaking is not enough new lines or spaces after commas.
Above all else, formatting is the sure sign of quality code that everyone can feel.

floatman
Mar 17, 2009
Hey terrible programming thread let's talk about gender.
Like, I have a customer management program CMS style thing. Because idiot hellfucker, it's one massive mono instance for multiple region worldwide.
Different users are going to have different gender management requirements. I.e. region A just wants to collect M, F, X. Region B wants to collect M, F, X, Y, P.
Anybody has any experience in implementing such a pattern?

One issue is the system is basically very poo poo database driven, so we got all the old data storing columns of old gender data consisting of M and Fs with some Xs.
I was thinking something like:
Create one master set of gender codes (M, F, X, Whatever) along with default mapping master set (M is male, F is female, X is unspecified, etc)
Each organisation can specify from the master set the subset of gender codes they wish to represent their customers (i.e. org A uses M and F, org B issues M,F,X org C uses M, F, X, A)
Each organisation can specify specific mapping overrides that will apply to their organisation only (M instead of Male is mapped to "Guy")
Create an abstraction which is basically a function that takes in the organisation id and returns a list of gender codes and mappings for that organisation id.

So far the main feeling I get from that idea is the gender codes are loosely enforced in the database i.e. you can get the configured acceptable gender codes, but you can store any gender code of the master set in the database record. I kinda don't care about this.
The other problem is custom mapping of labels i.e. if an organisation maps the code of "M", "Male" to "Guy" then that's fine, but the system technically allows them to map "M" to "Girl" as well, and I think that would gently caress things up when we need to aggregate across organisations gender information

floatman
Mar 17, 2009

Powerful Two-Hander posted:

in all seriousness though, you should use a join because if you don't and just have a single column someone is gonna be posting in this thread in future going "how the do I unfuck this single column that has some mysterious and arbitrary meaning"

Let me tell you a secret friend,
/* Leans in real close, in a quiet voice,
"It already IS a hosed up single column that has mysterious and arbitrary meaning."

I do appreciate the point of view for genders represented as IDs, therefore every organisation can specify their own genders and labels etc.
Unfortunately,
1) there's already information stored for existing customers as gender in a single column that has mysterious and arbitrary meaning,
2) the assignment of meaning is hardcoded everywhere in the code base (including a hilarious SQL statement that assumes IF GENDER NOT "F" THEN "M")
3) and the real talk thing is, there's going to be a need to aggregate customers by genders over organisations. So assuming I let organisations specify id 1 = male for org A, id 123 = male for org B, id XYZ = male for org N, I still need to have some master mapping that states that "all these codes are considered male across all organisations". As the software is brutally shoved down the users throat, I'm thinking might as well just remove the choice from them to stop them from doing special snowflake things.

Like, I get the ideological purity as genders represented as IDs, allowing for many genders per organisation but the reality of living in this cruel world with shitfuck sins of our forefathers architecture is pushing me to a solution where "M" is male.
The gender as id solution is just niceties but it doesn't seem to bring anything to the table but more work for me.

floatman
Mar 17, 2009
Oh, and efficiencies in join? Who cares!
Codebase is so poo poo anyway we're spending a fortune on raw computing power to give the illusion that the system is just slow rather than unusable.

floatman
Mar 17, 2009
Ultimately, each organisation's mapping of gender codes to labels will be sent from one system to another utilising a http endpoint that wraps comma separated string values in json to be further wrapped in an XML

I wish I was joking.

floatman
Mar 17, 2009

gonadic io posted:

defeats static analysis tools like find usages etc.

So, at JOB, we're using a plang.
Some genius actually heard that, you know, we should have basically something like enums to store string/int values to actual human readable names.

So they declare them as static variables in a class.
To be more precise, these constants are declared in A STATIC ARRAY IN THE CLASS, so instead of poo poo like user::FOO_VALUE, we got to do user::CONSTRAINTS_ARRAY['fooValue']
So if you want to find out what's referencing the value of food value, welp time to search for strings~

floatman
Mar 17, 2009

uncurable mlady posted:

that said 'effective logging and metrics' is also a skill that isn't taught very well

I worked on a website that somehow lived for 5 years without anybody doing application level debugging.
There just wasn't any framework placed inside whatsoever.
What developers did when they wanted to figure out what was happening in its complex internals was to Jerry rig a call to save strings into a database table on their local development environments.
There was no such thing in production so production was this magical black box of poo poo whenever anything went wrong.
When I asked them where's the logging they pointed me to this "database row change tracker" which was this giant convulted extension to the ORM that hijacked delete/update calls to create an XML representation of the table before update, do the update, create an XML representation of the table after update, then figure out what data changed.
So what you had was this table full of information such as "value 1 has changed to 0", or "value Bob has changed to Bobb" but you could never figure out the INTENT of the change.
God help you if one update affected multiple rows of data since there was no way to link them together.

floatman
Mar 17, 2009

Powerful Two-Hander posted:

reading this post made me feel stressed

Don't get me wrong, my current job is great, except for the one main thing which is the codebase for it.
Designed by idiot hellfucker with a hard-on for databases, and the more and more I uncover, the more I realised for all the poo poo they laud aforementioned idiot for his databases, turns out his databases are just as loving bad as his software (lmao "custom" ORM overrides that cast every value of plang into a unicode string. This means all nulls become 0s when shoved into into columns, also casting ints into unicodes means database has to cast it back, row by row, to do comparisons.)

I'm just barely contained rage on most days.

floatman
Mar 17, 2009
Oh yeahhhhh
first, imagine a class called Result. You can create a new result, and the constructor has an argument where you can shove in a constant. So you can do:
new Result(TypeFailed), new Result(TypePassed).
This result has a getType function to retrieve the result as well.
Now, imagine someone creates a class called ResultPassed, which inherits from Result. It also has a constructor where you can specify the constant. So although it is meant to represent a passed result, nobody is stopping you from doing new ResultPassed(TypeFailed).

In the codebase, sometimes a passed result is checked by calling ResultPassed.getType(). Other times, a passed result is check by checking if the result object is of type ResultPassed. There is no consistency, and I've lost my will to live.

floatman
Mar 17, 2009

It's amazing that a user of the name "AggressivlyStupid" can identify that it is problematic, yet the "code craftsmanship culture" here apparently is all okay with it.

floatman
Mar 17, 2009
Application doesn't have application logging even though has been in development and running for 5 years.
I add application logging via a psr 3 compatible interface. Yes this is loving PHP.
All logs go to PHP stdout because "12 factor application and MY CONTAINERS". But the people running the infrastructure so far has never bothered with the follow up to actually get and output the logs the application does.
Fast forward to today database is on fire timings slow. We don't know how slow. Wouldn't it be great if there was some way to show visibility of what the application is doing?

Well, I just found a ticket where they want to
* Somehow magically "get" database query timings for each request
* Attach the database timing into the header of the request
* Let nginx pick up the header information
* Then aggregate it into the log aggregator Splunk

It's like they're aggressively trying not to do application logging

floatman
Mar 17, 2009
"Why would anybody want application logging lol imagine you're developing locally and you get so many log data lmao"

I give up

floatman
Mar 17, 2009
Tell them that they should get rid of all staging servers, push everything to production because what profit do you get from testing???

floatman
Mar 17, 2009

ratbert90 posted:

I would much MUCH instead use Docker and K8s and manage microservices than have a vast monolithic application that has a million lines of code that nobody can handle adequately.


What if my vast monolithic application that has a million lines of code is managed in a docker compose with all kinds of poo poo containers such as an active directory container for the PHP application,
That is so loving complicated new developers have to set up various env values before they run docker-compose build if not it all fucks up,
But is ultimately NOT deployed on AWS with docker but instead some other poo poo ops decided to use so basically Dev Environment is not the same as staging Environment is not the same as production environment?

floatman
Mar 17, 2009
SQL good. But sometimes things done with SQL very bad.

floatman
Mar 17, 2009

Ciaphas posted:

i wonder what language causes the most revulsion in devs when they see examples of something truly foul being done

i don't think it's sql but it's sure up there

There was recently a jobs queue system implemented in SQL.
Jobs went in a table, then something picked jobs out. Jobs were then done.
Because there was multiple users who could queue jobs, a random picking algorithm was required so that the job system doesn't get stuck doing all the jobs for user A before user B (gotta be fair).
BUT then there was the concept that some jobs were of higher priority than other jobs, so there must be a random picking of jobs, but "randomly" pick the higher priority jobs first. And it was decided that random was not fair enough, so we must pick user jobs by round robin (i.e. high priority job for user A, high priority job for userB, normal job for user A, etc).

So how did this pan out? Some loving poo poo giant rear end SQL statement embedded in the codebase that used recursive CTEs. Since jobs were either waiting status or completed status, whenever a job was picked, the job record needed a lock to lock the job so if the SQL did run again while this job was running it wouldn't pick up the same job. This lock also touched the other tables that were involved in the job (I E. Updating of user details, etc you can imagine the loving shitshow). The recursive CTE query also had this group/order by statement that did JACK poo poo and was not needed but formed 90% of the query plan. Database just loving mysteriously died one-day once the amount of data that needed to be scanned through surpassed the query planned limits and the SQL server just loving gave up and decided to chug along row by row.
And here's the kicker:
The codebase called that SQL query to pick up a "semi-random" job if it wanted to get the next job to process. But if the codebase KNEW the primary key of the job it wanted to process and needed to get that job, it would still call this idiot hellfucker SQL query but with the primary key specified so even though you're getting just that one record you could have gotten with a straight up select you're still going to pay the costs of the recursive CTE and the 90% group/order by statement.

All these loving problems could have been avoided if we just did a naive random select from a queue of jobs. "But what about higher priority jobs," You may ask.
Welp, JUST USE TWO SEPARATE QUEUES and empty the priority one first duhhhhhhhhhhhhhh

floatman
Mar 17, 2009

Cold on a Cob posted:

lol

please tell me this was in sql server

Yes.

Cold on a Cob posted:

wait

they left the jobs in the "queue" ?!?!?!

:cripes:

Yes

floatman
Mar 17, 2009
Lmao just wait till the people at work hear that MSSQL actually has queues. It's ridiculous that the "chief architect" who's claim to fame was being a database wizzzzzzard choose to implement some idiot rubbish obscure custom bespoke queue system instead of just using the built in idiot rubbish obscure queue system.

floatman
Mar 17, 2009
Eh I don't know what to think already. I do agree with the points you said about what the purpose of queues should be, but nowadays after $JOB I'm just so jaded.
Like I'm starting to think that it's all fine and dandy to keep in mind "best practices" or theoretical good design and all that, but sometimes an anti pattern is necessary because if you leave the clowns to their own devices, you get poo poo like recursive CTE SQL queries for bad queue implementation or wrapping csvs in Json to be sent as XML strings.
It's not even good enough to reach the standard of bad. And most Devs here have Stockholm and they actually think everything is fine.

To put it this way I would have rather had a badly managed rabbitmq implementation to fix and debug rather than some idiot bespoke MSSQL server queue that one guy was so loving proud he did but didn't stay in the company long enough to see first hand the destruction it wrought

floatman
Mar 17, 2009

Kevin Mitnick P.E. posted:

theres actually zero difference between good & bad things. you imbecile. you loving moron

I'm sorry did I do something that offended you personally?

floatman
Mar 17, 2009

Boiled Water posted:

at this point i'm not even sure recursive anything should be allowed in sql. Just write out what you're attempting so the next person in line doesn't have to guess and guess

We actually paid a database consultant to come in to fix db problems. He wanted more context on the recursive CTE so we dug up the pull request which had the original writers plans and intents. His comments was, "wow what he says he sets out to do is entirely different from what was done."

floatman
Mar 17, 2009

Ah okay I'll go check it out for context thanks

floatman
Mar 17, 2009
I am so sick of bikeshedding.
Just because the software is poo poo and riddled with legacy problems that nobody wants to solve, all pull requests devolve into:
"WHY NOT MAKE THIS A ONE LINER INSTEAD?"
"WHY IS THIS A FUNCTION?"
"WHY IS THIS NOT A FUNCTION?"
"PLEASE ADD SPACES/ REMOVE SPACES/ PUNCTUATION"

meanwhile, nobody seems to care that all this poo poo code is going into a giant god abstract class. Seems that the pioneers of this software decided that if they're using a MVC framework, that means the only classes you can have HAVE to be models, views or controllers only. We got really fat controllers, which actually inherit some god base "abstract controller" which contains everything. EVERYTHING.

Adbot
ADBOT LOVES YOU

floatman
Mar 17, 2009

akadajet posted:

sort your goddamn using statements!

*Sorts by import part character length instead of alphabetical because "code aesthetics make pleasing"

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply