Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Sapozhnik
Jan 2, 2005

Nap Ghost
My main problem with caddy is that they're visibly really really really desperate to start generating some sweet RoI for their investors and best of luck to them but I don't want to get hitched to their poo poo in case they ram some other crap down my throat.

Traefik's devs for whatever reason seem less desperate. nginx is somewhat desperate but not nearly as much as caddy.

Adbot
ADBOT LOVES YOU

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

Back in 1.0, when the main guy was still developing it in his spare time, they briefly tried adding a HTTP header in the free binary release "Caddy server made possible by {sponsor company}". I thought that was totally fine, since it was not much different from a "powered by WordPress" at the bottom of your blog, it was clearly announced in advance, and you could easily build from source to remove it, but Hacker News absolutely lost their poo poo over it like it was malware and still keep bringing it up.

Nowadays there's a couple of small companies paying the devs a salary, one of them being ZeroSSL, which is basically "let's encrypt but with an optional paid plan". Caddy is pretty clearly a marketing tool for them, since I had no idea they existed until a Caddy released explained they were now supported as an alternative ACME provider. If Caddy ever deprecates Let's Encrypt support in favour of ZeroSSL only, that'll be the sign that they started to make compromises to generate cash.

Nomnom Cookie
Aug 30, 2009



NihilCredo posted:

do not take this as a suggestion you should switch ingress controller on a forum whim, _especially_ if you have a working kuby goldbernetes setup going for god's sake

but

i'm an aggressive hypeman for caddyserver so i want to just point out that the sample - but fully valid and complete - config i posted already includes automatic acme certificate management:

code:
mysite.com # <- that's the entire 'please get me a ssl cert for mysite.com and renew it automatically' configuration

handle_path /some-service/* {
    reverse_proxy some-service:8080
}
e: for others reading this who are blissfully unaware of modern devops, here's how you configure acme certificates renewal with cert-manager on k8s

the caddy ingress controllers are toys. anyone who tries to shove that config in a cluster is rapidly going to discover they have quite some work to do before they have something usable

Sapozhnik
Jan 2, 2005

Nap Ghost
what were your Concerns about my design anyway, I'm curious

redleader
Aug 18, 2005

Engage according to operational parameters
we all live in prisons of our own past decisions, but infrastructure people have it 100x worse

Nomnom Cookie
Aug 30, 2009



Sapozhnik posted:

what were your Concerns about my design anyway, I'm curious

you're asking ingresses to do something that isn't in the spec. even if everything works out fine now, you're no longer depending on ingress in general but traefik in particular. that complicates migrating to a different ingress controller. it's also a weird thing to do, which means either you have a truly unusual use case, or you're overcomplicating your design in pursuit of aesthetics. unusual use cases exist, but so do devs who fall in love with a pretty design and convince themselves that it's necessary

Sapozhnik
Jan 2, 2005

Nap Ghost
I'm not actually doing anything nonstandard with Ingress, although that's more than I can say for every example I've seen documented out in the wild; everybody seems to use vendor-specific annotations.

The prefix behavior is idiotic but that's what the spec says so I added code to my services to make their URL prefixes configurable from the same environment variable that controls their public base URLs.

I don't think it's an unusual use case at all, it's just pointless busywork. As far as the service is concerned it listens on http://0.0.0.0:8080/ or whatever, it shouldn't need to know where it's mounted in that particular deployment's public-facing URL namespace and be forced to listen under http://0.0.0.0:8080/api/whateverservice/. To make a really tortured analogy, when I mount a filesystem somewhere on a unix system I expect that filesystem's root directory to get mounted, not some deep subdirectory that matches whatever mount point I'm targeting. What on earth is the point of requiring that?

Nomnom Cookie
Aug 30, 2009



i agree that its ugly but unless you expect the prefix to change at some point, is it causing a problem. how much effort is it worth to make your URLs pretty

if you want a metaphor, it's mapping, not mounting

MrMoo
Sep 14, 2000

At a minimum it ensures some level of consistency. WebWorker scope badly broke that.

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat

ikanreed posted:

At major bank name omitted, there was a major trading application where the manager in charge of the project insisted "lock statements cause slowness" banned them from the code. This resulted in a solution the team's devs called "Luck based synchronization".

i once was hired by a friend of my boss to do a contracting gig to review some code and figure out why their database was so slow. the devs at the company kept fighting me, saying i didnt need to see the code to figure out why it was slow. they kept sandbagging me non stop, trying to keep me from seeing whatever they have written. eventually their manager told them to show me the code or else they would be fired. it sure felt like they were afraid that i was going to expose their terrible programming.

in about 30 minutes i found a job that was running that would do a bunch of prep work, loading some file and sorting it in memory, etc before doing any queries. they were explicitly locking their biggest table before all of that stuff, which caused everything else to halt from writing. the actual queries they were running took about 100ms. i moved the lock to right before the queries were executed, and it fixed all their problems.

locks good, but devs bad

shoeberto
Jun 13, 2020

which way to the MACHINES?
Maybe they were actually galaxy brain devs doing something like story 1 here
https://www.dodgycoder.net/2012/02/coding-tricks-of-game-developers.html?m=1
Just waiting for the right moment to move the lock, and you stole their glory, you monster.

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat
i made a few grand for 30 mins of actual work im ok with it

ikanreed
Sep 25, 2009

I honestly I have no idea who cannibal[SIC] is and I do not know why I should know.

syq dude, just syq!

CRIP EATIN BREAD posted:

i made a few grand for 30 mins of actual work im ok with it

Good old "knowing what button to push" billing.

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat

ikanreed posted:

Good old "knowing what button to push" billing.

well it did take almost a week of management fighting with their employees to let me see the code. so it was a week where i did nothing but work on other projects until they finally let me see the code.

MrMoo
Sep 14, 2000

Chome :chome: developers cap their virtual desktop size to 9,999 x 9,999, I need, let's see here, ..., 78K x 3,500 pixels.

Carthag Tuek
Oct 15, 2005

Tider skal komme,
tider skal henrulle,
slægt skal følge slægters gang



ugh this loving hyphenation code is hosed up

ive now considered like 5 different ways to handle it, but they all either immediately shows as unworkable or requires a ton of refactoring

ikanreed
Sep 25, 2009

I honestly I have no idea who cannibal[SIC] is and I do not know why I should know.

syq dude, just syq!

Carthag Tuek posted:

ugh this loving hyphenation code is hosed up

ive now considered like 5 different ways to handle it, but they all either immediately shows as unworkable or requires a ton of refactoring

Pushdown automata?

Carthag Tuek
Oct 15, 2005

Tider skal komme,
tider skal henrulle,
slægt skal følge slægters gang



ikanreed posted:

Pushdown automata?

im not sure how that would work

like creating an object that holds the individual parts of the hyphenated word, then have that object pretend to be the full word, by dynamically combining the properties of the parts?

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat

Bloody posted:

our biggest bottleneck is that we are stuck on 32-bit and a 64 bit migration requires a total rewrite

good luck.

one of my first projects i worked on was all C, written with a big endian cpu in mind (and it constantly did things like cast a long pointer to a char pointer for serialization, encoding, etc), and not a single thing was abstracted. they constantly re-implemented reading/writing bytes all over the project, about 1 million lines.

oh yeah, they also depended on divide by 0 not causing a fault and just returning 0. :smug:

we tried our damndest to port it to x86 but... well....

Bloody
Mar 3, 2013

simply override the division operator to include a if denominator == 0 check

Bloody
Mar 3, 2013

but yeah im pretty sure the 64-bit port is never gonna happen. i think we're more likely to do a rewrite/new product at that point

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

want to check if i'm designing a total horror or just a mild horror

our application uses RBAC with keycloak as openid authorization endpoint, so endpoints check keycloak's jwt tokens in the auth bearer header for permissions

i got asked to design the implementation of the following feature:

a regular user can ask permission to temporarily access $restricted_feature. when they do, the request is sent to their manager's app, who can approve or deny it. if they approve it, the user is granted the appropriate role for e.g. 30 minutes.

i don't want to have to manually check for overrides everywhere, so I am considering something like this

when the manager sends 'ok, grant $permission to user', the backend

1- logs in to keycloak's admin api using a backend-only set of elevated credentials with role-management and impersonation roles
2- grants the role to the user
3- impersonates the user
4- requests a jwt token with a custom (short) expiration date using this plugin
5- immediately revokes the role from the user
6- sends the jwt token to the user's app, which uses it in lieu of the one it has, until it expires and gets automatically refreshed as usual (the new token won't have the permission anymore)

pros:

- 100% transparent to the existing endpoints and auth flows, both client and server side

cons:

- if the user happens to log in between steps 2 and 5, he will be granted the permission for the duration of the regular access token. it might be possible to fix this by disabling the user's regular login during this operation, if this doesn't also block impersonation

- would likely break if we ever implemented jwt remote validation (i assume jwts with claims the user no longer possesses would not pass validation, same as if they were revoked)

distortion park
Apr 25, 2011


I would be worried about users getting into weird states if any of the account modification steps failed. Idk how keycloak works but monitoring for that sort of failure case might also be a challenge if you don't have an external source of truth. Could you clone the user to a new account without a login or something first?

You could also write your own plugin to allow overrides lol

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat
we are starting to look at various implementations to try to set up some SSO for things, but none of the options feel great.

keycloak seems to suffer from redhat syndrome and its huge and bulky, and requires a lot of static configuration files.

we were looking at fusionauth which seems to have some really nice APIs, but they charge you a monthly license to get LDAP/AD support (which is dumb).

finally I came across something called ORY Hydra which ONLY does the OAuth2 stuff (no user management), and requires you to set up a REST endpoint that does the login/logout/confirm access stuff, that you implement yourself and point Hydra at, and they handle all the JWT stuff, token generation, etc.

Xarn
Jun 26, 2015
Sounds like a big secfuck in making. Go for it.

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat
i really dont give a poo poo what happens to any of the users tbh

i'll jsut bail if there's a disaster

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

CRIP EATIN BREAD posted:

keycloak seems to suffer from redhat syndrome and its huge and bulky, and requires a lot of static configuration files.

fwiw we're running keycloak from the official docker image with zero configuration files. just environment variables pointing them to a postgres instance, then everything gets saved in the "keycloak" db

the stuff im considering would be the first time we have to add extra configuration (a command flag to enable the token_exchange feature since it's been in 'preview' since literally eight majors ago, idk either)

cool av
Mar 2, 2013

the only thing that sounds kinda off is the back-end pushing a new token to the client? (is there always just one client? will they definitely be logged in when the manager approves it?) and I guess if they need 2 temp. roles they have to request them both in a single transaction.

Nomnom Cookie
Aug 30, 2009



NihilCredo posted:

want to check if i'm designing a total horror or just a mild horror

our application uses RBAC with keycloak as openid authorization endpoint, so endpoints check keycloak's jwt tokens in the auth bearer header for permissions

i got asked to design the implementation of the following feature:

a regular user can ask permission to temporarily access $restricted_feature. when they do, the request is sent to their manager's app, who can approve or deny it. if they approve it, the user is granted the appropriate role for e.g. 30 minutes.

i don't want to have to manually check for overrides everywhere, so I am considering something like this

when the manager sends 'ok, grant $permission to user', the backend

1- logs in to keycloak's admin api using a backend-only set of elevated credentials with role-management and impersonation roles
2- grants the role to the user
3- impersonates the user
4- requests a jwt token with a custom (short) expiration date using this plugin
5- immediately revokes the role from the user
6- sends the jwt token to the user's app, which uses it in lieu of the one it has, until it expires and gets automatically refreshed as usual (the new token won't have the permission anymore)

pros:

- 100% transparent to the existing endpoints and auth flows, both client and server side

cons:

- if the user happens to log in between steps 2 and 5, he will be granted the permission for the duration of the regular access token. it might be possible to fix this by disabling the user's regular login during this operation, if this doesn't also block impersonation

- would likely break if we ever implemented jwt remote validation (i assume jwts with claims the user no longer possesses would not pass validation, same as if they were revoked)

dont manually check for overrides everywhere, just do it once. you have some function that checks whether some JWT has some permission, and do the override check there

Powerful Two-Hander
Mar 10, 2004

Mods please change my name to "Tooter Skeleton" TIA.


a team that consume messages we publish to a queue migrated their queue receiver and in the process somehow managed to change their processing from "if I encounter an error, take the message off and alert but continue to process" to "if there's an error, throw a cascade of alerts, crash the process and leave the message on the queue so when you restart it blows up again and the queue fills up"

none of their exceptions are being handled and are coming from the lack of any logical validation on what they're doing when they process a message (inserting rows in a table), so needless to say they have raised this as a sender side incident :rolleyes:

cool av
Mar 2, 2013

the solution to the problems caused by a queue is, of course, another queue

Zaxxon
Feb 14, 2004

Wir Tanzen Mekanik

NihilCredo posted:

want to check if i'm designing a total horror or just a mild horror

our application uses RBAC with keycloak as openid authorization endpoint, so endpoints check keycloak's jwt tokens in the auth bearer header for permissions

i got asked to design the implementation of the following feature:

a regular user can ask permission to temporarily access $restricted_feature. when they do, the request is sent to their manager's app, who can approve or deny it. if they approve it, the user is granted the appropriate role for e.g. 30 minutes.

i don't want to have to manually check for overrides everywhere, so I am considering something like this

when the manager sends 'ok, grant $permission to user', the backend

1- logs in to keycloak's admin api using a backend-only set of elevated credentials with role-management and impersonation roles
2- grants the role to the user
3- impersonates the user
4- requests a jwt token with a custom (short) expiration date using this plugin
5- immediately revokes the role from the user
6- sends the jwt token to the user's app, which uses it in lieu of the one it has, until it expires and gets automatically refreshed as usual (the new token won't have the permission anymore)

pros:

- 100% transparent to the existing endpoints and auth flows, both client and server side

cons:

- if the user happens to log in between steps 2 and 5, he will be granted the permission for the duration of the regular access token. it might be possible to fix this by disabling the user's regular login during this operation, if this doesn't also block impersonation

- would likely break if we ever implemented jwt remote validation (i assume jwts with claims the user no longer possesses would not pass validation, same as if they were revoked)

It's probably easier in the long run to just have the overrides be granted by a separate system that can better manage expiring and revoking them and stuff like that.

I would guess it's also pretty important for your logging and stuff that you know that person-x used this feature thanks to the override granted by manager y. So you want to explicitly check for an override to make sure you note that down.

the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





cool av posted:

the solution to the problems caused by a queue is, of course, another queue

i'm a staff data engineer and this is correct

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat
we use kafka for data streams/events and everything ive done with it rules but guys at work keep pushing 30mb messages onto the topic, for no good reason.

Nomnom Cookie
Aug 30, 2009



the reason is "its easier for us to do it this way" and it's the only good reason to do anything

shoeberto
Jun 13, 2020

which way to the MACHINES?
Anyone have any comparative experience with data pipeline tools? I was loving around with Dagster today since it's pretty turnkey but am curious if there's some good reason to go with Airflow or something instead.

bob dobbs is dead
Oct 8, 2017

I love peeps
Nap Ghost
i think airflow is gonna be the boring one that most peeps use, w luigi comin in second

Progressive JPEG
Feb 19, 2003

borat voice my husband likes prefect

the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





shoeberto posted:

Anyone have any comparative experience with data pipeline tools? I was loving around with Dagster today since it's pretty turnkey but am curious if there's some good reason to go with Airflow or something instead.

if you care about offloading poo poo to people who care more than you, airflow has hosted versions on basically every cloud you can think of

Adbot
ADBOT LOVES YOU

CRIP EATIN BREAD
Jun 24, 2002

Hey stop worrying bout my acting bitch, and worry about your WACK ass music. In the mean time... Eat a hot bowl of Dicks! Ice T



Soiled Meat

Nomnom Cookie posted:

the reason is "its easier for us to do it this way" and it's the only good reason to do anything

all the stuff they put in there is stdout and stderr output from a forked process

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply