|
I had a course in college that used matlab and p much agree. the language is an abomination but then you call do_the_complicated_math(my_data) and save a week over reimplementing that call in a more pleasant language
|
#
?
Dec 14, 2018 07:35
|
|
|
|
| # ? Apr 25, 2024 11:40 |
|
|
so how do goog handle storage (+ dbs, although i appreciate that dbs at goog are probably of a different scale) if k8s sucks at it
|
|
#
?
Dec 14, 2018 07:38
|
|
|
microservices are v good at solving the problem they were created to solve: alleviating the boredom of a bunch of engineers making yet another boring crud app
|
|
#
?
Dec 14, 2018 07:40
|
|
|
redleader posted:so how do goog handle storage (+ dbs, although i appreciate that dbs at goog are probably of a different scale) if k8s sucks at it my impression was that google uses borg and k8s is a vanity project for gce
|
|
#
?
Dec 14, 2018 07:47
|
|
|
redleader posted:microservices are v good at solving the problem they were created to solve: alleviating the boredom of a bunch of engineers making yet another boring crud app with unlimited private repos for $7/month it would be stupid to not make 1000 of them!
|
|
#
?
Dec 14, 2018 07:52
|
|
|
Kevin Mitnick P.E. posted:my impression was that google uses borg and k8s is a vanity project for gce huh. i knew they had borg, but given their investment in k8s i'm a bit surprised they aren't going in on k8s although it does fit into everything else i've heard about google, where you end up with 3 internal and 2 external projects doing fundamentally the same thing for Reasons
|
|
#
?
Dec 14, 2018 07:52
|
|
|
redleader posted:huh. i knew they had borg, but given their investment in k8s i'm a bit surprised they aren't going in on k8s the reason is one and its simple, google rewards initiatives and not deliveries
|
|
#
?
Dec 14, 2018 07:53
|
|
|
if your service needs transient working storage, k8s handles it just fine. if your service needs persistent storage, it shouldn't be doing it itself, it should talk to a different service or a database or something which handles that.
|
|
#
?
Dec 14, 2018 07:54
|
|
|
Jabor posted:if your service needs transient working storage, k8s handles it just fine. if your service needs persistent storage, it shouldn't be doing it itself, it should talk to a different service or a database or something which handles that. yeah thats what I said p much. PVC and friends are toys to check off features and the more you use them the more pain and regret you will experience
|
|
#
?
Dec 14, 2018 07:59
|
|
|
Krankenstyle posted:whew, think im finally starting to get the hang of tex scoping & expansion, a little kinda glad i don't understand this post tbh
|
|
#
?
Dec 14, 2018 08:54
|
|
|
gonadic io posted:truly i had not considered a system with all the convenience of microservices and all the code separation of a monolith a very accurate summary of our codebase
|
|
#
?
Dec 14, 2018 09:51
|
|
|
the modern world sounds complicated. i'm going to stick with copying files to the prod web server through explorer.exe
|
|
#
?
Dec 14, 2018 10:03
|
|
|
Jabor posted:if your service needs transient working storage, k8s handles it just fine. if your service needs persistent storage, it shouldn't be doing it itself, it should talk to a different service or a database or something which handles that. and that other service should in turn talk to another service to handle it, and so on, until your persistent storage is actually just a bunch of json perpetually flying around the cloud. delay line memory is back, and this time its webscale
|
|
#
?
Dec 14, 2018 10:09
|
|
|
redleader posted:the modern world sounds complicated. i'm going to stick with copying files to the prod web server through explorer.exe What's a prod Web server? I just start the thing on my laptop and try to remember not to turn it off while people are using it
|
|
#
?
Dec 14, 2018 10:17
|
|
|
gonadic io posted:What's a prod Web server? I just start the thing on my laptop and try to remember not to turn it off while people are using it this is, without a hint of exaggeration, how our operational monitoring works
|
|
#
?
Dec 14, 2018 10:24
|
|
|
redleader posted:this is, without a hint of exaggeration, how our operational monitoring works At $oldJob we had to fly a dev out to Milan to install a sftp server on one of $thirdParty's machines so they could have an intern put files in it (presumably after being emailed then). It was a csv that routinely had transcription errors so presumably was copy/pasted by hand
|
|
#
?
Dec 14, 2018 10:33
|
|
|
gonadic io posted:At $oldJob we had to fly a dev out to Milan to install a sftp server on one of $thirdParty's machines so they could have an intern put files in it (presumably after being emailed then). It was a csv that routinely had transcription errors so presumably was copy/pasted by hand Mind you, this is the same job where one of the other third parties refused to do even that we we had to get an intern on our end who would receive the email containing the xls 1997 version, and copy paste with a small adjustment into an internal web form where we parsed it and put it into our systems.
|
|
#
?
Dec 14, 2018 10:36
|
|
|
why do we do this poo poo
|
|
#
?
Dec 14, 2018 10:40
|
|
|
redleader posted:why do we do this poo poo its a living
|
|
#
?
Dec 14, 2018 10:44
|
|
|
redleader posted:why do we do this poo poo laziness, incompetence, lack of interest in improving the situation, fear of change, intern time is much cheaper than dev time E: wait do you mean why do companies do this poo poo or why to we do devving in general? Either way my answer applies gonadic io fucked around with this message at 10:54 on Dec 14, 2018 |
|
#
?
Dec 14, 2018 10:45
|
|
|
animist posted:kinda glad i don't understand this post tbh you should be, that way lies only pain and terror
|
|
#
?
Dec 14, 2018 10:56
|
|
|
redleader posted:so how do goog handle storage (+ dbs, although i appreciate that dbs at goog are probably of a different scale) if k8s sucks at it you just put everything in spanner. how does spanner handle storage? very carefully if you work with bigass blobs of unstructured data, you might put them in google cloud storage or maybe an older system built directly on colossus. idk I don't work with that kind of data.
|
|
#
?
Dec 14, 2018 11:10
|
|
|
redleader posted:the modern world sounds complicated. i'm going to stick with copying files to the prod web server through explorer.exe hello yes this happens too
|
|
#
?
Dec 14, 2018 11:46
|
|
|
Jabor posted:if your service needs transient working storage, k8s handles it just fine. if your service needs persistent storage, it shouldn't be doing it itself, it should talk to a different service or a database or something which handles that. So how does the DB handle its storage?
|
|
#
?
Dec 14, 2018 11:51
|
|
|
gonadic io posted:laziness, incompetence, lack of interest in improving the situation, fear of change, intern time is much cheaper than dev time i did mean it in the latter sense, but v good answer to both interpretations i find myself becoming more disillusioned with my choice of career every day, but cannot conceive of a better alternative so i'll just keep on slinging hacks on top of hacks on top of poo poo, taking some slight solace in the fact that ultimately competes 6 don't matter and all human progress is illusory
|
|
#
?
Dec 14, 2018 12:11
|
|
|
redleader posted:i did mean it in the latter sense, but v good answer to both interpretations We get paid way too much, generally less oversight than equivalent office jobs, also in general higher tolerance for failure than comparable jobs, and are highly in demand so moving jobs is better than equiv white collar jobs. I don't see myself leaving this career until the flood of current teens being taught this poo poo in school hits the job market. Peace. Also get therapy imo.
|
|
#
?
Dec 14, 2018 12:16
|
|
|
gonadic io posted:We get paid way too much, generally less oversight than equivalent office jobs, also in general higher tolerance for failure than comparable jobs, and are highly in demand so moving jobs is better than equiv white collar jobs. I don't see myself leaving this career until the flood of current teens being taught this poo poo in school hits the job market. Peace. Also get therapy imo. a lot of people think that computers are literal magic so they find it hard to understand how terrible we are in the same way they'd understand if a architect hosed up a house so it fell down
|
|
#
?
Dec 14, 2018 12:43
|
|
|
Xarn posted:So how does the DB handle its storage? very carefully but essentially, the point is if a service is responsible for persisting data, that's the *only* thing it should be responsible for. no processing, no business logic, purely persisting data for your other services to use and since that's not specific to your business domain, ideally you let someone else do the heavy thinking on that part
|
|
#
?
Dec 14, 2018 12:45
|
|
|
prisoner of waffles posted:in other news, I'm falling in love with MATLAB. no
|
|
#
?
Dec 14, 2018 14:43
|
|
|
Xarn posted:So how does the DB handle its storage? afaik you can spin up several postgres containers and set them up to operate as a replicated cluster. k8s has some sort of affinity thing in it to make sure it always starts up on those same physical hosts and uses some dedicated and persistent local storage areas on those hosts to keep its data around. if one of those physical hosts dies then you spin up another instance, add it to the cluster and let it get repopulated using replication. obviously you have to monitor things so that you never lose your last replica. while this is doable now it probably ought to be a one-click action in k8s but it's not really there yet. so in short Jabor posted:very carefully
|
|
#
?
Dec 14, 2018 14:44
|
|
|
I mean yeah matlab is great and a big part of that is the ide and the way it strikes a nice balance between being an ide and a repl and everything and plots are nice and the math works and is good but at this point Id rather jump through several annoying hoops to get python
|
|
#
?
Dec 14, 2018 14:45
|
|
|
Phobeste posted:I mean yeah matlab is great and a big part of that is the ide and the way it strikes a nice balance between being an ide and a repl and everything and plots are nice and the math works and is good but at this point Id rather jump through several annoying hoops to get python Matlab is amazing if you're an academic and things need to be janitored and babied to work (and also when you don't have to pay for their nickel and dime operation) when you're in an industrial setting ehhhh
|
|
#
?
Dec 14, 2018 15:00
|
|
|
Sapozhnik posted:afaik you can spin up several postgres containers and set them up to operate as a replicated cluster. k8s has some sort of affinity thing in it to make sure it always starts up on those same physical hosts and uses some dedicated and persistent local storage areas on those hosts to keep its data around. if one of those physical hosts dies then you spin up another instance, add it to the cluster and let it get repopulated using replication. obviously you have to monitor things so that you never lose your last replica. yeah see thats all v bad. dont use k8s to store anything important
|
|
#
?
Dec 14, 2018 15:21
|
|
|
lmao i just remembered a time in grad school when i failed a take-home coding test for a data science job because string munging in matlab is a hellscape for some reason (i didnt know any data science python at the time because i was bad)
|
|
#
?
Dec 14, 2018 15:52
|
|
|
I worBoiled Water posted:Matlab is amazing if you're an academic and things need to be janitored and babied to work (and also when you don't have to pay for their nickel and dime operation) I once worked on a project for a third party that had an algorithm designed in matlab, and wanted us to port it to a C library. when they learned it would take a whole week, they spent a week trying to embed matlab on the target platform.
|
|
#
?
Dec 14, 2018 15:52
|
|
|
CRIP EATIN BREAD posted:I wor licensing for matlab being what they are it would be way way more expensive. but then again corporate never made sense.
|
|
#
?
Dec 14, 2018 15:54
|
|
|
prisoner of waffles posted:in other news, I'm falling in love with MATLAB. The language is awful and really encourages making write-once monstrosities even though it provides better ways for organizing code, some of which pay off and make you look smart to the people who only ever write scripts. I like using their extremely robust standard library to reimplement a single function from a toolbox that costs $1000. I like chewing on physicsy data and doing stats or signal processing poo poo and I get paid for that now. I would love a job loving around with MATLAB all day. Beats the hell out of turd shunting enterprise nonsense.
|
|
#
?
Dec 14, 2018 15:56
|
|
|
Kevin Mitnick P.E. posted:yeah see that’s all v bad. don’t use k8s to store anything important how is it any different than running on bare metal? if a server dies you still need to replace it and spool up new replicas.
|
|
#
?
Dec 14, 2018 16:08
|
|
|
Finster Dexter posted:I blame microservices (and I'm not sure if that's good or bad, tbh) ah, thought this was a joke. i took a peek at it, and it looked really horrible
|
|
#
?
Dec 14, 2018 16:09
|
|
|
|
| # ? Apr 25, 2024 11:40 |
|
|
Sapozhnik posted:k8s has some sort of affinity thing in it to make sure it always starts up on those same physical hosts and uses some dedicated and persistent local storage areas on those hosts to keep its data around. This isn't how pods work at all. At least, not how it's configured to work ootb. k8s will gladly start moving pods around to different nodes if a node starts getting cpu or mem pressure. It's super fun when this happens to your web server and/or your bespoke Russian-built microservices and logs out everyone connected on that pod. And in terms of storage, because of this you can't count on anything on the file system being persistent (nor should you). If you want persistent, you have to spin up a persistent volume claim which ends up as EBS volumes. Although, even that isn't just flip a switch and go because if the pvc is spun up in a zone different then the node's zone, k8s throws weird errors about pvc not in zone or somesuch, and if all your nodes in that zone are already full of turd shunters lmao well now you have to do more config fuckery to get the ebs volumes to line up... CTO and I wasted a week on this pile of poo poo, and since I've only been using k8s for like a few months (yay DevOps) I have no idea if this is normal or what. So, maybe your k8s has magical affinity elves or something, but honestly, storage is just a giant loving pain in k8s.
|
|
#
?
Dec 14, 2018 16:10
|
|