|
helm template-value files can get a bit complex sometimes, so it can be nice to template them out as well for maximum simplicity and ease of use
|
#
?
Oct 19, 2021 09:06
|
|
|
|
| # ? Apr 26, 2024 17:48 |
|
|
that is, no poo poo, what our Platform Team decided to do. the template files are loving ghastly and the sole guy who built it left, so it’s unmaintainable
|
|
#
?
Oct 19, 2021 15:06
|
|
|
brb, just checking out my worker's taints
|
|
#
?
Nov 5, 2021 15:54
|
|
|
putting the nerd in containerd
|
|
#
?
Nov 8, 2021 07:18
|
|
|
echinopsis posted:sounds neat it isn’t
|
|
#
?
Nov 12, 2021 07:58
|
|
|
sounds poo poo then
|
|
#
?
Nov 12, 2021 10:16
|
|
|
are polling loops just the default way to manage internet scale distributed clusters nowadays?? what happened to pub/sub queue designs?? i mean openstack is a colossal pile of crap for a lot of reasons but they did at least base the cluster comms on a mq topology. why the hell does an idle 3 controller and 2 worker kubernetes cluster consume 6 cores of cpu capacity while doing nothing?? is there really that much work involved in understanding how much nothing is happening??? wtf?
|
|
#
?
Nov 12, 2021 14:21
|
|
|
I'm not an expert but IME pub/sub stuff sucks for synchronization. If you mess up handling a message you need some way to eventually do the thing - so you end up building a reconciliation system. The watch-loop stuff is just going straight to the reconciliation based system
|
|
#
?
Nov 12, 2021 20:30
|
|
|
really want to emphasize how far i am from being in any way knowledgeable about this though
|
|
#
?
Nov 12, 2021 20:30
|
|
|
for a queue you'd probably just want to run kafka but it's a memory hog and a pita to tune/maintain once you get significant load going through it
|
|
#
?
Nov 12, 2021 20:43
|
|
|
fresh_cheese posted:why the hell does an idle 3 controller and 2 worker kubernetes cluster consume 6 cores of cpu capacity while doing nothing?? is there really that much work involved in understanding how much nothing is happening??? wtf? sounds like you hosed something up bud
|
|
#
?
Nov 12, 2021 20:53
|
|
|
CRIP EATIN BREAD posted:sounds like you hosed something up bud that is highly likely I should let the smart cloud people handle this oh wait...
|
|
#
?
Nov 12, 2021 21:37
|
|
|
kube owns, and its multiplied my productivity because of how easy it is to deploy a new project. p much all you need to know is pods, deployments/statefulsets/daemonsets, services, and ingress. and then u copy paste yamls to do most of what u want. metallb too if ur baremetal (the only way to run bc cloud costs are unreasonable for personal projs) the only problem ive had so far is cloud storage. nfs/samba are complete poo poo for anything besides minio. for any db you pretty much have to deploy directly onto a specific node's ssd using hostpath. which is risky unless you replicaset it onto three nodes total for uptime. ive tried ceph multiple times, but its a complete joke. i might give up and do gluster, but idk. i really just want a zero maintenance file storage thats decent. and everything ive looked at so far wants u to waste ur life janitoring it
|
|
#
?
Nov 12, 2021 22:56
|
|
|
rook-ceph seems fine to me
|
|
#
?
Nov 12, 2021 23:33
|
|
|
can use local-path-provisioner instead of hostPath, much less scary while being functionally equivalent meanwhile for replicated storage ive been using longhorn and its worked fine even with my 4gb rpi nodes. but ive just used it for smaller stuff like configs where its not too io-intensive and where i don't want the pod to be stuck in pending if the host node dies
|
|
#
?
Nov 13, 2021 02:14
|
|
|
id use gce-pd or aws-ebs for db storage - i was surprised how few problems we had with gke storage considering the whole distributed stateful system thingie though preferably i'd provision a managed db
|
|
#
?
Nov 16, 2021 01:22
|
|
|
oh yeah if you're running it on one of them then just use their stuff, the above recommendations assume onprem
|
|
#
?
Nov 16, 2021 08:08
|
|
|
todays kubernetes s: Helm can install ValidatingAdmissionWebhooks that validate Secrets. When it does this, it's usually also installing the Pod that runs the webhook service. If that Pod is not running, all attempts to validate resources will fail, and Kubernetes will reject the change. Helm tries to update a Secret containing release info immediately after deploying the Pod that must be running for Kubernetes to allow that change, and it does not retry if that update fails. If the update fails, Helm will refuse to modify the release after, because the previous action is still "in progress" per the update it made to its metadata Secret before deploying the thing that prevented it from updating that Secret. thanks, Kubernetes. thubernetes
|
|
#
?
Nov 23, 2021 21:49
|
|
|
validating and mutating webhooks are a great way to brick your cluster. I guess they do other stuff too but I’ve mostly observed them causing problems
|
|
#
?
Nov 23, 2021 22:10
|
|
|
i think it says a lot about how much helm sucks when everyone is now just diying their own operators that do nothing other than deploy a service, rather than continuing to deal with helm's bullshit i still sparingly use helm but just because i don't quite hate it enough to bother with figuring out a replacement yet imo the best way to use it is "helm template | kubectl apply -f -" whenever possible, because helm's own handling of updates is super broken, e.g. treating updates to specs as purely additive. handling that part with "kubectl apply" avoids the broken logic in helm. but "helm template" itself will silently fail if you want to use lookup calls. it will treat the lookup items as not found rather than just safely failing the operation like you'd expect now that i've written all this i am really leaning towards dropping helm since i only use it for: - random one-time generated internal secrets via this one weird trick that helm devs hate (silently fails with "helm template" per above) - templating fixed/external secret values out into a separate yaml file that i keep in a password manager
|
|
#
?
Nov 23, 2021 23:17
|
|
|
Nomnom Cookie posted:validating and mutating webhooks are a great way to brick your cluster. I guess they do other stuff too but I’ve mostly observed them causing problems for validators at least, might be able to use "failurePolicy: Ignore" to fail open if the pod is down. for example ive been using (and happy with) opa/gatekeeper and that's their default idk about mutators, they always seemed like sort of a bad idea since suddenly it isn't WYSIWYG anymore but i haven't really seen much of them either
|
|
#
?
Nov 23, 2021 23:44
|
|
|
Nomnom Cookie posted:validating and mutating webhooks are a great way to brick your cluster. I guess they do other stuff too but I’ve mostly observed them causing problems ideally, if you need to validate a non-CRD resource, you use them with label selectors to not validate random stuff you'll never touch anyway. we, stupidly, use the presence of a specific key in a Secret for our filtering, so we can't do that. fortunately, you can also do "!value" in label selectors, so you can exclude Helm's Secrets. i just wish operators didn't need an actual operator deployment--you'd think Red Hat would have learned from everyone hating Tiller and Helm 3's (actually well done) state management. you should be able to just run operator CLI poo poo locally and have it vomit out YAML to apply, more or less. instead they've exacerbated the problem by making everyone roll their own Tiller Helm's additive/3-way diff thing is actually useful, but it shouldn't need to be. one of the shittier things about templates for users is that if you want to add in a field to some templated resource, it kinda just has to be in the template and exposed by values.yaml. the net effect of this is that your values.yaml slowly grows to just be the entire API spec for any templated resource, but organized differently than the spec for complicated historical reasons. the 3-way diff means that you can technically get around this, since your changes persist through upgrades despite not being in the template, but it's also poo poo because it's tied to the release and there's no standard way to persist it. the actual solution would be first-class support for kustomize, which, while it has its own serious limitations, is real good at taking a diff and applying it to some resource. Helm kinda pretends to have this by allowing you to run arbitrary post-processing scripts on template output, but that's more annoying to set up. you should be able to just provide the static diff files
|
|
#
?
Nov 23, 2021 23:55
|
|
|
I thought kustomize was bad and then I had to write a helm chart and kustomize is good enough actually
|
|
#
?
Nov 24, 2021 02:04
|
|
|
Progressive JPEG posted:for validators at least, might be able to use "failurePolicy: Ignore" to fail open if the pod is down. for example ive been using (and happy with) opa/gatekeeper and that's their default linkerd provides a mutating webhook and that works fine. it's not modifying your containers or anything
|
|
#
?
Nov 24, 2021 05:29
|
|
|
sharding? more like sharting
|
|
#
?
Jan 6, 2022 03:28
|
|
|
oh boy! work got us free subscriptions to professional development content and there's a "Certified Kubernetes Application Developer" course! wait, no, this course covers "how to write a Deployment spec in a not stupid way" and not "writing custom controller go code better and understanding the kubernetes libraries" eh, well, maybe the CKA course will be of some interest for learning something about the ops side that i don't deal with daily current annoyance/dilemma, which i think i've brought up before, is wanting to migrate from using Helm to an operator for our preferred deployment system. i am mad that the operator framework providing no option for running tasks outside an application in the cluster, as if it were fully impossible to run something to manage Deployments via an external client maybe utilizing static internal cluster state info Getting rid of Tiller without loss of functionality was the best thing Helm did, but operators are essentially stuck in world where you not only need to run Tiller still, you need to run a different Tiller instance for each application, with the Tiller version fully under the control of app devs. hopefully they keep it up to date, because users sure as hell can't!
|
|
#
?
Jan 6, 2022 04:12
|
|
|
operator framework doesnt support it because its a pretty deranged thing to want
|
|
#
?
Jan 6, 2022 06:37
|
|
|
like when you say "maybe utilizing static internal cluster state info" i start thinking that an operator probably isnt what you actually want to make, because the whole point of an operator is that it sits around watching events on a resource and reacting to them. if you don't care about getting updates from the apiserver then what you're making is not an operator more or less by definition
|
|
#
?
Jan 6, 2022 06:41
|
|
|
I’ve read this entire thread and I still don’t know what a kubernete is
|
|
#
?
Jan 6, 2022 21:04
|
|
|
Silver Alicorn posted:I’ve read this entire thread and I still don’t know what a kubernete is it's when u like cloud computing so much you run your own virtualized cloud on top of the one made and maintained by professionals
|
|
#
?
Jan 6, 2022 21:07
|
|
|
Silver Alicorn posted:I’ve read this entire thread and I still don’t know what a kubernete is a miserable pile of docker containers op
|
|
#
?
Jan 6, 2022 21:15
|
|
|
imagine a container crashing on a worker node, forever. that's the kubernetes vision of the future
|
|
#
?
Jan 6, 2022 22:29
|
|
|
akadajet posted:it's when u like cloud computing so much you run your own virtualized cloud on top of the one made and maintained by professionals sometimes it runs on top of the one made and maintained by your own it jackasses instead
|
|
#
?
Jan 6, 2022 23:20
|
|
|
lick the taint of my pod
|
|
#
?
Jan 6, 2022 23:21
|
|
|
Silver Alicorn posted:I’ve read this entire thread and I still don’t know what a kubernete is you know how a mainframe runs batch jobs to do transaction processing and it’s all coordinated by a job control language that describes the data flow so the OS can maximize the throughput its basically like that, except reinvented from first principles by UNIX nerds for doing web poo poo and implemented via the UNIX philosophy of “string together lots of components that do only one thing in a janky-rear end way that doesn’t quite fit the way the other components work so you need to introduce more points of failure in between to translate between them” and in theory the system will ensure scalability what I’m saying is that the UNIX folks have never bothered to learn about how mainframe and minicomputer transaction processing and database systems work, and have had to reinvent all of that stuff poorly, often without the OS support that enables the best performance and compatibility DEC VMS, HP MPE, Stratus VOS, the various IBM mainframe and midrange operating systems, etc. are really interesting to learn about without applying “UNIX is better” prejudice, especially in light of how you might use them to implement applications in the modern style
|
|
#
?
Jan 6, 2022 23:48
|
|
|
its an API and reference implementation for running things in containers on other computers the API acts like a config spec where you say what you want things to look like, then the implementation tries to get it there
|
|
#
?
Jan 7, 2022 00:23
|
|
|
Silver Alicorn posted:I’ve read this entire thread and I still don’t know what a kubernete is the problem it's solving is you have a bunch of servers and you want to schedule some processes amongst them, but you don't really care which server each goes to and you want it to handle failing nodes or processes automatically. also you want them all to be isolated (this is done using containers) the non-trivial parts of this are generally things like persistent storage and setting things up so the processes can all talk to each other over IP (each one gets its own IP address)
|
|
#
?
Jan 7, 2022 00:25
|
|
|
Don't forget that whoever implemented your docker orchestration system doesn't understand that routing is a thing so all the IPs have to be in the same network. This is why you can only ever have one Availability Zone in AWS, it's just impossible to work in any other way.
|
|
#
?
Jan 7, 2022 01:42
|
|
|
ate poo poo on live tv posted:Don't forget that whoever implemented your docker orchestration system doesn't understand that routing is a thing so all the IPs have to be in the same network. This is why you can only ever have one Availability Zone in AWS, it's just impossible to work in any other way. this is a weirdly specific gripe that i dont understand
|
|
#
?
Jan 7, 2022 05:09
|
|
|
|
| # ? Apr 26, 2024 17:48 |
|
|
Silver Alicorn posted:I’ve read this entire thread and I still don’t know what a kubernete is its a bunch of golang and yaml meant to abstract away the chore of scheduling containers onto a server cluster. its poo poo
|
|
#
?
Jan 7, 2022 07:55
|
|
