It's actually fun to see this. Running systems in a lot of different ways are just interesting. I do however get kinda sad at the hate at k8s because it's really good at what it does.
I've seen so many projects bending over backwards to avoid k8s and pay large cloud bills to avoid it at all costs. (ECS and app services are hopelessly expensive and bothersome)
K8s is really good, pretty easy to maintain, but a bit hard to understand. Mostly because distributed, zero down time systems are a bit hard to get by nature. But if you have someone that wants to take it own I've managed k8s clusters, solo, without incident, while doing lots of other stuff too (working with larger teams now though). Not to mention there's a lot of competence out there that can take over if I'd move on.
That said, no h8 towards going your own way! If your a solo developer (or small team) for a smallish project, don't feel the absolute need. If you get to the point you need it you should be earning enough to start paying someone to help ya get your app to a distributed system like k8s.
I think it's good to invest the time in understanding k8s though as a professional. Even if you won't directly run it it teaches you a lot about how to think about distributed, zero downtime systems. And what requirements that puts on an app.
I‘ve built this anti-k8s stance pre-LLMs, and just realized that, actually, agents should be pretty helpful in dealing with it? Is avoiding kubernetes still advisable for projects that will likely never use its full complexity, given how easy it is to maintain now?
There was an article a while back on HN about why web designers choose light grey on white background. Basically, it looks fine on their own monitor which has the contrast turned way up
The css is ".prose-invert" and there's a ".prose" that looks better, I wonder if something threw a switch to make it "invert" when it should be ".prose" because you're right, this is unreadable as-is. Interesting read though.
Its black background and white text on my screen. So either the OP saw the comments and fixed it or some people in this thread have weird settings. Or maybe I have weird settings...hmmm.
so op just recreated with sticks and tape a very basic feature what k8s does out of the box, and nobody else would be able to support his creation, because its handrolled adhoc with sparse documentation.
It's totally fine to not to use k8s. Personally, I think I have made several good decisions in my career to use/avoid k8s in different scenarios.
But if someone wrote a blog to brag about not using k8s, they can't stop people from wanting to compare their work against k8s. If there's any arrogance in the air, it feels stronger on the other side.
I worked at a place (a big name in a given vertical!), where the SRE looked at K8S and said "hold my beer".
Out came Docker, dnsmasq, miles of duct tape and a whole lot of swearing. Just to come nowhere close to reinventing something better folks were doing years prior.
Just because you can (or think you can) doesn't mean you should. I sure do hope no one is maintaining that NIH monstrosity now!
If this creation stays as is, then it's not very complicated and pretty easy to understand and support. But that is a big IF, and very likely won't be true. Over time people will add more useful features to it, then it becomes another Kubernetes (and if you don't have a strong engineering team, it will probably be much worse than Kubernetes).
Perhaps we should look the other way: why use K8s if podman-compose can do the same? Maybe we should deprecate it and move towards simpler and more robust solutions?
That isn’t a lot. You could easily run that from one host. The reason people reach for Kubernetes (and similar) is because they need to scale past that single host dependency.
You could, but they don't, meaning their argument is still sound (whether they _could_ use a single host is besides the point, they're not doing that).
The reason most people reach for Kubernetes is because it's cool. The entire infra the vast majority of Kubernetes users have could run on a single bare metal machine with a second one for redundancy.
To be fair: using Kubernetes anyways builds the skill just in case you become one of the 0.1% who actually need it down the line.
You can hire an Azure or Google Kubernetes devops guy and he will be equally comfortable on your AWS EKS kubernetes cluster. And when he leaves, you don't have a six week onboarding process with the new guy to learn all the ins and outs of your totally bespoke, non-standard container orchestration system that was cobbled together by two devs with no operations experience.
K3S takes about 5 minutes to setup the first time and you instantly have an entire universe of standardized operational tooling. I wouldn't touch docker compose with a 20 foot pole for production work.
As soon as you work in a team, it’s irrelevant whether the project actually needs it. There will be someone who convinces stakeholders that it is necessary and then you just have to fall in line and learn the skills knowing that it is most likely one of the 99.9% of projects where it is just overkill.
Until your project has some success, and it turns out all those "complex" features actually turn out to be extremely useful.
Which is exactly what is happening with us, too bad we didn't choose K8S from the get-go and stuck with a "simpler" tool (gaining very little in the process).
There's a mass delusion in the industry that using Kubernetes has to be hard, grossly over complex, and is always wrong if you are not Netflix scale. Is the system as described by the author significantly less complex and better than a small Kubernetes system? Sounds like they went through a lot of work to get it to their desired state.
Rolling your own zero-downtime deployments is as about as a good idea as rolling your own security... it's not a good idea.
I run our small system on a single EC2 instance with K3s. It runs a half-dozen or so services and does it quite well. I don't think it is particularly complex or over engineered. I like how easy it is to maintain the configuration in a helm package and quickly deploy it in different environments.
There is a learning curve for doing K8s well, but that's true for any non-trivial system.
> There's a mass delusion in the industry that you need Kubernetes to run a serious production service. You don't. At StatusDude, we serve thousands of monitoring checks per minute, run multi-region workers, and deploy multiple times a day
This is pretty small scale, Kubernetes comes in when you've got a larger workload.
“We don’t know how to scale Traefik so we went with haproxy”
Well doh. Haproxy is designed for this. You can make haproxy serve copious amount of traffic on a single arm core and a little bit of ram. Imagine what you can do with a few replicas on your large clusters.
This has nothing to do with the choice of CI/CD or docker versus kubernetes.
While I agree with you I'm not sure the rest of the world does.
Over the past decade, I'm seeing k8s used everywhere for everything, companies setting up clusters to run literally one simple app with couple of hundred requests per hour.
In theory. I read so many times now where people report they use it and don't really need it, and I've seen it myself now too. Still very anecdotal, but it seems somethings there
>P.S Nginx would do too, I just felt like getting haproxy up this time :)
I think this line summarizes better than anything. Perfect example of how move fast and break things begins gloriously at first then inevitably, the breaking you thought you were doing hasn't even started and you find out what that part means.
I've always been the one saying "this is going to be a problem in a couple months" then I get shot down for "being negative." Then in a couple of of months when it fails I start getting aggression thrown at me "oh i know you want to say I told you so" and such even though I've never said such a thing when something fails. No. I would just like you to hear out my thoughts even when they may not be what you want to hear. We are all working towards the same goal.
It's actually fun to see this. Running systems in a lot of different ways are just interesting. I do however get kinda sad at the hate at k8s because it's really good at what it does.
I've seen so many projects bending over backwards to avoid k8s and pay large cloud bills to avoid it at all costs. (ECS and app services are hopelessly expensive and bothersome)
K8s is really good, pretty easy to maintain, but a bit hard to understand. Mostly because distributed, zero down time systems are a bit hard to get by nature. But if you have someone that wants to take it own I've managed k8s clusters, solo, without incident, while doing lots of other stuff too (working with larger teams now though). Not to mention there's a lot of competence out there that can take over if I'd move on.
That said, no h8 towards going your own way! If your a solo developer (or small team) for a smallish project, don't feel the absolute need. If you get to the point you need it you should be earning enough to start paying someone to help ya get your app to a distributed system like k8s.
I think it's good to invest the time in understanding k8s though as a professional. Even if you won't directly run it it teaches you a lot about how to think about distributed, zero downtime systems. And what requirements that puts on an app.
Mandatory mention: https://www.macchaffee.com/blog/2024/you-have-built-a-kubern...
But yeah, pretty cool DNS resolving features in HAProxy, that's nifty
I‘ve built this anti-k8s stance pre-LLMs, and just realized that, actually, agents should be pretty helpful in dealing with it? Is avoiding kubernetes still advisable for projects that will likely never use its full complexity, given how easy it is to maintain now?
I remember a time where HN was quite critical to the complexity of k8s. After reading top comments, I can see the tide has shifted.
This or there's a lot of fresh blood ;-)
What's the thought behind having white and light grey text on a light grey background?
There was an article a while back on HN about why web designers choose light grey on white background. Basically, it looks fine on their own monitor which has the contrast turned way up
The css is ".prose-invert" and there's a ".prose" that looks better, I wonder if something threw a switch to make it "invert" when it should be ".prose" because you're right, this is unreadable as-is. Interesting read though.
Are the people here not looking at the article or bots? How on Earth does anyone read this?
Guessing it’s reading the system color scheme, because on my phone it’s white text on black.
(on iOS with dark mode enabled system wide)
Oh god, I just disabled dark mode and yeah, it looks awful. Looks great in dark mode though.
I tweaked the css in my browser with developer tools.
It’s white text on black bg for me. But to answer your question- reader mode!
Their dark mode is busted if you don't have JS enabled.
Looks like “dark mode” implementation attempt which missed setting the background to black.
Its black background and white text on my screen. So either the OP saw the comments and fixed it or some people in this thread have weird settings. Or maybe I have weird settings...hmmm.
so op just recreated with sticks and tape a very basic feature what k8s does out of the box, and nobody else would be able to support his creation, because its handrolled adhoc with sparse documentation.
sounds like ghetto engineering
You'd be amazed to learn how much of the world is running just fine free of k8s and arrogance
It's totally fine to not to use k8s. Personally, I think I have made several good decisions in my career to use/avoid k8s in different scenarios.
But if someone wrote a blog to brag about not using k8s, they can't stop people from wanting to compare their work against k8s. If there's any arrogance in the air, it feels stronger on the other side.
I am not sure the argument "muhh k8s is too complex, I will roll docker instead" flies well in the age of cloud managed k8s offerings
EKS is literally one click of a button away and you dont need to handroll this.
even if you dont know AWS console nor terraform, claude code with aws mcp can do that for you
The problem I have with Kunernetes in general is that everything is slow and configuring everything seems needlessly complex in common use cases.
I worked at a place (a big name in a given vertical!), where the SRE looked at K8S and said "hold my beer".
Out came Docker, dnsmasq, miles of duct tape and a whole lot of swearing. Just to come nowhere close to reinventing something better folks were doing years prior.
Just because you can (or think you can) doesn't mean you should. I sure do hope no one is maintaining that NIH monstrosity now!
If this creation stays as is, then it's not very complicated and pretty easy to understand and support. But that is a big IF, and very likely won't be true. Over time people will add more useful features to it, then it becomes another Kubernetes (and if you don't have a strong engineering team, it will probably be much worse than Kubernetes).
Perhaps we should look the other way: why use K8s if podman-compose can do the same? Maybe we should deprecate it and move towards simpler and more robust solutions?
If all you have is a hammer, everything looks like a nail.
> thousands of monitoring checks per minute
That isn’t a lot. You could easily run that from one host. The reason people reach for Kubernetes (and similar) is because they need to scale past that single host dependency.
> The reason people reach for Kubernetes (and similar) is because they need to scale past that single host dependency.
I have some stuff on single-node k3s. Because it's standard so I don't have to care.
100%. And a shared mental model. I love how I can scale up all my services the same way, across clouds.
It's great.
You could, but they don't, meaning their argument is still sound (whether they _could_ use a single host is besides the point, they're not doing that).
..and HA
The reason most people reach for Kubernetes is because it's cool. The entire infra the vast majority of Kubernetes users have could run on a single bare metal machine with a second one for redundancy.
To be fair: using Kubernetes anyways builds the skill just in case you become one of the 0.1% who actually need it down the line.
You can hire an Azure or Google Kubernetes devops guy and he will be equally comfortable on your AWS EKS kubernetes cluster. And when he leaves, you don't have a six week onboarding process with the new guy to learn all the ins and outs of your totally bespoke, non-standard container orchestration system that was cobbled together by two devs with no operations experience.
K3S takes about 5 minutes to setup the first time and you instantly have an entire universe of standardized operational tooling. I wouldn't touch docker compose with a 20 foot pole for production work.
Docker compose is hardly "totally bespoke".
Setting up K8s isn't rocket science, but maintaining it are offputting, to say the least.
As soon as you work in a team, it’s irrelevant whether the project actually needs it. There will be someone who convinces stakeholders that it is necessary and then you just have to fall in line and learn the skills knowing that it is most likely one of the 99.9% of projects where it is just overkill.
Until your project has some success, and it turns out all those "complex" features actually turn out to be extremely useful.
Which is exactly what is happening with us, too bad we didn't choose K8S from the get-go and stuck with a "simpler" tool (gaining very little in the process).
> The reason most people reach for Kubernetes is because it's cool.
This shittake was probably valid 10y ago, I would have agreed with you back then
> The entire infra the vast majority of Kubernetes users have could run on a single bare metal machine
Where are you pulling this out of? A large number of k8s users don't need it, but the alternative you have sounds hyperbolic.
Are you monitoring resource utilization per container? Do notifications get sent out when container(s) become unhealthy? How are you handling secrets?
These are things I'm trying to figure out at work using Podman. Would love to hear about any experience in these areas.
docker-rollout also works well: https://github.com/wowu/docker-rollout
The readme covers connection draining with Traefik which should solve one of the issues the author mentions
This just feels like mostly a complaint of missing features in Traefik.
There's a mass delusion in the industry that using Kubernetes has to be hard, grossly over complex, and is always wrong if you are not Netflix scale. Is the system as described by the author significantly less complex and better than a small Kubernetes system? Sounds like they went through a lot of work to get it to their desired state.
Rolling your own zero-downtime deployments is as about as a good idea as rolling your own security... it's not a good idea.
I run our small system on a single EC2 instance with K3s. It runs a half-dozen or so services and does it quite well. I don't think it is particularly complex or over engineered. I like how easy it is to maintain the configuration in a helm package and quickly deploy it in different environments.
There is a learning curve for doing K8s well, but that's true for any non-trivial system.
>>zero-downtime
the site is down for me.
> There's a mass delusion in the industry that you need Kubernetes to run a serious production service. You don't. At StatusDude, we serve thousands of monitoring checks per minute, run multi-region workers, and deploy multiple times a day
This is pretty small scale, Kubernetes comes in when you've got a larger workload.
The whole reasoning behind this is flawed.
“We don’t know how to scale Traefik so we went with haproxy”
Well doh. Haproxy is designed for this. You can make haproxy serve copious amount of traffic on a single arm core and a little bit of ram. Imagine what you can do with a few replicas on your large clusters.
This has nothing to do with the choice of CI/CD or docker versus kubernetes.
While I agree with you I'm not sure the rest of the world does.
Over the past decade, I'm seeing k8s used everywhere for everything, companies setting up clusters to run literally one simple app with couple of hundred requests per hour.
In theory. I read so many times now where people report they use it and don't really need it, and I've seen it myself now too. Still very anecdotal, but it seems somethings there
>P.S Nginx would do too, I just felt like getting haproxy up this time :)
I think this line summarizes better than anything. Perfect example of how move fast and break things begins gloriously at first then inevitably, the breaking you thought you were doing hasn't even started and you find out what that part means.
I've always been the one saying "this is going to be a problem in a couple months" then I get shot down for "being negative." Then in a couple of of months when it fails I start getting aggression thrown at me "oh i know you want to say I told you so" and such even though I've never said such a thing when something fails. No. I would just like you to hear out my thoughts even when they may not be what you want to hear. We are all working towards the same goal.