Wow, "hardened image" market is getting saturated. I saw atleast 3 companies offering this at Kubecon.
Chainguard came to this first (arguably by accident since they had several other offerings before they realized that people would pay (?!!) for a image that reported zero CVEs).
In a previous role, I found that the value for this for startups is immense. Large enterprise deals can quickly be killed by a security team that that replies with "scanner says no". Chainguard offered images that report 0 CVEs and would basically remove this barrier.
For example, a common CVE that I encountered was a glibc High CVE. We could pretty convincingly show that our app did not use this library in way to be vulnerable but it didn't matter. A high CVE is a full stop for most security teams. Migrated to a Wolfi image and the scanner reported 0. Cool.
But with other orgs like Minimus (founders of Twistlock) coming into this it looks like its about to be crowded.
There is even a govt project called Ironbank to offer something like this to the DoD.
Net positive for the ecosystem but I don't know if there is enough meat on the bone to support this many vendors.
Most likely yes. There are a lot enterprises out there that only trust paid subscriptions.
Paying for something “secure” comes with the benefit of risk mitigation - we paid X to give us a secure version of Y, hence its not our fault “bad thing” happenned.
Yep differentiation is tricky here. Chainguard are expanding out to VM images and programming language repos, but the core of hardened container images has a lot of options.
The question I'd be interested in is, outside of markets where there's a lot of compliance requirements, how much demand is there for this as a paid service...
People like lower CVE images, but are they willing to pay for them. I guess that's an advantage for Docker's offering. If it's free there is less friction to trying it out compared to a commercial offering.
First look shows me that this is not an easy drop in replacement. First thing is this requires a log-in and makes me wonder why this is required. Perhaps some upselling coming.
With Bitnami discontinuing their offer, we recently switched to other providers. For some we are using a helm chart and this new offer provides some helm charts but for some software just the image. I would be interested to give this a try but e.g. the python image only various '(dev)' images while the guide mentions the non-dev images. So this requires some planning.
EDIT: Digging deeper, I notice it requires a PAT and a PAT is bound to a personal account. I guess you need the enterprise offering for organisation support. I am not going to waste my time to contact them for an enterprise offer for a small start-up. What is the use case for CVE hardened images that you cannot properly run in an CICD and only on your dev machine? Are there companies that need to follow compliance rules or need this security guarantee but don't have CICD in place?
The enterprise hardened images license seems to be a different offering for offline mirroring or more strict compliance…
The main reason for CVE hardened images is that it’s hard to trust individuals to do it right at scale, even with CI/CD. You’re having to wire together your own scan & update process. In practice teams will use pinned versions, delays in fixing, turn off scanning, etc. This is easy mode
The news: Docker Hardened Images (DHI) are now free to use for everyone. No reason not to use them.
Offering image hardening to custom images looks like a reasonable way for Docker to have a source of sustained income. Regulated industries like banks, insurers, or governmental agencies are likely interested.
After their last rug pull when they started charging projects for registry after parading it as a fully free service for almost a decade, it has become hard to trust anything free.
Bait and switch once the adoption happens has become way too common in the industry.
Docker is a company I just can’t hate on. They’ve completely transformed how software is deployed. Containers gained so much momentum it kind of outgrew them and they lost a lot of potential business. I would hardly call beginning to charge after a decade of free service a rug pull, especially now that dependence on Docker’s registry is shrinking all the time.
Given the wealth and productivity creation that they're responsible for enabling across the industry, they deserve to be paid for it. There is no way for them to have achieved this with zero friction.
For oss projects with heavy pulls, the (free) dsos programme removes all rate limits on their public images, the intention was never to impact projects, but rather mega corporations using hub as free hosting:
> 100 pulls per 6 hours for unauthenticated users and 200 pulls per 6 hours for Docker Personal users
Not a problem for casual users but even a small team like mine, a dozen people with around a dozen public images, can hit the pull limit deploying a dozen landscapes a day. We just cache all the public images ourselves and avoid it.
I am a little confused because I got a 401 when I tried to pull an image from there. Do I need a login or something? For a free image it sure doesn't feel that way.
There's an excellent reason: They're login gated, which is at best unnecessary friction. Took me straight from "oh, let me try it" to "nope, not gonna bother".
hardened images are cool, definitely, but I'm not sure what it actually means? just systems with the latest patches or stricter config rules as well?for example: would any of these images have mitigated or even prevented Shai-Hulud [12]?
Docker Hardened Images are built from scratch with the minimal packages to run the image. The hardened images didn't contain any compromised packages for Shai-Hulud.
Hardened base images don't restrict what you add on top of them. That's where scanners like Docker Scout, Trivy, Grype, and more come in to review the complete image that you have built.
My guess is that it's a response to "Chainguard are growing so fast that VCs have fought each other to give them hundreds of millions in 3 years despite having no AI play".
CVE response time is a toss up, they all patch fast. Chainguard can only guarantee zero active exploits because they control their own exploit feed, and don't publish anything on it until they've patched. So while this makes it look better, it may not actually be better
I work at Chainguard. We don't guarantee zero active exploits, but we do have a contractual SLA we offer around CVE scan results (those aren't quite the same thing unfortunately).
We do issue an advisory feed in a few versions that scanners integrate with. The traditional format we used (which is what most scanners supported at the time) didn't have a way to include pending information so we couldn't include it there.
The basic flow was: scanner finds CVE and alerts, we issue statement showing when and where we fixed it, the scanner understands that and doesn't show it in versions after that.
so there wasn't really a spot to put "this is present", that was the scanner's job. Not all scanners work that way though, and some just rely on our feed and don't do their own homework so it's hit or miss.
We do have another feed now that uses the newer OSV format, in that feed we have all the info around when we detect it, when we patch it, etc.
FWIW - A whole host of the pre-IPO GitLab folks went to Chainguard. A lot of them, many in leadership roles. Most importantly, In Sales Leadership. These are people whom don’t really believe in high-pressure sales. Rather they aim to show the value and not squeeze customers for profit or making a number on a chart go up.
Wow, "hardened image" market is getting saturated. I saw atleast 3 companies offering this at Kubecon.
Chainguard came to this first (arguably by accident since they had several other offerings before they realized that people would pay (?!!) for a image that reported zero CVEs).
In a previous role, I found that the value for this for startups is immense. Large enterprise deals can quickly be killed by a security team that that replies with "scanner says no". Chainguard offered images that report 0 CVEs and would basically remove this barrier.
For example, a common CVE that I encountered was a glibc High CVE. We could pretty convincingly show that our app did not use this library in way to be vulnerable but it didn't matter. A high CVE is a full stop for most security teams. Migrated to a Wolfi image and the scanner reported 0. Cool.
But with other orgs like Minimus (founders of Twistlock) coming into this it looks like its about to be crowded.
There is even a govt project called Ironbank to offer something like this to the DoD.
Net positive for the ecosystem but I don't know if there is enough meat on the bone to support this many vendors.
The real question isn't whether the market is saturated, it's whether it still exists once Docker gives away the core value prop for free.
Most likely yes. There are a lot enterprises out there that only trust paid subscriptions.
Paying for something “secure” comes with the benefit of risk mitigation - we paid X to give us a secure version of Y, hence its not our fault “bad thing” happenned.
Yep differentiation is tricky here. Chainguard are expanding out to VM images and programming language repos, but the core of hardened container images has a lot of options.
The question I'd be interested in is, outside of markets where there's a lot of compliance requirements, how much demand is there for this as a paid service...
People like lower CVE images, but are they willing to pay for them. I guess that's an advantage for Docker's offering. If it's free there is less friction to trying it out compared to a commercial offering.
First look shows me that this is not an easy drop in replacement. First thing is this requires a log-in and makes me wonder why this is required. Perhaps some upselling coming.
With Bitnami discontinuing their offer, we recently switched to other providers. For some we are using a helm chart and this new offer provides some helm charts but for some software just the image. I would be interested to give this a try but e.g. the python image only various '(dev)' images while the guide mentions the non-dev images. So this requires some planning.
EDIT: Digging deeper, I notice it requires a PAT and a PAT is bound to a personal account. I guess you need the enterprise offering for organisation support. I am not going to waste my time to contact them for an enterprise offer for a small start-up. What is the use case for CVE hardened images that you cannot properly run in an CICD and only on your dev machine? Are there companies that need to follow compliance rules or need this security guarantee but don't have CICD in place?
I think Docker for Teams is $15/month per seat. https://www.docker.com/pricing/
The enterprise hardened images license seems to be a different offering for offline mirroring or more strict compliance…
The main reason for CVE hardened images is that it’s hard to trust individuals to do it right at scale, even with CI/CD. You’re having to wire together your own scan & update process. In practice teams will use pinned versions, delays in fixing, turn off scanning, etc. This is easy mode
The news: Docker Hardened Images (DHI) are now free to use for everyone. No reason not to use them.
Offering image hardening to custom images looks like a reasonable way for Docker to have a source of sustained income. Regulated industries like banks, insurers, or governmental agencies are likely interested.
After their last rug pull when they started charging projects for registry after parading it as a fully free service for almost a decade, it has become hard to trust anything free.
Bait and switch once the adoption happens has become way too common in the industry.
Docker is a company I just can’t hate on. They’ve completely transformed how software is deployed. Containers gained so much momentum it kind of outgrew them and they lost a lot of potential business. I would hardly call beginning to charge after a decade of free service a rug pull, especially now that dependence on Docker’s registry is shrinking all the time.
I don't hate them. But I don't want to depend on them for any product I manage.
Have you checked out Dagger?
It's what the people who created OG Docker are building now
Dagger is one of those things I want to like, but find incredibly painful to use in practice.
Given the wealth and productivity creation that they're responsible for enabling across the industry, they deserve to be paid for it. There is no way for them to have achieved this with zero friction.
Feels like they're trying to put the cat back in the bag and recoup a fraction of the exodus from the registry thing.
Projects are not charged for hub usage
When was this?
https://www.docker.com/developers/free-team-faq/
> Is Docker sunsetting the Free Team plan?
> No. Docker communicated its intent to sunset the Docker Free Team plan on March 14, 2023, but this decision was reversed on March 24, 2023.
For oss projects with heavy pulls, the (free) dsos programme removes all rate limits on their public images, the intention was never to impact projects, but rather mega corporations using hub as free hosting:
https://www.docker.com/community/open-source/application/
> 100 pulls per 6 hours for unauthenticated users and 200 pulls per 6 hours for Docker Personal users
Not a problem for casual users but even a small team like mine, a dozen people with around a dozen public images, can hit the pull limit deploying a dozen landscapes a day. We just cache all the public images ourselves and avoid it.
https://www.docker.com/blog/revisiting-docker-hub-policies-p...
I am a little confused because I got a 401 when I tried to pull an image from there. Do I need a login or something? For a free image it sure doesn't feel that way.
> No reason not to use them.
There's an excellent reason: They're login gated, which is at best unnecessary friction. Took me straight from "oh, let me try it" to "nope, not gonna bother".
This smells like LLM generated
hardened images are cool, definitely, but I'm not sure what it actually means? just systems with the latest patches or stricter config rules as well?for example: would any of these images have mitigated or even prevented Shai-Hulud [12]?
Docker Hardened Images are built from scratch with the minimal packages to run the image. The hardened images didn't contain any compromised packages for Shai-Hulud.
https://www.docker.com/blog/security-that-moves-fast-dockers...
Note: I work at Docker
yeah, but if you would have installed with npm your software, would the postinstall script have been executed?
Hardened base images don't restrict what you add on top of them. That's where scanners like Docker Scout, Trivy, Grype, and more come in to review the complete image that you have built.
Is this the response to the Bitnami/VMWare/Broadcom Helm charts thing?
My guess is that it's a response to "Chainguard are growing so fast that VCs have fought each other to give them hundreds of millions in 3 years despite having no AI play".
I appreciate what they're doing here, which is something I haven't seen other vendors doing.
Thanks for only doing this like, ten years later after all the damage is done.
I went to "Hardened Images Catalog" and searched for pgbouncer, not found (https://hub.docker.com/hardened-images/catalog?search=pgboun...)
There's a "Make a request" button, but it links to this 404-ing GitHub URL: https://github.com/docker-hardened-images/discussion/issues
oh well. hope its good stuff otherwise.
Thanks for reporting, team is fixing it, the right url is: https://github.com/docker-hardened-images/catalog/issues/
At $work, we switched everything to Redhat’s ubi images (micro and minimal) for that.
But, we pay for support already.
Nice from docker!
no need for chainguard/bitnami anymore?
Bitnami is in broadcom hell, nobody should use that.
Chainguard still has better CVE response time and can better guarantee you zero active exploits found by your prod scanners.
(No affiliation with either, but we use chainguard at work, and used to use bitnami too before I ripped it all out)
CVE response time is a toss up, they all patch fast. Chainguard can only guarantee zero active exploits because they control their own exploit feed, and don't publish anything on it until they've patched. So while this makes it look better, it may not actually be better
Hey!
I work at Chainguard. We don't guarantee zero active exploits, but we do have a contractual SLA we offer around CVE scan results (those aren't quite the same thing unfortunately).
We do issue an advisory feed in a few versions that scanners integrate with. The traditional format we used (which is what most scanners supported at the time) didn't have a way to include pending information so we couldn't include it there.
The basic flow was: scanner finds CVE and alerts, we issue statement showing when and where we fixed it, the scanner understands that and doesn't show it in versions after that.
so there wasn't really a spot to put "this is present", that was the scanner's job. Not all scanners work that way though, and some just rely on our feed and don't do their own homework so it's hit or miss.
We do have another feed now that uses the newer OSV format, in that feed we have all the info around when we detect it, when we patch it, etc.
All this info is available publicly and shown in our console, many of them you can see here: https://github.com/wolfi-dev/advisories
You can take this example: https://github.com/wolfi-dev/advisories/blob/main/amass.advi... and see the timestamps for when we detected CVEs, in what version, and how long it took us to patch.
FWIW - A whole host of the pre-IPO GitLab folks went to Chainguard. A lot of them, many in leadership roles. Most importantly, In Sales Leadership. These are people whom don’t really believe in high-pressure sales. Rather they aim to show the value and not squeeze customers for profit or making a number on a chart go up.
Do with that knowledge what you may.
Thanks for sharing. This kind of "color" isn't always easy to ascertain, but (for me, at least) it plays a part in vendor selection.