The impact of this incident is very unevenly spread.
Over 10 years ago, Mozilla shipped OneCRL [1]. We remediated this incident in a couple of hours for Firefox users who were protected automatically without the need for updates. Chrome, Apple and Microsoft have similar tooling. Since then, we've built more advanced tooling to handle revocation at scale for leaf certs (CRLite [2]).
However, for the rest of the ecosystem, there has been little to no investment in revocation and for those clients, this is a massive problem. Hopefully, we'll see folks invest now. Efforts like upki [3] look promising.
Firefox Mobile 158.0 on Android 17 (Pixel 7) connects to all the demo sites with zero warnings. Any idea why the "automatic protection without updates" doesn't seem to be working with that combination?
I'm hoping some organizations will step forward to fund work on this.
(Note that curl on Ubuntu 26.10 now has CRLite support baked via upki -- unfortunately that doesn't help with this because AIUI CRLite is only for end entities, not for intermediate certificates.)
You mean you and Chrome stopped supporting normal CRLs and OCSP, replacing (for normal users) these distributed-trust technologies with central lists controlled by browsers
You mean the response was uneven. The impact was a complete and total potential compromise of literally everyone. Every domain. Every client using one of the public root programs. Every user of those clients.
I mean, it's an emergency, but the acute severity is debatable; many times a month the major browsers quietly deal with vulnerabilities that yield attackers direct control of victim computers; here, attackers get the ability to spoof websites if they're in a position to hijack connections.
The middlebox situation, with non-updatable roots, is very annoying. But middleboxes also have unfixable RCEs, so...
It's totally fair to evaluate different projects by how well positioned they are to update their WebPKI trust anchors. I'm just saying, keep perspective.
Keep in mind that a _lot_ of software out there uses TLS as its only trust anchor for updates, commands or similar.
This can and will be weaponized for take-overs of systems. They will be of the consumer kind, but they will also be of the industry kind. Let's pray that they will not be of the state or military kind.
So it other words, it allowed attackers to do the one thing that Web PKI is supposed to prevent.
Being able to quickly update trust roots is at best a hack to paper over the issue when there is no good reason why every podunk CA should have authority to generate certificates for every domain in the first place. It's an inexcusable design flaw. Or a feature that gives power to root stores at the expense of everyones security, from the perspective of the root store operators.
The Go and Rustls upgrades specifically block the intermediate certs because neither of these use the system revocation lists, I believe. Once you upgrade to a trust store without the root cert for the CA this will also solve the issue. Not sure about other ecosystems.
Linux - the platform which offers no ability to revoke an intermediate certificate, thus necessitating every certificate library to implement its own blocking?
Windows is already blocking the compromised CAs as of today with no need for users to upgrade anything.
Every certificate library has to implement its own blocking no matter which platform you run on. Windows provides an API to do this at the system level, which means nothing if a given application vendors their own TLS library.
It's uncommon on Windows for applications to use a third party certificate verification library because Windows provides an API that does it. On Linux, you have no choice because all the platform provides is a bundle of PEM certificates.
trying to understand where this service is mainly used. judging by a quick google search ...
is this related to Government of India related websites and their services/websites like tax filing etc? is the impact expected to be concentrated around indian citizens availing these services?
Web PKI is cursed. Why should anyone have to ship a new version of Go, Electron, webpki-roots, etc. (possibly over a now-compromised TLS connection!) just to revoke a certificate? Why are so many packages independently shipping high-sensitivity trust-bootstrap credentials? And why can any CA still vouch for any domain instead of each being limited to a specific ccTLD or other non-total shard of subject names?
Also, it's quietly, in the email space, that we're seeing real innovation while browser vendors just say "no" to everything. We should be distributing trust, not centralizing it in browser vendor security teams and Let's Encrypt.
Web PKI doesn't mandate that every client ship with its own root certs. The clients you listed can be configured to use a centrally managed trust store, if that's what you want. The fact that they don't by default is a deliberate tradeoff that the clients made, not some standard's fault.
This isn't, like, one misbehaving client. It's the way the ecosystem has evolved. GP didn't specify whether they were criticizing the standard or the ecosystem more broadly, but even if they were criticizing the standard, it's pretty fair to evaluate a standard at least partly based on how it gets implemented in practice. There's a reason cryptography tools for the last decade or more have been focused on being misuse-resistant.
Yes. I am criticizing the de facto web PKI, not the somewhat better design intent. The design intent is still bad though: we should never have given CAs universal issuance privileges.
The web has continued to go the wrong way in practice. (For example, browser makers rejected RFC9102 DNSSEC stapling, which could have enabled an incremental migration away from CA-centric hell.) Everything fix the existing web PKI community makes just seems to centralize trust more.
For example, Let's Encrypt has been useful for moving people off unencrypted services, but side it's now ubiquitous on the internet and had to work within the broken universal-grant CA model, it now represents a terrifying central point of failure. Had browser makers supported DANE instead, and had we embraced DNSSEC instead of casting aspersions on it, we wouldn't have needed Let's Encrypt because ownership of a domain name would have already, by that very fact, let you authenticate to anyone.
> The design intent is still bad though: we should never have given CAs universal issuance privileges.
This is sort of right and sort of wrong. It's true that you could have a DNSSEC-style regime where only one entity could issue for a given domain, but as long as you're going to allow third party issuance at all, then you have to contend with the fact that most CAs want to issue in the big gTLDs (.com, .org, etc.) and so narrowing issuance doesn't help that much in many cases.
> Had browser makers supported DANE instead, and had we embraced DNSSEC instead of casting aspersions on it, we wouldn't have needed Let's Encrypt because ownership of a domain name would have already, by that very fact, let you authenticate to anyone.
This is ahistorical. By the time that DANE was designed, the WebPKI model was well entrenched and there was a huge population of browsers that couldn't really be upgraded and also only supported the WebPKI. If you wanted something that was acceptable to those browsers, it had to be WebPKI and it had to be cross-signed, hence Let's Encrypt.
I suppose you could argue that at some point after that one could have tried to deploy DANE and phase out LE and the rest of the WebPKI. I don't think that's true for the reasons that have been discussed on HN about DNSSEC, but that's really not the same as you wouldn't have needed LE in the first place.
> An attacker with the private key of any of the compromised intermediate CAs could impersonate any website or service.
And how many more times do we need to read this before we replace Web PKI with a system that isn't as completely insane?
We already have a functioning system for delegating domain ownership in a hierarchical fashion where compromises are limited to only the subtree under the compromised entity. Let's start using it for domain validation sooner rather than later.
It's been obvious for a while now that root-of-trust schemes create juicy targets, and that juicy targets attract all sorts of bad behavior that wouldn't be economical if we used a web-of-trust.
Building a web-of-trust scheme that works might not be easy, but also it's not like we've been trying exceptionally hard lately either.
DANE replaces this system with one that has the same basic cryptographic properties, but in which you can't eject bad actors --- or even monitor them with a mandatory transparency log. It's strictly worse than this.
It has the property of aligning cryptographic authentication with the domain name hierarchy. Yes, you trust the hierarchy. You trust LE too, so what? We need to move away from any authority being able to authenticate anyone, and DNS naturally aligns with the goal.
You've been vocal for a long time against DNSSEC deployment too for no discernable reason.
Why are you against every single thing that tries to make DNS useful for securing anything?
People have debated DANE endlessly here, so I'm not sure how much more I have to add at this point. However, browsers, at least, do swiftly handle revocation for CAs, as indicated in this comment: https://news.ycombinator.com/item?id=50167244. They also handle EE revocation to varying extents, with Firefox having complete coverage and other browsers having partial coverage.
The impact of this incident is very unevenly spread.
Over 10 years ago, Mozilla shipped OneCRL [1]. We remediated this incident in a couple of hours for Firefox users who were protected automatically without the need for updates. Chrome, Apple and Microsoft have similar tooling. Since then, we've built more advanced tooling to handle revocation at scale for leaf certs (CRLite [2]).
However, for the rest of the ecosystem, there has been little to no investment in revocation and for those clients, this is a massive problem. Hopefully, we'll see folks invest now. Efforts like upki [3] look promising.
[1] https://blog.mozilla.org/security/2015/03/03/revoking-interm...
[2] https://hacks.mozilla.org/2025/08/crlite-fast-private-and-co...
[3] https://github.com/rustls/upki
Firefox Mobile 158.0 on Android 17 (Pixel 7) connects to all the demo sites with zero warnings. Any idea why the "automatic protection without updates" doesn't seem to be working with that combination?
I'm on the same Firefox version on android and they are blocked now
Pixel 10 with 158.0.1 still connects to the emSign ones..
More on upki:
https://discourse.ubuntu.com/t/an-update-on-upki/77063
My interest in building OneCRL support into upki has definitely increased this week. Issue:
https://github.com/rustls/upki/issues/229
I'm hoping some organizations will step forward to fund work on this.
(Note that curl on Ubuntu 26.10 now has CRLite support baked via upki -- unfortunately that doesn't help with this because AIUI CRLite is only for end entities, not for intermediate certificates.)
You mean you and Chrome stopped supporting normal CRLs and OCSP, replacing (for normal users) these distributed-trust technologies with central lists controlled by browsers
You mean the response was uneven. The impact was a complete and total potential compromise of literally everyone. Every domain. Every client using one of the public root programs. Every user of those clients.
I mean, it's an emergency, but the acute severity is debatable; many times a month the major browsers quietly deal with vulnerabilities that yield attackers direct control of victim computers; here, attackers get the ability to spoof websites if they're in a position to hijack connections.
The middlebox situation, with non-updatable roots, is very annoying. But middleboxes also have unfixable RCEs, so...
It's totally fair to evaluate different projects by how well positioned they are to update their WebPKI trust anchors. I'm just saying, keep perspective.
Keep in mind that a _lot_ of software out there uses TLS as its only trust anchor for updates, commands or similar.
This can and will be weaponized for take-overs of systems. They will be of the consumer kind, but they will also be of the industry kind. Let's pray that they will not be of the state or military kind.
So it other words, it allowed attackers to do the one thing that Web PKI is supposed to prevent.
Being able to quickly update trust roots is at best a hack to paper over the issue when there is no good reason why every podunk CA should have authority to generate certificates for every domain in the first place. It's an inexcusable design flaw. Or a feature that gives power to root stores at the expense of everyones security, from the perspective of the root store operators.
Here's a question for Mozilla et. al.:
It's cute you have a magic DIY CRL. You accepted these assholes into your root program.
They lost the KEYS.
What is the screening process for CA's that can't be bothered to use a HSM?
Even Honest Achmed wasn't this careless.
IIRC up until very recently the screening process was "they report that they're good boys."
From what the internet is telling me, the keys were found on https://repository.emsign.com/certs/<redacted>.crt
Which, if true, is definitely a lot of fun.
Hope you people are ready for lots and lots of suddenly hackable IoT.
Source? That would be an incredible blunder, but that also appears to be the kind of URL one would expect to find certs at.
That's actually a good point - tons of IoT is "secure enough" to sign things, but not secure enough to bother pinning anything ...
Also who hasn't left their private keys hanging around web-accessible, eh? lol
Running Go? Upgrade to 1.27.3 or 1.26.10.
Rust with rustls? Upgrade to rustls-webpki 0.103.16 and/or webpki-roots/webpki-root-certs 1.0.10.
Python? Upgrade certifi to 2026.10.19.
Homebrew ca-certificates? 2026-09-25_1.
Electron? 44.9.1, 43.8.1, or 42.11.16.
The Go and Rustls upgrades specifically block the intermediate certs because neither of these use the system revocation lists, I believe. Once you upgrade to a trust store without the root cert for the CA this will also solve the issue. Not sure about other ecosystems.
Windows? Upgrade to Linux.
Linux - the platform which offers no ability to revoke an intermediate certificate, thus necessitating every certificate library to implement its own blocking?
Windows is already blocking the compromised CAs as of today with no need for users to upgrade anything.
Every certificate library has to implement its own blocking no matter which platform you run on. Windows provides an API to do this at the system level, which means nothing if a given application vendors their own TLS library.
It's uncommon on Windows for applications to use a third party certificate verification library because Windows provides an API that does it. On Linux, you have no choice because all the platform provides is a bundle of PEM certificates.
are there any consequences to eMudhra?
trying to understand where this service is mainly used. judging by a quick google search ...
is this related to Government of India related websites and their services/websites like tax filing etc? is the impact expected to be concentrated around indian citizens availing these services?
The way the web works, any CA (Certificate Authority) can create TLS certificates for almost any website.
This means that the people with these keys could MITM almost any https connection people make, not only the ones that this CA issued.
"any CA (Certificate Authority) can create TLS certificates for almost any website."
This has always been incredibly stupid.
Why? How do you pin a website to a specific cert provider? And what happens when you want to switch?
It’s used by Indian companies and also they provide digital signature services. It’s a widely used platform.
Web PKI is cursed. Why should anyone have to ship a new version of Go, Electron, webpki-roots, etc. (possibly over a now-compromised TLS connection!) just to revoke a certificate? Why are so many packages independently shipping high-sensitivity trust-bootstrap credentials? And why can any CA still vouch for any domain instead of each being limited to a specific ccTLD or other non-total shard of subject names?
Also, it's quietly, in the email space, that we're seeing real innovation while browser vendors just say "no" to everything. We should be distributing trust, not centralizing it in browser vendor security teams and Let's Encrypt.
Web PKI doesn't mandate that every client ship with its own root certs. The clients you listed can be configured to use a centrally managed trust store, if that's what you want. The fact that they don't by default is a deliberate tradeoff that the clients made, not some standard's fault.
This isn't, like, one misbehaving client. It's the way the ecosystem has evolved. GP didn't specify whether they were criticizing the standard or the ecosystem more broadly, but even if they were criticizing the standard, it's pretty fair to evaluate a standard at least partly based on how it gets implemented in practice. There's a reason cryptography tools for the last decade or more have been focused on being misuse-resistant.
Yes. I am criticizing the de facto web PKI, not the somewhat better design intent. The design intent is still bad though: we should never have given CAs universal issuance privileges.
The web has continued to go the wrong way in practice. (For example, browser makers rejected RFC9102 DNSSEC stapling, which could have enabled an incremental migration away from CA-centric hell.) Everything fix the existing web PKI community makes just seems to centralize trust more.
For example, Let's Encrypt has been useful for moving people off unencrypted services, but side it's now ubiquitous on the internet and had to work within the broken universal-grant CA model, it now represents a terrifying central point of failure. Had browser makers supported DANE instead, and had we embraced DNSSEC instead of casting aspersions on it, we wouldn't have needed Let's Encrypt because ownership of a domain name would have already, by that very fact, let you authenticate to anyone.
> The design intent is still bad though: we should never have given CAs universal issuance privileges.
This is sort of right and sort of wrong. It's true that you could have a DNSSEC-style regime where only one entity could issue for a given domain, but as long as you're going to allow third party issuance at all, then you have to contend with the fact that most CAs want to issue in the big gTLDs (.com, .org, etc.) and so narrowing issuance doesn't help that much in many cases.
> Had browser makers supported DANE instead, and had we embraced DNSSEC instead of casting aspersions on it, we wouldn't have needed Let's Encrypt because ownership of a domain name would have already, by that very fact, let you authenticate to anyone.
This is ahistorical. By the time that DANE was designed, the WebPKI model was well entrenched and there was a huge population of browsers that couldn't really be upgraded and also only supported the WebPKI. If you wanted something that was acceptable to those browsers, it had to be WebPKI and it had to be cross-signed, hence Let's Encrypt.
I suppose you could argue that at some point after that one could have tried to deploy DANE and phase out LE and the rest of the WebPKI. I don't think that's true for the reasons that have been discussed on HN about DNSSEC, but that's really not the same as you wouldn't have needed LE in the first place.
Why don't these massive package managers use the host OS CA store, with an optional preconfigured "additional" store if clients need them?
Is the update cadence of CAs that different between the sources? Is the list of CAs included so different from OS vendors and the package managers?
> Also, it's quietly, in the email space, that we're seeing real innovation […]
Can you elaborate on this innovation in the email space around PKI?
> What are the consequences?
> An attacker with the private key of any of the compromised intermediate CAs could impersonate any website or service.
And how many more times do we need to read this before we replace Web PKI with a system that isn't as completely insane?
We already have a functioning system for delegating domain ownership in a hierarchical fashion where compromises are limited to only the subtree under the compromised entity. Let's start using it for domain validation sooner rather than later.
>And how many more times do we need to read this before we replace Web PKI with a system that isn't as completely insane?
You say this like it's easy and simple and we are simply not choosing to do so.
It's been obvious for a while now that root-of-trust schemes create juicy targets, and that juicy targets attract all sorts of bad behavior that wouldn't be economical if we used a web-of-trust.
Building a web-of-trust scheme that works might not be easy, but also it's not like we've been trying exceptionally hard lately either.
> We already have a functioning system for delegating domain ownership in a hierarchical fashion
Well, yeah. DANE should have won. NameConstraints would be nice tooo, but who's going to make CAs use it?
And FFS, clients could at least check CRLs once in a while.
DANE replaces this system with one that has the same basic cryptographic properties, but in which you can't eject bad actors --- or even monitor them with a mandatory transparency log. It's strictly worse than this.
It has the property of aligning cryptographic authentication with the domain name hierarchy. Yes, you trust the hierarchy. You trust LE too, so what? We need to move away from any authority being able to authenticate anyone, and DNS naturally aligns with the goal.
You've been vocal for a long time against DNSSEC deployment too for no discernable reason.
Why are you against every single thing that tries to make DNS useful for securing anything?
People have debated DANE endlessly here, so I'm not sure how much more I have to add at this point. However, browsers, at least, do swiftly handle revocation for CAs, as indicated in this comment: https://news.ycombinator.com/item?id=50167244. They also handle EE revocation to varying extents, with Firefox having complete coverage and other browsers having partial coverage.