The nice thing about Python packaging is that it’s standardized: if uv’s build backend doesn’t fit into the hole you want, then you can use any number of other backends that do an admirable job of conforming to the packaging standards.
With that said, I don’t understand why Gentoo needs to rebuild every single release of uv-build. It doesn’t change as much as uv itself, and it’s intentionally compatible across uv versions. Why not set a policy of only updating once weekly, or even less frequently, with exceptions for important breaking changes or bug fixes?
(For context: the only reason uv-build releases as often as it does is because it’s easier to keep its internals in lockstep with the rest of uv. But you can do a normal version bound in your Python build backend metadata and uv will happily use whatever version of uv-build is already present.)
> The nice thing about Python packaging is that it’s standardized
But it isn't. Once upon a time, it was: you used distutils. But of course distutils had a lot of problems. Instead of adopting a single standardized solution for those problems, the Python developers punted and let third parties do it. So now we do not have a single standardized solution for building Python packages; we have a bunch of third party solutions, none of which is in the Python standard library.
Yes, all those third party solutions have to be PEP 517 compatible, but that's not standardization on a tool; it's just standardization on an interface between the backend and the frontend. Which is nice, but doesn't solve the problem under discussion here.
> if uv’s build backend doesn’t fit into the hole you want, then you can use any number of other backends that do an admirable job of conforming to the packaging standards.
As the project developer, sure. But as a Linux distro building your python package, Gentoo has to use the build backend you specify. It can't say, well, this particular package is just pure python, so really all we need is a minimal tool to build a pure python none-any wheel. It has to use the backend you specified.
> I don’t understand why Gentoo needs to rebuild every single release of uv-build.
Because if you don't pin a version of uv-build in your project, that means "use the latest version available". Gentoo can't second guess you--even if it were able to verify somehow that the new version of uv-build that just got released will end up producing an identical wheel for your project as the previous version, without actually using the new version of uv-build.
> you can do a normal version bound in your Python build backend metadata
Sure, but how many users of uv-build actually do that? Seems like the answer to that is "not enough to matter".
The problem is packages that use the build backends, and Gentoo is generally not in the business of maintaining that kind of patch.
Admittedly rustification like that is a problem not only for gentoo, I have commercially deployed systems where one of the most annoyings elements of updating is presence of rust in python deps...
By a search, Gentoo accounts for less than 1% of all developers. If uv-build is more conformant to people, probably volunteers (and if they're using it, it probably is), I don't think the Gentoo people have any right to want them not to use it because it makes their lives harder. They can request, and they do request nicely, but they also present the drawbacks for them as if they are absolute, and even claim that "its benefits don’t outweigh the drawbacks".
I'm sure it accounts for a much larger proportion of people using exotic or legacy systems. In any case, I think most people are using uv-build either because it's a default for uv now or just because it's hip and new. If a project absolutely needed it then so be it, but if you knew it was going to make life difficult even just for one person and no benefit to you then you'd probably choose something else.
UV build is distributed as a binary/executable, e.g. manylinux wheels on pypi: https://pypi.org/project/uv-build/#files. If a user or distro's conventions are to build things from source, this is a consequence they have implicitly accepted. For the vast majority of computer users and uses, installing binaries makes more sense.
Looks like it's being built with fat LTO by default (and gentoo patched to thin LTO). Which is fine if you want to spend CI cycles once to ship executables to thousands of users.
If you compile from source and run the tool maybe once then they should consider going further and run with maybe opt-level=1, lto=off; halfway between release and debug. This is how rustc itself builds some internal buildtools that get recompiled frequently relative to how often they run.
I'm not sure that skipping SPARC is a bad thing. Who even has SPARC machines sitting around to validate builds? Even Fujitsu moved to ARM for their later cores.
i got a debian maintainer for old IBM mainframes asking me to fix bugs and run tests for that architecture. ¯\_(ツ)_/¯ these people exist, but i'm not going to support them. feel free to fork me!
I don't think I agree with the author's assessment here:
> The other reason is performance and that argument makes even less sense. Even if it mattered, we’re realistically talking about a difference in the ballpark of 100 ms, for an action executed rarely; it is equally possible that any gain will be outweighed by the time needed to fetch the 1.5 MiB wheel for uv-build.
Modern CI architectures are building Docker stacks from scratch and using a readthrough cache of wheels. Downloading may still be the dominant factor, but the second-dominant factor does start to add up, especially when the downloading gets cached at various layers.
Yes, optionally apparently but I think we'll end up with forks of Python if the Rust enthusiasts push it any further. I think the option will be off for me.
I think mgorny is aware of that; he’s been active on the relevant DPO threads.
(At the end of the day, this is exactly what support tiers are for. CPython, at least to my eyes, has been very transparent about not guaranteeing support for every platform that has a C compiler.)
Maybe it's time to grow up and download binary builds like a normal person. I had my 'let's compile the linux kernel from source for 3.5% hypothetical performance gains' phase but you can't expect professional devs to cater to source-obsessed hobbyists
I doubt this oft-repeated stereotype was every particularly accurate. I'm not aware of any current Gentoo users on the distro specifically for performance reasons. Gentoo users are in it for control over their system and customization.
Gentoo’s decision to support source builds seems eminently reasonable to me, and I don’t think we should reward it with condescension.
The bigger problem, to me, is that it’s not really clear why this is actually a problem for Gentoo. I am glad they perform user-facing source builds, however, because they see things that other distributors don’t.
Gentoo made a lot of sense WAY back in a pre-docker world, where you had to run the whole world on the host OS. Being able to compile it all meant you could link it all together with whatever weird combination of optional libraries you wanted. Things like getting sendmail to auth over LDAP, that nobody packaged in the base distro, become possible. And with Gentoo's tooling, you could build a coherent distro to your own unique demands.
Now though? It's way easier to have a herd of unicorn docker containers running that each handle one weird thing in isolation.
To copy my response on Lobsters:
(I am one of uv’s maintainers.)
The nice thing about Python packaging is that it’s standardized: if uv’s build backend doesn’t fit into the hole you want, then you can use any number of other backends that do an admirable job of conforming to the packaging standards.
With that said, I don’t understand why Gentoo needs to rebuild every single release of uv-build. It doesn’t change as much as uv itself, and it’s intentionally compatible across uv versions. Why not set a policy of only updating once weekly, or even less frequently, with exceptions for important breaking changes or bug fixes?
(For context: the only reason uv-build releases as often as it does is because it’s easier to keep its internals in lockstep with the rest of uv. But you can do a normal version bound in your Python build backend metadata and uv will happily use whatever version of uv-build is already present.)
> The nice thing about Python packaging is that it’s standardized
But it isn't. Once upon a time, it was: you used distutils. But of course distutils had a lot of problems. Instead of adopting a single standardized solution for those problems, the Python developers punted and let third parties do it. So now we do not have a single standardized solution for building Python packages; we have a bunch of third party solutions, none of which is in the Python standard library.
Yes, all those third party solutions have to be PEP 517 compatible, but that's not standardization on a tool; it's just standardization on an interface between the backend and the frontend. Which is nice, but doesn't solve the problem under discussion here.
> if uv’s build backend doesn’t fit into the hole you want, then you can use any number of other backends that do an admirable job of conforming to the packaging standards.
As the project developer, sure. But as a Linux distro building your python package, Gentoo has to use the build backend you specify. It can't say, well, this particular package is just pure python, so really all we need is a minimal tool to build a pure python none-any wheel. It has to use the backend you specified.
> I don’t understand why Gentoo needs to rebuild every single release of uv-build.
Because if you don't pin a version of uv-build in your project, that means "use the latest version available". Gentoo can't second guess you--even if it were able to verify somehow that the new version of uv-build that just got released will end up producing an identical wheel for your project as the previous version, without actually using the new version of uv-build.
> you can do a normal version bound in your Python build backend metadata
Sure, but how many users of uv-build actually do that? Seems like the answer to that is "not enough to matter".
The problem is packages that use the build backends, and Gentoo is generally not in the business of maintaining that kind of patch.
Admittedly rustification like that is a problem not only for gentoo, I have commercially deployed systems where one of the most annoyings elements of updating is presence of rust in python deps...
Is Gentoo one of those compile-everything-from-source distros?
By a search, Gentoo accounts for less than 1% of all developers. If uv-build is more conformant to people, probably volunteers (and if they're using it, it probably is), I don't think the Gentoo people have any right to want them not to use it because it makes their lives harder. They can request, and they do request nicely, but they also present the drawbacks for them as if they are absolute, and even claim that "its benefits don’t outweigh the drawbacks".
You think Gentoo isn't important, they think the build backend doesn't matter. They're asking. Nobody's compelled to listen.
This is their request
I'm sure it accounts for a much larger proportion of people using exotic or legacy systems. In any case, I think most people are using uv-build either because it's a default for uv now or just because it's hip and new. If a project absolutely needed it then so be it, but if you knew it was going to make life difficult even just for one person and no benefit to you then you'd probably choose something else.
UV build is distributed as a binary/executable, e.g. manylinux wheels on pypi: https://pypi.org/project/uv-build/#files. If a user or distro's conventions are to build things from source, this is a consequence they have implicitly accepted. For the vast majority of computer users and uses, installing binaries makes more sense.
Why would anyone install an OpenAI binary from an insecure platform?
> the Gentoo uv ebuild already carries two hacks to make it possible to build uv on 32-bit platforms).
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/uv...
Looks like it's being built with fat LTO by default (and gentoo patched to thin LTO). Which is fine if you want to spend CI cycles once to ship executables to thousands of users. If you compile from source and run the tool maybe once then they should consider going further and run with maybe opt-level=1, lto=off; halfway between release and debug. This is how rustc itself builds some internal buildtools that get recompiled frequently relative to how often they run.
I'm not sure that skipping SPARC is a bad thing. Who even has SPARC machines sitting around to validate builds? Even Fujitsu moved to ARM for their later cores.
i got a debian maintainer for old IBM mainframes asking me to fix bugs and run tests for that architecture. ¯\_(ツ)_/¯ these people exist, but i'm not going to support them. feel free to fork me!
people who are specially interested in packaging and therefore do it for free care about SPARC.
I don't think I agree with the author's assessment here:
> The other reason is performance and that argument makes even less sense. Even if it mattered, we’re realistically talking about a difference in the ballpark of 100 ms, for an action executed rarely; it is equally possible that any gain will be outweighed by the time needed to fetch the 1.5 MiB wheel for uv-build.
Modern CI architectures are building Docker stacks from scratch and using a readthrough cache of wheels. Downloading may still be the dominant factor, but the second-dominant factor does start to add up, especially when the downloading gets cached at various layers.
Wait until he discovers that CPython is actually taking Rust in, see recent PyCon talks on the matter.
Yes, optionally apparently but I think we'll end up with forks of Python if the Rust enthusiasts push it any further. I think the option will be off for me.
I think mgorny is aware of that; he’s been active on the relevant DPO threads.
(At the end of the day, this is exactly what support tiers are for. CPython, at least to my eyes, has been very transparent about not guaranteeing support for every platform that has a C compiler.)
Maybe it's time to grow up and download binary builds like a normal person. I had my 'let's compile the linux kernel from source for 3.5% hypothetical performance gains' phase but you can't expect professional devs to cater to source-obsessed hobbyists
There's a long history of people telling Linux users "to grow up" and "do what normal people do."
Linux wouldn't exist if people submitted to that argument.
I doubt this oft-repeated stereotype was every particularly accurate. I'm not aware of any current Gentoo users on the distro specifically for performance reasons. Gentoo users are in it for control over their system and customization.
Gentoo’s decision to support source builds seems eminently reasonable to me, and I don’t think we should reward it with condescension.
The bigger problem, to me, is that it’s not really clear why this is actually a problem for Gentoo. I am glad they perform user-facing source builds, however, because they see things that other distributors don’t.
Or adopt the Nix approach of caching common binary artifacts in a content addressable store and rebuild from source only when needed.
Gentoo made a lot of sense WAY back in a pre-docker world, where you had to run the whole world on the host OS. Being able to compile it all meant you could link it all together with whatever weird combination of optional libraries you wanted. Things like getting sendmail to auth over LDAP, that nobody packaged in the base distro, become possible. And with Gentoo's tooling, you could build a coherent distro to your own unique demands.
Now though? It's way easier to have a herd of unicorn docker containers running that each handle one weird thing in isolation.
Time to grow up and use Windows like a normal person. The bulk of professional work is done on Windows.