What is it about Python that makes developers love fragmentation so much? Sending HTTP requests is a basic capability in the modern world, the standard library should include a friendly, fully-featured, battle-tested, async-ready client. But not in Python, stdlib only has the ugly urllib.request, and everyone is using third party stuff like requests or httpx, which aren't always well maintained. (See also: packaging)
You would think that sending HTTP requests is a basic capability, but I've had fun in many languages doing so. Long ago (2020, or not so long ago, depending on how you look at it) I was surprised that doing an HTTP request on node using no dependencies was a little awkward:
```
const response = await new Promise( (resolve, reject) => {
const req = https.request(url, {
}, res => {
let body = "";
res.on("data", data => {
body += data;
});
res.on('end', () => {
resolve(body);
});
});
req.end();
});
HTTP client is at the intersection of "necessary software building block" and "RFC 2616 intricacies that are hard to implement". Has nothing to do with Python really.
> Then I found out it was broken. I contributed a fix. The fix was ignored and there was never any release since November 2024.
This seems like a pretty good reason to fork to me.
> Sending HTTP requests is a basic capability in the modern world, the standard library should include a friendly, fully-featured, battle-tested, async-ready client. But not in Python,
Or Javascript (well node), or golang (http/net is _worse_ than urllib IMO), Rust , Java (UrlRequest is the same as python's), even dotnet's HttpClient is... fine.
Honestly the thing that consistently surprises me is that requests hasn't been standardised and brought into the standard library
>The Requests package is recommended for a higher-level HTTP client interface.
Which was fine when requests were the de-facto-standard only player in town, but at some point modern problems (async, http2) required modern solutions (httpx) and thus ecosystem fragmentation began.
Well, the reason for all the fragmentation is because the Python stdlib doesn't have the core building blocks for an async http or http2 client in the way requests could build on urllib.
The h11, h2, httpcore stack is probably the closest thing to what the Python stdlib should look like to end the fragmentation but it would be a huge undertaking for the core devs.
it's the gitmoji thing, I really don't like it, it was a mistake. Thinking to stop it soon. I was inspired by fastapi in the early days. I prefer conventionalcommits.org
Please don't be too much inspired by FastAPI - at least regarding maintainer bus factor and documentation (FastAPI docs are essentially tutorial only), and requiring dozens of hoops to jump through to even open an issue.
It is indeed a shame that niquests isn't used more, I think trying to use the (c'est Français) argument to in French will bring you many initial users needed for the inertia
I think that it owes its success to be first "port" of python requests to support async, that was a strong need.
But otherwise it is bad: API is not that great, performance is not that great, tweaking is not that great, and the maintainer mindset is not that great also.
For the last point, few points were referenced in the article, but it can easily put your production project to suddenly break in a bad way without valid reason.
Without being perfect, I would advise everyone to switch to Aiohttp.
aiohttp is an excellent library. very stable. I concurs, but!
it's too heavily tied to HTTP/1, and well, I am not a fan of opening thousands of TCP conn just to keep up with HTTP/2 onward. niquests easily beat aiohttp just using 10 conn and crush httpx see https://gist.github.com/Ousret/9e99b07e66eec48ccea5811775ec1...
I guess you mean htmx. Same here. I read the article for a while, and was confused by "HTTPX is a very popular HTTP client for Python." and wondering "why is OpenAI using htmx", until I eventually realized what's going on.
>thread to call out Read the Docs for profiting from MkDocs without contributing back.
>They also point out that not opening up the source code goes against the principles of Open Source software development
I will never stop being amused when people have feelings like this and also choose licenses like BSD (this project). If you wanted a culture that discouraged those behaviors, why would you choose a license that explicitly allows them? Whether you can enforce it or not, the license is basically a type of CoC that states the type of community you want to have.
Right, my suspicion was correct. When I interacted with them a few years ago they seemed perfectly nice and friendly, but seem to have gone off the rails more recently. It's an uncomfortable situation and I've a feeling people are afraid to discuss this kind of thing but we really need to. People are a risk factor in software projects and we need to be resilient to changes they face. Forking is the right way, but places like GitHub have sold people on centralisation. We need to get back to decentralised dev.
Always remember that open-source is an author’s gift to the world, and the author doesn’t owe anything to anyone. Thus, if you need a feature that for whatever reason can’t or won’t go upstream, forking is just about the only viable option. Fingers crossed!
I'm not a lawyer, but are there any potential trademark issues? AFAIK in general you HAVE to change the name to something clearly different. I consider it morally OK, and it's probably fine, but HTTPXYZ is cutting it close. It's too late for a rebrand, but IMO open-source people often ignore this topic a bit too much.
He would probably win in a legal case, but is he actually going to take it to court? I doubt it. Also I wouldn't be too offended about the name if I were him and for users it's better because it makes the link clearer.
I think if had named it HTTPX2 or HTTPY, that would be much worse because it asserts superiority without earning it. But he didn't.
The lack of a well-maintained async HTTP client in Python's stdlib has been a pain point for a while. Makes sense someone eventually took it into their own hands
Do you see yourself taking over httpcore as well as it's likely to have the same maintainership problem? It would certainly instill more confidence that this is a serious fork.
This certainly wouldn't be the first time an author of a popular library got a little too distracted on the sequel to their library that the current users are left to languish a bit.
It's a shame, httpx has so much potential to be the default Python http library. It's crazy that there isn't one really. I contributed some patches to the project some years ago now and it was a nice and friendly process. I was expecting a v1 release imminently. It looks like the author is having some issues which seem to afflict so many in this field for some reason. I notice they've changed their name since I last interacted with the project...
What is it about Python that makes developers love fragmentation so much? Sending HTTP requests is a basic capability in the modern world, the standard library should include a friendly, fully-featured, battle-tested, async-ready client. But not in Python, stdlib only has the ugly urllib.request, and everyone is using third party stuff like requests or httpx, which aren't always well maintained. (See also: packaging)
You would think that sending HTTP requests is a basic capability, but I've had fun in many languages doing so. Long ago (2020, or not so long ago, depending on how you look at it) I was surprised that doing an HTTP request on node using no dependencies was a little awkward:
``` const response = await new Promise( (resolve, reject) => { const req = https.request(url, { }, res => { let body = ""; res.on("data", data => { body += data; }); res.on('end', () => { resolve(body); }); }); req.end(); });
```
These days node supports the fetch API, which is much simpler.
HTTP client is at the intersection of "necessary software building block" and "RFC 2616 intricacies that are hard to implement". Has nothing to do with Python really.
> Then I found out it was broken. I contributed a fix. The fix was ignored and there was never any release since November 2024.
This seems like a pretty good reason to fork to me.
> Sending HTTP requests is a basic capability in the modern world, the standard library should include a friendly, fully-featured, battle-tested, async-ready client. But not in Python,
Or Javascript (well node), or golang (http/net is _worse_ than urllib IMO), Rust , Java (UrlRequest is the same as python's), even dotnet's HttpClient is... fine.
Honestly the thing that consistently surprises me is that requests hasn't been standardised and brought into the standard library
What's wrong with Go's? I've never had any issues with it. Go has some of the best http batteries included of any language
What, Go's net/http is fantastic. I don't understand that take. Many servers are built on it because it's so fully featured out of the box.
Your java knowledge is outdated. Java's JDK has a nice, modern HTTP Client https://docs.oracle.com/en/java/javase/11/docs/api/java.net....
>Honestly the thing that consistently surprises me is that requests hasn't been standardised and brought into the standard library
Instead, official documentation seems comfortable with recommending a third party package: https://docs.python.org/3/library/urllib.request.html#module...
>The Requests package is recommended for a higher-level HTTP client interface.
Which was fine when requests were the de-facto-standard only player in town, but at some point modern problems (async, http2) required modern solutions (httpx) and thus ecosystem fragmentation began.
Well, the reason for all the fragmentation is because the Python stdlib doesn't have the core building blocks for an async http or http2 client in the way requests could build on urllib.
The h11, h2, httpcore stack is probably the closest thing to what the Python stdlib should look like to end the fragmentation but it would be a huge undertaking for the core devs.
Node now supports the Fetch API.
the http landscape is rather scary lately in Python. instead of forking join forces... See Niquests https://github.com/jawah/niquests
I am trying to resolve what you've seen. For years of hard work.
Is it knee-quests or nigh-quests?
I've started seeing these emoji-prefixed commits lately now too, peculiar
There is a series of extensions for Vscode that add this functionality like https://github.com/ugi-dev/better-commits
ah ok, I am familiar with and not exactly against (non-emoji) commit message prefixes
it's the gitmoji thing, I really don't like it, it was a mistake. Thinking to stop it soon. I was inspired by fastapi in the early days. I prefer conventionalcommits.org
Please don't be too much inspired by FastAPI - at least regarding maintainer bus factor and documentation (FastAPI docs are essentially tutorial only), and requiring dozens of hoops to jump through to even open an issue.
agreed. as I said, it was a mistake from my end. and clearly looking to better myself.
nee-quests, I am French native.
I guess kind of obvious now noticing the rhyme
It is indeed a shame that niquests isn't used more, I think trying to use the (c'est Français) argument to in French will bring you many initial users needed for the inertia
ahah, "en effet"! je m'en souviendrai.
more seriously, all that is needed is our collective effort. I've done my part by scarifying a lot of personal time for it.
The basis of httpx is not very good at all.
I think that it owes its success to be first "port" of python requests to support async, that was a strong need.
But otherwise it is bad: API is not that great, performance is not that great, tweaking is not that great, and the maintainer mindset is not that great also. For the last point, few points were referenced in the article, but it can easily put your production project to suddenly break in a bad way without valid reason.
Without being perfect, I would advise everyone to switch to Aiohttp.
aiohttp is an excellent library. very stable. I concurs, but! it's too heavily tied to HTTP/1, and well, I am not a fan of opening thousands of TCP conn just to keep up with HTTP/2 onward. niquests easily beat aiohttp just using 10 conn and crush httpx see https://gist.github.com/Ousret/9e99b07e66eec48ccea5811775ec1...
fwiw, HTTP/2 is twelve years old, just saying.
aiohttp is for asynchronous contexts only
Can confirm, more features, a breeze to switch.
Somehow I confused httpx with htmlx
I guess you mean htmx. Same here. I read the article for a while, and was confused by "HTTPX is a very popular HTTP client for Python." and wondering "why is OpenAI using htmx", until I eventually realized what's going on.
I've been reading the whole article wrong too.
And also htmlx with htmx I guess?
Same! Only just realized it thanks to your comment.
I guess the Discussion on Hacker News href should be "https://news.ycombinator.com/item?id=47514603" instead of "news.ycombinator.com/item?id=47514603"
More related drama: The Slow Collapse of MkDocs (https://fpgmaas.com/blog/collapse-of-mkdocs/)
>thread to call out Read the Docs for profiting from MkDocs without contributing back.
>They also point out that not opening up the source code goes against the principles of Open Source software development
I will never stop being amused when people have feelings like this and also choose licenses like BSD (this project). If you wanted a culture that discouraged those behaviors, why would you choose a license that explicitly allows them? Whether you can enforce it or not, the license is basically a type of CoC that states the type of community you want to have.
Oh i recognised one of the involved people immediately, drama person.
I still think that hijacking the mkdocs package was the wrong way to go though.
The foss landscape has become way too much fork-phobic.
Just fork mkdocs and go over your merry way.
Right, my suspicion was correct. When I interacted with them a few years ago they seemed perfectly nice and friendly, but seem to have gone off the rails more recently. It's an uncomfortable situation and I've a feeling people are afraid to discuss this kind of thing but we really need to. People are a risk factor in software projects and we need to be resilient to changes they face. Forking is the right way, but places like GitHub have sold people on centralisation. We need to get back to decentralised dev.
Drama around Starlette. Drama around httpx. Drama around MkDocs. I just hope that DRF is not next, I still have some projects that depend on it.
What's the drama around starlette? (Can't find anything)
Congratulations on forking!
Always remember that open-source is an author’s gift to the world, and the author doesn’t owe anything to anyone. Thus, if you need a feature that for whatever reason can’t or won’t go upstream, forking is just about the only viable option. Fingers crossed!
This is not merely open-source, but taking part in a huge package ecosystem in a foundational role in an XKCD 2347 type of way for HTTP requests.
Put your side project on your personal homepage and walk away - fine.
Make it central infrastructure - respond to participants or extend or cede maintainership.
I'm not a lawyer, but are there any potential trademark issues? AFAIK in general you HAVE to change the name to something clearly different. I consider it morally OK, and it's probably fine, but HTTPXYZ is cutting it close. It's too late for a rebrand, but IMO open-source people often ignore this topic a bit too much.
Don't you need to register and actively defend you trademark for it to apply?
Is httpx trademarked? I couldn't find anything indicating it was.
I don't think HTTPX is a registered trademark.
He would probably win in a legal case, but is he actually going to take it to court? I doubt it. Also I wouldn't be too offended about the name if I were him and for users it's better because it makes the link clearer.
I think if had named it HTTPX2 or HTTPY, that would be much worse because it asserts superiority without earning it. But he didn't.
The lack of a well-maintained async HTTP client in Python's stdlib has been a pain point for a while. Makes sense someone eventually took it into their own hands
Good line from the blog post ...
"So what is the plan now?" - "Move a little faster and not break things"
Do you see yourself taking over httpcore as well as it's likely to have the same maintainership problem? It would certainly instill more confidence that this is a serious fork.
This certainly wouldn't be the first time an author of a popular library got a little too distracted on the sequel to their library that the current users are left to languish a bit.
Another abandoned project hurting users: https://github.com/benweet/stackedit
> Visitor 4209 since we started counting
Loved that little detail, reminds me of the old interwebs :)
It's gone from 45 when I looked at it an hour ago to 261 just now.
It's a shame, httpx has so much potential to be the default Python http library. It's crazy that there isn't one really. I contributed some patches to the project some years ago now and it was a nice and friendly process. I was expecting a v1 release imminently. It looks like the author is having some issues which seem to afflict so many in this field for some reason. I notice they've changed their name since I last interacted with the project...
Hi Michiel!
Just a small headsup: clicking on the Leiden Python link in your About Me page give not the expected results.
And a small nitpick: it's "Michiel's" in English (where it's "Michiels" in Dutch).
Thanks for devoting time to opensource... <3
smells like supply chain attack