And it should go without saying: Oils simply uses libc - we don't need to support system with a broken getcwd()!
Although a funny thing is that I just fixed a bug related to $PWD that AT&T ksh (the original shell, that bash is based on) hasn't fixed for 30+ years too!
The Bourne Again shell is not based upon the Korn shell. There's a very big clue in its name which prior shell the Bourne Again shell started out to clone. (-:
The unequivocal direct evidence that the Bourne Again shell is based upon the Bourne shell is Richard Stallman saying so, back when it was written in 1988.
"Foundation staff member Brian Fox is now implementing an imitation of the Bourne shell."
Most of the stuff that configure scripts check is obsolete, and breaks in situations like this as the checks are often not workable without running code.
It is likely the check does not apply to any system that has existed for decades. Lots of systems have disabled eg Nix in 2017 [1]
I had a look at the bash source code a few years back, and there are tons of hacks and workarounds for 1980s-era systems. Looking at the git log, GETCWD_BROKEN was added in bash 1.14 from 1996, presumably to work around some system at the time (a system which was perhaps already old in 1996, but it's not detailed which).
Also, that getcwd.c which contains the getcwd() fallback and bug is in K&R C, which should be a hint at how well maintained all of this is. Bash takes "don't fix it if it ain't broke" to new levels, to the point of introducing breakage like here (the bash-malloc is also notorious for this – no idea why that's still enabled by default).
For a long time, inode numbers from readdir() had certain semantics. Supporting overlay filesystems required changing those semantics. Piles of software were written against the old semantics; and even some of the most common have not been upgraded.
The opposite, if anything. Very little was written against the old semantics, with most of the time the supplied C library providing what was needed, and so the code that did rely upon old semantics barely got exercised. A little-used shim that had been broken wasn't noticed, in other words, until just the right combination of circumstances got the shim being used on a platform where it would break.
What there are piles of, are softwares that reinvent the C library, all too often in little bits of conditionally-compiled code that have either been reinvented or nicked from some old C library and sit unused in every platform that that application is nowadays ported to. Every time that I see a build log dutifully informing me that it has checked for <string.h> or some other thing that has been standard for 35 years I wonder (a) why that is thought to be necessary in 2025, and (b) what sort of shims would get used if the check ever failed.
> what sort of shims would get used if the check ever failed.
Most programs will probably just fail to compile: "#undef HAVE_STRING_H" gets added to config.h, but it's never checked. Or something along those lines. It's little more than "failed to find <string.h>" with extra steps.
The exceptions are older projects which support tons of systems: bash, Vim, probably Emacs, that type of thing. A major difficulty is that it can be very hard to know what is safe to remove. So to use your strings.h example, bash currently does:
#if defined (HAVE_STRING_H)
# include <string.h>
#endif /* !HAVE_STRING_H */
#if defined (HAVE_STRINGS_H)
# include <strings.h>
#endif /* !HAVE_STRINGS_H */
And Vim has an even more complex check:
// Note: Some systems need both string.h and strings.h (Savage). However,
// some systems can't handle both, only use string.h in that case.
#ifdef HAVE_STRING_H
# include <string.h>
#endif
#if defined(HAVE_STRINGS_H) && !defined(NO_STRINGS_WITH_STRING_H)
# include <strings.h>
#endif
Looks like that NO_STRINGS_WITH_STRING_H gets defined on "OS/X". Is that still applicable? Probably not?
Is any of this still needed? Who knows. Is it safe to remove? Who knows. No one is really tracking any of this. There is no "caniuse" for this, and even the autoconf people aren't sure on what systems autoconf does and doesn't work. There is no way to know who is running what on what, and people do run some of these programs on pretty old systems.
So ... people don't touch any of this because no one knows what is or isn't broken and what does and doesn't break if you touch it.
Aside: people love to complain about telemetry, sometimes claiming it's never useful, but this is where telemetry would absolutely be very useful.
I've seen both, over the years; programs that have shims, and programs whose checks are ridiculous because they'll just fail at compile time if the standard language feature they are checking for is not there.
But a lot of the time this isn't because people are afraid to take the checks out. It's because they were put in by cargo cult programming in the first place. Autotools et al. are so complex that people will copy existing setups to start new projects rather than begin with a blank slate and add things as and when they come up.
Which means that projects don't start out massively portable because of this stuff being used; but rather they start out inherently massively portable, with useless checks, parroted from somewhere else, on standard library stuff that was always there, sometimes decades before the project was even begun.
It's not really "Who knows". We actually do know. Well, those of us who lived through the process know. Checks for <stdlib.h> or <string.h> aren't needed. There was a period of about half a decade when they were, but by the early to middle 1990s it was perfectly adequate to just assume this pretty basic level of standard conformance.
The "old systems" argument is a specious one. I've worked on the old systems when they weren't nearly so old, including things like porting from Unix to MS-DOS and Big Iron, and experience teaches that the existence of standard headers like <string.h> et al., quickly adopted as soon as they were invented and solved by the likes of Henry Spencer for the rare non-adoption cases, was never a long-term problem; certainly not one long-term enough that it has lasted until 2025.
> It's not really "Who knows". We actually do know.
I'm not so sure about that to be honest. A few years ago (around 2020 or 2021 IIRC) autotools switched from the old deprecated backquotes to $(..) to run programs in shell, and people complained that it no longer worked with their SunOS shell or some such. I've also seen some Vim bug reports about stuff breaking on obscure old systems. In short, I think you may be underestimating the "long tail" of weird/old systems.
Personally, I'd say it's perfectly reasonable to just say "sorry, we're not supporting that any more, you can keep using the existing version" because the number of people doing this seems quite small. But they do seem to exist.
FWIW, if you are cross-compiling, while you might get a vaguely usable result by ignoring all of the warnings and letting worst-common-denominator defaults get applied, you absolutely should be paying more attention and either manually providing autoconf the answers it needs or (if at all possible, as this is more general) make sure to tell it how to run a binary on the target system (maybe in an emulator or over ssh)... you shouldn't just be YOLOing a cross-compile like this and expecting it to work (not to say that this wasn't a good bug in the fallback to fix, just that the premise is awkward).
Like for example when compiling Linux (plus user space) from Windows XP using only the official Services for Unix package from Microsoft as a starting point.
It looks easy on the surface to roll down support for any kind of operating system there is, based on auto-detection and then #if HAVE_THIS or #if HAVE_THAT, but it breaks in ways that maybe really hard to untangle later.
I'd rather have a limited set set of configurations targeting specific platforms/flavors, and knowing that no matter how I compile it, I would know what is `#define`-d and what is not, instead of guessing on what the "host" might have.
Interesting investigation, good read. Definitely illustrates how new paradigms (i.e. overlay filesystems) can subtly affect behaviors in ways that are complex to track down.
Wow great bug!
> Bash forgot to reset errno before the call. For about 30 years, no one noticed
I have to say, this part of the POSIX API is maddening!
99% of the time, you don't need to set errno = 0 before making a call. You check for a non-zero return, and only then look at errno.
But SOMETIMES you need to set errno = 0, because in this case readdir() returns NULL on both error and EOF.
I actually didn't realize this before working on https://oils.pub/
---
And it should go without saying: Oils simply uses libc - we don't need to support system with a broken getcwd()!
Although a funny thing is that I just fixed a bug related to $PWD that AT&T ksh (the original shell, that bash is based on) hasn't fixed for 30+ years too!
(and I didn't realize it was still maintained)
https://www.illumos.org/issues/17442
https://github.com/oils-for-unix/oils/issues/2058
There is a subtle issue with respect to:
1) "trusting" the $PWD value you inherit from another process
2) Respecting symlinks - this is the reason the shell can't just call getcwd() !
Basically, the shell considers BOTH the inherited $PWD and the value of getcwd() to determine its $PWD. It can't just use one or the other!The Bourne Again shell is not based upon the Korn shell. There's a very big clue in its name which prior shell the Bourne Again shell started out to clone. (-:
I'm saying the behavior of bash is largely based on ksh. See
https://pages.oils.pub/spec-compat/2025-06-19/renamed-tmp/sp...
(which I created)
Even the upcoming bash 5.4 implement ksh command sub ${ echo hi; }, which is more evidence that bash is based on ksh.
They're still implementing ksh 35 years later ...
The unequivocal direct evidence that the Bourne Again shell is based upon the Bourne shell is Richard Stallman saying so, back when it was written in 1988.
"Foundation staff member Brian Fox is now implementing an imitation of the Bourne shell."
ksh is a superset of Bourne shell
https://www.oilshell.org/archive/ksh-usenix.pdf
bash implemented essentially all Bourne shell features a long time ago -- and for the last decade+, and currently, it's implementing ksh features
I'm not sure why it's hard to hold those 2 ideas in your head at the same time
> and I didn't realize it was still maintained
ksh _was_ unmaintained for ages. It stopped effectively in 2012, with some very small attempts at reviving it in 2016, 2018 and 2020.
Then it was picked up for active development in 2021, and it lives here now:
https://github.com/ksh93/ksh
If you didn't already, you should open an issue there with your findings.
--
Shameless plug: I keep some docker images with all those versions for testing, and many other shells too both historical and active (including osh!)
https://github.com/alganet/shell-versions/blob/main/.github/...
The response to the bug on the mailing list is disheartening. Report goes "set errno=0 so your error message makes sense". Didn't get a thanks, fixed.
Instead there's objections on the basis "filesystems shouldn't work like that".
There seem to be a bunch of toxic people on the bash mailing list, and I think many or all of them don't even contribute code
The person who responded dismissively later says "I'm just another user."
---
Every commit since they started using git in 2009 is attributed to one person:
https://cgit.git.savannah.gnu.org/cgit/bash.git/log/
I think occasionally contributed patches are applied, but this is not apparent in source control.
I was attacked on the bash mailing list a several years ago, so I don't go there anymore :-)
Especially when the man page for readdir() mentions that you should set errno=0 if you want to distinguish errors from end-of-stream.
Not like setting errno=0 before calling readdir() is a novel idea...
Remember, folks: It's not enough to check $WEARING_PANTS before stepping outside. You need to check !$PANTS_BROKEN && !$SOLARIS too.
> Once the bug report becomes publicly visible, it will be linked here.
Here it is: https://lists.gnu.org/archive/html/bug-bash/2025-06/msg00149...
Most of the stuff that configure scripts check is obsolete, and breaks in situations like this as the checks are often not workable without running code. It is likely the check does not apply to any system that has existed for decades. Lots of systems have disabled eg Nix in 2017 [1]
[1] https://github.com/NixOS/nixpkgs/commit/dff0ba38a243603534c9...
I had a look at the bash source code a few years back, and there are tons of hacks and workarounds for 1980s-era systems. Looking at the git log, GETCWD_BROKEN was added in bash 1.14 from 1996, presumably to work around some system at the time (a system which was perhaps already old in 1996, but it's not detailed which).
Also, that getcwd.c which contains the getcwd() fallback and bug is in K&R C, which should be a hint at how well maintained all of this is. Bash takes "don't fix it if it ain't broke" to new levels, to the point of introducing breakage like here (the bash-malloc is also notorious for this – no idea why that's still enabled by default).
Saving this to explain why software is hard.
For a long time, inode numbers from readdir() had certain semantics. Supporting overlay filesystems required changing those semantics. Piles of software were written against the old semantics; and even some of the most common have not been upgraded.
The opposite, if anything. Very little was written against the old semantics, with most of the time the supplied C library providing what was needed, and so the code that did rely upon old semantics barely got exercised. A little-used shim that had been broken wasn't noticed, in other words, until just the right combination of circumstances got the shim being used on a platform where it would break.
What there are piles of, are softwares that reinvent the C library, all too often in little bits of conditionally-compiled code that have either been reinvented or nicked from some old C library and sit unused in every platform that that application is nowadays ported to. Every time that I see a build log dutifully informing me that it has checked for <string.h> or some other thing that has been standard for 35 years I wonder (a) why that is thought to be necessary in 2025, and (b) what sort of shims would get used if the check ever failed.
> what sort of shims would get used if the check ever failed.
Most programs will probably just fail to compile: "#undef HAVE_STRING_H" gets added to config.h, but it's never checked. Or something along those lines. It's little more than "failed to find <string.h>" with extra steps.
The exceptions are older projects which support tons of systems: bash, Vim, probably Emacs, that type of thing. A major difficulty is that it can be very hard to know what is safe to remove. So to use your strings.h example, bash currently does:
And Vim has an even more complex check: Looks like that NO_STRINGS_WITH_STRING_H gets defined on "OS/X". Is that still applicable? Probably not?Is any of this still needed? Who knows. Is it safe to remove? Who knows. No one is really tracking any of this. There is no "caniuse" for this, and even the autoconf people aren't sure on what systems autoconf does and doesn't work. There is no way to know who is running what on what, and people do run some of these programs on pretty old systems.
So ... people don't touch any of this because no one knows what is or isn't broken and what does and doesn't break if you touch it.
Aside: people love to complain about telemetry, sometimes claiming it's never useful, but this is where telemetry would absolutely be very useful.
I've seen both, over the years; programs that have shims, and programs whose checks are ridiculous because they'll just fail at compile time if the standard language feature they are checking for is not there.
But a lot of the time this isn't because people are afraid to take the checks out. It's because they were put in by cargo cult programming in the first place. Autotools et al. are so complex that people will copy existing setups to start new projects rather than begin with a blank slate and add things as and when they come up.
Which means that projects don't start out massively portable because of this stuff being used; but rather they start out inherently massively portable, with useless checks, parroted from somewhere else, on standard library stuff that was always there, sometimes decades before the project was even begun.
It's not really "Who knows". We actually do know. Well, those of us who lived through the process know. Checks for <stdlib.h> or <string.h> aren't needed. There was a period of about half a decade when they were, but by the early to middle 1990s it was perfectly adequate to just assume this pretty basic level of standard conformance.
The "old systems" argument is a specious one. I've worked on the old systems when they weren't nearly so old, including things like porting from Unix to MS-DOS and Big Iron, and experience teaches that the existence of standard headers like <string.h> et al., quickly adopted as soon as they were invented and solved by the likes of Henry Spencer for the rare non-adoption cases, was never a long-term problem; certainly not one long-term enough that it has lasted until 2025.
> It's not really "Who knows". We actually do know.
I'm not so sure about that to be honest. A few years ago (around 2020 or 2021 IIRC) autotools switched from the old deprecated backquotes to $(..) to run programs in shell, and people complained that it no longer worked with their SunOS shell or some such. I've also seen some Vim bug reports about stuff breaking on obscure old systems. In short, I think you may be underestimating the "long tail" of weird/old systems.
Personally, I'd say it's perfectly reasonable to just say "sorry, we're not supporting that any more, you can keep using the existing version" because the number of people doing this seems quite small. But they do seem to exist.
FWIW, if you are cross-compiling, while you might get a vaguely usable result by ignoring all of the warnings and letting worst-common-denominator defaults get applied, you absolutely should be paying more attention and either manually providing autoconf the answers it needs or (if at all possible, as this is more general) make sure to tell it how to run a binary on the target system (maybe in an emulator or over ssh)... you shouldn't just be YOLOing a cross-compile like this and expecting it to work (not to say that this wasn't a good bug in the fallback to fix, just that the premise is awkward).
Like for example when compiling Linux (plus user space) from Windows XP using only the official Services for Unix package from Microsoft as a starting point.
Autoconf is the prime example of easy vs simple.
It looks easy on the surface to roll down support for any kind of operating system there is, based on auto-detection and then #if HAVE_THIS or #if HAVE_THAT, but it breaks in ways that maybe really hard to untangle later.
I'd rather have a limited set set of configurations targeting specific platforms/flavors, and knowing that no matter how I compile it, I would know what is `#define`-d and what is not, instead of guessing on what the "host" might have.
Interesting investigation, good read. Definitely illustrates how new paradigms (i.e. overlay filesystems) can subtly affect behaviors in ways that are complex to track down.