The last paragraph says:
> At some point your docs will outgrow a single folder, and then all bets are off. You’ll want a separate repo with its own build process...
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
I also put everything in one repo nowadays, no matter the usage, the language etc. All is synced, and all is accessible by my LLM. There are a lot of tools to manage monorepos, and frankly most of the time you don't even need them.
I'm no fan of GitHub add-ons and I agree with the premise here but...
I can think of one other possibility. It's easier to write in a wiki via the browser. I can open that on my phone and edit docs. I can open it in my browser and edit docs.
On desktop it's a tiny bit more to pull the repo and open it in your editor (and you might already be there) but that tiny bit can be enough to stop you from writing documentation. For me, writing documentation must be totally painless so that I'll actually do it.
Why am I not a fan of the add-ons like PR's, wiki's, discussions, projects, and issues? Because they each introduce vendor lock-in to varying degrees.
Fossil (https://fossil-scm.org/home/doc/trunk/www/index.wiki) solves this pretty nicely. You can have documentation as files or in a special wiki namespace and it's versioned both ways, and every repository clone gets everything. Even better than that, your in-tree documentation files are rendered and browseable in exactly the same way as the dedicated wiki namespace.
The linked URL to the home page there can even serve as an example: the "trunk" is a check-in name (https://fossil-scm.org/home/doc/trunk/www/checkin_names.wiki) that points to the newest check-in on the "trunk" branch. You can replace it with any other reference to get the old version; eg, version-2.20 would work to get the version 2.20 of the docs, 2015-03-14 would work to get the version from 14 March 2015, etc.
Fossil was written for Sqlite in the same way that git was written for Linux. It's really a shame that more projects don't use it. I think that a github competitor (with social features, PRs, CI, etc) with a fossil backend would be very popular.
I think there is a missing pro here on the wiki side: trivial edits are trivial. Even fixing a typo in the docs directory requires PR + approvals + CI. Effectively limiting your docs contributors to folks who are comfortable with a code editor, and git, is a decision
I've been trying for years to get the contents indexed, accessible to search engines or even AI. When I started, no wiki was ever indexed but it seems GitHub backed off a bit since then. Still, there's a bunch that is still not indexed for various reasons with valuable data in them.
I do agree, most users should look into docs instead.
The second paragraph neatly triggered my confirmation bias:
The initial version of this post opened with “You can use the wiki or a docs folder for your GitHub project, both are valid choices” but as I wrote more, I realised that there is a single reason to use a wiki, and many more reasons not to use the wiki. So many in fact, that I consider using the wiki on GitHub is an anti-pattern.
A very straightforward example of McCulloch's quote that "Writing is thinking."
I agree with this post. I’ve never found the GitHub wiki experience to be particularly ergonomic. I don’t have any issues with it, but it’s no more convenient than a simple /docs folder. And from there, it’s almost trivial to turn /docs into GitHub pages. Similar effort for a much better end product.
Wikis typically connote distributed, anonymous edits. This feature is partially covered by git already.
> I’ve never found the GitHub wiki experience to be particularly ergonomic.
That's because the original sin of GitHub "wikis" is that they weren't (and most of them still aren't) even wikis. There's this perverse thing that happened during the wiki age, where people unable or unwilling to get on board decided to just start calling things "wikis" even though they exemplify the very thing that the wiki was invented as a response to. The reckless debasing of the word then infected adjacent spaces. Sourcehut's "read-only wikis" (wat) aren't even designed to be edited in the browser; on Sourcehut, "Publishing your changes is as easy as committing them and pushing them upstream." Newsflash: That's not a wiki.
In my experience, the docs for something like setting up a dev env are typically greatly improved by the second person who sets up the dev env, not the personal who originally wrote the docs.
In that case, when the docs are not associated with a code change, you want to make getting those improvements into the docs as frictionless as possible, otherwise the changes aren't going to get made.
Personally, I've found that making docs updates incredibly fast + easy to be far more valuable than anything you get from forcing doc changes through the full SDLC process. If someone has feedback on your docs changes they would have shared in a review, they can just update the docs instead.
> In my experience, the docs for something like setting up a dev env are typically greatly improved by the second person who sets up the dev env, not the personal who originally wrote the docs.
In my experience, this is also true of a lot of code as well. Your dev scripts should probably have much more relaxed standards than your service source or CI/CD. Ideally I could define merge requirements by directory without doing some weird shenanigans with the CODEOWNERS file and a bot.
I've never understood why people even _generate_ from the docs folder. If you've written it in markdown (which they nearly always do) then it's already rendered properly in Github. Or is it because they then publish those docs somewhere else?
A lot of GitHub’s secondary features are like this. The Issues tab and Discussions tab are so similar, with slightly different feature sets. And users will happily use both for feature requests and bug reports, with varying degrees of quality, so then I just have yet another thing to stay on top of. I think Discussions were made to reduce noise in very busy repos, but I generally find something useful in the noise. It’s really just a way to ignore users. As a result, I always turn off Discussions and just let people file issues when they feel it’s appropriate.
I agree, but my only gripe is I hate the ceremony required for doc-only updates. It needs a review and CI. The review is a good thing in most cases (want your docs to be correct), but that often takes my team like two days (I'm realizing this is likely our fault now that I'm typing this). For CI, I've just added in a step to all our GitHub actions to skip Markdown only changes. Anyone have any better ideas?
Indeed, and also now with agents everywhere the docs can be updated and checked more regularly. If it's in the wiki, you can clone it locally and put an AGENTS.md line informing of where the docs are, but it's still a separate repo to deal with. Credit where due, it was a revolution (even a godsend) in it's time, but at this point I agree a /docs is better.
I really wanted the wiki to at minimum have directories so they would be easier to structure and allow even for major versions that are maintained in parallel.
Interesting because I just added a wiki for one of my projects. I'm not using it for docs, since the project already has in-tree docs. I'm using it more as a public scratchpad of ideas / experiments to try that aren't well-defined enough (or known to be worth) opening as an issue yet.
Yes, and the rest of it is too; there's another outage today and my CI is blocked. I guess I can read the wikis while I wait, eh?? https://www.githubstatus.com/
The Github wiki is pretty bad. However had, it is easier to use than issues and coordinating them. Github issues require too much cross-communication and not everyone has the time to meta-coordinate many different issues in many different projects. A wiki lowers the entry-barrier too, so the idea of a wiki is, in principle, good.
If I were Github I would improve the wiki, a bit stylistically, to make it
more visually pleasing to use (but not much, those designers always go overboard when making changes in my experience), but much more importantly so, to make the wiki a more flexible addition, including API-wise, usage examples, documentation and so forth. People can, in principle, do so on their own, but also from experience, most people stop doing so after a while, and then the wiki decays into outdated information. That's bad too. Spawning more issues to manage the wiki also does not work well.
It sounds like the whole argument here is that the commits in the docs repo aren't identical commits in the source code repo. But you can ameliorate that with something like tags or submodules.
The last paragraph says: > At some point your docs will outgrow a single folder, and then all bets are off. You’ll want a separate repo with its own build process...
My question is, why is this taken as a given? Is it so hard to have docs and code live together in version control after a certain scale? If so, what is the specific problem and what is the cause?
I ask because I've never been that satisfied with the various ways I've tried to organize projects in git. Recently I've been trying to keep the source, tests and docs together in the same tree so that changes are more localized. It seems to be helping me keep track of things, especially with coding agents so eager to make changes all over the place. I find their proclivity to repeat the same idea in multiple locations (agent instructions, docs, docstrings, help strings, comments) especially problematic.
I also put everything in one repo nowadays, no matter the usage, the language etc. All is synced, and all is accessible by my LLM. There are a lot of tools to manage monorepos, and frankly most of the time you don't even need them.
I'm no fan of GitHub add-ons and I agree with the premise here but...
I can think of one other possibility. It's easier to write in a wiki via the browser. I can open that on my phone and edit docs. I can open it in my browser and edit docs.
On desktop it's a tiny bit more to pull the repo and open it in your editor (and you might already be there) but that tiny bit can be enough to stop you from writing documentation. For me, writing documentation must be totally painless so that I'll actually do it.
Why am I not a fan of the add-ons like PR's, wiki's, discussions, projects, and issues? Because they each introduce vendor lock-in to varying degrees.
Fossil (https://fossil-scm.org/home/doc/trunk/www/index.wiki) solves this pretty nicely. You can have documentation as files or in a special wiki namespace and it's versioned both ways, and every repository clone gets everything. Even better than that, your in-tree documentation files are rendered and browseable in exactly the same way as the dedicated wiki namespace.
The linked URL to the home page there can even serve as an example: the "trunk" is a check-in name (https://fossil-scm.org/home/doc/trunk/www/checkin_names.wiki) that points to the newest check-in on the "trunk" branch. You can replace it with any other reference to get the old version; eg, version-2.20 would work to get the version 2.20 of the docs, 2015-03-14 would work to get the version from 14 March 2015, etc.
Why is this easier or more effective than just a /docs directory?
Parent comment linked to that answer.
Fossil is the best. Sqlite uses it
Fossil was written for Sqlite in the same way that git was written for Linux. It's really a shame that more projects don't use it. I think that a github competitor (with social features, PRs, CI, etc) with a fossil backend would be very popular.
I think there is a missing pro here on the wiki side: trivial edits are trivial. Even fixing a typo in the docs directory requires PR + approvals + CI. Effectively limiting your docs contributors to folks who are comfortable with a code editor, and git, is a decision
https://github-wiki-see.page/
I've been trying for years to get the contents indexed, accessible to search engines or even AI. When I started, no wiki was ever indexed but it seems GitHub backed off a bit since then. Still, there's a bunch that is still not indexed for various reasons with valuable data in them.
I do agree, most users should look into docs instead.
The second paragraph neatly triggered my confirmation bias:
The initial version of this post opened with “You can use the wiki or a docs folder for your GitHub project, both are valid choices” but as I wrote more, I realised that there is a single reason to use a wiki, and many more reasons not to use the wiki. So many in fact, that I consider using the wiki on GitHub is an anti-pattern.
A very straightforward example of McCulloch's quote that "Writing is thinking."
I agree with this post. I’ve never found the GitHub wiki experience to be particularly ergonomic. I don’t have any issues with it, but it’s no more convenient than a simple /docs folder. And from there, it’s almost trivial to turn /docs into GitHub pages. Similar effort for a much better end product.
Wikis typically connote distributed, anonymous edits. This feature is partially covered by git already.
> I’ve never found the GitHub wiki experience to be particularly ergonomic.
That's because the original sin of GitHub "wikis" is that they weren't (and most of them still aren't) even wikis. There's this perverse thing that happened during the wiki age, where people unable or unwilling to get on board decided to just start calling things "wikis" even though they exemplify the very thing that the wiki was invented as a response to. The reckless debasing of the word then infected adjacent spaces. Sourcehut's "read-only wikis" (wat) aren't even designed to be edited in the browser; on Sourcehut, "Publishing your changes is as easy as committing them and pushing them upstream." Newsflash: That's not a wiki.
Yeah you can configure gh “wikis” to be freely editable but that’s not the default and most of them are not,
In my experience, the docs for something like setting up a dev env are typically greatly improved by the second person who sets up the dev env, not the personal who originally wrote the docs.
In that case, when the docs are not associated with a code change, you want to make getting those improvements into the docs as frictionless as possible, otherwise the changes aren't going to get made.
Personally, I've found that making docs updates incredibly fast + easy to be far more valuable than anything you get from forcing doc changes through the full SDLC process. If someone has feedback on your docs changes they would have shared in a review, they can just update the docs instead.
> In my experience, the docs for something like setting up a dev env are typically greatly improved by the second person who sets up the dev env, not the personal who originally wrote the docs.
In my experience, this is also true of a lot of code as well. Your dev scripts should probably have much more relaxed standards than your service source or CI/CD. Ideally I could define merge requirements by directory without doing some weird shenanigans with the CODEOWNERS file and a bot.
That could be easily be corrected by relaxing merge gates for changes only to the `docs` folder (or some suitable naming pattern).
You can even do live edits on the web if you don't want to use a command line.
Corrections and improvements to docs are just a bugfix though?
You can set up automation and/or configuration such that changes to the docs folder don't require code review.
I've never understood why people even _generate_ from the docs folder. If you've written it in markdown (which they nearly always do) then it's already rendered properly in Github. Or is it because they then publish those docs somewhere else?
A lot of GitHub’s secondary features are like this. The Issues tab and Discussions tab are so similar, with slightly different feature sets. And users will happily use both for feature requests and bug reports, with varying degrees of quality, so then I just have yet another thing to stay on top of. I think Discussions were made to reduce noise in very busy repos, but I generally find something useful in the noise. It’s really just a way to ignore users. As a result, I always turn off Discussions and just let people file issues when they feel it’s appropriate.
I agree, but my only gripe is I hate the ceremony required for doc-only updates. It needs a review and CI. The review is a good thing in most cases (want your docs to be correct), but that often takes my team like two days (I'm realizing this is likely our fault now that I'm typing this). For CI, I've just added in a step to all our GitHub actions to skip Markdown only changes. Anyone have any better ideas?
Use CODEOWNERS files to route docs/ reviews differently.
Biggest thing for me is wiki edits skip code review, so docs rot silently while a /docs PR at least shows up in the diff next to the change.
Indeed, and also now with agents everywhere the docs can be updated and checked more regularly. If it's in the wiki, you can clone it locally and put an AGENTS.md line informing of where the docs are, but it's still a separate repo to deal with. Credit where due, it was a revolution (even a godsend) in it's time, but at this point I agree a /docs is better.
im having trouble connecting the dots here. How does adding the friction of code review reduce rot?
You can spot a code change without a docs change or vice versa
To consider, there are different docs:
in repo (can be wiki format):
- dev docs
- user manual
different repo:
- community usage wiki
I really wanted the wiki to at minimum have directories so they would be easier to structure and allow even for major versions that are maintained in parallel.
nit: ”Using the /docs folder is the highest effort-to-reward ratio option” shouldn’t it be lowest or reward-to-effort instead?
Interesting because I just added a wiki for one of my projects. I'm not using it for docs, since the project already has in-tree docs. I'm using it more as a public scratchpad of ideas / experiments to try that aren't well-defined enough (or known to be worth) opening as an issue yet.
i was going to say the biggest reason wasn't mentioned here, that github sets Disallow: /*/wiki*
however I think maybe this has changed? I don't see it in https://github.com/robots.txt now
They're not indexed if they're publicly editable
also the repo needs 500+ stars https://docs.github.com/en/communities/documenting-your-proj...
It also needs to be non publically editable.
In other words, spiritually not a wiki.
In Gitlab the wiki is just a separate git repo. Is this not the case with GitHub?
It is. Although it’s a hidden git repo with none of the GitHub tooling.
(2022)
Yes, and the rest of it is too; there's another outage today and my CI is blocked. I guess I can read the wikis while I wait, eh?? https://www.githubstatus.com/
The Github wiki is pretty bad. However had, it is easier to use than issues and coordinating them. Github issues require too much cross-communication and not everyone has the time to meta-coordinate many different issues in many different projects. A wiki lowers the entry-barrier too, so the idea of a wiki is, in principle, good.
If I were Github I would improve the wiki, a bit stylistically, to make it more visually pleasing to use (but not much, those designers always go overboard when making changes in my experience), but much more importantly so, to make the wiki a more flexible addition, including API-wise, usage examples, documentation and so forth. People can, in principle, do so on their own, but also from experience, most people stop doing so after a while, and then the wiki decays into outdated information. That's bad too. Spawning more issues to manage the wiki also does not work well.
You own it now, I'll quote you on that!
In Forgejo, wikis are just another repo, so you have versioning there.
That's also true on GitHub: https://docs.github.com/en/communities/documenting-your-proj...
and on GitLab: https://docs.gitlab.com/user/project/wiki/
It sounds like the whole argument here is that the commits in the docs repo aren't identical commits in the source code repo. But you can ameliorate that with something like tags or submodules.
Duh.
It's likely also better that agents seeing versioned /docs can understand the context of changes to the codebase better.