> There are two main schools of thought in software development about how to build really big, complicated stuff.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
Iceberg is, primarily, a spec [0]. It defines exactly what data is stored and how it is interacted with. The community debates broadly on spec changes first, see a recent one on cross-platform SQL UDFs [1].
We have yet to see a largely llm driven language implementation, but it is surely possible. I imagine it would be easier to tell the llm to instead translate the Java implementation to whatever language you need. A vibe-coded language could do major damage to a companies data.
> I can imagine open source projects that will revolve entirely around specifications
This is a really good observation and I predict you will be correct.
There is a consequence of this for SaaS. You can imagine an example SaaS that one might need to vibecode to save money. The reason its not possible now is not because Claude can't do it, its because getting the right specs (like you suggested) is hard work. A well written spec will not only contain the best practices for that domain of software but also all the legal compliance BS that comes along with it.
With a proper specification that is also modular, I imagine we will be able to see more vibecoded SaaS.
Interested in ideas for this. I've mulled over different compact DSLs for specs, but unstructured (beyond file-specific ownership boundaries) has served me better.
“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”
Gall’s Law
But this is about the first systems? I tend to tell people, the fourth try usually sticks.
The first is too ambitious and ends in an unmaintainable pile around a good core idea.
The second tries to "get everything right" and suffers second system syndrome.
The third gets it right but now for a bunch of central business needs. You learned after all. It is good exactly because it does not try to get _everything_ right like the second did.
The fourth patches up some more features to scoop up B and C prios and calls it a day.
Sometimes, often in BigCorp:
Creators move on and it will slowly deteriorate from being maintenaned...
The Evolution method outlined also seems born from the Continuous Delivery paradigm that was required for subscription business models. I would argue Engineering is the superior approach as the Lean/Agile methods of production were born from physical engineering projects whose end result was complete. Evolution seems to be even more chaotic because an improper paradigm of 'dev ops' was used instead of organically emerged as one would expect with an evolving method.
Ai assistance would seem to favor the engineering approach as the friction of teams and personalities is reduced in favor of quick feasibility testing and complete planning.
Software cannot be built like skyscrapers because the sponsors know about the malleability of the medium and treat it like a lump of clay that by adding water can be shaped to something else.
You're mixing up design and manufacturing. A skyscraper is first completely designed (on paper, cad systems, prototypes), before it is manufactured. In software engineering, coding is often more a design phase than a manufacturing phase.
Designers need malleability, that is why they all want digital design systems.
But software is in fact not very malleable at all. It's true the medium supports change, it's just a bunch of bits, but change is actually hard and expensive, perhaps more than other mediums.
How rapidly has business software changed since COVID? Yet how many skyscrapers remain partially unoccupied in big cities like London, because of the recent arrival of widespread hybrid working?
The buildings are structurally unchanged and haven't been demolished to make way for buildings that better support hybrid working. Sure office fit outs are more oriented towards smaller simultaneous attendance with more hot desking. Also a new industry boom around team building socials has arrived. Virtual skeet shooting or golf, for example.
On the whole, engineered cities are unchanged, their ancient and rigid specifications lacking the foresight to include the requirements that accommodate hybrid working. Software meanwhile has adapted and as the OP says, evolved.
A major factor supporting evolution over big up-front design is the drift in system requirements over time. Even on large military like projects, apparently there's "discovery"--and the more years that pass, the more requirements change.
> There are two main schools of thought in software development about how to build really big, complicated stuff.
> The most prevalent one, these days, is that you gradually evolve the complexity over time. You start small and keep adding to it.
> The other school is that you lay out a huge specification that would fully work through all of the complexity in advance, then build it.
I think AI will drive an interesting shift in how people build software. We'll see a move toward creating and iterating on specifications rather than implementations themselves.
In a sense, a specification is the most compact definition of your software possible. The knowledge density per "line" is much higher than in any programming language. This makes specifications easier to read, reason about, and iterate on—whether with AI or with peers.
I can imagine open source projects that will revolve entirely around specifications, not implementations. These specs could be discussed, with people contributing thoughts instead of pull requests. The more articulated the idea, the higher its chance of being "merged" into the working specification. For maintainers, reviewing "idea merge requests" and discussing them with AI assistants before updating the spec would be easier than reviewing code.
Specifications could be versioned just like software implementations, with running versions and stable releases. They could include addendums listing platform-specific caveats or library recommendations. With a good spec, developers could build their own tools in any language. One would be able to get a new version of the spec, diff it with the current one and ask AI to implement the difference or discuss what is needed for you personally and what is not. Similarly, It would be easier to "patch" the specification with your own requirements than to modify ready-made software.
Interesting times.
Iceberg is, primarily, a spec [0]. It defines exactly what data is stored and how it is interacted with. The community debates broadly on spec changes first, see a recent one on cross-platform SQL UDFs [1].
We have yet to see a largely llm driven language implementation, but it is surely possible. I imagine it would be easier to tell the llm to instead translate the Java implementation to whatever language you need. A vibe-coded language could do major damage to a companies data.
[0] https://iceberg.apache.org/spec/ [1] https://lists.apache.org/thread/whbgoc325o99vm4b599f0g1owhgw...
> I can imagine open source projects that will revolve entirely around specifications
This is a really good observation and I predict you will be correct.
There is a consequence of this for SaaS. You can imagine an example SaaS that one might need to vibecode to save money. The reason its not possible now is not because Claude can't do it, its because getting the right specs (like you suggested) is hard work. A well written spec will not only contain the best practices for that domain of software but also all the legal compliance BS that comes along with it.
With a proper specification that is also modular, I imagine we will be able to see more vibecoded SaaS.
Overall I think your prediction is really strong.
Interested in ideas for this. I've mulled over different compact DSLs for specs, but unstructured (beyond file-specific ownership boundaries) has served me better.
I think it has to be modular and reusable. Like GDPR compliance spec should be opensourced and reused by all SaaS specs.
“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.” Gall’s Law
Ah, the Second System Effect, and the lesson learned from it.
But this is about the first systems? I tend to tell people, the fourth try usually sticks.
The first is too ambitious and ends in an unmaintainable pile around a good core idea.
The second tries to "get everything right" and suffers second system syndrome.
The third gets it right but now for a bunch of central business needs. You learned after all. It is good exactly because it does not try to get _everything_ right like the second did.
The fourth patches up some more features to scoop up B and C prios and calls it a day.
Sometimes, often in BigCorp: Creators move on and it will slowly deteriorate from being maintenaned...
So true.
The Evolution method outlined also seems born from the Continuous Delivery paradigm that was required for subscription business models. I would argue Engineering is the superior approach as the Lean/Agile methods of production were born from physical engineering projects whose end result was complete. Evolution seems to be even more chaotic because an improper paradigm of 'dev ops' was used instead of organically emerged as one would expect with an evolving method.
Ai assistance would seem to favor the engineering approach as the friction of teams and personalities is reduced in favor of quick feasibility testing and complete planning.
Software cannot be built like skyscrapers because the sponsors know about the malleability of the medium and treat it like a lump of clay that by adding water can be shaped to something else.
Funny you bring up the clay analogy.
It was discussed here just 2 days ago intensively.
https://news.ycombinator.com/item?id=46881543
You're mixing up design and manufacturing. A skyscraper is first completely designed (on paper, cad systems, prototypes), before it is manufactured. In software engineering, coding is often more a design phase than a manufacturing phase.
Designers need malleability, that is why they all want digital design systems.
But software is in fact not very malleable at all. It's true the medium supports change, it's just a bunch of bits, but change is actually hard and expensive, perhaps more than other mediums.
I'd argue it's more malleable than a skyscraper.
How rapidly has business software changed since COVID? Yet how many skyscrapers remain partially unoccupied in big cities like London, because of the recent arrival of widespread hybrid working?
The buildings are structurally unchanged and haven't been demolished to make way for buildings that better support hybrid working. Sure office fit outs are more oriented towards smaller simultaneous attendance with more hot desking. Also a new industry boom around team building socials has arrived. Virtual skeet shooting or golf, for example.
On the whole, engineered cities are unchanged, their ancient and rigid specifications lacking the foresight to include the requirements that accommodate hybrid working. Software meanwhile has adapted and as the OP says, evolved.
With LLMs it's becoming very malleable.
A major factor supporting evolution over big up-front design is the drift in system requirements over time. Even on large military like projects, apparently there's "discovery"--and the more years that pass, the more requirements change.
Even if the requirements are indeed fixed your understanding of the problem domain evolves.