We built EvoWeb, an open-source benchmark for long-horizon web-app coding.
It includes 32 apps with 10 cumulative feature requests each. After every coding-agent update, a browser agent tests the live application for new functionality, integration, and regressions.
The interesting finding is that even frontier models like Fable 5 and Kimi K3 struggle when new features depend on earlier ones.
We generate realistic tasks using features collected from real websites and dependency graphs to ensure meaningful interdependencies.
We hope this approach can also help benchmark other long-horizon agents.
We built EvoWeb, an open-source benchmark for long-horizon web-app coding.
It includes 32 apps with 10 cumulative feature requests each. After every coding-agent update, a browser agent tests the live application for new functionality, integration, and regressions.
The interesting finding is that even frontier models like Fable 5 and Kimi K3 struggle when new features depend on earlier ones.
We generate realistic tasks using features collected from real websites and dependency graphs to ensure meaningful interdependencies.
We hope this approach can also help benchmark other long-horizon agents.