They used their own proprietary benchmark called sudo L7, which purports to evaluate staff-level engineering judgment more than the correctness of the code itself.
Not surprising to me that higher thinking will outperform coding harnesses here.
Maybe all the extra features and scaffolding helped when he first started building it... But I'd bet that raw Claude Code and Codex are smart enough now that they make things worse.
Skills and plugins have become less and less important in the ecosystems. Superhuman seems to be the outlier in both this benchmark and empirical evidence.
The challenge is improving performance while decreasing cost. There are many token condensers and model routers that attempt to do decrease cost, but they end up sacrificing performance.
They used their own proprietary benchmark called sudo L7, which purports to evaluate staff-level engineering judgment more than the correctness of the code itself.
Not surprising to me that higher thinking will outperform coding harnesses here.
I wonder how gstack stacks up (Garry Tan's special coding setup, which he spends $200K/yr on: https://techcrunch.com/2026/11/02/y-combinators-garry-tan-us...)
Maybe all the extra features and scaffolding helped when he first started building it... But I'd bet that raw Claude Code and Codex are smart enough now that they make things worse.
Skills and plugins have become less and less important in the ecosystems. Superhuman seems to be the outlier in both this benchmark and empirical evidence.
The challenge is improving performance while decreasing cost. There are many token condensers and model routers that attempt to do decrease cost, but they end up sacrificing performance.
interesting. there are so many different types of staff engineers. im wondering if the models have optimized for it.
lol maybe more thinking = higher desire to get promoted or an equity refresh.
TIL about Superpowers!
[flagged]