I agree and that’s why we need and indeed have an ever evolving landscape of benchmarks
> Private, refreshed test sets attack the mechanism itself, and in my view they are the only intervention that does. If the questions have never touched the public Web, they can’t be in the training data; if they rotate, memorizing this year’s set doesn’t help next year.
That’s what we have. A fresh public benchmark is also good, and teams do make efforts to decontaminate training data but there’s likely just no great way around leakage.
Btw, lots more issues in benchmarks than the ones discussed; for instance you can leak answers from the questions themselves or in the case of e.g. multiple choice formats in the actual answers. You just pass the MCQ choices themselves to the model and it may be able to guess way above chance. Coding agent benchmarks sometimes forget to delete .git. They mention e.g. a 6.9% error rate in one of the benchmark items, this seems pretty typical and I would actually be fine shipping that.
Benchmarks are very ugly, but if they didn’t exist we would need to invent them. All of the problems above and more do not explain the progress we see. There are probably 50,000 benchmarks in the literature and new ones get created frequently with varying levels of quality and usefulness.
I think Goodhart's law is just a consequence of correlation vs causation.
It is very easy to find a metric that is correlated with what you want. But once you start trying to influence a system, you quickly push it out of the range where the correlation holds.
In order to optimize for something, you need to maximize the actual causative variable. This is much harder.
I think the difference is that Goodhart's law describes how the causal chain _changes_ as a result of management behavior, and in particular the incentives they design for the labor they manage. Incentives are a causal variable for outcomes, and what happens is that people find much easier ways to produce the outcomes you thought you wanted.
Like if you manage a call center and set up KPIs around average call time, reps will start hanging up on customers. Employees could always have done that, and the causal link was always there, there was just no reason to.
IMO the problem is executives want (and perhaps need) their directs to report and track one big number month over month. If you give them five metrics they'll never know if you're making progress or just oscillating between a few local minima. And if each of their ten directs has five metrics, you now have 50 numbers and no idea what time it is[1].
I agree and that’s why we need and indeed have an ever evolving landscape of benchmarks
> Private, refreshed test sets attack the mechanism itself, and in my view they are the only intervention that does. If the questions have never touched the public Web, they can’t be in the training data; if they rotate, memorizing this year’s set doesn’t help next year.
That’s what we have. A fresh public benchmark is also good, and teams do make efforts to decontaminate training data but there’s likely just no great way around leakage.
Btw, lots more issues in benchmarks than the ones discussed; for instance you can leak answers from the questions themselves or in the case of e.g. multiple choice formats in the actual answers. You just pass the MCQ choices themselves to the model and it may be able to guess way above chance. Coding agent benchmarks sometimes forget to delete .git. They mention e.g. a 6.9% error rate in one of the benchmark items, this seems pretty typical and I would actually be fine shipping that.
Benchmarks are very ugly, but if they didn’t exist we would need to invent them. All of the problems above and more do not explain the progress we see. There are probably 50,000 benchmarks in the literature and new ones get created frequently with varying levels of quality and usefulness.
“When you place a tangible value on trust, trust becomes a commodity to be bought and sold.”
— <https://news.ycombinator.com/item?id=27432186>
I think Goodhart's law is just a consequence of correlation vs causation.
It is very easy to find a metric that is correlated with what you want. But once you start trying to influence a system, you quickly push it out of the range where the correlation holds.
In order to optimize for something, you need to maximize the actual causative variable. This is much harder.
I think the difference is that Goodhart's law describes how the causal chain _changes_ as a result of management behavior, and in particular the incentives they design for the labor they manage. Incentives are a causal variable for outcomes, and what happens is that people find much easier ways to produce the outcomes you thought you wanted.
Like if you manage a call center and set up KPIs around average call time, reps will start hanging up on customers. Employees could always have done that, and the causal link was always there, there was just no reason to.
IMO the problem is executives want (and perhaps need) their directs to report and track one big number month over month. If you give them five metrics they'll never know if you're making progress or just oscillating between a few local minima. And if each of their ten directs has five metrics, you now have 50 numbers and no idea what time it is[1].
[1]: https://en.wikipedia.org/wiki/Segal%27s_law "A man with two watches never knows what time it is"
obviously, the best benchmark is the one you tell no one about.
Jokes on them, I don't trust benchmarks
Once something becomes a benchmark it is no longer a good benchmark.
No, it's when a benchmark becomes a target. You might have a private benchmark that you tell no one about. Would you not trust it?