What is METR RE-Bench?

RE-Bench is an evaluation suite from METR that measures how well autonomous agents perform open-ended machine learning research engineering work, placing them in environments with an explicit scoring function and a fixed time budget in which to improve a solution. Human experts attempt the same environments under comparable conditions, so agent scores can be read against a human baseline rather than an absolute threshold.