pragma.vision Technology observatory & strategic foresight

Verification register Security & Identity

Readiness verdict

METR RE-Bench

A dated reading of what is claimed, reported, and independently verified in the current evidence.

As of
2026-09-30
Revision
3
Method
v1.4.0

Current reading

Readiness band and full integer triple

AI-assisted assembly · derived results

Readiness band

Watch

Primary summary from verified readiness

Confidence
87% · stale
Computed at
2026-09-30T21:06:04.529782+00:00
Claimed
55

Public ambition and stated capability

Reported
45

Observed practitioner reporting

Verified
35

Independently supported evidence

Gap
+20

Claimed leads verified

Evidence strength Growing

Decision

What the current evidence supports

Agent-assisted assessment · independently reviewed · 2026-09-30

Use as one component of a research-agent evaluation.

Why
The environments are usable, but reward hacking and narrow task coverage limit conclusions about real research ability.
Next
Pin the task and harness versions, inspect trajectories for integrity violations, and pair results with held-out research tasks.

Constraints

Blockers

  • Reward hacking can invalidate apparent task success.
  • Seven environments provide limited research-domain coverage.
  • Harness-port differences affect comparability.
  • GPU requirements increase evaluation cost.
  • General research-productivity validity remains uncertain.

Evidence summary

Derived counts

AI-assisted assembly

Total
10
Tier 1
8
Tier 2
2
Tier 3
0
Supports
4
Contradicts
3
Context
3
Latest observed
2026-05-08

Counts and dates only. Raw signals, private excerpts, trust records, and internal corpus material are not published here.

Publication record

Revisions

Revision 3

  1. 2026-07-19 Reading moved from ready to ready.
  2. 2026-09-22 Reading moved from ready to ready.
  3. 2026-09-30 Reading moved from ready to watch.

Your opinion

Tell us anything.

What works, what doesn't, what's missing — especially about our watches, lenses, and the register itself. Anonymous is fine; leave an email if you'd like a reply.