pragma.vision Your technology observatory

Verification register Security & Identity

Readiness verdict

METR RE-Bench

A dated reading of what is claimed, reported, and independently verified in the current evidence.

As of
2026-06-28
Revision
1
Method
v1.0.0

Current reading

The readiness gap, in one scan

AI-assisted assembly · derived results

Claimed
80

Public ambition and stated capability

Reported
75

Observed practitioner reporting

Verified
70

Independently supported evidence

Gap
+10

Claimed minus verified

Evidence strength Strong

Decision

What the current evidence supports

Human editorial judgment · 2026-06-28

Adopt with guardrails

Why
Most mature of the three — peer-reviewed (ICLR/OpenReview), MIT-licensed, actively maintained with continual frontier-model updates through 2026 and an established human baseline. Reward-hacking and saturation are real but manageable with manual transcript review.
Next
Pilot RE-Bench (plus HCAST/SWAA time-horizon tasks) as a capability tripwire in any internal frontier-model-routing or agent-autonomy gate; pair scores with reward-hacking transcript inspection rather than trusting normalized scores alone.

Constraints

Blockers

No named blocker is present in the current public projection.

Evidence summary

Derived counts

AI-assisted assembly

Total
7
Tier 1
0
Tier 2
0
Tier 3
7
Supports
2
Contradicts
2
Context
3
Latest observed
2026-05-08

Counts and dates only. Raw signals, private excerpts, trust records, and internal corpus material are not published here.

Publication record

Revisions

Initial public reading

This is the initial public reading. No earlier readiness change is recorded.

Your opinion

Tell us anything.

What works, what doesn't, what's missing — especially about our watches, lenses, and the register itself. Anonymous is fine; leave an email if you'd like a reply.