pragma.vision Technology observatory & strategic foresight

Verification register AI & Agents

Readiness verdict

SWE-bench Multimodal v2

A dated reading of what is claimed, reported, and independently verified in the current evidence.

As of
2026-09-22
Revision
1
Method
v1.3.0

Current reading

Readiness band and full integer triple

AI-assisted assembly · derived results

Readiness band

Ready

Primary summary from verified readiness

Confidence
54% · fresh
Computed at
2026-09-22T00:12:00.533021+00:00
Claimed
70

Public ambition and stated capability

Reported
66

Observed practitioner reporting

Verified
59

Independently supported evidence

Gap
+11

Claimed leads verified

Evidence strength Strong

Decision

What the current evidence supports

Human editorial judgment · 2026-09-22

Track; not yet

Why
Legitimate, primary-sourced, and directly relevant (agentic coding + multimodal UI evaluation intersects jm/pv agent tooling), but the leaderboard is too thin (4 models) and the contamination-resistance claim is inherited from Verified critiques rather than proven for this exact task set — premature to adopt as a decision input.
Next
Re-check the Multimodal v2 leaderboard once >=8-10 frontier models have reported scores (currently only 4) and watch for an independent contamination audit of the v2 480-task set specifically, before citing it as an evaluation gate for any Pragma.Vision coding-agent work.

Constraints

Blockers

No named blocker is present in the current public projection.

Evidence summary

Derived counts

AI-assisted assembly

Total
5
Tier 1
0
Tier 2
1
Tier 3
4
Supports
2
Contradicts
2
Context
1
Latest observed
2026-09-21

Counts and dates only. Raw signals, private excerpts, trust records, and internal corpus material are not published here.

Publication record

Revisions

Initial public reading

  1. 2026-09-21 Reading moved from ready to ready.

Your opinion

Tell us anything.

What works, what doesn't, what's missing — especially about our watches, lenses, and the register itself. Anonymous is fine; leave an email if you'd like a reply.