pragma.vision Technology observatory & strategic foresight

Verification register AI & Agents

Readiness verdict

Berkeley Function Calling Leaderboard V4 (BFCL V4)

A dated reading of what is claimed, reported, and independently verified in the current evidence.

As of
2026-10-02
Revision
4
Method
v1.4.0

Current reading

Readiness band and full integer triple

AI-assisted assembly · derived results

Readiness band

Emerging

Primary summary from verified readiness

Confidence
87% · fresh
Computed at
2026-10-02T00:16:37.404911+00:00
Claimed
55

Public ambition and stated capability

Reported
35

Observed practitioner reporting

Verified
25

Independently supported evidence

Gap
+30

Claimed leads verified

Evidence strength Strong

Decision

What the current evidence supports

Agent-assisted assessment · independently reviewed · 2026-10-02

Use BFCL V4 conditionally for exploratory model comparisons.

Why
Independent researchers have used its task subsets, but audits found grading errors, brittle state matching and stale answers that can distort rankings.
Next
Pin the evaluator and search configuration, inspect relevant task labels and failed traces, and supplement results with repeated tests on your own workflows.

Constraints

Blockers

  • Incorrect or brittle grading can reject valid tool use or accept invalid responses.
  • Live-web questions can outlast their recorded answers.
  • Search-provider and harness differences limit comparability across runs.
  • Repeated tool descriptions limit corpus diversity.
  • Full trace-level evidence for one major audit remains unavailable.
  • Benchmark scores require validation against the intended application workflows.

Evidence summary

Derived counts

AI-assisted assembly

Total
13
Tier 1
11
Tier 2
2
Tier 3
0
Supports
6
Contradicts
4
Context
3
Latest observed
2026-09-10

Counts and dates only. Raw signals, private excerpts, trust records, and internal corpus material are not published here.

Publication record

Revisions

Revision 4

  1. 2026-07-19 Reading moved from mature to mature.
  2. 2026-09-22 Reading moved from mature to mature.
  3. 2026-09-30 Reading moved from mature to watch.
  4. 2026-10-02 Reading moved from watch to emerging.

Your opinion

Tell us anything.

What works, what doesn't, what's missing — especially about our watches, lenses, and the register itself. Anonymous is fine; leave an email if you'd like a reply.