Agentic AI Eval Research — Phase 1, Part 1 · March 2026

The Blind Spot in Every AI Eval Framework

97M+ Monthly MCP
SDK Downloads
40% Enterprise AI with
Agents by 2026
14.2% Step Attribution
Accuracy (ICML 2025)
0 Frameworks
Solving This
VC
Viola Cao · March 2026 · 7 min read
Eval Frameworks MCP Tool Layer Attribution
Part of series: Part 1 — The Blind Spot → Part 2 — APEX Framework Solution
scroll
In this article
  1. The Eval Framework Blind Spot
  2. What the Research Confirms

Why This Matters

The Eval Framework Blind Spot

97M+
Python + TypeScript SDKs combined
40%
Gartner, Aug 2025
14.2%
Zhang et al., ICML 2025 Spotlight
✓ WHAT FRAMEWORKS COVER ✓ Model output quality Faithfulness, relevance, coherence ✓ RAG pipeline evaluation Context precision, retrieval accuracy ✓ Agent goal completion Did the task finish? End-to-end pass/fail ✓ Observability & tracing Spans, latency, token usage ✗ WHAT NO FRAMEWORK COVERS ✗ Semantic arg correctness SQL runs but returns wrong data — L2 ✗ Tool output trust validation Stale, injected, or partial results — L3 ✗ Chain-level failure attribution Which step caused failure? — L4 (14.2%) ✗ MCP protocol-level failures Tool selection, routing, injection via MCP
Fig. 1 — Coverage comparison. Every major eval framework covers the left column. None cover the right column — which maps directly to Layers 2, 3, and 4 of the taxonomy in Part 2.
PRACTITIONER ADMISSION

Standing on Shoulders

What the Research Confirms

ICML 2025 SPOTLIGHT · PENN STATE / DUKE
"Which Agent Causes Task Failures and When?"
LAYER 4 · CHAIN ATTRIBUTION
MICROSOFT AI RED TEAM · WHITEPAPER 2025
Taxonomy of Failure Modes in Agentic AI Systems
SECURITY FRAMING · ALL LAYERS
arXiv · FEBRUARY 2026
"MCP Tool Descriptions Are Smelly!"
LAYER 1 · TOOL SELECTION
arXiv · SEPTEMBER 2025
Diagnosing Failure Root Causes in Agentic Platforms
LAYER 4 · ROOT CAUSE GAP
NDSS 2026 · INJECAGENT / TOOLHIJACKER
Indirect Prompt Injection Benchmarks
LAYER 3 · INJECTION SECURITY
NIST · FEBRUARY 2026
AI Agent Standards Initiative
STANDARDS · REGULATORY SIGNAL
Continue Reading

Part 2: The Taxonomy & The Solution

Read Part 2 → Work Together