api documentation

I graded 36 popular MCP servers on agent usability. A third got a D or F

2 months ago

The author scanned 36 popular MCP servers and found that a third are failing in agent usability despite protocol compliance.
Key finding: Most failures stem from missing parameter descriptions in tool schemas, often because zod or OpenAPI definitions lack `.describe()` calls.
The mcpgrade tool provides a Lighthouse-style scorecard for MCP servers, running with one command and no API key.
Results: 15 servers scored A (e.g., brave-search, exa), 11 scored D/F (e.g., MongoDB, Notion, Airtable), and 2 were unscannable.
Finding 1: The ecosystem has an 'undocumented-parameter epidemic' — servers like firecrawl had 132 of 134 errors from missing descriptions.
Finding 2: Small catalogs tend to score higher, but discipline (e.g., shrimp-task-manager with 15 tools scored A/96) proves large, well-documented catalogs are possible.
Finding 3: Compliance checkers cannot measure usability; archived servers with manual documentation can outperform actively developed ones.
Finding 4: Static scores correlate with real model behavior — well-documented servers achieved 100% tool selection accuracy, while fuzzy catalogs like firecrawl dropped to 84% and refused only 50% of out-of-scope tasks.
Best practices include: every tool description answers what/ when/ returns; every parameter has description with format and example; use enum for fixed values; verb_object naming; error messages that enable self-correction.
mcpgrade offers 24 rules with concrete fixes; maintainers can rescan after improvements.