Benchmarking Humans and AI in Contract Drafting

7 hours ago

Copy Link

AI tools matched or outperformed human lawyers in producing reliable first drafts, with the top AI tool achieving a 73.3% reliability rate compared to 70% for the top human lawyer.
Legal AI tools were more likely to identify material risks, raising explicit risk warnings in 83% of outputs compared to 55% for general-purpose AI tools, while humans raised none.
Specialized legal AI tools did not significantly outperform general-purpose AI in output reliability or usefulness but excelled in platform workflow support, integrating better into lawyers' workflows.
Human lawyers demonstrated strengths in interpreting legal instructions, exercising commercial judgment, and managing complex, multi-source inputs, areas where AI struggled.
AI tools were faster and more consistent in routine drafting tasks, producing outputs in seconds compared to nearly 13 minutes per task for humans.
Lawyers are increasingly using multiple AI tools, with 83% of surveyed lawyers using more than one tool for different tasks.
Workflow integration and context management were more critical to lawyers than raw accuracy, with only 6% requiring 100% accuracy from AI tools.
The future of contract drafting lies in combining the speed and consistency of AI with the judgment and contextual understanding of human lawyers.

Hasty Briefsbeta