Show HN: Agent Draw: An agent draws while you talk, built on TLDraw

2 days ago

Agent Draw is built on tldraw's infinite-canvas SDK for React, allowing an AI agent to draw on the canvas while a user presents.
The tool works by capturing a rectangle on the canvas, recording speech via a browser mic, transcribing it to text using Mistral's Voxtral model, and queuing multiple drawings for sequential processing.
Implementation involves a state machine-based canvas tool, an audio recorder, a transcription route in a Cloudflare Worker, and integration with tldraw's official Agent starter kit for the drawing logic.
Using agent.prompt instead of agent.request ensures the AI completes entire drawings in one go without needing follow-up turns, and removing non-essential actions like setMyView and review reduces model calls.
The system prompt instructs the AI to interpret requests visually (e.g., diagrams, illustrations) rather than transcribing speech verbatim, encouraging immediate drawing without stalling.
Performance varies by AI model: Claude Opus 4.8 handles complex scenes well, while smaller models like Claude Haiku 4.5 may produce simpler compositions, and weaker models like Gemini 2.5 Flash Lite might give up.
The feature is added to tldraw's toolbar via its override API, making it accessible with an icon and keyboard shortcut for user interaction.
Licensing: The code is MIT, but tldraw SDK requires a proprietary license key for public deployment, with free hobby licenses available for non-commercial projects.

Hasty Briefsbeta