An experimental guide to Answer Engine Optimization

3 days ago

Answer Engine Optimization (AEO) focuses on making content easily understandable and citable by AI models, as traditional search engine ranking may no longer suffice.
The author rebuilt their content pipeline using markdown as the foundation, as AI models are trained on it and it avoids the complexity of HTML with nested elements.
Implemented llms.txt, a curated index for AI agents similar to a sitemap, to help models efficiently navigate and fetch relevant site content.
Added middleware to detect AI agents via user-agent headers or Accept: text/markdown, serving raw markdown instead of HTML to improve content accessibility.
Enriched markdown with metadata in frontmatter, such as business details and structured data, to provide AI models with clear, verifiable context about the content.
Set permissions using robots.txt and Content-Signal headers to control how AI systems can use the content, balancing access with restrictions on training.
The approach includes practical steps like supporting .md extensions for explicit markdown requests and handling duplicate content with canonical headers.
Caveats include potential issues with user-agent sniffing, the evolving list of AI bots, and the complexity of merging metadata from various sources.
The overall goal is to create a site that is both user-friendly and AI-accessible, positioning content for future trends in AI-mediated information discovery.

Hasty Briefsbeta