Do you put rules or examples in your LLM context?
a day ago
- #SQL Generation
- #LLM Context Optimization
- #Example Libraries
- Derivation distance refers to how much transformation and composition an LLM needs to perform to get from given facts to an answer.
- Completeness of context is crucial; missing business rules leads to poor results.
- Adding a full example library significantly improves accuracy for smaller models (e.g., Haiku from 70% to 82%).
- Stronger models like Opus perform well with minimal context (e.g., 98% with complete English descriptions).
- Consistency matters more than volume; contradictions can erase benefits of additional context.
- A broad library covering many query patterns is more effective than a few relevant examples.
- The example library helps by providing reusable solution paths and disambiguating underspecified questions.
- Cost considerations include token usage, model pricing, and maintenance efforts to avoid inconsistencies.