- Steering involves manipulating LLM activations mid-inference to guide outputs, made practical for local models like DeepSeek-V4-Flash.
- Two methods to extract steering vectors: naive subtraction of activations between prompted pairs, and advanced sparse autoencoders to map features to concepts.
- Steering is intriguing but often outcompeted by prompt engineering for basic adjustments, as prompts already achieve fine-grained control.
- Potential value in steering for concepts like 'intelligence' or 'codebase knowledge' that are hard to prompt, though these may require fine-tuning instead.
- Open-source community may explore steering more with tools like DwarfStar 4, especially for removing refusals or boosting specific behaviors without damaging model capabilities.