Show HN: Cogitator – A Python Toolkit for Chain-of-Thought Prompting
a year ago
- #Python
- #LLM
- #CoT-Prompting
- Cogitator is a Python toolkit for experimenting with chain-of-thought (CoT) prompting in large language models (LLMs).
- CoT prompting improves LLM performance on complex tasks by generating intermediate reasoning steps before the final answer.
- The toolkit supports OpenAI and Ollama as LLM providers and includes Pydantic validation for structured model output.
- It provides a customizable benchmarking framework and implementations of popular CoT strategies like Self-Consistency.
- Installation can be done via pip or by cloning the GitHub repository and setting up a Python environment with Poetry.
- Example usage includes running Self-Consistency CoT with Ollama, demonstrated with a sample question and prompt.
- The project includes documentation, examples, and a benchmarking framework for evaluating CoT strategies on datasets like GSM8K.
- Contributions are welcome, and users are encouraged to cite the project if used in research.
- The toolkit is licensed under the MIT License.