ThinkMesh: A Python lib for parallel thinking in LLMs
17 days ago
- #Python
- #LLM
- #Parallel Reasoning
- ThinkMesh is a Python library for parallel reasoning with confidence gating and budget reallocation.
- Supports offline Hugging Face Transformers and vLLM/TGI, as well as hosted APIs like OpenAI and Anthropic.
- Features include async execution, reducers (majority/judge), pluggable verifiers, caching, and JSON traces.
- Early development phase; breaking changes may occur.
- Install via `pip install -e ".[dev,transformers]"` and clone the GitHub repository.
- Configurable with `ThinkConfig` for model, strategy, reducer, and budget settings.
- Supports strategies like DeepConf, self-consistency, debate, and tree-based reasoning.
- Emits JSON traces, Prometheus metrics, and OpenTelemetry spans.
- MIT licensed; encourages citation in academic work.