Ollama vs. Llama.cpp – Quick Benchmark
11 hours ago
- #performance comparison
- #LLM serving
- #AI benchmarks
- Ollama and llama-server benchmark on Tesla V100 using Qwen3 8B Q4_K_M GGUF.
- Generation speeds nearly identical: llama-server at 111.00 tok/s, Ollama at 110.61 tok/s.
- Prompt processing faster in llama-server: 1255.29 tok/s vs 1028.02 tok/s for Ollama.
- Ollama used prompt caching, inflating results if not disabled via unloading.
- Benchmark limited: short prompt (60 tokens), one model/GPU, no concurrent requests.
- llama-server recommended for long prompts; Ollama comparable for generation speed.
- vLLM suggested as another candidate but unsupported on V100's Volta architecture.