Alibaba Cloud claims to reduce Nvidia GPU use by 82%
10 hours ago
- #AI
- #GPU Optimization
- #Cloud Computing
- Alibaba Cloud's new Aegaeon system reduces Nvidia GPU usage by 82%.
- Aegaeon was tested in Alibaba Cloud’s model marketplace, cutting required GPUs from 1,192 to 213.
- The system serves dozens of large language models (LLMs) more efficiently.
- Researchers highlight excessive costs of serving concurrent LLM workloads.
- Most GPUs are underutilized, with 17.7% serving only 1.35% of requests.
- Global efforts focus on pooling GPU power to improve efficiency.