Running our Docker registry on-prem with Harbor
14 days ago
- #DevOps
- #Containerization
- #Infrastructure
- 37signals migrated from Dockerhub and Amazon ECR to an on-premise Harbor registry to address cost, performance, security, and independence issues.
- Key issues with external registries included high costs, slow deployment times due to bandwidth limits, security risks, and API limitations.
- Harbor was chosen for its open-source nature, reliability, performance, and ease of setup, with minimal extra tooling required.
- The setup involved using S3-compatible storage (Pure FlashBlade), configuring multiple Harbor instances for redundancy, and implementing retention policies to minimize storage footprint.
- Replication between Harbor instances was configured using Terraform, with a two-way replication scheme to keep data in sync.
- A scripted approach was used to migrate Dockerhub repositories to Harbor in batches to avoid API throttling.
- Performance improvements included up to 25-second faster image pulls and 15-second faster deploy times, along with significant cost savings.
- The new setup reduced storage usage from 9 TiB to 1.5 TiB and saved approximately $5k/year in subscription fees.
- Harbor has proven reliable, serving over 32,000 pulls in two months, demonstrating the benefits of moving away from SaaS and public cloud providers.