Iroh Global Content Discovery
3 hours ago
- The motivation for global content discovery stems from IPFS's ability to publish content globally, but the shutdown of IPFS Shipyard adds urgency.
- BitTorrent is the current leading system for permissionless global content discovery, with its Mainline DHT providing fast lookups (under 1 second).
- BitTorrent's transfer protocol uses .torrent files with piece hashes (SHA-1), while iroh-blobs replaces this with BLAKE3 verified streaming for more efficient validation.
- Mainline DHT is minimalistic: single-packet UDP, provider records are just host:port. To use it with iroh's Ed25519 endpoint identities, a UDP address index service maps host:port to EndpointId.
- The workflow involves announcing content via Mainline and address index, then discovering via get_peers and address index to get endpoints, followed by iroh connection.
- For websites, a link syntax using blake3.net is combined with a browser plugin that rewrites links to a local gateway, which serves content-addressed data.
- Pkarr provides permissionless mutable naming (pkarr.net), allowing owners of Ed25519 keys to publish DNS records on Mainline DHT.
- The system is experimental and has limitations: scalability, unencrypted traffic, lack of privacy, and no post-quantum security, but it is a starting point.
- Future work includes improving the DHT (possibly extending Mainline or writing a new one) and achieving 1.0 releases for core protocols.