Just landed: streaming ingestion on Cloudflare with Arroyo and Pipelines
a year ago
- #StreamProcessing
- #Cloudflare
- #DataIngestion
- Launch of Pipelines open beta for streaming ingestion into R2.
- Pipelines allows high-volume, real-time data ingestion without infrastructure management.
- Acquisition of Arroyo to enhance data transformation and processing capabilities.
- Pipelines supports tens of thousands of requests per second (RPS).
- Integration with R2 Data Catalog for efficient data querying.
- Simple setup with Wrangler command for pipeline creation.
- Scalable architecture using Durable Objects and SQLite for immediate data persistence.
- Future plans include SQL-based transformations and additional data sinks.
- Initial beta phase offers free usage with future pricing based on data volume.
- Community engagement through Discord and example projects.