Hasty Briefsbeta

All tags

#database

200 stories total

Bilingual

Turso SQLite Offline Sync Public Beta
a year ago
- Turso Offline Sync is now in public beta, allowing applications to function seamlessly offline.
- Local database operations continue normally and sync automatically when connectivity is restored.
- Turso uses Embedded Replicas to keep local databases in sync with Turso Cloud.
- Previously, sync was unidirectional; now, local databases can accept writes and sync later.
- Offline Sync enables local-first applications, making them fast and resilient to network failures.
- Turso's architecture simplifies local-first apps by syncing the full database.
- Use cases include on-device apps, field data collection, and more.
- Public beta features include TypeScript and Rust support, with more languages planned.
- Setup involves creating a new database in AWS and enabling offline mode in the client.
- Expo integration is available for building offline-first mobile apps.
- Future features include bidirectional sync, conflict resolution, and more.
- Turso Offline Sync is not yet recommended for production use during the beta.
Declarative Schemas for simpler database management
a year ago
- Release of declarative schemas to simplify managing complex database schemas.
- Declarative schemas allow defining database structure in centralized, version-controlled .sql files.
- Benefits include single pane of glass for schema management, versioned migrations, and concise code reviews.
- Comparison between declarative schemas and traditional migrations, highlighting complexity in large schemas.
- Example of a complex projects table with RLS and ABAC, including views and trigger functions.
- Process of adding a new column in traditional migrations vs. declarative schemas.
- Adoption of declarative schemas in production reduces development time and simplifies updates.
- Introduction of schema diff tools like migra to automate migration file generation.
- Declarative schemas now available on Supabase, with documentation and CLI tools.
- Encouragement to try declarative schemas for simplified database development.
SpacetimeDB v1.0: Multiplayer at the speed of light
a year ago
- SpacetimeDB is a combined database and server system.
- It allows uploading application logic directly into the database via 'modules'.
- Clients connect directly to the database, eliminating the need for separate web or game servers.
- All application logic, including permissions and authorization, is written in Rust and deployed as a single binary.
- SpacetimeDB is optimized for real-time applications like games and chat, with low latency and high speed.
- It holds all application state in memory while persisting data in a write-ahead-log (WAL) for recovery.
- SpacetimeDB can be installed on macOS, Linux, and Windows via simple scripts or from source.
- The system is used in the MMORPG BitCraft Online for real-time data processing and synchronization.
- SpacetimeDB supports multiple programming languages for module development.
- It is licensed under BSL 1.1, converting to AGPL v3.0 with a linking exception after a few years.
Cloudflare Acquires Outerbase
a year ago
- Cloudflare has acquired Outerbase, integrating its database management tools into Cloudflare's platform.
- Outerbase simplifies database interaction with features like table browsing, query editing, and visualization, making data accessible to both developers and non-developers.
- Starbase, a SQLite-compatible database built on Cloudflare's Durable Objects, offers automatic REST APIs, row-level security, and WebSocket support.
- The integration will bring Outerbase's data explorer, REST APIs, and query editor to Cloudflare's dashboard, enhancing D1 and Durable Objects.
- Outerbase's hosted cloud will shut down on October 15, 2025, but self-hosting options will remain available.
- Future updates include improved Durable Objects experience, Workflow visualization, and pre- and post-query hooks for D1.
- The acquisition aims to make data more accessible and intuitive, leveraging Cloudflare's infrastructure and Outerbase's developer experience.
SpacetimeDB
a year ago
- SpacetimeDB stores a full-transaction history for applications.
- Allows resetting the database state to any previous point in time.
- Enables replaying transactions from any historical point.
- Provides replays for free, likened to time-travel.
Vertical Sharding Sucks
a year ago
- Vertical sharding reduces load on the main database but complicates the application architecture.
- Vertical sharding decreases uptime because multiple databases must be operational for the app to function.
- Application-layer joins become necessary when tables are split across databases, increasing complexity and potential for bugs.
- Vertical sharding can lead to duplicated data and increased latency as developers try to work around the limitations.
- The Postgres ecosystem lacks a built-in OLTP sharding solution, unlike MySQL with Vitess or DynamoDB.
- PgDog is an open-source project aimed at providing a sharding solution for Postgres to improve productivity.
Show HN: HelixDB – Native graph and vector types for RAG and retrieval
a year ago
- HelixDB is a high-performance, multi-model database built in Rust with LMDB as its storage engine.
- Designed for developer experience and efficient data operations, ideal for AI and vector-based applications.
- Features include fast performance, native vector data support, developer-friendly tools, and ACID compliance.
- Available as a fully managed cloud service with enterprise support options.
- Helix CLI tool facilitates local deployment, query checking, and instance management.
- Current focus areas: enhancing vector capabilities, query language, developer tools, and performance.
- Licensed under GPL-3.0.
Elasticsearch 9.0 has been released
a year ago
- Elasticsearch 9.0 introduces the experimental `rank_vectors` field type for late-interaction ranking with dense vectors.
- ES|QL LOOKUP JOIN is now in technical preview, enabling data enrichment and cross-index joins without complex queries.
- The `semantic_text` field type is now generally available (GA), facilitating semantic search setup with minimal ramp-up time.
- New features include authentication updates (SSHA-256 for API keys), authorization enhancements, and CCS improvements.
- Data stream enhancements include reindexing optimizations, migration APIs, and consistent OTel log/event mappings.
- ES|QL receives significant updates: cross-cluster telemetry, nulls support in Categorize, and LOOKUP JOIN field-caps integration.
- Machine Learning adds DeBERTa-V2/V3 tokenizer, Jina AI API support, and enterprise license checks for Inference API.
- Security updates include Kibana role expansions, reserved role queryability, and `.reindexed-v8-*` index management.
- Snapshot/Restore improvements: IMDSv2 support for S3, separate operation/request stats, and throttled CAS upload retries.
- Bug fixes span ES|QL, data streams, mappings, inference services, and search functionalities for stability and performance.
Show HN: Advanced-Alchemy – A framework agnostic library for SQLAlchemy
a year ago
- Advanced Alchemy is a companion library for SQLAlchemy, offering sync and async repositories with CRUD and bulk operations.
- Supports integration with major web frameworks like Litestar, Starlette, FastAPI, and Sanic.
- Includes custom-built Alembic configuration, CLI, and utility base classes with audit columns and primary keys.
- Features optimized JSON types, support for UUID6/UUID7, and Nano ID with optional extras.
- Tested with multiple database backends including SQLite, Postgres, MySQL, Oracle, and more.
- Provides pre-configured base classes, synchronous/asynchronous repositories, and service classes for easier data handling.
- Includes examples for Litestar, Flask, FastAPI, Starlette, and Sanic integration.
- Encourages community contributions with guidelines available in the contribution guide.
Instant SQL for results as you type in DuckDB UI
a year ago
- Instant SQL allows real-time SQL query updates as you type, eliminating the need for a run button.
- Available in Preview in MotherDuck and DuckDB Local UI, it aims to make SQL writing less tedious and slow.
- Features include real-time result set previews, CTE inspection and editing, and complex column expression debugging.
- Works with various data sources like DuckDB tables, MotherDuck, parquet files in S3, Postgres, SQLite, MySQL, Iceberg, and Delta.
- Includes AI-powered edit suggestions with immediate previews of changes.
- Built on DuckDB's local-first design and MotherDuck’s dual execution architecture for low latency.
- Utilizes DuckDB's JSON extension for AST generation and sophisticated local caching strategies.
- New prompt-based Edit feature available for MotherDuck users.
- MotherDuck is hiring for roles focused on building data-intensive interfaces.
Anatomy of a SQL Engine
a year ago
- SQL engines perform parsing, binding, plan simplification, join exploration, plan costing, execution, and spooling results.
- Parsing converts a query into an abstract syntax tree (AST) using left-recursive Yacc grammar for efficiency.
- Binding matches AST identifiers to database catalog symbols, similar to variable scoping in programming languages.
- Plan simplification regularizes SQL syntax into a canonical form for better execution performance.
- Join exploration involves searching for optimal join configurations and costing them based on database statistics.
- Execution converts the final plan into an executable format, often using row-based iterators.
- Spooling translates result rows into the appropriate format for network transmission.
- Future improvements may include unifying intermediate representations and optimizing memory management.
Wikipedia: Database Download
a year ago
- Wikipedia offers free copies of all content under Creative Commons Attribution-ShareAlike 4.0 License (CC-BY-SA) and GNU Free Documentation License (GFDL).
- Offline Wikipedia readers include Kiwix, XOWA, WikiTaxi, and others for various platforms.
- Database dumps are available in SQL and XML formats from dumps.wikimedia.org and the Internet Archive.
- Multistream dumps are recommended for efficient access to articles without decompressing the entire file.
- Images and media files are available under various licenses and can be downloaded from mirrors.
- Compressed dump files require specific decompression tools like bzip2, 7-Zip, or WinRAR.
- File system and operating system limits affect the handling of large Wikipedia dump files.
- Tips for detecting corrupted files include checking MD5 sums and using updated download tools.
- Dynamic HTML generation from local XML dumps is possible with tools like XOWA and WikiFilter.
- WikiTaxi and BzReader are offline readers for Windows that allow browsing Wikipedia without an internet connection.
- EPWING format allows reading Wikipedia offline on various platforms, including mobile devices.
- Mirror building tools like WP-MIRROR enable local browsing of Wikipedia content.
Understanding transaction visibility in PostgreSQL clusters with read replicas
a year ago
- Jepsen report highlights transaction visibility behavior in Amazon RDS for PostgreSQL Multi-AZ clusters.
- Issue involves Long Fork anomaly, affecting visibility order between primary and replicas.
- No data loss or corruption; issue known to PostgreSQL community since 2013.
- Example scenario: Alice and Bob see different rankings due to visibility order differences.
- AWS is working with PostgreSQL community to propose a fix using Commit Sequence Numbers (CSN).
- Long Fork anomaly impacts distributed systems, query routing, and data synchronization.
- AWS commits to PostgreSQL's success, with a dedicated PostgreSQL Contributors Team.
- Recommendations: review application assumptions, use explicit synchronization, contact AWS Support.
- Further reading available on PostgreSQL community work and Aurora solutions.
Fast, extensible and composable SQL sigils in Elixir
a year ago
- Extensible SQL parser and sigil for Elixir.
- Automatic parameterized queries for confident SQL writing.
- Lowers barrier for DBAs to contribute without translating SQL to Ecto.Query.
- Composable queries without needing to remember SQL clause order.
- Interpolation-al queries to avoid fiddling with fragments.
- Example usage with SQL sigil (~SQL) for query composition.
- Integration with Ecto.Repo for executing SQL queries.
- Enumerable protocol implementation for SQL queries.
- Benchmark results available for performance evaluation.
- Installation via Hex package manager (mix.exs dependency).
- Documentation generation with ExDoc and availability on HexDocs.
Show HN: AnuDB– Backed on RocksDB, 279x Faster Than SQLite in Parallel Workloads
a year ago
- AnuDB is a lightweight, serverless document database for C++ applications, using MessagePack for JSON document serialization.
- Built on RocksDB, AnuDB ensures atomicity, durability, and consistency, with adjustable memory/CPU usage for performance tuning.
- Supports Docker and MQTT for containerized environments and real-time IoT data communication.
- Features include embedded & serverless operation, JSON document storage, flexible querying, indexing, and transactional properties.
- AnuDB offers high concurrency with 32 worker threads for MQTT requests, TLS security, and cloud MQTT broker compatibility.
- Installation involves cloning the repository, building with CMake, and optional ZSTD compression support.
- Provides a comprehensive API for database operations, including document CRUD, collection management, and querying.
- Supports cross-compilation for embedded platforms and includes examples for quick start.
- Performance tips include indexing frequently queried fields and using ZSTD compression on embedded devices.
- Limitations include being embedded-only with no built-in replication or sharding.
SQLite 3.49.2 Released
a year ago
- SQLite Release 3.49.2 includes several enhancements and bug fixes.
- Query planner improvements for WITHOUT ROWID tables and large star-query joins.
- Enhanced iif() SQL function to accept any number of arguments greater than or equal to two.
- Session extension now works with databases using generated columns.
- Replaced Autotools with Autosetup for the configure script in the precompiled amalgamation tarball.
- Three new options added to sqlite3_db_config(): SQLITE_DBCONFIG_ENABLE_ATTACH_CREATE, SQLITE_DBCONFIG_ENABLE_ATTACH_WRITE, SQLITE_DBCONFIG_ENABLE_COMMENTS.
- Bug fixes include memory errors, incorrect query answers, and issues with the generate_series() extension function.
- Incremental improvements to configure/make scripts.
QueryHub
a year ago
- QueryHub is an AI-powered platform designed to simplify database interactions using natural language.
- It addresses challenges like complex SQL query writing, time-consuming database analysis, and limited access to database expertise.
- Core capabilities include an AI-powered chatbot, automatic SQL generation, schema understanding, query management, and data visualization.
- QueryHub democratizes data access, accelerates development, and enhances data understanding.
- Getting started is seamless with a free tier offering 1 database connection and 100 AI queries per month.
- Security features include AES-256-CBC encryption, secure key management, and TLS protocols for data transmission.
ToyDB rewritten: a distributed SQL database in Rust, for education
a year ago
- toyDB is a distributed SQL database written in Rust as an educational project.
- Features include Raft distributed consensus, ACID transactions with MVCC-based snapshot isolation, and pluggable storage engines.
- Includes an iterator-based query engine with heuristic optimization and time-travel support.
- Supports SQL features like joins, aggregates, and transactions.
- Originally created in 2020 to learn about database internals, rewritten based on professional experience.
- Designed for simplicity and correctness, not performance or scalability.
- Includes architecture guide, SQL examples, SQL reference, and research materials.
- Can run a local five-node cluster and includes a command-line client.
- Uses Golden scripts for testing and comes with a workload benchmark tool.
- Debugging support via VSCode and CodeLLDB extension.
Show HN: Helixdb – Open-source vector-graph database for AI applications (Rust)
a year ago
- HelixDB is an open-source, high-performance graph-vector database written in Rust, designed for RAG and AI applications.
- Powered by LMDB via Heed3, it offers reliability and modern features tailored for AI and vector-based applications.
- Performance highlights include being 1000x faster than Neo4j, 100x faster than TigerGraph, and on par with Qdrant for vectors.
- Features native support for graph and vector data types, making it ideal for RAG applications.
- Provides the easiest way to store relationships between nodes, vectors, or both.
- Ensures data integrity with ACID compliance and robust storage via LMDB.
- Includes a CLI tool for installation, setup, query compilation, and deployment.
- Supports TypeScript and Python SDKs for querying the database.
- Current focus areas include expanding vector data type capabilities, enhancing query language robustness, and implementing a test suite.
- Long-term projects include developing an in-house graph-vector storage engine and network protocol libraries.
- Licensed under AGPL, with managed services and enterprise support available for selected users.
Databricks and Neon
a year ago
- Databricks announces acquisition of Neon, a serverless Postgres company.
- Neon's architecture separates storage and compute, designed for modern developers and AI systems.
- Neon's team includes Postgres experts and database veterans.
- Neon was founded to disrupt the database industry with a focus on developer experience.
- Neon's architecture enables features like database branching similar to Git.
- AI agents create over 80% of databases on Neon, up from 30% last year.
- Neon's founders have strong backgrounds in databases and open source.
- Databricks and Neon share a commitment to open source and technical innovation.
- OLTP databases are a $100B market ripe for disruption by developers and AI.
- Existing Neon customers will continue to receive support and innovation under Databricks.
- Databricks plans to share more at the Data + AI Summit in June.

firstprev0next

About|Login

#database

Turso SQLite Offline Sync Public Beta

Declarative Schemas for simpler database management

SpacetimeDB v1.0: Multiplayer at the speed of light

Cloudflare Acquires Outerbase

SpacetimeDB

Vertical Sharding Sucks

Show HN: HelixDB – Native graph and vector types for RAG and retrieval

Elasticsearch 9.0 has been released

Show HN: Advanced-Alchemy – A framework agnostic library for SQLAlchemy

Instant SQL for results as you type in DuckDB UI

Anatomy of a SQL Engine

Wikipedia: Database Download

Understanding transaction visibility in PostgreSQL clusters with read replicas

Fast, extensible and composable SQL sigils in Elixir

Show HN: AnuDB– Backed on RocksDB, 279x Faster Than SQLite in Parallel Workloads

SQLite 3.49.2 Released

QueryHub

ToyDB rewritten: a distributed SQL database in Rust, for education

Show HN: Helixdb – Open-source vector-graph database for AI applications (Rust)

Databricks and Neon