Hasty Briefsbeta

All tags

#python

387 stories total

Bilingual

Dataframely: A polars-native data frame validation library
a year ago
- QuantCo migrated data pipelines from pandas to polars for performance gains.
- Legacy codebases lacked clarity in data frame invariants, leading to inefficiencies.
- Existing validation libraries like pandera and patito had limitations for polars.
- dataframely was developed to address these shortcomings with polars-native support.
- Features include schema definition, validation, interdependent data frame checks, and test data generation.
- Soft-validation and failure introspection improve debugging in production pipelines.
- dataframely enhances code readability, robustness, and static type checking.
- Used successfully in multiple teams for analytical and production pipelines.
- Open-sourced to benefit the broader data engineering community.
Making PyPI's test suite 81% faster
a year ago
- Trail of Bits collaborated with PyPI to improve test suite performance, reducing execution time from 163 seconds to 30 seconds.
- Key optimizations included parallelizing test execution with pytest-xdist (67% reduction), using Python 3.12’s sys.monitoring for coverage (53% reduction), optimizing test discovery, and removing unnecessary imports.
- The test suite grew from 3,900 to 4,700 tests while achieving an 81% performance improvement.
- Parallelization required solutions for database fixtures, coverage reporting, and test output readability.
- Optimizing coverage with sys.monitoring in Python 3.12 further reduced test execution time by 53%.
- Strategic use of testpaths configuration reduced test discovery time by 66%.
- Removing unnecessary imports like ddtrace provided a modest 3.4% improvement.
- A proof-of-concept for database migration squashing showed potential but was not merged due to complexity.
- Faster tests improve security by encouraging frequent and thorough testing.
- Quick wins for other projects include parallelizing tests, optimizing coverage, speeding up test discovery, and eliminating unnecessary imports.
Converting a Git repo from tabs to spaces (2016)
a year ago
- The author converted Yelp's Python codebase from tabs to four spaces in 2013.
- Key steps included fixing inconsistent indentation, distributing a Git filter, and performing the conversion during a low-activity period.
- Git filters were used to ensure no new tabs were introduced, with both clean and smudge filters converting tabs to spaces.
- Merging and rebasing strategies were adjusted to handle the whitespace conversion, with some complications for old branches and stashes.
- The conversion caused minor issues like phantom-modified files and slow Git status checks, but these were manageable.
- The author recommends final cleanup steps like adding a pre-commit hook to reject tabs outright.
ePub-utils: a Python library and CLI tool for inspecting ePub from the terminal
a year ago
- A Python CLI and utility library for manipulating EPUB files.
- Parse and validate EPUB container and package files.
- Extract metadata like title, author, and identifier.
- Command-line interface for quick file inspection.
- Syntax highlighted XML output.
- Install the package using pip: `pip install epub-utils`.
- Use as a CLI tool to inspect container, package, and table of contents.
- Use as a Python library to load and access EPUB metadata.
Minimum Viable Blog
a year ago
- Instructions for creating a simple blog using Python and Markdown.
- Uses a template.html file for the blog structure.
- Script reads Markdown files, converts them to HTML, and renders them into blog posts.
- Includes a method to generate an index.html page listing all posts.
- Late-night music recommendation: Alice Danger - Give 'em Hell.
Show HN: Pipask – safer pip without compromising convenience
a year ago
- Pipask is a secure drop-in replacement for pip that performs security checks before installing Python packages.
- It relies on PyPI metadata to avoid unnecessary code execution and asks for consent when execution is required.
- Pipask can be installed via pipx (recommended) or pip, and used just like pip with commands like `pipask install requests`.
- For convenience, users can alias pip to pipask in their shell configuration.
- Pipask performs multiple security checks including repository popularity, package age, known vulnerabilities, download statistics, and metadata verification.
- It uses PyPI's JSON API and collects security data from pypistats.org, GitHub/GitLab, OSV.dev, and PyPI's integrity API.
- A dry-run mode (`--dry-run`) allows checking security without installation.
- Only explicitly requested packages undergo full checks; transitive dependencies only get vulnerability checks.
- If approved, pipask delegates the actual installation to standard pip.
Urtext: The Python plaintext library for people who've tried everything else
a year ago
- Urtext is an open-source library for plaintext writing, research, and documentation.
- It supports various uses like knowledge bases, journaling, Zettelkasten, and note-taking.
- Plaintext is fast, human-readable, flexible, cross-platform, and future-proof.
- Urtext can embed other syntaxes, including markup and programming languages.
- Extensible with Python, allowing for self-modifying or arbitrary code execution.
- Future-proof storage format as plain text, interpretable in any capable language.
- Freeform, flexible syntax combining content, structure, and instructions.
- Local-first operation, not dependent on cloud or subscription services.
- Minimum UI with theming and syntax highlighting, avoiding popups and menus.
- Handles file creation, naming, saving, and organization automatically.
- Python interpreter for Urtext runs on Python 3.3 or later.
Semantic unit testing: test code without executing it
a year ago
- Semantic unit testing evaluates if a function's implementation matches its documented behavior using LLMs.
- The `suite` Python library enables semantic unit testing by analyzing code and docstrings with LLMs.
- Example usage: `tester = suite(model_name="openai/o3-mini")` tests functions like `multiply` for semantic correctness.
- How it works: Extracts function info, dependencies, builds a prompt, and queries an LLM for evaluation.
- Reasons to avoid: LLMs can be unreliable, expensive for large codebases, and traditional tests are more dependable.
- Reasons to use: Broad coverage, catches edge cases early, integrates with pytest, and supports async/local models.
- Conclusion: Semantic testing complements but doesn't replace traditional tests; explore it as a supplementary tool.
FastAPI Cloud: deploying FastAPI apps with just a single command
a year ago
- FastAPI Cloud allows deploying FastAPI apps with a single command, handling automatic deployments, HTTPS, autoscaling, and more.
- FastAPI provides a great developer experience, velocity, and certainty with features like Pydantic validation and automatic API docs.
- Deploying to the cloud traditionally requires learning Kubernetes and cloud-native tools, which is time-consuming and complex.
- FastAPI Cloud simplifies cloud deployment, offering certainty and optimization for FastAPI and Python.
- The team behind FastAPI Cloud includes experienced developers with a strong track record in Python and open-source projects.
- FastAPI Cloud ensures no vendor lock-in, allowing users to deploy FastAPI apps anywhere while providing an excellent developer experience.
- The company raised $4.2M in seed funding, backed by notable investors and advisors from the Python and AI communities.
- FastAPI Cloud enables developers to build and deploy apps quickly, from simple projects to sophisticated products.
Kate and Python Language Server
a year ago
- Setting up Python language server (python-lsp-server) in Kate editor for virtual environments.
- Using python-lsp-ruff plugin to integrate Ruff linter and formatter with the language server.
- Creating a bash script (pylsp_in_env) to detect and activate virtual environments (.venv or venv) before running the language server.
- Configuring Kate's LSP settings to use the custom script and enable Ruff.
- Kate's %{Project:NativePath} argument provides the full path to the Python project.
- Solution differs slightly from Kate's documentation by checking multiple virtual environment folder names.
- Note: Poetry or other tools may require different setup.
Declarative Tcl/Tk UI Library for Python
a year ago
- Declarative Tcl/Tk UI library for Python with React-like features.
- Well-typed and closely maps to Tcl/Tk for debugging.
- Zero dependency and small enough to understand fully.
- Incomplete state with missing functionality.
- Example usage includes creating a counter and a TODO list application.
- Packer is the main method for arranging widgets.
- Future improvements include efficient diffing algorithms and potential Rust offloading.
- Development setup involves mypy and pytest for testing.
Show HN: PageQL – Embed SQL directly in HTML
a year ago
- PageQL is an experimental template language and micro Python web framework that allows embedding SQL inside HTML directly.
- Inspired by ColdFusion, Handlebars/Mustache, and HTMX.
- Install via pip: `pip install pageql`.
- Core philosophy: Simplifies web development with relational data by embedding SQL-like queries in HTML.
- Minimal approach: Embraces SQL and HTML, focusing on common web features (lists, forms, CRUD).
- Declarative focus: Encourages clarity by discouraging extensive scripting.
- Leverages SQL: Uses SQL engine for expression evaluation instead of a custom language.
- Key tags include `#from`, `#insert`, `#update`, `#delete`, `#set`, `#param`, `#import`, `#partial`, `#render`, `#if`, `#else`, `#redirect`.
- Example code demonstrates creating a table and querying data directly in HTML.
- Future plans: Embeddable, simple, performant, with potential for reactivity and client-based offline mode.
- Designed for extensibility, with possible reimplementation in other languages (C/C++/Rust/JavaScript).
- Reactive mode leverages declarative nature of HTML and SQL for easier reasoning.
- Client-server complexity reduced via SQL change synchronization protocol.
- Supports templating APIs similar to Mustache/Handlebars/Jinja.
- Currently for hobbyist projects, with future plans for advanced features like routing, authentication, and production deployment.
Sutton and Barto Book Implementation
a year ago
- Repository contains implementations of reinforcement learning algorithms from Sutton's book.
- Organized into modules covering topics like Multi-Armed Bandits, Policy Evaluation, and Temporal Difference.
- Includes model-free solvers requiring only states, actions, and a transition function.
- Example provided: Single State Infinite Variance Example 5.5.
- Monte Carlo Tree Search (MCTS) demonstrated with maze solving.
- Code is basic and not optimized; improvements are welcome.
- Useful for learning reinforcement learning from scratch.
Ty: A fast Python type checker and language server, written in Rust
a year ago
- An extremely fast Python type checker and language server written in Rust.
- The project is still in development and not ready for production use.
- Questions or bug reports should be submitted as issues in the repository.
- Development occurs in the Ruff repository; pull requests should be opened there for changes in the ruff submodule.
- The project is licensed under the MIT license.
- Contributions are also licensed under MIT without additional terms.
Show HN: US Routing – Python library for fast local routing in the United States
a year ago
- US Routing is a Python library for fast local routing in the United States.
- It bootstraps from the North American Roads dataset.
- Install via pip: `pip install us-routing` or via Poetry by cloning the repository.
- Example usage includes routing between cities, zip codes, or coordinates.
- Provides detailed route information like distance, duration, and steps.
- Includes functionality to download and process the North American Roads dataset.
- Development setup involves cloning the repo and installing dependencies with Poetry.
- Offers a BaseRouter class for custom routing graphs from shapefiles.
- Contributions are welcome via Pull Requests.
The Flask Mega-Tutorial, Part III: Web Forms
a year ago
- The Flask Mega-Tutorial, Part III focuses on working with web forms in Flask applications.
- Flask-WTF extension is introduced for handling web forms, integrating WTForms with Flask.
- Configuration management is discussed, emphasizing the importance of the SECRET_KEY for security.
- A LoginForm class is created with fields for username, password, remember_me checkbox, and a submit button.
- Form validation is implemented using DataRequired validator to ensure fields are not submitted empty.
- Templates for forms are created, including a login form that inherits from a base template.
- The login view function is updated to handle both GET and POST requests, validating form data upon submission.
- Flash messages are introduced to provide feedback to users upon form submission.
- Field validation errors are displayed next to the respective fields in the form template.
- The url_for() function is recommended for generating URLs dynamically, improving maintainability.
Show HN: Req Update Check
a year ago
- A Python tool named 'req-update-check' to check for package updates in requirements.txt files.
- Features include checking update severity (major/minor/patch), displaying package homepages and changelogs, and optional file caching for performance.
- Supports comments and inline comments in requirements.txt and ignores pre-release versions.
- Installation options include PyPI, direct repo installation, or from source.
- Basic usage involves running the tool against a requirements.txt file with options to disable caching or specify a cache directory.
- Example output shows package updates with version changes, severity, and links to Pypi, homepage, and changelog.
- Supports exact version specifiers (==) with plans to add other specifiers in future releases.
- Can be used as a Python library for programmatic checks.
- Development setup includes cloning the repo, creating a virtual environment, and installing development dependencies.
- Encourages contributions under the MIT License.
Show HN: Kit – open-source toolkit for building AI devtools
a year ago
- kit is a Python toolkit from Cased for building LLM-powered developer tools and workflows.
- It provides precise, accurate, and relevant context to LLMs for tasks like code reviewing, code generation, and building coding assistants.
- Features include mapping any codebase with structured views, file trees, and language-aware symbol extraction.
- Supports multiple search methods, combining fast text search with semantic vector search for optimal speed and accuracy.
- Offers fine-grained docstring context for finding code snippets, answering questions, and improving code generation.
- Includes ready-made utilities for code chunking, context retrieval, and interacting with LLMs to build AI workflows.
- MIT-licensed and available on GitHub.
How async/await works in Python
a year ago
- Async/await in Python allows programs to perform useful tasks while waiting for I/O operations to complete.
- Understanding async/await requires familiarity with concepts like concurrency, parallelism, event loops, and coroutines.
- Python's implementation of async/await involves generators, native coroutines, and the yield/yield from syntax.
- Concurrency enables a program to handle multiple tasks simultaneously, even if they don't run at the same physical time.
- Thread-based concurrency is easy to implement but doesn't scale well due to high memory usage.
- I/O multiplexing with selectors allows single-threaded programs to handle multiple connections efficiently.
- Generators can be used as coroutines to write concurrent code that resembles synchronous code.
- The yield from syntax simplifies running subgenerators, making coroutine calls more intuitive.
- Async/await, introduced in Python 3.5, provides syntactic sugar over generators to distinguish coroutines from regular functions.
- asyncio is Python's standard library for async programming, offering an event loop and utilities for coroutine management.
Python popularity climbs to highest ever – Tiobe
a year ago
- Python's popularity in the Tiobe index reached 25.35% in May 2025, the highest since Java in 2001.
- Python's lead over the second most popular language, C++, is 15 percentage points, the largest in Tiobe's history.
- Tiobe CEO Paul Jansen noted Python's drawbacks, such as low performance and being interpreted, but acknowledged its dominance in most domains.
- In 2001, Tiobe tracked only 20 languages compared to 282 today, making Java's high rating easier to achieve.
- The Tiobe index ranks languages based on skilled engineers, courses, and third-party vendors, using data from Google, Wikipedia, and others.
- The Pypl index, which ranks languages based on Google search frequency, also places Python at the top with a 30.41% share.

first prev2next

About|Login

#python

Dataframely: A polars-native data frame validation library

Making PyPI's test suite 81% faster

Converting a Git repo from tabs to spaces (2016)

ePub-utils: a Python library and CLI tool for inspecting ePub from the terminal

Minimum Viable Blog

Show HN: Pipask – safer pip without compromising convenience

Urtext: The Python plaintext library for people who've tried everything else

Semantic unit testing: test code without executing it

FastAPI Cloud: deploying FastAPI apps with just a single command

Kate and Python Language Server

Declarative Tcl/Tk UI Library for Python

Show HN: PageQL – Embed SQL directly in HTML

Sutton and Barto Book Implementation

Ty: A fast Python type checker and language server, written in Rust

Show HN: US Routing – Python library for fast local routing in the United States

The Flask Mega-Tutorial, Part III: Web Forms

Show HN: Req Update Check

Show HN: Kit – open-source toolkit for building AI devtools

How async/await works in Python

Python popularity climbs to highest ever – Tiobe