compiler

Bilingual

Octane – React's programming model, compiled
2 months ago
- Octane is a compiled frontend framework that removes the need for rules of hooks and dependency arrays, allowing hooks to be placed in conditions or early returns.
- It compiles templates to direct DOM writes and keyed @for lists for efficient updates, and supports .tsx files for gradual adoption.
- Octane maintains compatibility with React components, hooks, Suspense, context, portals, transitions, actions, controlled forms, and SSR, enabling migration one component at a time.
- Signals are optional; Octane keeps components as plain functions that read top-to-bottom, with the compiler handling performance optimization.
- The framework significantly outperforms React in most benchmarks, often being 2-3x faster, and is competitive with other modern frameworks like Svelte, Solid, and Vue Vapor.
- It includes a CLI tool, octane doctor, for diagnosing issues, and supports integration with existing React apps via OctaneCompat islands.
Overengineered Calculator: Zig and QBE
2 months ago
- Crafting Interpreters and Writing An Interpreter In Go teach tokenization, parsing, and interpretation but do not cover native code generation.
- QBE is a compact compiler backend providing 70% of the performance of industrial compilers in 10% of the code, ideal for learning code generation.
- The project uses a simple grammar for arithmetic expressions to define precedence and tokenization rules.
- The tokenizer is a state machine implemented in Zig, producing tokens like numbers, operators, and parentheses.
- The parser builds an AST with binary and unary operations based on grammar rules like Sum, Product, and Unary.
- The interpreter evaluates the AST directly by recursively computing values for binary and unary operations.
- The emitter generates QBE intermediate language in static single-assignment form, mapping operations to QBE instructions.
- The compiler stages include generating QBE SSA code, using QBE to produce assembly, and then compiling with Zig to create an executable.
- QBE optimizes by constant folding when possible, but division prevents full optimization, producing explicit assembly instructions.
- The project requires QBE and Zig packages to build and run, with source code available for further development.
Show HN: Learning Rust by writing a Markdown to HTML compiler
2 months ago
- Andrea, new to Rust, built a simple .md to .html compiler as a practical project, moving away from Hugo due to its complexity.
- The compiler was created in 12-13 hours without GenAI tools, relying on documentation and basic markdown syntax implementation.
- Compilation involves scanning the posts directory, parsing block and inline elements, building an AST, and outputting HTML with basic templates.
- Posts require YAML front matter with title and date; optional fields include description, tags, and draft status.
- CommonMark coverage includes headings, paragraphs, code blocks, lists, blockquotes, inline formatting, links, and images.
- The index.html displays all posts with search and tag filtering, plus vim-like navigation for browsing.
- The project is intended for personal use but was a fun learning experience.
Zig's Incremental Compilation Internals
2 months ago
- Zig's incremental compilation detects changed functions/declarations, recompiles only those, and patches the binary for fast rebuilds.
- The feature is now viable for real-world projects and used daily by the Zig core team.
- A demo shows rebuilds taking 50-70ms after an initial 5-second build.
- Per-file processing (parse, AstGen, ZIR) is embarrassingly parallel and cached on disk.
- Semantic analysis uses a dependency graph of analysis units (types, values, bodies) to minimize re-analysis.
- Code generation (AIR to MIR) is embarrassingly parallel and functions are independent.
- Incremental linking uses a MappedFile abstraction for resizing and patchability, avoiding full relinking.
- The linker amortizes movement costs via exponential growth, making slow updates rare.
- Flush includes a reference graph traversal; optimization opportunities remain.
- Usage: run `zig build --watch -fincremental` targeting x86_64-linux; works on Zig master branch.
Zig Parser
2 months ago
- The Zig parser uses a MultiArrayList for efficient memory and cache locality, storing each struct field in separate contiguous arrays.
- AST node data is stored in three locations: the token stream (e.g., identifiers), the node list (child AST nodes), and the extra data list (additional metadata).
- The parser maintains state in a struct called Parser, which includes token arrays, node list, extra data, errors, and scratch space.
- Parsing proceeds by inspecting the current token and using conditional logic to determine expected tokens, building AST nodes incrementally.
- Variable declarations are parsed by first consuming 'const' or 'var', then expecting an identifier, optional type expression, and optional initialization expression.
- Function definitions are parsed by reserving a node index, parsing parameters, calling convention, optional return type, and then setting the node with extra data.
- The final AST structure includes the source text, tokens, nodes, extra data, and accumulated errors, ready for the next compilation stage (AstGen).
Zig Sema: ZIR => AIR
2 months ago
- Sema is the compiler stage after AstGen that produces AIR (Analyzed Intermediate Representation) from ZIR, performing both type analysis and comptime evaluation.
- AIR is fully typed, generated per-function, and consists of instructions, extra data, and a values array for comptime-known constants.
- Key data structures include Value (comptime values), Type (comptime types), and TypedValue (value paired with its exact type).
- The core function analyzeBodyInner loops over ZIR instructions, mapping each to zero or more AIR instructions using the inst_map.
- Comptime evaluation enables constant folding, type coercion, and target-specific emulation, as demonstrated in the addition and type conversion examples.
- Sema is invoked lazily per referenced declaration, supporting Zig's lazy analysis for fast compilation and minimal codegen.
- After Sema, AIR is passed to codegen backends (LLVM, WASM, etc.) for final lowering to machine code.
A survey of inlining heuristics
2 months ago
- Inlining is crucial for optimizing dynamic language JIT compilers but introduces non-local effects and complexity.
- Compilers use heuristics to decide when to inline, balancing code size, compile time, and performance gains.
- Common heuristics include inlining small methods, limiting depth, and using cost budgets based on bytecode size or instruction count.
- Profiling with call context (e.g., trial inlining, bytecode inlining) helps avoid polymorphic pollution and improves inline decisions.
- Different JIT compilers (Cinder, V8, PyPy, HotSpot, etc.) implement inlining at various stages and with distinct strategies.
- Over-inlining can cause code bloat, cache thrashing, and block other optimizations, making careful tuning necessary.
- Research explores advanced techniques like machine learning, partial inlining, and interprocedural analysis to improve inlining decisions.
Matt Godbolt's blog
2 months ago
- Compilers often optimize loop code, making seemingly expensive operations fast through techniques like using induction variables.
- Loop optimizations like transforming multiplications into series of additions can sometimes be rejected by the compiler if multiplications are already fast and independent.
- Independent loop iterations allow the CPU to overlap calculations, improving throughput despite longer per-iteration latency.
- Tools like Compiler Explorer and llvm-mca can verify compiler optimizations and analyze CPU performance.
- Trust compilers for optimizations but always verify with benchmarks and analysis tools.
Matt Godbolt's blog
2 months ago
- Giving the compiler compile-time known loop count (e.g., via `std::span<int, 8>`) enables loop unrolling optimization.
- A dynamically-sized span (`std::span<int>`) results in a straightforward loop with counter and branch per iteration.
- Unrolling eliminates loop counter and conditional branch, saving instructions and allowing further pattern detection like using `ldmib` for multiple loads.
- Compilers may partially unroll, speculatively unroll, or fall back to regular looping depending on heuristics and iteration count (e.g., at 50 iterations it gives up).
- Checking hot loops and providing compile-time loop counts sets the compiler up for success; profile-guided optimization (PGO) can also help improve guesses.
Matt Godbolt's blog
2 months ago
- Switch statements can compile to jump tables, but compilers often use more clever optimizations like arithmetic or bitmasks.
- For dense case values, compilers may build lookup tables or use bitmasks to avoid branches entirely.
- For sparse values, compilers may use binary search trees of comparisons or fall back to conditional branches.
- Different compilers (e.g., GCC vs Clang) employ different optimization strategies for switch statements.
- Programmers should write clear switch statements and trust the compiler to choose the best implementation.
56,000 lines of DOOM, in a language I made up
2 months ago
- bet is a small, real programming language designed as a contained experiment.
- It compiles to native code via LLVM, self-hosts, manages memory, and can run DOOM.
- The language uses arena allocation (called cribs) for efficient memory management in games.
- DOOM was fully ported to bet with over 56,000 lines, achieving bit-for-bit identical gameplay.
- The project was built with no prior research and gated solely by tests, without code review.
- It served as a joke, an experiment, and a learning experience, with no plans for further development.
How Our Rust-to-Zig Rewrite Is Going
2 months ago
- The Roc compiler team has rewritten 300,000 lines of Rust code into Zig, recently achieving feature parity with the original compiler.
- Key new features in the Zig-based compiler include hot code loading during development, cross-compiled binaries, and type-safe string interpolation in pattern matching with zero heap allocations.
- The rewrite was motivated by architectural challenges, especially in implementing polymorphic defunctionalization, which required a scratch rewrite rather than incremental changes.
- Zig was chosen over Rust due to faster build times, better memory control with granular allocators, ecosystem relevance for compiler needs, and more assistance with memory-unsafe code.
- Memory safety comparison shows fewer memory corruption bug reports in the Zig compiler (10) vs. Rust (21), though most were miscompilations, not compiler itself issues.
- Build times are significantly faster with Zig's incremental builds (35ms) compared to Rust's (3.4s), despite a current stable release bug delaying full benefits.
- The compiler uses zero-parse deserialization for caching, leveraging array-based data structures for I/O-bound performance, enabled by Zig's memory control.
- Zig's ecosystem, including its compiler code, provided reusable components like an LLVM bitcode serializer, aligning with Roc's unusual needs.
- Missing aspects from Rust include automatic allocation in tests, private struct fields, and better dead code detection, but Zig offers benefits like control over data layouts and error handling.
- Next steps include releasing Roc version 0.1.0 later this year, with ongoing work on documentation and bug fixes, supported by the nonprofit Roc Programming Language Foundation.
History of T
2 months ago
- T was a highly influential Lisp implementation known for its clean design and optimization.
- Developed at Yale in the early 1980s by Jonathan Rees, T was an optimizing native-code Scheme system, a radical choice at the time.
- T introduced innovative implementation techniques like efficient data representations, a depth-first garbage collector, and fine-grain interrupt handling.
- The compiler, Orbit, used CPS (continuation-passing style) as an intermediate representation and was among the most optimizing Scheme compilers for years.
- T's development led to multiple PhD dissertations and influenced later languages and compilers, including SML/NJ and Scheme 48.
- The language design featured a clean object system with integrated assignment machinery and a consistent naming scheme.
- T's legacy includes contributions to functional programming, compiler technology, and garbage collection research.
Zig: All Package Management Functionality Moved from Compiler to Build System
3 months ago
- Package management functionality moved from compiler to build system, improving modularity and allowing patching without compiler rebuild.
- SPIR-V backend progressed with new @SpirvType builtin, execution mode via calling convention, and multi-threaded codegen.
- New @bitCast semantics defined by logical bit layout, making behavior endian-agnostic and enabling new operations like integer to vector conversion.
- ELF linker improvements enable fast incremental compilation and linking with external libraries, though DWARF debug info support is pending.
- Build system reworked to separate configurer and maker processes, significantly speeding up build times and improving caching.
- Incremental compilation now works with LLVM backend, speeding up error feedback during development.
- Type resolution redesign made compiler lazier, improved dependency loop error messages, and enhanced incremental compilation.
- io_uring and Grand Central Dispatch std.Io implementations landed, allowing easy swapping of I/O implementations.
- Package management enhancements: dependencies stored locally in zig-pkg directory and --fork flag added for easy dependency forking.
- Windows standard library now prefers native ntdll APIs over kernel32 to avoid unnecessary overhead and improve reliability.
- zig libc project replaces C source files with Zig wrappers, reducing redundancy and improving optimization.
Zig: Package Management Moved from Compiler to Build System
3 months ago
- Package management functionality moved from compiler to build system for flexibility and safety improvements.
- SPIR-V backend progress includes new builtins, execution modes, multi-threaded codegen, and object file linking.
- Redefined @bitCast semantics to use logical bit layout, improving cross-platform consistency.
- ELF linker improvements enable incremental compilation with external libraries, including LLVM and LLD.
- Build system rework separates configurer and maker processes, speeding up builds and enabling cached configurations.
- Package management workflow enhancements include local zig-pkg directory and --fork flag for dependency overrides.
- Windows std.Io bypasses kernel32.dll for direct ntdll calls, improving performance and reliability.
- Zig libc project replaces C source files with Zig wrappers, reducing code and enabling optimizations.
- New std.Io.Evented implementations (io_uring and GCD) enable experimental async I/O with fiber-based coroutines.
- Type resolution redesign improves laziness, dependency loop error messages, and incremental compilation performance.

compiler

Octane – React's programming model, compiled

Overengineered Calculator: Zig and QBE

Show HN: Learning Rust by writing a Markdown to HTML compiler

Zig's Incremental Compilation Internals

Zig Parser

Zig Sema: ZIR => AIR

A survey of inlining heuristics

Matt Godbolt's blog

Matt Godbolt's blog

Matt Godbolt's blog

56,000 lines of DOOM, in a language I made up

How Our Rust-to-Zig Rewrite Is Going

History of T

Zig: All Package Management Functionality Moved from Compiler to Build System

Zig: Package Management Moved from Compiler to Build System