Hasty Briefsbeta

All tags

debugging

20 stories total

Bilingual

Flax debugging: making a hash of things
2 months ago
- The author was debugging a JAX/Flax NNX training loop where loss remained constant at 10.82, indicating the model was essentially guessing randomly.
- To verify whether model parameters were being updated, the author used hashing of parameter arrays by converting them to numpy bytes and computing a hash to detect any changes.
- The root cause was using @jax.jit instead of @nnx.jit, which prevented the in-place parameter updates required by Flax NNX. Switching to @nnx.jit resolved the issue, allowing the loss to decrease.
The case of the invalid function pointer when shutting down the display control panel - The Old New Thing
2 months ago
- A crash occurs in the display control panel while destroying a property sheet, due to an invalid window procedure address.
- The invalid address results from a 64-bit pointer being truncated to 32 bits and then sign-extended, causing an invalid kernel-mode address.
- The correct window procedure is identified by matching the lower 32 bits with valid user-mode addresses in the process's module list.
- The bug is caused by a cast to LONG instead of LONG_PTR in C++ code, truncating the original pointer.
- The fix is a one-byte patch in the binary: changing the instruction from movsxd (sign-extend 32-bit) to mov (64-bit load).
AI-generated tools can make programming more fun
2 months ago
- AI-generated tools can make programming more enjoyable by enabling rapid creation of custom debug UIs.
- Using an AI-built interactive debugger, the author quickly identified bugs (e.g., infinite loop) in a Prolog interpreter.
- The fast turnaround and easy iteration of AI-assisted UI development helped the author stay focused on the main task.
- Custom UIs built with AI can turn limited AI capabilities into accelerants for advanced programming tasks.
- The experience highlights the dream of malleable software: creating tools tailored to immediate needs at the speed of thought.
The “Bug-O” Notation — overreacted
2 months ago
- The Bug-O notation measures how much an API or code structure slows debugging as a codebase grows, analogous to Big-O for algorithms.
- Imperative DOM manipulation without structure leads to O(n!) Bug-O due to combinatorial explosion of code paths, making debugging nearly impossible.
- Improving Bug-O can be achieved by resetting state before rendering, reducing complexity to O(n) where n is the number of rendering code paths.
- Libraries like React help by enforcing component boundaries and always recreating UI from scratch, reducing debugging trace to O(tree height).
- When evaluating APIs, consider the Bug-O of common debugging tasks to assess practical usability.
Why Isn’t X a Hook? — overreacted
2 months ago
- React Hooks include useState(), useEffect(), and useContext(), but not all APIs are Hooks.
- Key properties for Hook APIs are composition (custom Hooks should not conflict) and debugging (easy bug tracing).
- useState() supports composition and debugging by allowing independent state variables and clear cause-effect paths.
- A proposed useBailout() Hook fails because it breaks composition by blocking updates from other Hooks and complicates debugging by hiding the source of skipped updates.
- React provides useMemo() instead of a Hook version of memo() to avoid composition and debugging issues.
- Other APIs like useProvider(), useCatch(), or useSuspense() are not Hooks due to similar composition and debugging constraints.
How to Fix Any Bug — overreacted
2 months ago
- Describes a bug where adding a remote call to a button broke scrolling, causing jitter.
- Stresses the importance of having a reproducible case (repro) to verify fixes, which Claude lacked.
- Suggests narrowing the repro by measuring scroll position before and after the bug action.
- Emphasizes verifying the narrowed repro ensures a positive result is still possible (e.g., commenting out the network call fixes both).
- Advocates systematically removing code while keeping the bug present to reduce the bug's cause to its essence.
- Warns against forming theories prematurely and losing the original repro, akin to well-founded recursion.
- Concludes that the root cause was an old version of React Router's ScrollRestoration interfering due to revalidation.
<antirez>
2 months ago
- Lua scripting is the most successful Redis feature, enabling programmable and fast scripts that are cached and executed using the same interpreter.
- Two key features requested by users are a proper Lua debugger and script effects replication, both implemented by the author.
- The debugger is a remote, interactive, client-server tool that uses fork() to avoid blocking the server and enables rollback of dataset changes.
- Key debugger features include breakpoints, stepping, variable inspection, backtrace, and custom Lua functions like redis.breakpoint() and redis.debug().
- Script effects replication allows replicating only the write commands resulting from a script, improving efficiency and enabling use of non-deterministic commands like TIME.
- The feature supports opt-in via redis.replicate_commands(), with advanced control using redis.set_repl() to exclude certain commands from replication.
- Both features will be available in Redis 3.2, scheduled for release in December 2015.
<antirez>
2 months ago
- A long-running process crash is problematic, especially when it holds significant memory state; web programming frameworks that create new interpreters per page view offer a more reliable paradigm.
- Redis experienced a crash in its radix tree implementation, with error addresses zero-padded at the end of the heap, indicating a read overflow beyond memory bounds.
- Extensive fuzz testing failed to reproduce the crash because the bug was already fixed in a newer version of the code, but not backported to Redis 4.0 due to being bundled in a Streams commit.
- The crash analysis involved examining CPU registers and disassembly, revealing that the compiler-generated code did not match the C source due to version mismatch.
- Key lessons include the value of detailed crash reports for debugging system software, the necessity of understanding AMD64 assembler, the power of fuzz testing with reference comparisons, and careful management of bug fixes across branches.
- Seeking help from knowledgeable peers (e.g., Fedor Indutny) was crucial in resolving the mystery.
Pinball on 64-Bit Alpha AXP Windows NT
2 months ago
- Pinball (3D Pinball for Windows – Space Cadet) was removed from Windows Vista due to a reported collision detection bug in 64-bit versions.
- NCommander challenged the removal in 2021, finding 64-bit versions playable on XP and pre-release Vista, suggesting UI incompatibility with Vista design.
- Raymond Chen later clarified the bug was specific to the 64-bit Alpha AXP version, which had a severe collision detection bug.
- The author debugged Pinball on Alpha NT emulators (QEMU/ES40) after fixes enabled booting the 64-bit build 2210.
- Two bugs were found: a WNDCLASSA struct alignment issue causing crashes in RegisterClassA, and pointer truncation in mciseq.dll due to ZAPNOT instruction misuse.
- Patching these bugs made Pinball work flawlessly on 64-bit Alpha NT, without encountering the original collision detector bug.
- Author speculates the collision bug might have been introduced after build 2210 or only manifests in release builds.
We audited 1,751 "relationship milestones" our RAG extractor wrote 62% were junk
2 months ago
- A conversational AI with long-term memory stores (vector and fact) had an extractor that over-produced junk milestones (~6.5/day), causing prompt monoculture with forced romantic declarations.
- Diagnosis used a trace inspector and golden set; a keyword-as-necessary-condition rule blocked 94% of historical junk while passing 100% of positive controls.
- Fix A stopped the tap: keyword gate, roleplay guard, tightened dictionaries, anti-multi-label, deployed behind a test suite.
- Fix B cleaned existing data via reclassification (never delete) using an LLM judge, after avoiding prompt contamination and unsound shortcuts.
- Measurement showed new milestones/day dropped 92% (6.5 to 0.5), entity labels per message reduced to 1, and forced milestones per prompt dropped from 2.0 to 0.65.
- A post-fix incident revealed a relevance failure: retyped facts were force-injected unconditionally, causing accurate but irrelevant memories, highlighting a gap in context-aware injection.
- Qualitative contrast: before fix, assistant drifted romantic; after fix, it stayed on-topic, recalled specifics correctly, and admitted gaps instead of fabricating.
- Remaining issues include always-include inflation, extractor gate leaks, date over-extraction, parallel junk populations, debugger persona limitations, and untested scale/volume degradation.
- Key takeaways: memory can be accurate but wrong (relevance vs truth), instrument before tuning, and reversibility enables honest shipping and measurement.
A flaky test exposed a Redis client use-after-free
2 months ago
- Non-deterministic bugs are harder to debug because they require repeated execution and often involve data corruption that occurs before the crash.
- The bug was discovered in the redis-client library after a Redis gem upgrade caused flaky tests in Buildkite's CI system.
- The team used Test Engine to correlate test reliability, core dumps to inspect crash states, and ASAN to identify a heap use-after-free in hiredis C extension.
- The fix involved disabling the C extension for Redis connections and reporting the issue upstream.
- The debugging process combined skill, teamwork, and luck, with tools enabling detection of memory corruption that would otherwise take months to fix.
Show HN: Web-based Jujutsu revset explorer and debugger
2 months ago
- The page provides a playground for exploring and debugging jj revsets.
- Users can edit the graph and evaluate revsets.
- Setups can be shared via a permalink.
- Nodes can be selected with Enter/Space and moved with arrow keys.
- Nodes can be deleted with Delete, and Escape cancels selection.
- Edges can be selected and deleted similarly.
- A TOML 'revset-aliases' table can be specified.
Where did my segfault go?
2 months ago
- A segmentation fault message is printed by the shell when it reaps a child process that died from SIGSEGV, not by the crashing program itself.
- When using commands like 'bash -c', bash may exec directly into the command without forking, preventing the shell from being the parent and printing the error.
- Using a wrapper script or forcing a subshell (e.g., with parentheses) ensures a shell is present as the parent to detect and report the segmentation fault.
- Adding a command after the crashing one (e.g., 'true') also prevents exec optimization, allowing the shell to print the error message.
Game Boy Advance Dev: Logging to the Console
3 months ago
- mGBA allows GBA developers to use printf-like functionality for debugging by writing to specific memory-mapped registers.
- Logging is enabled by writing 0xC0DE to REG_LOG_ENABLE, writing the message to REG_LOG_BUFFER, and then sending a log level to REG_LOG_SEND.
- The log levels include ERROR, WARNING, INFO, and DEBUG, with each corresponding to a specific value for the REG_LOG_SEND register.
- Logs can be viewed in mGBA's Tools > View logs... window or output to the terminal if launched from the command line with --log-level.
- Advanced logging features include printf-style formatting using vsnprintf and conditional compilation with a macro to exclude logging from release builds.
The Safari MCP server for web developers
3 months ago
- Safari MCP server introduced in Safari Technology Preview 247.
- Enables agents to access browser data (DOM, network requests, screenshots, console logs).
- Speeds up debugging by reducing context switching between browser and terminal.
- Use cases include checking rendering, improving Safari compatibility, analyzing performance, and verifying accessibility.
- Available tools include console messages, network requests, page interactions, screenshots, and JavaScript evaluation.
- Setup requires installing Safari Technology Preview and enabling developer features.
- Local-only operation ensures data privacy and no external network calls.
- Goal is to enhance AI-assisted web development workflows for Safari.
No laptop? No problem. Code on the Go brings the debugger to your phone
3 months ago
- Code on the Go (CoGo™) enables debugging on a single Android device without a PC, cable, or external connections.
- It uses a local socket-based approach with Java Debug Wire Protocol (JDWP) instead of the traditional ADB bridge model.
- Privileged access is bootstrapped via a modified version of Shizuku, leveraging wireless debugging pairing for shell-level access.
- The target app is launched in debug mode using the --attach-agent flag to inject libjdwp.so, establishing a debug connection.
- The IDE communicates via JDI using sun-jdi.jar from AOSP, enabling breakpoints, stepping, variable inspection, and editing.
- Current limitations include Java-only debugging, primitive variable editing only, user-dependent bootstrapping, and reliance on system JDWP.
- Future plans include Kotlin support, conditional breakpoints, expression evaluation, and attaching to running processes.
It's not me, it's the compiler
3 months ago
- A programmer discovered a compiler bug in Rust while refactoring a JavaScript engine's parser.
- The bug was in a function where using 'x as u32' for boolean addition caused incorrect behavior, but using an explicit if-else worked.
- Assembly inspection confirmed the compiler omitted an increment instruction, leading to a miscompilation.
- The issue was reported as 'Suspected miscompilation' and quickly fixed within 18 hours by the Rust community.
- The bug was classified as p-critical and i-miscompile, highlighting rare but serious compiler errors.
Underappreciated Builtin: Grand Unified Debugger
3 months ago
- The Grand Unified Debugger (GUD) in Emacs is an underappreciated built-in feature that supports multiple debuggers like GDB, LLDB, Perl, Java, Python, and others.
- A demo using GDB with a Fibonacci program shows how to set breakpoints via fringe clicks, use toolbar buttons (Run, Next Line, Step Line, Up/Down Stack), and manage breakpoints in a dedicated window.
- GUD provides features like an I/O window for output, variable inspection by hovering, a locals window, and variable watching for automatic updates when values change.
DLL that was not present in memory despite not being formally unloaded
3 months ago
- A bug reported that shell32.dll was causing crashes in a third-party program, but investigation revealed shell32 was a victim.
- Crash dumps showed a stack overflow due to a recursive exception handling death spiral, starting from an access violation in combase!CoTaskMemFree.
- Analysis found that combase.dll had been forcibly unloaded from memory (likely via VirtualFree), not by FreeLibrary, causing subsequent calls to fail.
- Shell32 was blamed as it was the first to call into the unloaded combase.dll during process termination, triggering the crash loop.
- Reviewing 100 recent crashes showed 46% were due to similar rogue DLL force-unloads, with different DLLs affected, indicating a bucket spray issue.
- The root cause is unknown, but theories point to memory corruption or uninitialized variables in the third-party program or other components.
- Wil comments suggest avoiding static object destruction during shutdown and note existing shutdown-aware types in Microsoft code.
Fixing a kubelet memory leak in Kubernetes 1.36
3 months ago
- The author discovered a memory leak in Kubernetes 1.36 on a small test cluster, revealed due to high memory pressure.
- Investigation showed the kubelet process itself was growing in memory usage, not the application pods.
- Heap profiling using Go's pprof identified nearly a million leaked contexts as the primary cause, consuming most memory.
- A code change in Kubernetes 1.36 introduced a context leak by overwriting cancel functions without canceling old contexts in the pod reconciliation loop.
- AI tooling (Codex) helped pinpoint the regression in the codebase, specifically in the startPodSync function.
- A patch was developed and submitted; it was simplified to revert the immediate leak after integration test failures revealed additional context issues.
- The Kubernetes team was responsive, with the fix merged into the master branch for v1.37 and backported to v1.36.
- Restarting kubelet temporarily resolved memory pressure, but the underlying leak persisted until patched.
- Lessons include the value of heap profiling, the time dimension in long-running systems, and infrastructure-level issues.
- A one-liner command to monitor kubelet memory usage showed a significant drop from 974 MiB to 110 MiB after restart.

firstprev0next

debugging

Flax debugging: making a hash of things

The case of the invalid function pointer when shutting down the display control panel - The Old New Thing

AI-generated tools can make programming more fun

The “Bug-O” Notation — overreacted

Why Isn’t X a Hook? — overreacted

How to Fix Any Bug — overreacted

<antirez>

<antirez>

Pinball on 64-Bit Alpha AXP Windows NT

We audited 1,751 "relationship milestones" our RAG extractor wrote 62% were junk

A flaky test exposed a Redis client use-after-free

Show HN: Web-based Jujutsu revset explorer and debugger

Where did my segfault go?

Game Boy Advance Dev: Logging to the Console

The Safari MCP server for web developers

No laptop? No problem. Code on the Go brings the debugger to your phone

It's not me, it's the compiler

Underappreciated Builtin: Grand Unified Debugger

DLL that was not present in memory despite not being formally unloaded

Fixing a kubelet memory leak in Kubernetes 1.36