Hasty Briefsbeta

All tags

#go

117 stories total

Bilingual

Deadlocks in Go: the dark side of concurrency (2021)
a year ago
- Deadlocks in Go occur when parts of a program stop working because goroutines are waiting on each other to release locks.
- Go's deadlock detector is not always useful in real-world programs because it requires all goroutines to be asleep, which rarely happens.
- Debugging deadlocks involves collecting backtraces using tools like SIGQUIT on Unix or net/http/pprof on Windows.
- Deadlocks can occur with both Mutexes and channels, though Mutexes are more commonly involved.
- Common causes of deadlocks include recursive locking, overlapping locks, and sending/receiving on the same channel in the same goroutine.
- Strategies to avoid deadlocks include shortening lock windows, using read-only variables, and imposing total lock ordering.
- Tools like panic-parse and go-deadlock can help identify and debug deadlocks, though they have limitations.
- Single-threaded engines can eliminate deadlocks but may reduce performance.
- Comments and code reviews are essential for maintaining lock ordering invariants in complex systems.
rqlite turns 10: Lessons from a decade building Distributed Systems
a year ago
- rqlite is a lightweight, open-source, distributed relational database written in Go, using SQLite and Raft.
- Started 10 years ago to learn Go and explore distributed consensus, rqlite has grown in adoption and technical depth.
- Key successes include choosing Go, building rqlite.io, Docker integration, Kubernetes support, and stable API.
- Testing and incremental snapshotting were crucial for stability and large dataset support.
- Change Data Capture (CDC) is considered the last big feature, emphasizing clean and understandable systems.
- rqlite was ahead of its time, combining Raft with SQLite before others, but lacked early marketing.
- A SQLite bug uncovered by rqlite validated its progress from a side project to a stress-testing tool.
- Distributed systems are often misunderstood, with reliability and performance frequently conflated.
- The author continues to work on rqlite for enjoyment, learning, credibility, and therapeutic reasons.
Show HN: F2 – Cross-Platform CLI Batch Renaming Tool
a year ago
- F2 is a cross-platform command-line tool for batch renaming files and directories, written in Go.
- Key advantages include dry run by default, variable support (e.g., EXIF data, ID3 tags), comprehensive renaming options, and safety-first conflict resolution.
- High performance, undo functionality, and extensive documentation make F2 user-friendly and efficient.
- Installation options include Go install (v1.23+ required) or downloading pre-compiled binaries.
- Features cover installation guides, tutorials, real-world examples, built-in variables, file pair renaming, CSV file renaming, sorting, and conflict resolution.
- Bug reports and feature requests are encouraged, with issues preferred before pull requests.
- Created by Ayooluwa Isaiah under the MIT License.
Show HN: Fwtui – A terminal UI for managing UFW (built with Bubble Tea)
a year ago
- fwtui is a terminal-based UI built in Go for managing UFW (Uncomplicated Firewall) rules.
- Provides a clean, responsive, and keyboard-driven interface to view and manage firewall rules.
- Features include viewing active UFW rules, adding custom rules with specific ports, protocols, and traffic direction.
- Allows deletion of rules using keyboard shortcuts.
- Enables viewing and changing default policies for incoming and outgoing traffic.
- Supports creation and installation of reusable rule profiles.
- Full keyboard navigation, ideal for terminal lovers and remote server admins.
- Installation requires downloading the binary, setting permissions, and running with sudo.
- Keyboard shortcuts include navigation, editing, submitting changes, and deleting items.
New Go Garbage Collector
a year ago
- New experimental garbage collector for Go programs announced.
- The project is hosted on GitHub under the Go language repository.
- The announcement was made by Michael Knyszek on his personal site.
- The issue includes a comment link for further details.
- The post emphasizes the need for JavaScript due to the interactive nature of the web application.
Show HN: Git-Add–Interactive with Enhancements
a year ago
- Go port of Git's interactive add functionality with enhancements.
- Interactive staging and patch mode with commands like y/n/s/e/q/a/d.
- Hunk operations: split, edit, navigate between hunks.
- Multiple patch modes: stage, reset, checkout, stash, worktree operations.
- Git integration with full color configuration and repository support.
- Terminal UI with color-coded interface and keyboard shortcuts.
- Filter hunks across files using regex patterns.
- Automatically split hunks to maximum granularity.
- Accept all hunks across files after filtering and splitting.
- Local search within current file without affecting global filter.
- Status display shows filter and auto-split indicators.
- Cross-file filtering persists across all files in the session.
- Build and install instructions to replace Perl script with Go implementation.
- Commands to revert back to the original Perl implementation.
- Direct usage and patch mode commands explained.
- Codebase organization and main packages overview.
- Test suite and conventions followed.
- Comprehensive unit tests, Git integration, error handling.
Pure vs. Impure Iterators in Go
a year ago
- Go has standardized iterators with the introduction of generics in Go 1.18, enhancing flexibility and performance.
- Iterators in Go can be classified into 'pure' (stateless, no observable side effects) and 'impure' (stateful, with side effects).
- The official documentation introduces 'single-use iterators' as a subcategory of impure iterators, but this classification is seen as ambiguous and imprecise.
- Pure iterators are easier to reason about and can reduce heap allocations, making them preferable from a performance standpoint.
- However, consistency with related iterators (like those in the bytes package) may necessitate the use of impure iterators in some cases.
- The Go community is still establishing conventions around iterators, with ongoing discussions about terminology and best practices.
Structured Errors in Go
a year ago
- Go's error handling is based on returning values and any type that satisfies the error interface is considered an error.
- Structured logging is essential for diagnosing issues in medium and large systems, but current error handling in Go doesn't easily support structured metadata.
- Custom error types can include metadata, but creating them is cumbersome and not ergonomic, leading to inconsistent use.
- The author proposes a solution inspired by structured logging libraries like Logrus, allowing errors to be wrapped with metadata fields.
- Context in Go can be used to carry metadata down the call tree, which can then be attached to errors as they ascend, bridging the gap between context and error handling.
- A library called 'fault' is introduced, implementing structured error handling with metadata, making it easier to diagnose issues in production.
Show HN: A toy version of Wireshark (student project)
a year ago
- Vanta is a lightweight, fast, command-line-based network behavior analyzer.
- It reconstructs protocol-level flows and extracts structured activity from captured data.
- Supports HTTP, DNS, TLS (with partial fingerprinting).
- Automatically reconstructs bidirectional flows.
- Outputs clean JSON-formatted summaries.
- Single binary, no external dependencies.
- Developed and tested on a MacBook Air M1 by an undergraduate student.
- Project is a personal response to political pressure on international students.
- Thanks to professors and schools for standing up for students.
(On | No) Syntactic Support for Error Handling
a year ago
- Go's error handling is often criticized for being verbose, with frequent `if err != nil` checks.
- Multiple proposals for syntactic sugar to simplify error handling have been made, including `check/handle`, `try`, and Rust's `?` operator.
- None of the proposals gained sufficient consensus, leading to their decline.
- Arguments for maintaining the status quo include the risk of alienating users who prefer current practices and the principle of not providing multiple ways to do the same thing.
- The Go team has decided to stop pursuing syntactic changes for error handling for the foreseeable future.
Show HN: Vim-like text editor in go. (LSP, TreeSitter, Themes)
a year ago
- McWig is a modal, Vim-like text editor written in Go.
- Currently supports only .go files; caution advised as it's in early development.
- Features include LSP autocomplete, Tree-sitter support, color themes, macros, and Emacs org-mode-like functionality.
- Developed as a 'speed run' project to explore text editor design without extensive planning.
- Includes common Vim keybindings for daily source code editing.
- Plans to evolve into a stable, fully-featured Vim-like editor.
Show HN: I wrote a BitTorrent Client from scratch
a year ago
- BitTorrent client implementation in Go supporting torrent file parsing, peer discovery, and file downloading
- Bencode encoding/decoding with support for all Bencode types (strings, integers, lists, dictionaries)
- Torrent file processing including parsing, info hash calculation, and piece hash extraction
- Peer discovery and communication with HTTP tracker support and BitTorrent message protocol
- Download functionality with piece and block management, concurrent downloads, and progress tracking
- Project structure includes cmd/, internal/, and pkg/ directories
- Requires Go 1.21 or later
- Future features: Magnet link support, metadata exchange protocol, DHT support
Show HN: I built a tool to use my homelab apps remotely without a full VPN
a year ago
- prxy is a command-line reverse proxy written in Go for forwarding HTTP requests through an outbound proxy.
- Automatically rewrites the Host header to ensure requests reach the target service correctly.
- Designed to work with applications that don't support proxy configurations.
- Acts as a companion to wireproxy for split-tunneling on an application-by-application basis.
- Installation options include Homebrew, Docker, and direct binary download.
- Configuration can be done via command-line flags or environment variables.
- Contributions are welcome through GitHub issues and pull requests.
- Licensed under the MIT license.
Cmapv2: A high performance, concurrent map
a year ago
- Install the `cmapv2` package in your Go project using `go get github.com/sirgallo/cmapv2`.
- Run `go mod tidy` to install dependencies.
- Import the package in your Go code with `import "github.com/sirgallo/cmapv2"`.
- Initialize a concurrent map (`cMap`) or a sharded map (`sMap`) with optional shard count.
- Use methods like `Put`, `Get`, and `Delete` to manage key-value pairs.
- Run tests with `go test -v ./tests` or benchmarks with additional profiling flags.
- Analyze profiling results using `go tool pprof` with CPU and memory profiles.
I Wrote a Compiler
a year ago
- The author wrote a compiler for a simplified BASIC variant called toybasic over a weekend.
- The compiler is written in Go and translates BASIC into Go code.
- The compiler has three stages: Lexer, Parser, and Compiler.
- The Lexer uses nex to tokenize the source code.
- The Parser uses goyacc to build a syntax tree from tokens.
- The Compiler generates Go code by traversing the syntax tree.
- The project was inspired by TinyBASIC and modified to exclude the INPUT statement.
- The full code is available on GitHub.
- The author tested the compiler with a BASIC program using all language constructs.
- The project was a fun and educational experience for the author.
Calling Go from Elixir with a CNode in Crystal
a year ago
- Mozi needed to connect an Elixir Phoenix LiveView app to an existing Go backend.
- Options considered included NIFs (Native Implemented Functions) and Ports, but both had drawbacks.
- The chosen solution was a 'C Node' using Erlang's erl_interface library for BEAM distribution.
- A C wrapper was built to call Go code, with main() in C and Go initialized from there.
- The C node connects to the Elixir app via a well-known local address, managed by S6 in a container.
- Ergo, a Go implementation of BEAM distribution, was considered but not used due to compatibility and licensing issues.
- The C code was later rewritten in Crystal for better maintainability, with thin wrappers for erl_interface.
- The system is deployed as a single Docker container in Kubernetes, with independent builds for Elixir and Go/Crystal components.
- A custom Go compiler build was temporarily needed for Alpine Linux support, but this will soon be unnecessary.
- Interest was expressed in open-sourcing the Crystal wrapper for erl_interface.
How to store Go pointers from assembly
10 months ago
- The Go toolchain includes an assembler with Plan 9 syntax and custom names for platform-specific instructions.
- Go's garbage collector is concurrent, minimizing pauses by running alongside application code.
- Pointer stores are instrumented with write barriers to inform the GC of new references.
- Go avoids instrumenting stack pointer stores for performance, previously requiring a final stack rescan.
- Go 1.8 introduced a broader write barrier, reducing stop-the-world pauses significantly.
- Handling pointer stores in assembly requires manual write barrier implementation.
- Linking to runtime symbols for write barriers is restricted but possible with specific build tags.
- Allocating 128-bit aligned slots for concurrent hash tables is tricky in Go, requiring creative solutions.
- A workaround for alignment involves manipulating slice types and unsafe pointers.
Why Go Rocks for Building a Lua Interpreter
10 months ago
- The author built a custom Lua interpreter in Go because existing implementations didn't meet their needs.
- Lua is a dynamically typed language with several value types including nil, booleans, numbers, strings, userdata, tables, and functions.
- The interpreter is structured into three Go packages: lualex (lexical analysis), luacode (parsing and bytecode generation), and lua (execution).
- Go's interface types are used to represent Lua values, with specific types for each Lua data type.
- The interpreter includes a 'freezing' feature inspired by Starlark to prevent mutations on values and enable safe sharing across concurrent interpreters.
- The parser generates bytecode instructions directly, enabling efficient execution and on-the-fly optimizations.
- Go's standard library, garbage collection, and testing tools significantly simplified the interpreter's development compared to C Lua.
- Challenges included error handling, porting standard libraries, and dealing with Lua's garbage collector and weak tables.
- The interpreter is part of the zb project and is available under an MIT license, though not currently supported as a standalone package.
Show HN: A Go service that exposes a FIFO message queue in RAM
10 months ago
- A single-binary Go microservice for a RAM-based FIFO message queue.
- Balances simplicity and high-throughput with a plain slice + mutex and lock-free atomic counters.
- Supports HTTP/TLS, structured logging, and health endpoints for easy integration into micro-service stacks.
- Features ultra-low-latency queueing for transient messages (≤ 128 KB each).
- Enforces hard caps (maxBytes, maxMsgs) to guard RAM usage.
- Concurrency-safe design using a mutex + atomic counters.
- Provides JSON metrics, TLS option, and graceful shutdown.
- Includes endpoints for enqueue, dequeue, clear, persist, load, metrics, and health checks.
- Optimized for performance with O(1) operations for enqueue and dequeue.
- Supports graceful shutdown to ensure no message corruption during redeploys.
- Includes telemetry for monitoring queue size, Go goroutines, and FD usage.
- Designed for easy debugging with zero client codegen and trivial curl commands.
- Supports TLS for secure communication.
- Includes flags and environment variables for configuration.
- Provides best practices for deployment, including process supervision and alerting.
4-10x faster in-process pub/sub for Go
10 months ago
- High-performance, in-process event dispatcher for Go.
- Supports both synchronous and asynchronous processing.
- 4x to 10x faster than channels.
- Works with any type implementing the Event interface.
- Each subscriber runs in its own goroutine for non-blocking handling.
- Ideal for decoupling modules and lightweight pub/sub patterns.
- Not suitable for inter-process communication or event persistence.
- Provides a default global dispatcher for convenience.
- Benchmarks show high throughput with minimal allocations.
- Licensed under MIT License.

first prev1next

About|Login

#go

Deadlocks in Go: the dark side of concurrency (2021)

rqlite turns 10: Lessons from a decade building Distributed Systems

Show HN: F2 – Cross-Platform CLI Batch Renaming Tool

Show HN: Fwtui – A terminal UI for managing UFW (built with Bubble Tea)

New Go Garbage Collector

Show HN: Git-Add–Interactive with Enhancements

Pure vs. Impure Iterators in Go

Structured Errors in Go

Show HN: A toy version of Wireshark (student project)

(On | No) Syntactic Support for Error Handling

Show HN: Vim-like text editor in go. (LSP, TreeSitter, Themes)

Show HN: I wrote a BitTorrent Client from scratch

Show HN: I built a tool to use my homelab apps remotely without a full VPN

Cmapv2: A high performance, concurrent map

I Wrote a Compiler

Calling Go from Elixir with a CNode in Crystal

How to store Go pointers from assembly

Why Go Rocks for Building a Lua Interpreter

Show HN: A Go service that exposes a FIFO message queue in RAM

4-10x faster in-process pub/sub for Go