Bilingual

FreeBSD vs. SmartOS: Who's Faster for Jails, Zones, and bhyve VMs?
a day ago
- Benchmarks reflect specific hardware and workloads, not universally applicable results.
- A PCEngines APU running Proxmox failed after seven years due to power surges, prompting a replacement with an Intel N150 MiniPC.
- The author tested multiple virtualization options on the N150 and an i7-7500U for comparison.
- OS-level containers (jails and zones) showed negligible CPU overhead, near bare-metal performance.
- Memory benchmarks revealed significant differences: bhyve on i7 had 0.52% overhead, while on N150 it was 9%, likely due to missing optimizations.
- SmartOS native and LX zones delivered top memory performance, beating FreeBSD bare metal by up to 25%.
- Legacy KVM on SmartOS underperformed severely, making bhyve the preferred choice.
- FreeBSD demonstrated superior bhyve HVM performance on mature hardware, with near-zero overhead.
- The final recommendation depends on workload: SmartOS for container speed and Linux flexibility, FreeBSD for pure HVM performance.
- bhyve emerged as the key technology for the client's new server.
Sheetz is quitting VMware, migrating 11,000 virtual machines
13 days ago
- Sheetz, a US convenience store chain, is migrating 838 stores from VMware to StorMagic's SvHCI, affecting about 11,000 VMs.
- The move was driven by Broadcom's changes, including eliminating perpetual licenses, mandatory subscriptions, and projected price hikes, which increased uncertainty and vendor dependence.
- Sheetz has completed over 600 store migrations, averaging 200 per month, and expects to finish in four months, using existing Dell hardware without on-site technicians.
- StorMagic was chosen after successful use of SvSAN since 2019, offering resilience and centralized management for a distributed retail environment.
- Challenges remain in the migration process, though remote deployment without hardware upgrades is saving significant money.
Theo de Raadt: "You've been smoking something mind altering" (2007)
17 days ago
- Virtualization adds another kernel layer with potential new bugs.
- It relies on x86 architecture that has poor page protection.
- Claims that virtualization provides security benefits are misguided.
- Software engineers struggle with secure OS and application development.
- Expecting flawless virtualization layers is unrealistic.
Ditching Vagrant: VMs with KVM and Virsh on Debian
23 days ago
- The author decided to move away from Vagrant to using KVM and libvirt for virtual machine management, feeling it was overly complex and redundant since Linux can handle virtualization natively.
- KVM allows the Linux kernel to function as a hypervisor, while libvirt is a library that manages KVM and other virtualization platforms, with virsh as a command-line interface for interacting with libvirt.
- A practical example is given for creating a preseeded Debian VM using virt-install, which automates installation with predefined answers and handles network connectivity, mounting, and SSH agent forwarding.
- Key steps include checking KVM module status, installing libvirt and virsh, using preseed files for deterministic builds, and troubleshooting issues like serial output problems after OS upgrades.
- The author highlights the benefits of using libvirt for unified virtualization management and shares useful commands for VM lifecycle control, network setup, and file system mounting.
Binary Coverage the Wrong Way
24 days ago
- A third option of formally proving programs is dismissed as a lie.
- Fuzzers commonly use N-gram coverage to capture path-dependent information by combining the last N branches.
- Weird coverage metrics still essentially record which code was hit, making distinctions less significant.
- If branch history is recorded, N-gram coverage can likely be computed from it.
- Windows 11 runs the kernel as a virtualized guest, not on bare metal, which is an interesting fact.
- Mention of Varnish, an HTTP cache, as an example reference.
Smol Machines
a month ago
- SmolVM allows running workloads in fast, hardware-isolated Linux VMs with identical SDK and .smolmachine artifacts for local development and deployment.
- Key features include sandboxing untrusted code with hypervisor isolation, creating portable executables, persistent VMs, and GPU-accelerated workloads via Vulkan.
- Installation is via a shell script, with commands to run ephemeral VMs, interactive shells, and package workloads into binaries.
- It uses libkrun VMM with a custom kernel (libkrunfw), provides default 4 vCPUs and 8 GiB RAM, and supports macOS and Linux hosts with KVM.
NUMA: Cores, memory, and the distance between them
a month ago
- NUMA (Non-Uniform Memory Access) means memory access costs vary depending on CPU and memory bank locations, unlike UMA (Uniform Memory Access).
- NUMA emerged to scale beyond single-socket limits, with each socket having its own memory controller and interconnects like AMD's HyperTransport/Infinity Fabric and Intel's QPI/UPI.
- Modern servers can have multiple NUMA nodes per socket due to designs like AMD's EPYC with chiplets and Intel's Sub-NUMA Clustering, moving beyond the 'one socket, one node' model.
- Remote memory access on modern servers is 1.5x to 3x slower than local access in microbenchmarks, worsening under load due to interconnect contention, affecting tail latency.
- Memory interleaving spreads memory across nodes to flatten costs but sacrifices peak performance for predictability, making 'everything equally bad'.
- NUMA involves two affinities: CPU affinity (pinning processes to CPUs) and memory affinity (controlling memory allocation per node), which must align for optimal performance.
- Linux's 'first touch' policy allocates memory on the node of the CPU that first accesses it, which can cause remote accesses if allocation and usage threads are on different nodes.
- Xen's dom0 lacks NUMA awareness, leading to mismatches between guest vCPUs and memory placement, silently causing cross-interconnect access and performance issues.
- Virtualized systems like Xen have two stacks (hypervisor and guest) making separate placement decisions, unlike KVM where the host Linux kernel manages NUMA for VMs.
- Edera's work makes Xen NUMA-aware end-to-end, allowing guests to see accurate topology and optimize placement, avoiding the 'blind and interleave' dilemma.

virtualization

FreeBSD vs. SmartOS: Who's Faster for Jails, Zones, and bhyve VMs?

Sheetz is quitting VMware, migrating 11,000 virtual machines

Theo de Raadt: "You've been smoking something mind altering" (2007)

Ditching Vagrant: VMs with KVM and Virsh on Debian

Binary Coverage the Wrong Way

Smol Machines

NUMA: Cores, memory, and the distance between them