ai accelerators

Bilingual

Can AMD Break the CUDA Moat? AMD Advancing AI 2026
2 months ago
- AMD's software and hardware have significantly improved, shifting from a 0% to a high chance of success against Nvidia's CUDA moat, but key risks remain.
- Two major risks: 1) Helios rack production challenges due to cable design and Ethernet retimer requirements; 2) Lack of stable GPU clusters for internal software development and CI testing, slowing progress.
- AMD's MI455X silicon leads in logic and memory integration (2nm, 12 HBM4 stacks, 432GB), but microarchitecture and software still lag behind Nvidia's Rubin.
- AMD's software stack (ROCm, vLLM, SGLang) is improving rapidly, especially in single-node inference and documentation, but distributed inference (disaggregation, WideEP) needs more work.
- AI agents and open-source tools like ROCm.ai/GEAK are helping AMD close the software gap faster than traditional methods.
- AMD needs to prioritize stable CI/GPGPU clusters for internal teams, upstream distributed inference support, and ensure composability of optimizations across models.
- Anthropic and Microsoft/OpenAI have committed to AMD's MI455X Helios, with financial incentives like equity rebate discounts making TCO highly attractive.
- The CUDA moat is eroding as AI agents reduce the need for manual kernel engineering, benefiting AMD's open-source approach.
Reiner Pope – Chip design from the bottom up
2 months ago
- The talk explains how chips work from basic logic gates to AI accelerators, focusing on the multiply-accumulate (MAC) unit as the fundamental primitive for matrix multiplication.
- The Dadda multiplier uses AND gates and full adders to efficiently perform multiplication, with the number of full adders equaling the product of bit widths.
- Data movement costs dominate chip design—a mux for selecting register file inputs can require 24× the gate count of the MAC itself, motivating systolic arrays to maximize compute per communication.
- Systolic arrays store weight matrices locally and vector inputs temporarily, reducing bandwidth needs by reusing data across cycles.
- Clock cycles synchronize chip logic via pipeline registers, but loops (e.g., running sums) impose speed limits because feedback loops cannot be pipelined without changing computation.
- FPGAs use lookup tables (LUTs) and muxes to emulate arbitrary gates, but are 10× less efficient than ASICs due to overhead from configurability.
- Deterministic latency in chips often relies on scratchpads (explicit memory access) rather than caches (hardware-managed), which introduce non-determinism.
- CPU cores are larger than GPU cores due to branch predictors and extensive caches; GPUs strip these for more parallel compute units.
- The brain is slower and uses unstructured sparsity, while chips are clocked high for throughput; energy consumption scales with gate switching (dynamic power).
- A GPU can be seen as a collection of many small TPUs (tensor cores in SMs), each with local memory, while a TPU has fewer, larger systolic arrays with a central vector unit.
M-Chips: M7 with up to 1.5 TB – and why Apple is skipping the M6
2 months ago
- Apple is skipping M6 Pro, Max, and Ultra variants, planning only a standard M6 chip.
- The M7 series, starting in 2027, will feature major AI accelerator upgrades and include Pro, Max, and Ultra variants with up to 1.5 TB addressable RAM.
- Apple emphasizes local AI for cost savings, privacy, and technical advancements, aligning with trends away from cloud-dependent inference.
Scrying the AMD GFX1250 LLVM Tea Leaves
2 months ago
- AMD's new GFX1250 accelerator, part of MI455X for machine learning, shows similarities to RDNA4 including WGP structure and WMMA support, but lacks dynamic VGPR allocation.
- GFX1250 allows up to 1024 VGPRs per wave, a major improvement over CDNA's 512 and RDNA's 256, and features a merged 448KB WGP Cache for LDS and vector L0.
- Graphics support is minimal in GFX1250, with no rasterizer, texture, or raytracing instructions, making it a pure compute accelerator.
- Tensor operations in GFX1250 combine RDNA4's programming model with CDNA4's performance, supporting various data formats and sparse WMMA, but excluding fp64.
- New features include clusters for thread block coordination, cluster-level barriers and loads, cooperative atomics, and hardware support for tanh and explicit prefetching.
- Data dependency handling is redesigned with more granular counters (e.g., ASYNCcnt, TENSORcnt) to reduce stalls, improving efficiency over GCN/CDNA.
Who Needs DRAM? We Have Fiber
2 months ago
- Hyperscale data centers and generative AI are increasing DRAM demand and pricing pressure.
- Fiber Memory uses optical fiber as an active, recirculating delay-line memory for immutable data like LLM weights.
- The architecture employs multi-core fibers, passive optical tap-and-amplify interfaces, co-packaged optics, and all-optical regeneration.
- It eliminates redundant weight storage across thousands of AI accelerators.
- Compared to traditional HBM3e configurations, it reduces weight-delivery energy by over 70%.

ai accelerators

Can AMD Break the CUDA Moat? AMD Advancing AI 2026

Reiner Pope – Chip design from the bottom up

M-Chips: M7 with up to 1.5 TB – and why Apple is skipping the M6

Scrying the AMD GFX1250 LLVM Tea Leaves

Who Needs DRAM? We Have Fiber