Hasty Briefsbeta

All tags

ai infrastructure

28 stories total

Bilingual

The Wild Wild West of Lego Datacenters
2 months ago
- Modular construction is transforming datacenter building, with components like walls, power rooms, and even entire data halls arriving prefabricated, similar to assembling a LEGO set.
- Labor shortages, especially among electricians, are a key driver for modularization; modular methods can reduce construction time by ~36% and costs by ~8%.
- Hyperscalers like AWS (Project Houdini) and Meta (tent structures) are pioneering modular designs, while colocation providers like Compass and QTS are industrializing repeatable modular kits.
- The modular market includes subsystem modularization (power blocks, cooling skids) and whole-facility modularization (containerized datacenters, prefab blocks), with over 80 vendors.
- Nvidia's DSX reference designs are standardizing modular AI factories, enabling faster deployment and integration across the supply chain.
EU opens call for seven 'gigafactories' to train next-generation AI
2 months ago
- The European Commission is aiming to build up to seven AI gigafactories by mid-2028 to create sovereign AI infrastructure and reduce dependence on foreign cloud and chip suppliers.
- The initiative, part of a broader tech sovereignty push, has expanded from 4-5 to 7 gigafactories due to industry interest, with 76 consortia expressing preliminary interest.
- Public funding is limited, with the EU contributing roughly €5 billion and governments another €5 billion, while private sector provides about €20 billion; only €1 billion is available now, with the rest from the next budget.
- The projects face criticism for delays, favoring wealthier member states, and reliance on foreign chipmakers; the EU has signed deals with Nvidia, AMD, and Qualcomm to mitigate risks.
- Construction is expected to start in early 2027, with gigafactories operational by mid-2028, and access to AI compute will be shared with public projects and research centers.
Proposed UK data centres could pay £100M in deposits for power demands
2 months ago
- Ofgem proposes refundable deposits for data centre developers to connect to the UK electricity grid, ranging from £237,500 to £712,500 per megawatt.
- Projects seeking 1 GW of power would face upfront costs of hundreds of millions, with deposits refunded if the project is completed.
- Electricity capacity requests have surged from 41 GW to 125 GW in the past year, exceeding the UK's 2025 peak demand of around 46 GW.
- The proposal aims to deter speculative projects from blocking grid capacity, with key milestones required to maintain connection queue positions.
- Data centres are controversial due to concerns over noise, electricity use, and water consumption for cooling, despite being essential for AI and digital services.
How OpenAI Kills Oracle
2 months ago
- On January 21, 2025, Oracle CEO Larry Ellison, SoftBank CEO Masayoshi Son, and OpenAI CEO Sam Altman announced the Stargate project with President Trump, a $500 billion AI infrastructure venture in the U.S. claiming to create over 100,000 jobs.
- Stargate LLC has never been formed after 15 months; SoftBank and OpenAI have contributed no capital, and no government money or funding has been deployed, raising suspicions of a marketing conspiracy among the companies.
- The project's only real component is a data center campus in Abilene, Texas, which was initially for xAI before Elon Musk pulled out; Oracle now builds it with only two of eight planned buildings operational and behind schedule.
- Oracle is heavily indebted, raising over $38 billion in project financing for data centers in Wisconsin and Shackelford, keeping debt off balance sheets, while its core businesses plateau and cashflow is negative $24.7 billion.
- OpenAI, the single tenant for all Stargate data centers, is expected to lose over $167 billion by 2028 and cannot pay Oracle for compute; it also has massive commitments to Amazon, Microsoft, and others totaling hundreds of billions.
- All Stargate data centers are far behind schedule; Abilene won't finish before April 2027, and other sites are barely started (e.g., patches of land with few steel beams), with completion unlikely before 2029.
- The analysis concludes Stargate will bankrupt Oracle unless OpenAI becomes the most profitable company ever within two years; GPU capacity will be obsolete before debts are repaid, and Ellison's personal loans ($20 billion) collateralized by Oracle shares risk margin calls.
Building the enterprise environment for agentic AI
2 months ago
- Agentic AI in enterprises is a larger systems problem involving orchestration, data access, tool use, and scalability, not just LLM inference.
- Key metrics for evaluating agentic AI systems include task success rate, cost per task, time per task, task throughput, agent density, and latency.
- Plan capacity using agent density (agents per vCPU) rather than raw agent count, and monitor P95 task latency for better performance signals.
- Default to scale-out architecture for agent hosting, reserving scale-up for workloads with heavier per-agent compute or constraints.
- Production-grade agentic AI value comes from automating codified workflows like code creation, regression testing, ticket triaging, market analysis, and security review.
What happens when coding agents stop feeling like dialup?
2 months ago
- Coding agents initially felt magical but now feel like dialup due to reliability issues and infrastructure strain.
- Token usage is exploding (50x increase on OpenRouter) because agentic workflows consume ~1000x more tokens than chats.
- Current LLM speed (30-60 tok/s) is frustrating, while faster models (2000 tok/s) could enable more unsupervised parallel approaches.
- The future of coding agents may involve parallel attempts and automated evaluation, enabled by higher tok/s.
- Infrastructure scaling faces challenges from slower semiconductor improvements, leading to potential pricing model innovations like off-peak plans.
- Developers should stay curious and adapt; experienced developers often dismiss coding agents but benefit most from them.
Please ship APIs, not AI
2 months ago
- Prioritize exposing core functionality through a well-designed, accessible API with token efficiency, fine-grained permissions, human-in-the-loop hooks, documentation, and skill usage support.
- Advocate for shipping an API instead of (or alongside) product-specific AI chat bars, sidebars, assistants, or agents, despite potential existential threats to the company.
- Emphasize that infrastructure providers play a hard but necessary role, and APIs enable agents to overcome UI limitations and operate more efficiently.
- Encourage leaping to the next frontier by delivering outcomes directly via API, rather than dashboards, starting with the API as the first step.
AI bet goes awry: Oracle fires 21,000 employees
2 months ago
- The global AI race has become a costly corporate competition, with tech giants expected to spend about $600 billion on AI infrastructure by 2026, risking cash flows and pressuring companies to prove profits.
- Oracle took a major bet by signing a $300 billion AI computing contract with OpenAI, leading to severe cash strain, aggressive cost-cutting, and a workforce reduction of about 21,000 employees (13%) by fiscal 2026.
- Oracle's multi-billion-dollar data center project in Port Washington, Wisconsin faces financial hurdles due to collateral requirements of over $7 billion after its credit rating dropped to BBB-, sparking a court petition against the high costs.
Big Tech AI Spree Revives Accounting Devices That Toppled Enron
2 months ago
- Big Tech companies are using off-balance-sheet financing vehicles similar to those that enabled Enron's collapse to fund massive AI infrastructure spending, potentially hiding debt and risks from investors.
- These arrangements package debt tied to assets like chips, servers, and energy equipment, keeping substantial costs off parent companies' balance sheets and presenting a healthier financial picture.
- Examples include Meta's $46 billion Louisiana data center VIE, Alphabet's off-balance-sheet leases, Oracle's $3.3 billion backstop, and Nvidia's $119 billion purchase obligations.
- The accounting treatment is controversial, with critics warning of a 'house of cards' risk and auditors highlighting significant judgment in consolidation decisions.
- Investors must dig through footnotes to uncover these structures, as the information is available but not immediately obvious, echoing Enron's crime of hiding special purpose vehicles.
The Growing Compute Shortage
2 months ago
- Compute is becoming an indispensable resource akin to 'spice' in Dune, powering AI and constraining growth due to supply chain bottlenecks.
- AI workloads, especially agentic systems, now consume 100x to 1,000x more tokens than traditional chatbots, accelerating demand.
- Multiple simultaneous bottlenecks exist: GPU scarcity, memory price surges, TSMC capacity sold out, and power infrastructure constraints.
- Scarcity is repricing assets, such as bitcoin miners with power and grid access, which are now seen as strategically valuable 'Lazarus assets'.
- Access to compute is becoming a competitive moat, with companies securing long-term supply gaining advantages over those without.
- Higher inference costs may slow AI adoption, concentrating resources on high-value applications and disadvantaging smaller firms.
- The AI race has shifted from a software competition to an industrial-scale infrastructure race for chips, memory, electricity, and land.
Power companies are using eminent domain to seize land for data centers
2 months ago
- The US is experiencing both an AI boom and a data center construction boom, with thousands existing and many in development.
- Despite AI's economic and national security importance, many Americans oppose local AI data center projects due to concerns about higher utility costs, pollution, noise, loss of green space, and high water/energy consumption.
- Opposition also extends to associated infrastructure like transmission lines, which often require crossing private land, leading to eminent domain disputes.
- Eminent domain, the government's power to seize private property for public use with just compensation, is being invoked for data center power needs, raising questions about whether private data centers qualify as 'public use'.
- Data centers' massive electricity demand strains grids; power companies may use eminent domain to acquire land for new transmission lines to meet this growing demand.
- While the Supreme Court has broadly interpreted 'public use', many states enacted reforms or have stricter constitutional interpretations, potentially offering landowners more protection in state courts.
- Court rulings on eminent domain for transmission lines vary; seizing land for grid reliability often qualifies as public use, especially if it benefits in-state customers, but lines that don't benefit the state may be challenged successfully.
Manufact (YC S25) Is Hiring a Senior infra engineer to build the MCP cloud
2 months ago
- Manufact provides cloud infrastructure and tools for MCP servers and Claude/ChatGPT apps.
- The company built the mcp-use SDK, an open-source framework with 10k GitHub stars and 8M+ downloads, used by over 20% of US 500 companies.
- Manufact Cloud usage is growing rapidly, doubling monthly, with a focus on capturing a large share of AI tool calls through MCP.
- The team raised $6.3M in seed funding with PeakXV as lead investor, backed by prominent angels including Supabase founders.
- The platform enables quick development and deployment of MCP servers/apps, making them available to AI agents and ChatGPT via GitHub integration.
Nobody knows what a used GPU cluster is worth
2 months ago
- If xAI defaults on its $5B debt, lenders like Apollo can seize its 200,000 GPU cluster, Colossus, and rent it out, but the cluster's value depends heavily on operational factors that lenders cannot see.
- GPU clusters require constant maintenance; at scale, failures are routine (e.g., ~50 GPUs fail daily in a 200,000-GPU cluster), and operational knowledge resides with the team, making the asset's value as collateral uncertain.
- AI infrastructure is increasingly financed by GPU-collateralized debt (e.g., via SPVs), with lenders charging high premiums (e.g., 8.5% above benchmark) due to unmeasurable risks, as there is no mature price discovery or hedging market for GPUs.
- GPUs have three values: face value (based on depreciation schedules), liquidation value (lower in distress), and going-concern value (dependent on operational handoff), but only face value is currently priced, creating a significant risk gap.
- Key players like KKR avoid GPU-backed debt, investing instead in physical infrastructure (e.g., data centers), signaling that chips may not be durable assets, while the lack of standardized valuation tools (e.g., futures, secondary markets) keeps financing costs high.
Microsoft's AI Buildout Sent Carbon Emissions Up 25% Last Year
2 months ago
- Microsoft's carbon emissions increased by 25% to 20 million metric tons in the past year.
- The company attributes the rise to building new data centers for AI, which drives energy and resource demand.
- Microsoft's 2030 carbon-negative goal is challenged by the rapid expansion of AI infrastructure.
- Other tech giants, like Alphabet, also report rising emissions due to data centers and AI demand.
- The International Energy Agency forecasts global data center emissions to nearly double to 300 million metric tons by 2035.
Microsoft adjusts climate agenda as emissions leap
2 months ago
- Microsoft's carbon emissions increased by 25% in fiscal year 2025, largely due to data center expansion for AI and cloud services.
- Electricity-related emissions rose to 13% of the total footprint in 2025 from 2% in 2024, partly due to changes in renewable energy accounting.
- Microsoft introduced site-level transparency for water withdrawals and electricity use in its latest sustainability report for the first time.
- The company is adopting a 'Community-First AI Infrastructure' approach to address backlash and ensure responsible data center development.
- Microsoft consumed 37 million MWh of electricity in 2025, a 24% increase, with North America accounting for 56% of the total.
- Water withdrawals were 13 million cubic liters, with 14 million liters replenished as part of enhanced water management efforts.
- The company is investing in renewable energy, nuclear power, geothermal, and innovative technologies like superconducting cables to reduce emissions.
- Microsoft signed carbon removal contracts for 29 long-term projects in 2025, aiming for over 45 million metric tons of reductions over 30 years.
- Despite emissions growth, Microsoft reaffirms its climate commitments and emphasizes evolving tactics for durable sustainability outcomes.
Nvidia, CoreWeave, and Nebius: Inside the Circular Financing of the GPU Boom
2 months ago
- Neoclouds like CoreWeave and Nebius are experiencing rapid growth due to hyperscaler demand for AI infrastructure, but they face profitability challenges due to high debt and cash flow mismatches.
- Hyperscalers such as Microsoft and Meta are making massive long-term commitments to neoclouds, exceeding $145 billion, to quickly access latest GPUs, optimize utilization, and shift costs from capex to opex.
- Nvidia plays a key role through circular financing, investing in neoclouds like CoreWeave and Nebius while also serving as their GPU supplier and providing financial backstops for unsold capacity.
- CoreWeave relies heavily on GPU-backed debt, with rising interest expenses pressuring profitability, while Nebius has a stronger balance sheet but still faces significant funding needs for expansion.
- The neocloud model balances surging AI demand with risks from rising debt, interest rates, and dependence on Nvidia's support, raising questions about long-term sustainability.
Why the Next Era of AI Is About Infrastructure, Not Just Models
2 months ago
- The evolution of AI in enterprises has shifted from initial experimentation to production, with concerns now focusing on ROI, cost, and data governance.
- Production AI requires reliability, auditability, cost control, and governance, highlighting the need for infrastructure rather than just models.
- Recent changes include increased adoption velocity leading to fragmentation, cost opacity, and governance gaps as AI expands across sectors.
- Managing multiple AI providers creates operational complexity with custom failover, spreadsheets for cost tracking, and manual tuning, which is unsustainable.
- Cost visibility is critical; teams need to compare cost-per-outcome and set policies to manage spending effectively as AI scales.
- Control over AI infrastructure is becoming the new competitive advantage, emphasizing intelligent routing, observability, and policy enforcement.
- Mozilla built Otari as an open-source control plane to provide visibility, governance, and flexibility, addressing consolidation and opacity in AI systems.
- The agentic era requires systems to coordinate many AI agents, increasing complexity and necessitating a foundational control layer for scalable deployments.
- Otari aims to define a new category of AI infrastructure, enabling organizations to own their stack and avoid vendor lock-in in critical industries.
$130M Series A to Build the Open Superintelligence Stack
2 months ago
- Prime Intellect raises $130M in Series A funding, led by Radical Ventures, with total funding now over $150M.
- The company aims to build an open superintelligence stack for training, deploying, and continuously improving AI models.
- RL (Reinforcement Learning) enables companies to own their model optimization loop, creating a competitive advantage.
- Over 6,000 customers, including AI startups and enterprises, use the stack, generating over $100M in annualized revenue.
- Case study: Ramp trained a model on Prime Intellect's stack that outperformed closed frontier models in speed and cost.
- Future focus includes scaling compute clusters, advancing long-horizon agents (RLMs), automating AI research, and enabling continual learning.
- Prime Intellect is hiring and invites teams to collaborate on training frontier models and building open superintelligence infrastructure.
Ways to think about token pricing
2 months ago
- Token prices are currently in a state of supply crunch and instability, with all variables in play, leading to an uncertain future equilibrium.
- Supply is increasing due to massive data center and semiconductor investments, improved inference efficiency, and variable token efficiency in new models.
- Demand surge is driven mainly by software development, a relatively small field, leaving future use cases, scale, and token needs unknown.
- Inference currently has high gross margins, but profitability depends on covering training costs and uncertain future demand ROI.
- Bottom-up modeling of token pricing is challenging due to unknown variables like supply, demand, marginal costs, and ROI, similar to forecasting broadband in 1998.
- Top-down analysis considers factors like frontier model adoption, ROI for high-cost models, competition, and value capture by models versus surrounding tooling.
- Key uncertainties include whether frontier models will maintain competitive advantages, see reduced competition, or face commoditization like databases.
- Comparisons with fiber, mobile data, and semiconductors highlight risks of low-margin infrastructure with value captured elsewhere, but analogies lack predictive power.
- Structural uncertainty in AI is heightened by a lack of theoretical understanding of model improvements, making predictions about compute needs and demand volatile.
- For foundation models to avoid commoditization and achieve market dominance, significant changes such as network effects, reduced competition, or regulatory shifts are needed, but current dynamics point toward commodity infrastructure.
Stymied datacentre projects threaten global AI revolution
2 months ago
- The Prince William Digital Gateway datacentre project in Virginia faces delays due to its proximity to a Civil War battlefield and legal opposition, reflecting broader challenges in datacentre development.
- Global datacentre projects, especially those exceeding 100MW, are experiencing cancellations and delays due to factors like inexperienced developers, high energy and water consumption, supply chain issues, and local community resistance.
- AI firms are constrained by datacentre shortages, with Google admitting compute limitations, while projects in California and Amsterdam face grid capacity issues, highlighting conflicts over electricity allocation.
- Uptime Institute reports an unprecedented increase in datacentre power demand, with mega-gigawatt projects emerging, but warns that North American grids are struggling to meet these surging needs.
- Governments, such as the UK's, are pushing AI ambitions without adequate audits of resource tradeoffs, including electricity availability, leading to potential inefficiencies in site selection.
- Despite challenges, some experts like JLL's Andrew Batson are optimistic, citing innovations in battery storage and onsite power generation to overcome energy constraints, with ongoing lease and construction activities ahead of estimates.
- The largest planned datacentres globally propose 45GW of onsite power, primarily from gas, equaling the UK's peak demand, underscoring the scale of energy requirements and the shift toward onsite solutions.

firstprev0next

ai infrastructure

The Wild Wild West of Lego Datacenters

EU opens call for seven 'gigafactories' to train next-generation AI

Proposed UK data centres could pay £100M in deposits for power demands

How OpenAI Kills Oracle

Building the enterprise environment for agentic AI

What happens when coding agents stop feeling like dialup?

Please ship APIs, not AI

AI bet goes awry: Oracle fires 21,000 employees

Big Tech AI Spree Revives Accounting Devices That Toppled Enron

The Growing Compute Shortage

Power companies are using eminent domain to seize land for data centers

Manufact (YC S25) Is Hiring a Senior infra engineer to build the MCP cloud

Nobody knows what a used GPU cluster is worth

Microsoft's AI Buildout Sent Carbon Emissions Up 25% Last Year

Microsoft adjusts climate agenda as emissions leap

Nvidia, CoreWeave, and Nebius: Inside the Circular Financing of the GPU Boom

Why the Next Era of AI Is About Infrastructure, Not Just Models

$130M Series A to Build the Open Superintelligence Stack

Ways to think about token pricing

Stymied datacentre projects threaten global AI revolution