Bilingual

Your Site, your rules: new AI traffic options for all customers
a month ago
- Cloudflare announced the first Content Independence Day a year ago to help website owners block AI bots and launched a Pay-Per-Crawl marketplace.
- The conversation around AI bots has evolved from simply blocking training to seeking nuanced solutions, recognizing that website owners need more options than just blocking all automation.
- Small websites face a dilemma: allow AI training for discoverability or block it and risk being unfindable, which unfairly advantages large search providers.
- AI is now integrated into many services, like Google's answer engine, shifting search from AI-sorted results to direct answers on results pages.
- Cloudflare's updated approach classifies bots based on behavior—Search, Agent, and Training—rather than just labeling them as AI, to provide better transparency and control.
- New management options allow website owners to control AI traffic based on these three use cases, with defaults set to block Training and Agent bots on ad pages while allowing Search.
- Multi-purpose crawlers like Googlebot will be handled according to all their behaviors, and customers can opt out of default configurations before September 15, 2026.
- BotBase is launched as a visibility tool for Enterprise customers, tracking all known bots with classifications and enabling detailed management and filtering.
- A new 'content use' signal is introduced for robots.txt, allowing website owners to specify how bots can use their content—immediate, reference, or full—enhancing control over content reuse.
- The definition of 'Verified' bots is updated to align with category-based allowances, and the verification process is made more transparent and accessible to bot operators.
- Cloudflare experiments with transitive trust using the Forwarded header to maintain trust across intermediaries, ensuring that trusted operators retain access even through layers.
- These changes aim to give website owners more control over content usage, promote transparency, and adapt to the evolving web ecosystem, with all new options available now.
Web-based cryptography is always snake oil
23 days ago
- Web-based 'end-to-end' encryption is fundamentally flawed because the server operator, who you're supposedly securing against, distributes the client-side code and could push malicious updates.
- Such cryptosystems are incoherent as they fail to separate the entity being secured against from the one providing the implementation, making them essentially backdoored.
- Companies adopt snake oil cryptography primarily as a legal maneuver to avoid warrant and subpoena obligations, not to provide real security.
- Legal reliance on this model is risky, as government actions like in the Lavabit and FBI–Apple cases show that authorities can and do demand compromises.
- Web platform constraints inherently couple client-side code distribution with the service, making meaningful cryptosystems impossible without a paradigm shift.
- Potential fixes like using service workers with subresource integrity have limitations, such as trust-on-first-use issues and user verification challenges.
An Update on the scraper situation
17 days ago
- Scraper attacks from AI bot networks using residential proxies are overwhelming websites, creating excessive traffic by using millions of compromised devices, often without the owner's knowledge.
- Companies like Bright Data and others offer 'ethical' residential-proxy networks, exploiting free VPNs or SDKs to turn users' devices into endpoints for scraping.
- The scraping is believed to be fueled by a growing demand for training data for AI models, driven by various entities including corporations and government agencies.
- Websites are implementing defensive measures such as proof-of-work systems like Anubis, captchas, paywalls, and login gates to mitigate scraper impacts.
- LWN has adopted site optimizations and selective access controls to protect its site while minimizing disruption to legitimate users and search engines.
- Recent takedowns of networks like IPIDEA and NetNut have temporarily reduced attacks, but the problem persists, indicating the need for more lasting solutions.
- There is concern that the ongoing scraping threatens the open internet and may push more content behind restrictive barriers, compromising its creative potential.
Prismata: Confining cross-site prompt injection in web agents
17 days ago
- Prismata is a defense mechanism for web agents to prevent cross-site prompt injection attacks.
- It enforces contextual least privilege by deriving dynamic trust labels for page content and restricting agent capabilities.
- The system provides structural confinement guarantees to bound labeling errors, ensuring labels can only decrease in privilege.
- Prismata requires no developer annotations, making it applicable to a wide range of websites.
- It significantly reduces attack success rates while maintaining utility for benign tasks.
What's the best way to do authentication in modern applications
17 days ago
- JWTs are commonly used for authentication but storing them in localStorage makes them vulnerable to XSS attacks, as any JavaScript on the page can read localStorage.
- Using HTTP-only cookies with flags like Secure and SameSite=Lax prevents JavaScript from reading tokens, reducing the risk of token exfiltration during XSS attacks.
- CSRF attacks are a risk with cookies, but can be mitigated using CSRF tokens, SameSite attributes, and checking request headers like Origin or Sec-Fetch-Site.
- Stateless JWTs have revocation issues; server-side sessions with opaque session IDs in HTTP-only cookies allow immediate session invalidation and are more secure for single applications.
- For OAuth or third-party tokens, use a Backend for Frontend (BFF) pattern where tokens are stored server-side, and the browser only holds an HTTP-only session cookie, preventing token exposure to JavaScript.
- Refresh token rotation and reuse detection enhance security by invalidating stolen refresh tokens and forcing re-authentication.
- Device Bound Session Credentials (DBSC) bind sessions to hardware, preventing stolen cookies from being used long-term, though adoption is currently limited.
- For React applications, centralize token handling in an API wrapper to manage token refresh and prevent concurrent refresh requests from causing logout.

Hasty Briefsbeta

web security

Your Site, your rules: new AI traffic options for all customers

Web-based cryptography is always snake oil

An Update on the scraper situation

Prismata: Confining cross-site prompt injection in web agents

What's the best way to do authentication in modern applications