ai detection

Bilingual

Substackers Say New AI Detection Tool Is a 'Witch Hunt'
2 months ago
- Substack's new AI detection features face backlash from creators who view them as a 'witch hunt' and fear false accusations.
- Some writers who use AI for assistance argue it improves their writing quality and refuse to apologize for its use.
- Critics highlight that AI detectors like Pangram are inaccurate, risking reputational damage from false positives, especially for non-native speakers and neurodiverse writers.
- Substack CEO Chris Best claims the tool aims to increase transparency, not penalize AI use, and offers a 'How I make this' statement for disclosure.
- Supporters of the feature see it as a way to combat unlabeled AI-generated content ('slop'), but the backlash shows the challenge of balancing detection with creator trust.
Professor Created a Trap in His Midterm to Catch Students Using AI
2 months ago
- Dr. Jason Gibson of Alcorn State University planted the hidden word 'Madagascar' in white font within an exam prompt to catch students using AI to generate essays without proofreading.
- 32 out of 35 students failed the midterm section because their AI-generated responses included the hidden word, revealing they had copied directly from a chatbot.
- Gibson's TikTok videos about the trap went viral, sparking debate on AI's role in education, with some praising his integrity lesson and others criticizing the tactic.
- Only two students appealed the grades, and one had her grade changed because dark mode made the white text invisible on her screen.
- Gibson emphasized his goal was not to fail students but to highlight the issue of students outsourcing learning, as many responses contained nonsensical AI-generated phrases.
I'm So Tired of "Is This AI?"
2 months ago
- AI detection is unreliable and easily gamed with frequent false positives.
- AI-generated content now surpasses human-created work, making detection pointless.
- Intent behind AI use matters more than detection—slop is bad, assisting with editing is fine.
- Substack's partnership with Pangram for AI detection may lead to public 'witch hunts' and unfair targeting.
- Focus should be on content value and enjoyment, not on whether AI was involved.
- AI detectors cannot measure effort, thinking, or editing, only AI presence.
I wouldn't say Pangram is broken, but I would say that it's brittle
2 months ago
- Pangram produces contradictory results: the same text can be flagged as 100% AI or 100% human depending on context, creating a 'Russian doll' of inconsistent outcomes.
- The percentage meter is misleading because many users interpret '100% AI' as high confidence, but it actually indicates the portion of text suspected to be AI-generated; the meter also frequently outputs 100% for hybrid texts.
- False positives are easy to induce: the author wrote his own text that Pangram flagged as 100% AI with high confidence, and AI-generated text can be made to appear human by embedding it in human writing.
- Even a low error rate (e.g., 1 in 10,000) leads to many false accusations when applied to large volumes of text, especially every paragraph becomes a separate test.
- The tool's design encourages a culture of 'gotcha' accusations among writers, exploiting professional jealousy and resentment rather than providing reliable evidence.
- Pangram's founder acknowledged the issue with short text segments and the inevitability of adversarial false positives, but the author argues for epistemological modesty and warns against treating any detection tool as a final authority.
Scanning for Pangram Errors
2 months ago
- Pangram is an AI detection tool with a claimed false positive rate of 1 in 10,000.
- The author tested Pangram on 2.9 million words from 45 old books that were never digitized, to check for memorization of training data.
- Pangram did not flag any human-written content as AI (false positives) except for segments where OCR (Mistral) hallucinated text, which Pangram correctly identified as AI-generated.
- The results suggest Pangram is not simply memorizing its training set but is detecting AI writing patterns.
- Human review is recommended when using Pangram for important decisions, as the tool itself is not infallible.
Substack's new tool tells you who's been writing their newsletters with AI
2 months ago
- Substack has launched an AI detection feature using Pangram to estimate how much of a post is human-written versus AI-generated.
- In the short term, this could hurt trust by revealing AI-written newsletters, but long-term it may help maintain quality and user trust.
- The feature is part of a broader trend of platforms labeling AI-generated content, including social media and music streaming services.
- Substack CEO Chris Best endorsed the tool, emphasizing that the hard part—ideas worth reading—should remain human, while software handles the rest.
- The detection works on posts, notes, replies, and comments over 100 characters, and writers can add an optional AI author's note.
- The tool is not punitive; it encourages transparency and allows publishers to scan drafts and report errors.
"No AI" Statements Are More Than Mere Statements
2 months ago
- The author argues that 'No AI' statements are necessary and useful, disagreeing with a post that claims viewers should distinguish human from AI content and that quality/voice alone suffice.
- Handmade and human-made labels have existed long before AI, because consumers cannot reliably tell the difference between machine-made and handmade products, and the same applies to AI-generated content.
- Quality, voice, and style are not reliable indicators of human creation, as AI can replicate them; human effort is valuable regardless of perceived quality, and some works (e.g., technical writing) lack distinct voice yet are still valuable.
- The author's personal experiences show that viewers and AI detectors often falsely label human-written content as AI-generated, making explicit labels necessary.
- Declaring 'No AI' is a stance that signals the creator values human effort and the creative process, and it supports transparency and pride in human-made work.
- The creative process itself is meaningful and irreplaceable, and creators should disclose their use of AI to show support for human effort.
Against Claudefishing – AI detection feature on Substack
2 months ago
- Substack is launching a feature with Pangram to detect AI-generated text, aiming to maintain human authenticity.
- The problem is not AI use per se, but the mismatch between reader expectations and reality, especially when readers unknowingly engage with AI content.
- Substack is introducing tools for transparency, including a detection scan and a 'How I make this' statement for creators.
- The feature is available on web and iOS, with Android coming soon; future plans include customizable preferences for AI content.
- The article highlights a debate: detection tools can help transparency but may become a purity test, as AI-assisted writing can still involve human creativity.
Over 30% of new ArXiv submissions now read as AI-written
2 months ago
- A study measured AI-written content in arXiv papers using a calibrated detector with a 0.4% false-positive rate on pre-ChatGPT human text.
- About a third (32%) of recent arXiv papers from 2023-2026 read as machine-written, with peaks near 39% in early 2026, showing a significant rise post-ChatGPT.
- Field analysis reveals wide variation: Computer science leads at 65% flagged, while mathematics is lowest at 0.7%, partly due to detector limitations with notation-heavy prose.
- Limitations include small control sample sizes per field, potential detector blind spots (especially in mathematics), and inability to distinguish between heavy AI assistance and full generation.
- The detector provides a lower bound on AI adoption; reported prevalence is conservative due to incomplete coverage of diverse models and prompts used by authors.
How Does Pangram Work?
2 months ago
- Pangram is a classifier model that distinguishes AI-generated text from human writing by analyzing writing style patterns.
- It operates by mapping writing styles, placing similar authors close together; AI and human texts form distinct clusters on this map.
- Pangram evaluates entire texts, not just individual words, avoiding pitfalls of perplexity-based detectors which often misclassify historical documents.
- Training involves pairing human-written texts with AI-generated doubles to refine detection, focusing on subtle stylistic boundaries.
- The model requires a minimum word count for accuracy, and its performance is statistically validated through internal benchmarks and third-party testing.
Detecting LLM-Generated Texts with “Classical” Machine Learning
2 months ago
- Mainstream LLM-generated text as of early 2026 shows strong statistical patterns, detectable by traditional machine learning models.
- An online demo using a simple SVM classifier achieves ~85% single-sentence accuracy, trained on human and LLM-generated data.
- Data was gathered from pre-2022 human texts and LLM-regenerated summaries via multiple low-cost/free APIs from models like Gemini, Qwen, and GLM.
- Seven binary classifiers were trained, using majority voting; detection remains robust against common anti-detection methods like translation or rewriting prompts.
- Testing on unseen models and human texts shows low false positives; analysis of Lofter content suggests significant undisclosed AI-generated material.
- The author expresses skepticism about LLMs' creativity, viewing AI-generated content as repetitive and shallow, and reflects on broader implications of AI misuse.
Probably the best AI detector tool is ready
2 months ago
- AI text detector analyzes linguistic patterns to detect AI-generated text with 99.5% accuracy, showing the reasoning behind each verdict.
- Features include lightning-fast results (under 10 seconds), unlimited checks, real linguistic analysis using proven techniques, detection of over 10 AI models, and 100% privacy with encrypted, non-permanent data transmission.
- Offers sentence-by-sentence color-coded analysis and PDF reports with highlights and methodology breakdown.
- Built for educators, content creators, and businesses to verify authenticity and maintain integrity; includes transparent, cancel-anytime pricing plans.
- User testimonials from students, professors, writers, and editors praise its accuracy, low false positive rate, and usefulness in academic and professional settings.
- Based on peer-reviewed science using metrics like perplexity (lower in AI text), burstiness (sentence variation), and entropy (syntactic complexity) from institutions like Stanford and UCLA.
- Helps prevent language model collapse and machine-generated content dilution; includes quotes from researchers on the importance of detection tools.
- FAQs explain the detector as a tool analyzing text via linguistic metrics to distinguish human from AI writing, trusted by thousands and offering Premium upgrades for bulk scanning and API access.
Show HN: How clanker are you? A reverse Turing test
3 months ago
- A tool called "How clanker" analyzes Twitter (X) posts to measure how AI-like (predictable) someone's writing is.
- It pulls recent posts from a user, excluding retweets and replies, and scores each word based on its surprisal (predictability) under language models.
- Low surprisal (high predictability) results in a "clanker" (AI-like) score, while high surprisal indicates more human-like writing.
- The method uses log probabilities from models, normalizing scores against baselines, and the overall score reflects the nearest model's assessment.
- In demo mode, scores are simulated using a deterministic stand-in instead of real models, and the tool is presented as fun, not a judgment of the person.

ai detection

Substackers Say New AI Detection Tool Is a 'Witch Hunt'

Professor Created a Trap in His Midterm to Catch Students Using AI

I'm So Tired of "Is This AI?"

I wouldn't say Pangram is broken, but I would say that it's brittle

Scanning for Pangram Errors

Substack's new tool tells you who's been writing their newsletters with AI

"No AI" Statements Are More Than Mere Statements

Against Claudefishing – AI detection feature on Substack

Over 30% of new ArXiv submissions now read as AI-written

How Does Pangram Work?

Detecting LLM-Generated Texts with “Classical” Machine Learning

Probably the best AI detector tool is ready

Show HN: How clanker are you? A reverse Turing test