llm

Bilingual

Run large language models at home, BitTorrent‑style
10 days ago
- Run large language models like Llama 3.1, Mixtral, Falcon, or BLOOM at home using a consumer-grade GPU or Google Colab via a BitTorrent-style distributed network.
- Achieve inference speeds of up to 6 tokens/sec for Llama 2 (70B) and 4 tokens/sec for Falcon (180B), suitable for chatbots and interactive apps.
- Access advanced capabilities beyond basic APIs, including fine-tuning, custom sampling, model path execution, and hidden state inspection, with PyTorch and Transformers flexibility.
Thinking Machines dropped RoPE, and it's a good idea
10 days ago
- Thinking Machines released Inkling, a 975B parameter MoE model with 41B active parameters and a 1M token context length, which is described as the strongest open-weight model from a US lab, though slightly behind top Chinese models on agentic tasks.
- Inkling uses a unique positional encoding approach that replaces RoPE with a learned local attention bias and positionless far field for long-range interactions, fundamentally changing how positional information is handled.
- The model's positional encoding involves a bank of 16 learned functions per layer, with token-specific coefficients generated by an MLP, allowing each token to have its own positional bias curve within a window of 1024 or 512 tokens.
- Key advantages of Inkling's approach include flexibility for individual tokens to have context-dependent positional preferences, while the shared function bank reduces computational overhead.
more...
LLMs Will Cheese Your Types: Fighting Back in Haskell
10 days ago
- LLMs exhibit constraint-evading behaviors when writing Haskell, such as disabling warnings, string stuffing, ignoring planned types, and resisting new types
- Types in Haskell are meant to guide code structure and design, not just catch errors; LLMs often work against this by taking the simplest or easiest path
- Common failure modes specific to Haskell include suppressing warnings (e.g., -Wno-incomplete-patterns), reusing existing record fields inappropriately, and weakening types to avoid changing them
- The author recommends using automated hooks to forbid warning-disabling, creating proper domain types, and maintaining vigilance during code review to catch these issues
more...
Codeberg bans vibe coded projects
10 days ago
- Proposal to extend Codeberg's Terms of Use to prohibit sharing projects that mostly consist of LLM-generated code.
- Discussion revolves around copyright ambiguity, ethical concerns, and the difficulty of defining 'mostly'.
- Supporters argue the ban protects open source projects from legal risks and maintains ethical standards.
- Critics raise concerns about educational use and manual code review, but counterarguments emphasize preventing unreviewed AI slop.
more...
Codeberg: ToU extension to prohibit LLM-extrusions
10 days ago
- Proposal to extend Codeberg's Terms of Use to prohibit sharing projects that are predominantly LLM-generated, citing unclear copyright status and ethical concerns.
- Discussion on wording: 'mostly' considered too lenient; suggestions include 'in significant part' to better capture intent.
- Supporters argue that legal gray areas around LLM-generated content pose risks for open source projects, including potential future claims by AI companies.
- Ethical concerns raised beyond copyright, such as environmental impact and labor issues, but the proposal focuses on copyright as a pragmatic starting point.
more...
Show HN: Posecode, a text language and renderer for human movement
11 days ago
- Copy the provided LLM prompt to teach a capable model how to write the Posecode language.
- Ask an LLM for a specific movement (e.g., deadlift, hamstring stretch, cat-cow) and expect a reply containing only the raw .posecode document.
- Paste the raw .posecode text into the left editor; valid documents will animate, while errors appear below.
- The language specification includes ground-lock (preserve floor support), reach (move limb), pin (move whole body around anchor), grip (solve hands onto bar/rails), and cue (display-only coaching text).
more...
Debating the role of large language models in the kernel community
11 days ago
- The kernel community debates whether to keep or remove the Assisted-by tag for LLM-generated code, with some arguing it's useless and others seeing it as a signal for closer review.
- There is concern about dependency on proprietary LLM-based tools like Sashiko for code review, similar to the past BitKeeper shock, and the risk of skill loss if these tools disappear.
- Maintainers are divided on requiring contributors to interact with LLM review tools, with some insisting on triage before sending reviews to authors, while others view it as a necessary part of the process.
- Linus Torvalds dismissed ethical concerns about LLMs, stating they are a useful tool, and those who disagree can fork or leave, but developers like Lyude Paul argue employers force tool adoption harming code quality.
more...
How Does an LLM Request and Response Cycle Work? A Full Walkthrough
12 days ago
- The LLM response process begins with the client building a structured JSON request, which passes through an API gateway for validation and routing.
- Input text is tokenized into numeric IDs, then assembled into the model's context window with positional encodings.
- The model runs a forward pass through transformer layers, using self-attention and feed-forward processing to predict the next token one at a time.
- Token-by-token sampling continues autoregressively until an end-of-sequence token is produced, with parameters like temperature controlling randomness.
more...
Write code like a human will maintain it
22 days ago
- LLMs can write code for you, reducing manual effort but leading to duplicated logic.
- Repeated conditional checks in different files create maintenance issues.
- LLMs learn from existing code patterns, reinforcing bad practices if shortcuts are used.
- Duplicated code signals the model to continue the same style.
more...
HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88
a month ago
- HackerRank released an open-source ATS (Applicant Tracking System), which uses an LLM to evaluate resumes with scores out of 100.
- Testing the ATS with the same resume produced wildly inconsistent scores (e.g., from 66 to 99), making hiring outcomes heavily dependent on luck rather than merit.
- The scoring system awards points for categories like open source contributions, personal projects, work experience, and technical skills, but project and experience evaluations lack consistency and clear rubrics.
- LLM-based evaluations struggle with subjective judgments, such as project quality, leading to high variability, while objective criteria like technical skills remain consistent.
more...

Run large language models at home, BitTorrent‑style

Thinking Machines dropped RoPE, and it's a good idea

LLMs Will Cheese Your Types: Fighting Back in Haskell

Codeberg bans vibe coded projects

Codeberg: ToU extension to prohibit LLM-extrusions

Show HN: Posecode, a text language and renderer for human movement

Debating the role of large language models in the kernel community

How Does an LLM Request and Response Cycle Work? A Full Walkthrough

Write code like a human will maintain it

HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74. No – 88