Noma Labs discovered GitLost, a critical prompt injection vulnerability in GitHub's new Agentic Workflows.
Exploit allows an unauthenticated attacker to exfiltrate data from private repositories by posting a crafted issue in a public repository of the same organization.
The GitHub AI agent, triggered by workflow events, treats malicious instructions hidden in issue content as trusted commands.
Attackers can leverage keywords like 'Additionally' to bypass GitHub's guardrails and leak private data.
Leaked data was posted as a public comment, making it accessible to anyone.
Vulnerability highlights fundamental security challenges in agentic AI systems where the context window becomes an attack surface.
Recommendations include not trusting user-controlled content, scoping permissions minimally, and isolating user input from instruction context.
Prompt injection is the top AI security threat because LLMs cannot distinguish between legitimate and malicious instructions.
Current guardrails only mitigate damage without solving the root cause of separating trusted from untrusted sources.
Push-based attacks target individuals (e.g., via email) and are limited in scale, while pull-based attacks (e.g., from websites) have also been limited in scale.
HalluSquatting is a new pull-based attack that exploits LLM hallucinations to target AI coding assistants and agents, enabling large-scale exploits like botnet assembly and DDoS attacks.
The attack works by predicting hallucinated resource identifiers, registering them with malicious code, and infecting devices indiscriminately, affecting tools such as Cursor, GitHub Copilot, and others.
Lawyers in Brazil attempted to deceive the justice system by hiding instructions in white text within petitions, targeting the AI system Galileu used by the court. The AI detected the attempt, leading to sanctions.
This 'prompt injection' technique is not science fiction but an expected development, with legal frameworks like Spain's already considering AI use in courts.
Prompt injection involves inserting hidden prompts to manipulate AI systems, and similar tactics have been used in education and hiring to detect or influence AI behavior.
The legal world is particularly vulnerable due to the high volume of documents and the state's authority, with prior cases of lawyers using AI to fabricate case law.
Galileu assists judges by drafting rulings but requires human review; the fraud aimed to eliminate evidence before it reached the judge.
Other Brazilian courts have found similar attempts, including a prompt ordering the AI to grant all requested measures.
The real danger is 'invisible fraud'—altering the process of reading and interpretation, not just falsifying content.
Risk extends to law firm document exchanges, raising concerns about professional ethics and the fine line between zealous defense and deception.