source: Simon Willison: Quoting Boris Cherny
level: technical
Anthropic’s Opus 5 model demonstrates notable resistance to prompt injection attacks, according to Boris Cherny. He stated that the model is the company’s least prompt-injectable release so far. This claim is supported by details in the system card, specifically on page 73, which covers prompt injection evaluations and red teaming results. The improvement addresses a persistent security challenge for large language models deployed in real-world applications.
The system card reports that Opus 5 performed well across prompt injection evaluations and red teaming exercises. These tests measure how easily an attacker can manipulate the model into ignoring its instructions or revealing sensitive information. While exact scores are not disclosed in the quotation, the emphasis on reduced vulnerability suggests a meaningful advance over previous versions. This matters because prompt injection remains a top concern for safely integrating LLMs into products.
Prompt injection has been a known weakness since the rise of instruction-following models, often exploited to bypass safety filters or extract system prompts. Anthropic’s focus on hardening Opus 5 reflects broader industry efforts to make AI systems more trustworthy. As companies embed LLMs into customer-facing tools and internal workflows, resistance to such attacks becomes critical. The improvement may influence adoption in security-sensitive sectors like finance, healthcare, and enterprise software.
why it matters: Stronger prompt injection resistance reduces the risk of AI systems being manipulated to ignore safety rules or leak data, making them safer for production use.