Measuring LLMs’ Ability to Perform Cryptanalysis
Overview
Researchers have developed a new benchmark called CryptanalysisBench to evaluate the ability of large language models (LLMs) to conduct cryptanalysis, which is the process of finding vulnerabilities in cryptographic schemes. The study found that models like Anthropic's Claude Opus 4.8 and others demonstrated a significant capability, breaking a majority of tested cryptographic primitives. The benchmark includes tasks across various cryptographic systems, ranging from well-known weaknesses to challenges involving advanced algorithms. Notably, some models not only replicated existing attacks but also discovered new vulnerabilities, including a key-recovery method exploiting a design flaw in the SpoC AEAD. This research is significant as it raises concerns about the security of cryptographic systems that are foundational to digital security, particularly as AI continues to advance in its reasoning capabilities.
Key Takeaways
- Affected Systems: Cryptographic primitives, including those from NIST standardization competitions such as block ciphers and hash functions.
- Action Required: Continued monitoring and assessment of cryptographic algorithms for vulnerabilities, implementation of stronger cryptographic standards as needed.
- Timeline: Newly disclosed
Original Article Summary
There’s new benchmark measuring AI’s ability to perform mathematical cryptanalysis. Anthropic’s frontier model actually found new attacks. The benchmark: “CryptanalysisBench: Can LLMs do Cryptanalysis?” The idea is to benchmark the ability of LLMs to discover new mathematical cryptanalytic attacks against a series of historical algorithms. Abstract: Cryptanalysis—the task of finding attacks against cryptographic schemes—its at the intersection of mathematical reasoning and cybersecurity, two areas where LLMs have advanced fastest. Cryptanalysis represents both a clean testbed for frontier reasoning (as practical attacks can be automatically verified) and a domain with unusually high stakes, since the primitives under study underpin our digital security. In this paper we ask whether LLMs can do cryptanalysis, and find that the answer is increasingly yes. We introduce CryptanalysisBench, 191 tasks across six families of cryptographic primitives (block ciphers, hash functions, etc.) drawn primarily from four NIST standardization competitions. Our benchmark consists of three tiers: (i) primitives with known practical breaks; (ii) primitives with no known practical break, evaluated both at full strength and as scaled-down variants; and (iii) a challenge set of production primitives at the frontier of cryptanalysis. Five frontier models (Claude Opus 4.8, Sonnet 5, Mythos 5, GPT-5.5, and the open-weights GLM-5.2) break 65%86% of Tier 1 schemes, 612 Tier-2 schemes at full strength, and 2461 across all scaled-down variants. Beyond deriving known results, models produce novel cryptanalysis, such as a key-recovery attack that exploits a design flaw in the SpoC AEAD and an error in KINDI’s published CCA-security proof, both to the best of our knowledge not previously known...
Impact
Cryptographic primitives, including those from NIST standardization competitions such as block ciphers and hash functions.
Exploitation Status
No active exploitation has been reported at this time. However, organizations should still apply patches promptly as proof-of-concept code may exist.
Timeline
Newly disclosed
Remediation
Continued monitoring and assessment of cryptographic algorithms for vulnerabilities, implementation of stronger cryptographic standards as needed.
Additional Information
This threat intelligence is aggregated from trusted cybersecurity sources. For the most up-to-date information, technical details, and official vendor guidance, please refer to the original article linked below.