We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them
8 hours ago
- #AI Security
- #Malware Detection
- #Binary Analysis
- AI agents can detect some hidden backdoors in binary executables, but the approach is not production-ready.
- Claude Opus 4.6 found backdoors in small/mid-size binaries only 49% of the time, with high false positive rates.
- Binary analysis involves reverse engineering machine code into assembly and pseudo-C, a tedious process.
- The benchmark used open-source projects with artificially injected backdoors to test AI detection capabilities.
- Claude successfully identified a backdoor in lighttpd by tracing popen() usage and analyzing decompiled code.
- AI models often miss obvious backdoors, rationalizing them as legitimate functions, like in the DHCP backdoor example.
- Current LLMs lack strategic focus, often decompiling random functions instead of prioritizing high-risk areas.
- False positives are a significant issue, with models reporting non-existent backdoors 28% of the time.
- Open-source tools like Ghidra and Radare2 lag behind commercial alternatives, especially for Rust and Go binaries.
- AI can make initial security audits more accessible but is not yet reliable for end-to-end malware detection.