Interpretable Adversarial Attacks in Superposition

[carlospolop]

🤖 Automated Content Update

This PR was automatically generated by the HackTricks News Bot based on a technical blog post.

📝 Source Information

• Blog URL: https://nccgroup.com/research/interpretable-adversarial-attacks-in-superposition
• Blog Title: Interpretable Adversarial Attacks in Superposition
• Suggested Section: AI > AI Security > Mechanistic-Interpretability-Guided Adversarial Attacks / Superposition Feature Interference

🎯 Content Summary

Classification and scope. The NCC Group research article, published on July 17, 2026, describes a research-stage white-box adversarial machine-learning technique rather than a conventional software vulnerability. It has no CVE, targets neural networks that represent more semantic features than available activation dimensions, and uses mechanistic interpretability to construct human-understandable adversarial inputs. The demonstrated attack targets a deliberately small MNIST model; exploit...

🔧 Technical Details

Superposition feature-interference attack. When a neural network or an interpretable representation encodes more semantic features than available activation dimensions, feature vectors become non-orthogonal. A white-box attacker can calculate decoder-vector similarities, select several legitimate features whose projections point toward a target direction, and construct an input that activates those contributors together. If their combined projection exceeds the target’s normal activation threshold, the linear decoder and ReLU reconstruct the absent target feature, causing downstream computation to treat it as present. A useful geometric test is Iₜ = Σᵢ≠ₜ |aᵢ(vᵢ·vₜ)/||vₜ|| | > max(a), while accounting for signed cancellation, decoder bias, and ReLU thresholds.

Interpretability-guided input synthesis. Recover feature directions with encoder inspection, sparse autoencoders, circuit analysis, or similar tools; render inputs that maximise individual fe...

🤖 Agent Actions

Implemented the HackTricks update.

Changes

• Added src/AI/AI-Mechanistic-Interpretability-Adversarial-Attacks.md
  • Superposition interference model
  • White-box assessment workflow
  • Signed decoder contribution analysis
  • NumPy triage snippet
  • MNIST proof of concept
  • Gemma Scope geometric analysis
  • Practical limitations and citations
• Linked the page from src/AI/README.md.
• Added it to src/SUMMARY.md.

Validation

• Internal links, citations, references, and forbidden-source checks passed.
• Reference URLs are reachable.
• git diff --check passed.
• mdbook was unavailable in the environment, so a full build was not run.

---

This PR was automatically created by the HackTricks Feed Bot. Please review the changes carefully before merging.

[carlospolop]

🔗 Additional Context

Original Blog Post: https://nccgroup.com/research/interpretable-adversarial-attacks-in-superposition

Content Categories: Based on the analysis, this content was categorized under "AI > AI Security > Mechanistic-Interpretability-Guided Adversarial Attacks / Superposition Feature Interference".

Repository Maintenance:

• MD Files Formatting: 1032 files processed

Review Notes:

• This content was automatically processed and may require human review for accuracy
• Check that the placement within the repository structure is appropriate
• Verify that all technical details are correct and up-to-date
• All .md files have been checked for proper formatting (headers, includes, etc.)

Bot Version: HackTricks News Bot v1.0