TopoGuard: Graph Theory Based Defenses Against Split-Knowledge Attacks on RAG
TopoGuard introduces a graph theory-based defense mechanism to detect "split-knowledge attacks" in Retrieval Augmented Generation (RAG) systems, where individually benign documents create malicious associations when combined. The method constructs a semantic similarity graph from retrieved documents to identify contexts with malicious topology, proving effective against threats that bypass traditional per-document filters like LlamaGuard. Experimental results on the HotpotQA dataset show TopoGua
Analysis
TL;DR
- TopoGuard introduces a graph theory-based defense mechanism to detect "split-knowledge attacks" in Retrieval Augmented Generation (RAG) systems, where individually benign documents create malicious associations when combined.
- The method constructs a semantic similarity graph from retrieved documents to identify contexts with malicious topology, proving effective against threats that bypass traditional per-document filters like LlamaGuard.
- Experimental results on the HotpotQA dataset show TopoGuard-$\lambda_2$+Entity achieves 32.6% recall at 1% False Positive Rate, catching 21 times more attacks than LlamaGuard-2-8B.
- The proposed variants operate with sub-millisecond latency and demonstrate robustness against adaptive adversaries and noisy inputs, offering a computationally efficient alternative to large language model-based detection systems.
Why It Matters
This research addresses a critical blind spot in current RAG security architectures by highlighting how adversarial attacks can exploit the aggregation of multiple documents rather than individual content. For AI practitioners, it underscores the necessity of moving beyond single-document filtering to holistic, structural analysis of retrieved context to prevent sophisticated injection attacks. The efficiency of the solution also suggests that high-performance security measures can be integrated into production RAG pipelines without significant latency overhead.
Technical Details
- Attack Vector Analysis: The paper defines "split-knowledge attacks," where adversaries inject documents that are safe in isolation but form false or harmful associations when processed together by an LLM, rendering standard filters ineffective.
- Graph-Based Detection: TopoGuard builds a semantic similarity graph from the set of retrieved documents. It analyzes the topological structure of this graph to detect anomalies indicative of malicious coordination between documents.
- Performance Metrics: On the HotpotQA dataset, TopoGuard-$\lambda_2$+Entity demonstrated a 32.6% attack recall rate at a 1% False Positive Rate (FPR), significantly outperforming LlamaGuard-2-8B which achieved only 1.5% recall under the same conditions.
- Efficiency and Robustness: The system operates at sub-millisecond latency, making it suitable for real-time applications. It maintains robustness against adaptive adversaries and performs well even with noisy input data.
Industry Insight
- Shift in Security Paradigm: Organizations deploying RAG systems must evaluate their security stacks beyond simple content moderation filters. Implementing structural or topological analysis of retrieved contexts is becoming essential to mitigate emerging multi-document attack vectors.
- Operational Viability: The sub-millisecond latency of graph-theory-based defenses proves that advanced security measures do not necessarily require heavy computational resources or large language models, allowing for scalable deployment in high-throughput environments.
- Future-Proofing Against Adaptive Threats: As attackers evolve to exploit the aggregation phase of RAG pipelines, adopting defense mechanisms that understand the relationships between retrieved documents will be crucial for maintaining system integrity and user trust.
Disclaimer: The above content is generated by AI and is for reference only.