Black-Box Inference of LLM Architectural Properties with Restrictive API Access
Researchers introduce NightVision, a black-box attack capable of inferring LLM architectural properties like hidden dimension, depth, and parameter count using only restrictive API access. The method utilizes a novel "common set prompting" technique combined with spectral analysis of log probabilities to estimate hidden dimensions with high accuracy (within 23% average relative error). Depth and parameter counts are estimated by correlating end-to-end Time To First Token (TTFT) measurements with
Analysis
TL;DR
- Researchers introduce NightVision, a black-box attack capable of inferring LLM architectural properties like hidden dimension, depth, and parameter count using only restrictive API access.
- The method utilizes a novel "common set prompting" technique combined with spectral analysis of log probabilities to estimate hidden dimensions with high accuracy (within 23% average relative error).
- Depth and parameter counts are estimated by correlating end-to-end Time To First Token (TTFT) measurements with the inferred hidden dimension.
- Empirical evaluation on 32 open-source models demonstrates that current commercial API restrictions are insufficient to fully obfuscate model architecture.
Why It Matters
This research highlights a significant security and intellectual property vulnerability in the AI industry, demonstrating that proprietary model details can be reverse-engineered even when providers limit API exposure to single-token logits. For AI practitioners and enterprise clients, this implies that competitive advantages derived from unique model architectures may be compromised through simple inference attacks, necessitating more robust API design or detection mechanisms.
Technical Details
- NightVision Algorithm: A black-box inference technique that exploits restrictive API access, requiring only single logit outputs per token without logit biasing capabilities.
- Common Set Prompting: A novel prompting strategy where multiple prompts are designed to expose log probabilities for the same set of output tokens, enabling the extraction of structural information.
- Spectral Analysis: Used to analyze the extracted log probability distributions to accurately infer the hidden dimension of the feed-forward network.
- TTFT Correlation: Estimates model depth and total parameter count by measuring end-to-end Time To First Token (TTFT) and correlating it with the previously estimated hidden dimension.
- Performance Metrics: Achieved 9% average relative error on Mixture-of-Experts (MoE) models for hidden dimension recovery and within 53% error for depth and parameter counts on models exceeding three billion parameters.
Industry Insight
- API Security Review: LLM providers must reassess their API designs, as exposing timing data (TTFT) alongside even minimal probabilistic data can leak significant architectural secrets.
- Obfuscation Strategies: Implementing noise injection, rate limiting variations, or artificial latency padding could mitigate the effectiveness of timing-based inference attacks.
- Competitive Defense: Organizations relying on proprietary model structures should consider that "restrictive" APIs are not equivalent to "secure" APIs, and additional layers of protection against side-channel attacks are required.
Disclaimer: The above content is generated by AI and is for reference only.