Curvature Cryptanalysis of Smooth Transformer Feed-Forward Networks
Smooth two-layer FFNs in transformers (GELU/SiLU) leak internal parameter geometry through second-order curvature (Hessian) information accessible via black-box chosen-input queries Projected input Hessians decompose into mixtures of symmetric rank-one factors from hidden input weights, enabling structural recovery via partially symmetric decomposition Only 16 projected Hessians (8,193 black-box queries) recover FFN directions with >0.94 average cosine alignment on CIFAR-10 vision transformers R
Analysis
TL;DR
- Smooth two-layer FFNs in transformers (GELU/SiLU) leak internal parameter geometry through second-order curvature (Hessian) information accessible via black-box chosen-input queries
- Projected input Hessians decompose into mixtures of symmetric rank-one factors from hidden input weights, enabling structural recovery via partially symmetric decomposition
- Only 16 projected Hessians (8,193 black-box queries) recover FFN directions with >0.94 average cosine alignment on CIFAR-10 vision transformers
- Recovered structure enables functional extraction: fitting remaining parameters yields >93% top-1 agreement and accuracy within 0.62–0.90% of targets
- Output rounding and Gaussian noise degrade recovery, but adaptive finite-difference step selection restores alignment to ~0.94–0.96
Why It Matters
This work reveals a previously unidentified side-channel in transformer architectures: second-order curvature leakage that bypasses traditional black-box protections. For AI practitioners deploying proprietary models, it demonstrates that behavioral fidelity alone cannot prevent structural model extraction, necessitating new defensive considerations around Hessian-level information disclosure.
Technical Details
- Attack model: Chosen-input raw-output oracle at the FFN branch with no access to parameters, gradients, or internal activations; exploits second-order leakage through projected input Hessians
- Mathematical framework: Hessian collection formalized as a partially symmetric decomposition, establishing conditions for local identifiability and stability; vector-output stencil reuse reduces query cost by 16×
- Experimental setup: Evaluated on independently trained CIFAR-10 vision transformers with GELU and SiLU activations across all transformer blocks
- Recovery performance: 95.1% of GELU directions and 91.9% of SiLU directions exceed 0.90 cosine alignment; robust across repeated runs and independent training seeds
- Functional extraction: Fixing recovered directions and fitting remaining FFN parameters produces high-fidelity substitutes with >93% top-1 agreement; noise resilience achieved through adaptive finite-difference step adjustment
Industry Insight
- Model protection strategies relying solely on API-level output access are insufficient; defenses must account for second-order information leakage through finite-difference Hessian estimation
- The 16-query efficiency (via stencil reuse) makes this attack practically feasible at scale, raising urgency for curvature-aware obfuscation techniques in production deployments
- As vision transformers become more widely deployed as black-box services, this work establishes a template for similar cryptanalytic approaches targeting other smooth activation regimes
Disclaimer: The above content is generated by AI and is for reference only.