Inverting Self-Triggered Control: Adversarial Reinforcement Learning for Sparse Denial-of-Service Attacks
An adversarial RL agent is trained to learn the sparsest jamming/DoS schedule that destabilizes self-triggered reinforcement learning control (RL-STC) systems, inverting the defender's objective of finding the sparsest stabilizing schedule. The authors prove a plant-property lower bound on the minimum jam count required for an immediate hold-last MAC adversary to force a crash against STC controllers satisfying a Lyapunov contract. The learned adversary achieves 100% crash rate across all defend
Analysis
TL;DR
- An adversarial RL agent is trained to learn the sparsest jamming/DoS schedule that destabilizes self-triggered reinforcement learning control (RL-STC) systems, inverting the defender's objective of finding the sparsest stabilizing schedule.
- The authors prove a plant-property lower bound on the minimum jam count required for an immediate hold-last MAC adversary to force a crash against STC controllers satisfying a Lyapunov contract.
- The learned adversary achieves 100% crash rate across all defenders (LQR and three RL-STC variants) on all three plants (Pendulum, CartPole, Quadrotor2D), outperforming greedy and periodic baselines by up to 2.8x on jam-time-per-failure.
- Robustness ablations confirm the adversary maintains 100% failure rates even under significant Gaussian observation noise and position-only observation constraints.
- This extends DoS-scheduling count-budget analysis from periodic and linear-time-invariant systems to the more general class of self-triggered controllers.
Why It Matters
This work bridges adversarial machine learning and cyber-physical system security, demonstrating that RL-based adversaries can exploit the very sparsity mechanisms designed to make self-triggered controllers efficient. For practitioners building safe autonomous systems, it highlights that Lyapunov-based stability guarantees can be systematically inverted by an adaptive adversary, necessitating new robustness-aware design paradigms for control systems operating under potential jamming threats.
Technical Details
- Adversarial formulation: The paper inverts RL-STC by training an adversarial RL agent with a Lyapunov-increase admissibility predicate that mirrors the defender's Lyapunov-decreasing safety certificate, learning sparse DoS schedules that destabilize the closed-loop system.
- Theoretical contribution: A plant-property lower bound is proven on the minimum jam count required for an immediate hold-last medium-access-control adversary to force a crash, recovering a certificate-level analog of consecutive-grouping optimality from prior count-budget DoS scheduling as a corollary.
- Experimental setup: Four fixed defenders per plant (one LQR and three RL-STC variants) evaluated on Pendulum, CartPole, and Quadrotor2D benchmarks, with comparisons against greedy and periodic jamming baselines.
- Performance results: The learned adversary achieves 100% crash rate universally, while greedy baselines miss Quadrotor2D LQR on 42% of episodes and periodic baselines miss Pendulum LQR on 97% of episodes, with up to 2.8x improvement in jam-time-per-failure.
- Robustness analysis: Ablation studies show the adversary remains effective under Gaussian observation noise exceeding initial-state magnitude and position-only observation scenarios, maintaining both 100% failure rates and strict superiority over baselines.
Industry Insight
- Security-conscious control system designers should treat Lyapunov-based safety certificates as potentially invertible under adaptive adversarial conditions, and incorporate adversarial training or robust verification into the control synthesis pipeline rather than relying solely on nominal stability guarantees.
- The 2.8x efficiency advantage of learned adversarial schedules over periodic/greedy baselines suggests that resource-constrained attackers can achieve dramatically higher impact with fewer jamming actions, making sparse DoS attacks a serious threat to energy-efficient cyber-physical systems like autonomous vehicles and drone networks.
- The robustness of the adversary under severe observation noise and partial state information indicates that even degraded sensing conditions do not neutralize learned jamming strategies, implying that defense mechanisms must account for adversarial adaptability rather than assuming information asymmetry will protect the system.
Disclaimer: The above content is generated by AI and is for reference only.