$E^3$-Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference

This paper presents a research contribution introducing a new system architecture, E^3-Agent, for managing AI inference workloads on edge devices. The analysis focuses on its core innovation: a hybrid control system designed to navigate the fundamental tension between reactivity and stability in non-stationary environments.

The Brittleness of Static Optimization

The central problem identified is the failure of traditional, offline-tuned resource managers in real-world edge deployments. Performance is not only unknown initially but also non-stationary, shifting due to:

• Semantic events: Changing user input patterns altering computational load.
• Device churn: Phones or tablets joining or leaving the network.
• Hidden drift: Gradual changes in background load or hardware state.

A system optimized for a fixed regime becomes "brittle," requiring expensive manual recalibration whenever the environment changes. The authors argue this makes static management fundamentally ill-suited for the volatile edge ecosystem.

The Dual-Path Control Architecture

E^3-Agent's solution is an executable and evolving agent with a clear separation of concerns between two control loops:

1. Fast-Path Router: A lightweight, learned model that makes millisecond-level dispatch decisions. Its sole job is to match incoming AIGC inference requests to the best available device-model pair based on current performance predictions. It operates continuously at high speed.
2. Slow-Path LLM Meta-Controller: A larger language model that acts as a higher-level "brain." It does not make routine routing decisions. Instead, it monitors system-wide performance and event streams, intervening only to mitigate regime shifts. It uses a defined tool interface to perform three key actions:
   • Risk Gating: Putting the system into a safe mode during detected crises.
   • Router Configuration: Updating the parameters of the fast-path router.
   • Rapid Performance Calibration: Re-estimating device-model performance metrics.

This structure allows the system to be both responsive (via the fast path) and adaptive to major shifts (via the slow path), with the LLM focusing its powerful reasoning on infrequent, high-impact strategic adjustments.

Online Learning and the Control Surface

A key insight is that the meta-controller's power is not in free-form reasoning but in its interaction with a small, explicit control surface. By limiting its actions to a few well-defined levers (gating, configuration, calibration), the system ensures the LLM's interventions are structured and executable. The agent learns from execution feedback—observing the results of its own actions—to continuously update its performance models. This online learning loop is what allows it to adapt to unknown and time-varying conditions without manual retraining.

Evaluation in Dynamic Simulations

The evaluation, conducted in a discrete-event simulator using real-world measurement priors, tests E^3-Agent against its core premise: managing volatility. The results are compelling:

• It reduces average latency by 65%-73% compared to the best static baseline, directly validating its adaptive advantage.
• It remains within 7%-10% of a theoretical online Oracle with perfect information, indicating near-optimal performance given the constraints.
• It effectively suppresses stutter rate during semantic degradation, a critical quality-of-service metric for user experience.

The testing across three distinct dynamic regimes—semantic dynamics, device churn, and hidden drift—demonstrates that the architecture is general-purpose, not tuned to one specific type of change. The primary independent judgment from this paper is that the separation of a fast, learned router from a slow, strategic LLM meta-controller is a powerful pattern for building adaptive autonomous systems. It effectively bounds the latency of critical decisions while reserving sophisticated reasoning for when it is truly needed to handle paradigm shifts, offering a scalable blueprint for managing other complex, non-stationary distributed systems.