MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination

Background

In multi-agent deployments, coordinating agents effectively requires accurate trust in their responses. The standard approach relies on self-reported confidence levels from the agents, but recent evidence indicates that these confidences are systematically mis-calibrated and often inversely related to actual performance on challenging tasks. Design-time calibration methods such as temperature scaling or histogram binning fail under distribution shifts, further complicating reliable multi-agent coordination.

Key Points

MARGIN addresses these limitations by continuously learning per-agent, per-confidence-band calibration factors directly from the task stream in real-time. This method uses symmetric exponentially weighted moving averages with Bayesian shrinkage blending and requires minimal hyperparameters. The study evaluates MARGIN across 19 foundation models, eight benchmarks, and over 50,000 observations.

• Dynamic Calibration: MARGIN dynamically adjusts confidence scores based on the current task stream.
• No Model Access Required: Unlike design-time methods, MARGIN operates without access to the underlying model's training data or parameters.
• Symmetric Exponentially Weighted Moving Averages (EWMAs): These are used to track and adjust confidence scores over time.
• Bayesian Shrinkage Blending: This technique helps in combining different EWMAs efficiently, improving convergence speed and accuracy.

Significance

The performance of MARGIN is highly significant for multi-agent systems. It achieves a 3-6x reduction in calibration error compared to the best design-time baseline under distribution shifts. Importantly, it improves pairwise resolution from as low as 45-56% using raw verbalized confidence to 70-89%. In some benchmarks, MARGIN even surpasses the performance of an always-best-model oracle.

Key Insights:

• Improved Pairwise Resolution: Raw confidence scores perform poorly in high-stakes tasks (45-56%), but MARGIN corrects this, significantly improving resolution to 70-89%.
• Optimal Convergence and Tracking Speed: Six formal propositions characterize the convergence properties of MARGIN, highlighting its robustness under various conditions.

These findings underscore the necessity for runtime calibration methods in dynamic multi-agent environments, particularly where task distributions may change over time.