Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation
Crayotter introduces Group-Relative Preference Backpropagation (GRPB), a novel training framework that converts subjective, delayed feedback in long-horizon video editing into ordinal, task-local preference signals The method transforms same-task rankings into zero-sum advantages and redistributes them as bounded credit across semantic editing segments, avoiding the ambiguity of global scalar objectives A lagged allocator and guarded transmission mechanism prevent current judgments or unreliable
Analysis
TL;DR
- Crayotter introduces Group-Relative Preference Backpropagation (GRPB), a novel training framework that converts subjective, delayed feedback in long-horizon video editing into ordinal, task-local preference signals
- The method transforms same-task rankings into zero-sum advantages and redistributes them as bounded credit across semantic editing segments, avoiding the ambiguity of global scalar objectives
- A lagged allocator and guarded transmission mechanism prevent current judgments or unreliable estimates from directly shaping the same rollout group, ensuring stable credit assignment
- The authors construct a project-disjoint, horizon-stratified suite of realistic editing tasks for training and controlled evaluation, with code and materials publicly released
- The resulting 9B-parameter Crayotter model surpasses several proprietary systems on AgenticVBench, validated through matched baselines, credit interventions, external benchmarking, and blinded human evaluation
Why It Matters
Long-horizon agentic tasks in creative domains like video editing suffer from delayed, subjective feedback that resists traditional reward modeling; GRPB offers a principled, practical solution by leveraging within-task ordinal comparisons rather than absolute quality scores. This approach is directly transferable to other domains where outcomes are multi-step, subjective, and lack reliable scalar ground truth—such as code generation, design, and narrative creation. The public release of the 9B model and evaluation suite also provides the community with a concrete benchmark and baseline for future work in agentic video understanding and generation.
Technical Details
- Group-Relative Preference Backpropagation (GRPB): Instead of learning from absolute quality scores, GRPB ranks multiple editing trajectories produced for the same task and converts these rankings into zero-sum advantages. These advantages are then redistributed as bounded credit signals over semantic editing segments (e.g., scene transitions, object manipulations, color grading steps), enabling fine-grained credit assignment across long horizons.
- Lagged Allocator and Guarded Transmission: To prevent feedback contamination, the allocator uses a lagged mechanism so that judgments from one rollout group cannot influence the same group during training. Guarded transmission further ensures that unreliable or noisy preference estimates are filtered before being used for gradient updates, stabilizing learning.
- Task-Local Preference Reduction: The core insight is that fixing the request, source materials, and production constraints transforms a subjective global objective into a set of directly comparable alternatives, making preference learning tractable without requiring calibrated quality metrics.
- Evaluation Suite: A manually constructed, project-disjoint, horizon-stratified benchmark of realistic video editing tasks is introduced for both training and controlled evaluation, addressing the lack of standardized long-horizon video editing benchmarks.
- Model: A 9B-parameter model trained with GRPB, evaluated on AgenticVBench against proprietary systems, with additional validation through credit intervention ablations and blinded human evaluation.
Industry Insight
- The GRPB framework demonstrates that preference-based learning can scale to complex, multi-step creative tasks without relying on expensive or ambiguous reward models—making it a viable path for building agentic systems in domains where human feedback is the only reliable signal.
- The lagged allocator and guarded transmission mechanisms address a critical instability in long-horizon reinforcement learning; these techniques should be adopted broadly in any agent system that learns from delayed, noisy, or subjective outcomes.
- The public release of a horizon-stratified benchmark and a 9B open model positions Crayotter as a potential new standard for agentic video editing, likely accelerating research and commercial development in AI-assisted creative production tools.
Disclaimer: The above content is generated by AI and is for reference only.