Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs 深思熟虑的进化：基于大语言模型的代理推理以实现样本高效符号回归

The entire field of AI-driven scientific discovery is obsessed with a seductive but deeply flawed premise: that you can force a language model to be both the mad scientist and the meticulous lab manager simultaneously. This new paper, "Deliberate Evolution," finally puts a name to the dysfunction and offers a potent corrective. It’s not just an incremental improvement in symbolic regression; it’s a blueprint for how we need to restructure AI agents to tackle problems of genuine complexity.

The core critique is devastatingly simple and correct. Look at most LLM-based evolutionary methods today. You hand the model a starting expression and a score, like a Mean Squared Error. The entire burden of progress falls on that single scalar number. The LLM must now simultaneously perform a suite of distinct cognitive tasks: imagine a novel candidate expression (proposal), deduce why the previous one was flawed based on one number (diagnosis), and figure out how to navigate a vast mathematical search space toward a better solution (guidance). It’s like asking a chef to invent a new recipe, taste the soup once, be told only "too salty," and then be expected to perfectly adjust the seasoning, cooking time, and core ingredients all at once. The signal-to-noise ratio is atrocious. This "conflation" of roles isn't just inefficient; it's a fundamental architectural bug that caps performance.

Deliberate Evolution (DE) proposes a surgical fix: decouple the thinker from the tinkerer. It separates the "proposal" phase—the LLM generating a mathematical candidate—from the "control" phase, which handles search direction, structural diagnosis, and long-term learning. This is more than a technical tweak. It’s a philosophical shift from viewing the LLM as an all-or-nothing oracle to treating it as a specialized component within a larger, deliberate system. The framework introduces adaptive operators to guide the search (think: smart mutators that know whether to tweak a coefficient or perform a radical simplification), analytical tools to diagnose structural flaws (like symbolic derivatives or dimension checks that provide concrete, non-scalar feedback), and a reflective memory to learn from entire trajectories, not just isolated scores.

The results are telling. Hitting top performance on the LLM-SRBench with only 40% of the sample budget isn't just "better." It’s a indictment of the brute-force, sample-hungry approach that currently dominates. It suggests that for every dollar spent on compute in traditional methods, sixty cents is wasted on the model stumbling around in the dark, trying to juggle its multifaceted role. DE’s efficiency points toward a more sustainable and, frankly, more intelligent path for AI in science. It moves us away from the fantasy of a single, monolithic "science model" and toward a reality of curated, collaborative agent teams, each with a clear job description.

What’s truly exciting is the implication beyond symbolic regression. This is a case study in building trustworthy AI systems for any domain requiring causal reasoning or discovery. In physics, materials science, or drug design, you can't just reward a model with a final "success/failure" score and expect it to learn the nuanced, underlying principles. You need systems that can say, "My proposal failed not just because the error was high, but because it violates this conservation law," or "This molecular structure is unstable because of this specific bond angle." DE’s architecture—providing structured, diagnostic feedback to a generative component—is a prototype for this kind of robust, transparent reasoning.

Critics might argue this adds complexity, and they’re right. Integrating symbolic analyzers, adaptive operators, and reflective memory is more engineering than a single elegant prompt. But this is the necessary growing pain of moving from brittle, demo-worthy systems to robust, reliable tools. The "simplicity" of the old approach was a false economy that traded capability for ease of implementation.

Ultimately, Deliberate Evolution should be read as a warning and an inspiration. It warns that stuffing more reasoning into a single LLM call is hitting a wall of diminishing returns. It inspires by showing that when you decompose the problem and give each agent component a clear, focused task with rich feedback, the whole becomes greater than the sum of its parts. The future of AI in science isn't about finding a bigger, smarter monolith. It's about building better teams of specialized, well-coordinated thinkers—and DE just drafted an excellent playbook.

让LLM搞符号回归，就像让一个天才少年同时扮演发明家和实验室主管——结果往往是两者都做不好。最近一份arXiv预印本直接戳破了这个尴尬：当前基于大语言模型（LLM）的符号回归方法，样本效率低得惊人。问题出在哪？研究者一针见血：LLM被强行塞进了一个既要“提出候选数学表达式”，又要“根据反馈指导自己下一步搜索”的双重角色里。它拿到的全部指导，可能就是一个MSE（均方误差）分数。这好比你只告诉一位厨师“这道菜不行”，却不告诉他盐放多了还是火候过了，更别提历史上所有失败的食谱了。指望LLM从一个孤立的分数里，同时“悟”出如何创新、诊断错误和复用经验，简直是天方夜谭。这种设计，本质上是把LLM当成了一个“会写代码的计算器”，一个需要自己顿悟整个演化法则的黑盒苦力。

这篇名为《Deliberate Evolution》的论文提出的解决方案，思路清晰得令人舒适：把活儿分开干。它搞了一个叫DE的智能体框架，核心就是解耦——让LLM专心做它擅长的“符号生成”，即提出新的数学表达式猜想。而搜索方向的引导、表达式结构问题的诊断、以及历史探索经验的记忆和复用，则交给另外一套专门设计的机制：自适应算子、分析工具和反思记忆。这就像把那位手忙脚乱的少年，拆分成了一个纯粹的“灵感源泉”和一个老练的“实验室主管”。主管手里有导航图（自适应算子）、有诊断仪器（分析工具）、还有实验记录本（反思记忆），他只需要告诉天才少年：“根据之前的失败记录和当前趋势，我们下一步试试往多项式方向调整，另外上次那个结构在x=0处有奇点，这次避开它。”LLM的工作变得纯粹：听指挥，出方案。

这框架在LLM-SRBench上的表现确实亮眼：性能全面超越其他LLM符号回归方法，而且只用了40%的标准样本预算。这省下的60%预算，在科研里意味着真金白银的计算资源和时间。论文的功劳在于，它点破了一个行业幻觉：LLM并非无所不能的“通用问题解决器”。在需要严密逻辑和结构化搜索的科学发现任务上，把LLM直接丢进强化学习或进化算法的循环里当主角，是一种懒惰的架构设计。DE的价值在于，它重新厘清了人机（或者说，LLM与传统算法）协作的边界：LLM是强大的“模式提议者”和“概念联想者”，但搜索策略、经验归纳这些“元认知”任务，或许更适合用确定性更强、可解释性更高的传统方法模块来承担。

然而，赞扬之余，也不免有些更尖锐的疑问。DE框架本身会不会变得过于复杂？它引入的“自适应算子”和“反思记忆”，其设计和调参本身是否又成了一个新的研究难题？我们是否只是用一个更复杂的系统，去解决一个由过于简单的系统引发的问题？这种高度模块化的“智能体”架构，在解决符号回归问题上展现了优势，但它是否揭示了一个更深层的现实：当前LLM在严格逻辑推理和结构化探索方面的能力，存在根本性的天花板？我们不得不承认，LLM在“创造”候选表达式时，其发散思维能力或许很强，但在“评估”和“规划”如何系统地逼近真理时，其内在的随机性和黑盒特性，可能反而是个弱点。DE的成功，某种程度上是对LLM自身“智能”局限性的一种温和承认——我们需要给它套上一个更传统的“算法缰绳”，才能更好地驾驭它。

从更广的视角看，DE这类工作预示着AI4Science（AI驱动科学）领域一个必要的范式转变：从追求“端到端”的、看似全能的AI黑盒，转向构建“人机协同”或“多方法混合”的可解释系统。未来的科学发现工具，很可能就是DE的扩大版：LLM负责提出大胆、跨领域的假设和公式草案；专业的数学分析工具进行严格验证和约束；进化算法或贝叶斯优化负责在庞大的表达式空间中进行高效搜索；而一个不断增长的知识图谱或案例库，则提供领域知识和历史经验。LLM不再是舞台中央的独角戏演员，而是才华横溢的“客座明星”，需要与其他专业“演员”和精良的“导演系统”配合。

这篇论文最辛辣的潜台词或许是：别再无脑地把LLM塞进每个问题里当“主角”了。有时候，让它当个听话且强大的“副驾驶”，整个系统反而能跑得更快、更稳。在通往真正智能的科研之路上，知道什么时候该用LLM，和知道什么时候不该用，可能同样重要。DE提供了一个不错的实践范本，但关于LLM在严肃科学发现中的角色和边界，争论才刚刚开始。