On the Use of LLMs for Specialised Terminology: A Good Alternative to Corpora? 关于在专业术语中使用大型语言模型：语料库的良好替代方案？

TL;DR

• The study evaluates four proprietary LLMs (GPT-4o, GPT-5.2, Claude Sonnet 4.5, DeepSeek) for specialized terminology translation from English to French in two domains: Earth, Environmental and Planetary Sciences (EEPS) and Natural Language Processing (NLP).
• Two prompting strategies were tested: terminology mode and translation mode, with 80 terms per domain.
• Claude Sonnet 4.5 achieved the best results in the most favorable configuration, while DeepSeek demonstrated greater stability across tests.
• Confidence estimates provided by models were only a partial indicator of terminological accuracy.
• LLMs can assist specialized translators but cannot currently replace specialized corpora.

Why It Matters

This research is highly relevant to AI practitioners and researchers working on natural language processing, particularly those focused on machine translation and terminology management. It provides empirical evidence on the capabilities and limitations of current LLMs in specialized domains, which can inform both tool development and practical applications in professional translation workflows. The findings also highlight the importance of continued improvement in model reliability and confidence estimation mechanisms.

Technical Details

• Four proprietary LLMs were evaluated: GPT-4o, GPT-5.2, Claude Sonnet 4.5, and DeepSeek.
• Testing was conducted in two specialized domains: Earth, Environmental and Planetary Sciences (EEPS) and Natural Language Processing (NLP).
• Each domain included 80 test terms, totaling 160 terms across both fields.
• Two prompting strategies were compared: terminology-focused prompts and translation-context prompts.
• Performance metrics likely included accuracy of term equivalents, consistency across domains, and correlation between model confidence scores and actual correctness.
• The study emphasizes that while some models perform well under optimal conditions, none consistently match the reliability of curated specialized corpora.

Industry Insight

Specialized translation professionals should consider integrating LLMs as supplementary tools rather than replacements for established terminological resources like domain-specific corpora or glossaries. Model selection matters—Claude Sonnet 4.5 shows promise in high-performance scenarios, while DeepSeek offers more predictable behavior, making it suitable for consistent output needs. Future developments should focus on improving confidence calibration and reducing variability across domains to make LLMs more trustworthy for critical translation tasks.

TL;DR

• 研究评估了四种主流大语言模型（GPT-4o、GPT-5.2、Claude Sonnet 4.5、DeepSeek）在专业术语翻译中的表现，对比其与传统语料库的替代潜力。
• 实验覆盖地球环境与行星科学（EEPS）和自然语言处理（NLP）两个领域，每领域80个术语，采用术语模式与翻译模式两种提示策略。
• Claude Sonnet 4.5在最优配置下表现最佳，DeepSeek则展现出更高的稳定性；置信度评分仅能部分反映术语准确性。
• LLMs可作为专业翻译的有效辅助工具，但目前尚无法完全取代专业语料库。
• 该研究为未来探索LLM在实际工作与教育场景中的实用性奠定基础。

为什么值得看

本文针对专业翻译领域中术语资源获取难、成本高痛点，系统评估了当前最先进LLMs在跨语言术语等价生成上的能力，为从业者提供实证依据以判断是否可将LLM纳入工作流。同时揭示了模型间差异及提示工程的重要性，对AI开发者优化垂直领域应用具有指导意义。

技术解析

• 研究选取四个商业闭源模型：GPT-4o、GPT-5.2、Claude Sonnet 4.5、DeepSeek，在EEPS与NLP两个高专业度领域进行基准测试。
• 每个领域构建80个英法术语对样本，分别测试“术语模式”（直接询问术语对应词）与“翻译模式”（将术语置于句子中翻译）两种prompt策略。
• 评估指标包括术语匹配准确率、模型输出一致性、以及内部置信度分数与实际正确率的相关性分析。
• 结果显示不同模型在特定领域存在显著性能分化，如Claude在EEPS领域优势明显，而DeepSeek在跨域稳定性上更优。
• 置信度得分虽有一定预测力，但并非可靠代理——高置信度结果仍可能错误，表明需结合人工校验或外部验证机制。

行业启示

• 专业翻译机构可考虑引入LLM作为初步术语筛选工具，尤其适用于高频重复性任务，但必须保留专家审核环节以确保质量。
• AI厂商应针对垂直领域开发专用微调版本或增强检索增强生成（RAG）架构，以提升术语一致性与上下文理解能力。
• 教育与培训场景中，LLM可用于辅助学生快速建立术语认知框架，但不能替代系统性术语库学习与实践训练。