Naver-News-KO: A Korean News Summarization Dataset for Open-Source Fine-Tuning of Summarization Models Naver-News-KO：用于开源摘要模型微调的韩国新闻摘要数据集

TL;DR

• Naver-News-KO is a Korean news summarization dataset containing 27,400 document-summary pairs collected from Naver News in July 2022.
• The dataset focuses on Economy (77%) and IT/Science (23%) categories with a mean compression ratio of 6.03x.
• It includes reproducible baselines using KoBART and Gemma-2B-ko with LoRA, alongside standard extractive references.
• A significant data leakage issue was identified: 16.8% near-duplicate title-Jaccard overlap exists between the test and train sets.
• The resource is hosted on Hugging Face Hub and has seen approximately 33,000 monthly downloads as of May 2026.

Why It Matters

This technical report provides essential transparency for researchers using Naver-News-KO, particularly regarding the significant data leakage between training and testing splits which could inflate performance metrics. By documenting collection protocols, statistics, and providing baseline models, it establishes a standardized reference point for evaluating open-source Korean summarization models.

Technical Details

• Dataset Composition: 27,400 pairs split into 22,194 (train), 2,466 (validation), and 2,740 (test) records.
• Domain Focus: Primarily Economy (77%) and IT/Science (23%) news articles.
• Baseline Performance: Lead-3 extractive baseline achieved ROUGE-1 of 55.1; KoBART fine-tuning reached ROUGE-1 56.6 and BERTScore-F1 81.5; Gemma-2B-ko with LoRA achieved ROUGE-1 55.3 and BERTScore-F1 78.3.
• Data Quality Issue: Measured 16.8% near-duplicate title-Jaccard overlap between test and train sets, indicating potential evaluation bias.
• Accessibility: Publicly available on Hugging Face Hub with release-time training scripts and clear licensing terms.

Industry Insight

Practitioners must account for the 16.8% data leakage when benchmarking models on this dataset to avoid overestimating generalization capabilities. The availability of standardized baselines like KoBART and Gemma-2B-ko allows for more consistent comparative analysis across different Korean NLP research efforts. Future work should prioritize datasets with stricter temporal or content-based separation to mitigate such overlap issues.

TL;DR

• 发布Naver-News-KO数据集，包含27,400对韩语新闻摘要数据，涵盖经济与IT/科学两类。
• 提供详细的统计信息，包括文档到摘要的平均压缩比为6.03倍及测试集与训练集间的标题重复率警告。
• 建立基准性能，包括Lead-3抽取式参考点及KoBART和Gemma-2B-ko微调模型的具体评估指标。
• 旨在为下游研究提供可引用的参考资源，而非提出新的基准测试标准。
• 数据集自2023年1月起在Hugging Face Hub公开，月下载量约33,000次，社区维护了多个基于此的模型变体。

为什么值得看

该报告为韩语自然语言处理社区提供了经过验证的高质量摘要数据集及其详细的使用指南和基线结果。对于需要构建或评估韩语新闻摘要系统的研究者而言，这份资料明确了数据分布特征和潜在偏差（如标题重复），有助于避免过拟合并提升模型泛化能力。

技术解析

• 数据集构成：Naver-News-KO包含27,400个(文档, 摘要)对，数据源自2022年7月Naver新闻，分为经济(77%)和IT/科学(23%)两个类别，划分为训练集(22,194)、验证集(2,466)和测试集(2,740)。
• 统计特征与风险：文档到摘要的平均字符压缩比为6.03倍；值得注意的是，测试集与训练集之间存在16.8%的近重复标题Jaccard重叠，使用者需警惕由此产生的数据泄露风险。
• 基准性能：提供了Lead-3抽取式方法的ROUGE-1得分55.1和ROUGE-L得分50.6作为参考；微调基线中，KoBART取得ROUGE-1 56.6和BERTScore-F1 81.5，Gemma-2B-ko (LoRA)取得ROUGE-1 55.3和BERTScore-F1 78.3。
• 开源与复现：附带发布时的训练脚本，明确许可范围和预期用途，并在Hugging Face Hub上托管，支持社区复现和进一步开发。

行业启示

• 韩语NLP生态中，高质量、标注规范的开源数据集仍是推动模型发展的关键基础设施，特别是针对特定领域（如新闻摘要）的数据资源。
• 数据集发布方应主动披露数据分布中的潜在偏差（如训练/测试集重叠），以帮助开发者更客观地评估模型真实性能，避免虚高的基准分数。
• 社区驱动的模型微调（如基于Gemma和KoBART）表明，轻量级开源模型结合特定领域数据微调，是解决垂直领域语言任务的高效路径。