CrisisKD: Five-Stage Knowledge Distillation for Aspect-Level Sentiment and Emotion Analysis in Crisis Discourse
CrisisKD introduces a five-stage teacher-student knowledge distillation framework for aspect-level sentiment and emotion analysis (ABSA) on unannotated social media data during crises A teacher LLM generates aspect-level labels and reasoning traces across five stages: aspect extraction, syntactic parsing, opinion extraction, sentiment classification, and emotion classification The authors release a dataset of 50,615 aspect-level labels along with open-source annotation and fine-tuning scripts Th
Analysis
TL;DR
- CrisisKD introduces a five-stage teacher-student knowledge distillation framework for aspect-level sentiment and emotion analysis (ABSA) on unannotated social media data during crises
- A teacher LLM generates aspect-level labels and reasoning traces across five stages: aspect extraction, syntactic parsing, opinion extraction, sentiment classification, and emotion classification
- The authors release a dataset of 50,615 aspect-level labels along with open-source annotation and fine-tuning scripts
- The Qwen2.5-7B student model achieves significant gains: +7.9 F1 on aspect extraction, +17.0 accuracy on emotion, and +6.5 accuracy on sentiment over the untuned baseline
- CrisisKD also improves the ICL baseline on the external ABEA benchmark by 2.8 F1 (ATE) and 3.8 F1 (joint ATE+AEC)
Why It Matters
This work addresses a critical gap in crisis informatics: understanding public sentiment and emotion at the aspect level during health-related emergencies, where knowing what specific topics drive public concern is essential for targeted interventions. The teacher-student distillation approach offers a practical pathway for deploying capable ABSA systems in resource-constrained settings, making advanced NLP accessible beyond large-scale API-dependent pipelines.
Technical Details
- Five-stage distillation pipeline: The framework sequentially distills capabilities across aspect extraction, syntactic parsing, opinion extraction, sentiment classification, and emotion classification, with each stage supervised by reasoning traces from a teacher LLM
- Teacher-student architecture: A large teacher LLM generates labeled data and chain-of-thought reasoning on unannotated social media posts, which then supervises the fine-tuning of a smaller Qwen2.5-7B student model
- Dataset: CrisisKD corpus contains 50,615 aspect-level labels derived from crisis-related social media data, with a manually annotated 500-tweet gold standard for evaluation
- Evaluation: Tested on both the internal gold set and the external ABEA benchmark, measuring ATE (Aspect Term Extraction), AEC (Aspect Emotion Classification), and joint ATE+AEC performance
- Open-source release: Annotation scripts, fine-tuning code, and the dataset are publicly available
Industry Insight
- The five-stage distillation design demonstrates that decomposing complex ABSA tasks into sequential sub-tasks can yield more effective knowledge transfer than end-to-end fine-tuning, a pattern likely to generalize to other multi-component NLP pipelines
- CrisisKD's focus on health-related crisis discourse highlights an emerging need for culturally and linguistically aware sentiment analysis tools, especially as crisis response organizations increasingly rely on real-time social media monitoring
- The substantial emotion accuracy gain (+17.0 points) suggests that distilling reasoning traces is particularly valuable for nuanced classification tasks like emotion detection, where surface-level patterns are insufficient—practitioners should prioritize trace-based supervision for affective computing applications
Disclaimer: The above content is generated by AI and is for reference only.