DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods

Background

DreamerNLplus was developed for the CLPsych 2026 shared task, focusing on modeling mental health dynamics from social media timelines. The system aims to address three main tasks: psychological state modeling, temporal change detection, and sequence-level summarization.

Key Points

• Task 1 (Psychological State Modeling): DreamerNLplus combines LLM-based data augmentation with DeBERTa classification and Random Forest regression for structured state prediction.
• Task 2 (Temporal Change Detection): It uses few-shot prompting with a locally deployed Llama 3.1 model to detect Switch and Escalation events using short-term temporal context.
• Task 3: The system explores both deterministic rule-based summarization pipelines and few-shot LLM-based approaches, ranking second overall. Its RAG method excels in Task 3.2, achieving first place for Improvement and third place for Deterioration.

Significance

The results of DreamerNLplus highlight the challenges in modeling mental health dynamics from social media data:

• Performance Mismatch: There is a noted discrepancy between classification and regression performance.
• Temporal Transitions: Modeling temporal transitions remains complex due to their complexity and variability.
• Evaluation Metrics Disagreement: Semantic and similarity-based evaluation metrics often disagree, complicating the assessment of model performance.

These findings underscore the need for more unified evaluation frameworks in future research. The system's code and prompts are publicly available at https://github.com/4dpicture/CLPsych2026, facilitating further advancements in this field.