A Social Media Analysis of Discourse on the Israel--Palestine Conflict on Telegram
Multi-method computational analysis of 87,617 messages across 16 Telegram channels (8 pro-Israel, 8 pro-Palestine) from May 2021 to June 2026 Three stance detection paradigms compared: keyword matching, zero-shot DeBERTa via NLI, and fine-tuned BERTweet model Fine-tuned BERTweet achieved best performance at 72.1% accuracy and 0.721 macro F1, outperforming label-free baselines by 8–11 points Both communities use identical death- and victim-related vocabulary but in opposite emotional registers Pr
Analysis
TL;DR
- Multi-method computational analysis of 87,617 messages across 16 Telegram channels (8 pro-Israel, 8 pro-Palestine) from May 2021 to June 2026
- Three stance detection paradigms compared: keyword matching, zero-shot DeBERTa via NLI, and fine-tuned BERTweet model
- Fine-tuned BERTweet achieved best performance at 72.1% accuracy and 0.721 macro F1, outperforming label-free baselines by 8–11 points
- Both communities use identical death- and victim-related vocabulary but in opposite emotional registers
- Pro-Israel channels adopt predominantly neutral, report-style framing while pro-Palestine channels are markedly more negative, reflecting acting-party vs. affected-party discourse positions
Why It Matters
This study demonstrates that off-the-shelf NLP models hit a hard performance ceiling when applied to in-domain political discourse, underscoring the necessity of fine-tuning for stance detection in specialized domains. It also provides a methodological blueprint for multi-method computational analysis of conflict-related social media, combining sentiment, stance, and framing in ways that reveal nuanced patterns invisible to any single approach.
Technical Details
- Dataset: 87,617 messages from 16 Telegram channels (8 pro-Israel, 8 pro-Palestine), spanning May 2021 to June 2026, covering multiple conflict escalations
- Evaluation: 736 manually annotated messages used for evaluation under 5-fold cross-validation
- Stance detection methods compared: (1) keyword matching, (2) zero-shot DeBERTa via natural language inference, (3) fine-tuned BERTweet model
- Best model: fine-tuned BERTweet at 72.1% accuracy and 0.721 macro F1; keyword matching and zero-shot DeBERTa stalled in the low-to-mid 60s
- Combined analytical pipeline: sentiment analysis + stance detection + framing analysis, with the key finding emerging only from the tripartite synthesis
Industry Insight
- Domain adaptation is critical for NLP tasks in political and conflict discourse; practitioners should invest in fine-tuning rather than relying on zero-shot or label-free baselines for high-stakes applications.
- Multi-method approaches that triangulate sentiment, stance, and framing yield richer insights than any single technique, suggesting that layered analytical pipelines should become standard practice in computational social science.
- The broadcast architecture of platforms like Telegram offers a uniquely direct record of deliberate political communication, making them valuable targets for systematic monitoring and early-warning applications.
Disclaimer: The above content is generated by AI and is for reference only.