Transformer-based segmentation of prosodic boundaries in Brazilian Portuguese
Introduction of SAMPA, a novel Whisper-based model designed specifically for automatic prosodic boundary segmentation in Brazilian Portuguese. Fine-tuning of the Whisper large-v3 architecture on the NURC-SP dataset enables simultaneous transcription and insertion of explicit prosodic boundary markers. The model demonstrates competitive performance, achieving an F1 score of 0.731 on held-out test splits and 0.796 on the out-of-distribution MuPe-Diversidades dataset. Analysis confirms that SAMPA e
Analysis
TL;DR
- Introduction of SAMPA, a novel Whisper-based model designed specifically for automatic prosodic boundary segmentation in Brazilian Portuguese.
- Fine-tuning of the Whisper large-v3 architecture on the NURC-SP dataset enables simultaneous transcription and insertion of explicit prosodic boundary markers.
- The model demonstrates competitive performance, achieving an F1 score of 0.731 on held-out test splits and 0.796 on the out-of-distribution MuPe-Diversidades dataset.
- Analysis confirms that SAMPA effectively leverages morphosyntactic, semantic, and acoustic-visual cues to detect prosodic boundaries accurately.
Why It Matters
This research addresses a significant gap in Natural Language Processing by moving beyond rule-based methods for Brazilian Portuguese, offering a robust deep learning solution for prosody. For developers working on voice assistants or speech-to-text systems targeting Lusophone markets, integrating SAMPA can significantly enhance the naturalness and interpretability of synthesized or transcribed speech.
Technical Details
- Model Architecture: Utilizes the Whisper large-v3 encoder-decoder transformer, fine-tuned to output text with inserted markers indicating terminal prosodic boundaries.
- Datasets: Training was conducted on the NURC-SP dataset, which contains manually segmented recordings. Evaluation included out-of-distribution testing on the MuPe-Diversidades dataset to assess generalization.
- Performance Metrics: The best-performing configuration achieved an F1 score of 0.731 on the standard held-out test split and improved to 0.796 on the diverse MuPe-Diversidades dataset.
- Analysis Methods: The study employed n-gram analysis and acoustic-visual techniques to verify that the model’s decisions align with linguistic and phonetic cues rather than relying solely on statistical artifacts.
Industry Insight
- Low-Resource Language Support: This approach demonstrates how leveraging powerful multilingual base models like Whisper can be effectively adapted for specific linguistic tasks in languages that lack extensive specialized tooling.
- Enhanced Speech Synthesis Quality: Integrating prosodic boundary detection into transcription pipelines allows for more nuanced text-to-speech generation, reducing robotic intonation in Brazilian Portuguese applications.
- Generalization Strategy: The strong performance on out-of-distribution data suggests that fine-tuning on high-quality, manually annotated datasets is crucial for building robust models that generalize across different speaking styles and demographics.
Disclaimer: The above content is generated by AI and is for reference only.