Volume 12, Number 20, November 2022

Streaming Punctuation for Long-Form Dictation with Transformers

  Authors

Piyush Behre, Sharman Tan, Padma Varadharajan and Shuangyu Chang, Microsoft Corporation, USA

  Abstract

While speech recognition Word Error Rate (WER) has reached human parity for English, longform dictation scenarios still suffer from segmentation and punctuation problems resulting from irregular pausing patterns or slow speakers. Transformer sequence tagging models are effective at capturing long bi-directional context, which is crucial for automatic punctuation. Automatic Speech Recognition (ASR) production systems, however, are constrained by real-time requirements, making it hard to incorporate the right context when making punctuation decisions. In this paper, we propose a streaming approach for punctuation or re-punctuation of ASR output using dynamic decoding windows and measure its impact on punctuation and segmentation accuracy across scenarios. The new system tackles over-segmentation issues, improving segmentation F0.5-score by 13.9%. Streaming punctuation achieves an average BLEU-score improvement of 0.66 for the downstream task of Machine Translation (MT).

  Keywords

Automatic punctuation, automatic speech recognition, re-punctuation, speech segmentation.