Reference Publications

Nist Hosts Large Vocabulary Conversational Speech Recognition Workshop - National Institute of Standards and Technology - Brief Article

Journal of Research of the National Institute of Standards and Technology, July-August, 2001

NIST hosted the 2001 Workshop on Large Vocabulary Conversational Speech Recognition, in May 2001, in Linthicum, MD. Workshop participants reviewed the results of this year's evaluation conducted by the division in cooperation with DoD sponsors. Participating sites developed systems to automatically generate word-level transcriptions of recorded telephone conversations; these were scored against official reference transcriptions.

Eight research groups from the United States and Europe participated in parts of the evaluation and discussed their work at the workshop.

This year's test set was the largest ever in size, involving 60 5-minute telephone conversations from three different corpora. One of these was the new Switchboard-Cellular Corpus, marking the first time evaluation has been done on cellular telephone data. NIST researchers presented an analysis of the evaluation results and compared these results to those of previous evaluations.

Overall performance results of the best systems, measured by word error rate, were the best outcome ever achieved on the Switchboard-2 Corpus. As in recent years, the system achieving the lowest word error rates on each corpus this year was that developed by Cambridge University. As in the 2000 evaluation, a separate non-competitive evaluation was conducted on a subset of the test data, in which sites were asked to generate transcriptions at both the word and phonetic levels. Researchers at the International Computer Science Institute in Berkeley, CA analyzed the results, and the findings were presented at the workshop. The website i s http://www.nist.gov/speech/tests/ctr/h5_2001/index.htm.>