Assessing the prosody of non-native speakers of English: Measures and feature sets

Eduardo Coutinho; Florian Hönig; Yue Zhang; Simone Hantke; Anton Batliner; Elmar Nöth; Björn Schuller

Assessing the prosody of non-native speakers of English: Measures and feature sets

Eduardo Coutinho
, Florian Hönig
, Yue Zhang
, Simone Hantke
, Anton Batliner
, Elmar Nöth
, Björn Schuller

Imperial College London
University of Liverpool
Friedrich Alexander Universität Erlangen-Nürnberg
Technical University of Munich
Universität Passau

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

20 Scopus citations

Abstract

In this paper, we describe a new database with audio recordings of non-native (L2) speakers of English, and the perceptual evaluation experiment conducted with native English speakers for assessing the prosody of each recording. These annotations are then used to compute the gold standard using different methods, and a series of regression experiments is conducted to evaluate their impact on the performance of a regression model predicting the degree of naturalness of L2 speech. Further, we compare the relevance of different feature groups modelling prosody in general (without speech tempo), speech rate and pauses modelling speech tempo (fluency), voice quality, and a variety of spectral features. We also discuss the impact of various fusion strategies on performance. Overall, our results demonstrate that the prosody of non-native speakers of English as L2 can be reliably assessed using supra-segmental audio features; prosodic features seem to be the most important ones.

Original language	English
Title of host publication	Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016
Editors	Nicoletta Calzolari, Khalid Choukri, Helene Mazo, Asuncion Moreno, Thierry Declerck, Sara Goggi, Marko Grobelnik, Jan Odijk, Stelios Piperidis, Bente Maegaard, Joseph Mariani
Publisher	European Language Resources Association (ELRA)
Pages	1328-1332
Number of pages	5
ISBN (Electronic)	9782951740891
State	Published - 2016
Externally published	Yes
Event	10th International Conference on Language Resources and Evaluation, LREC 2016 - Portoroz, Slovenia Duration: 23 May 2016 → 28 May 2016

Publication series

Name	Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

Conference

Conference	10th International Conference on Language Resources and Evaluation, LREC 2016
Country/Territory	Slovenia
City	Portoroz
Period	23/05/16 → 28/05/16

Keywords

Feature evaluation
Index terms: non-native speech
Prosody

Cite this

Coutinho, E., Hönig, F., Zhang, Y., Hantke, S., Batliner, A., Nöth, E., & Schuller, B. (2016). Assessing the prosody of non-native speakers of English: Measures and feature sets. In N. Calzolari, K. Choukri, H. Mazo, A. Moreno, T. Declerck, S. Goggi, M. Grobelnik, J. Odijk, S. Piperidis, B. Maegaard, & J. Mariani (Eds.), Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016 (pp. 1328-1332). (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016). European Language Resources Association (ELRA).

Coutinho, Eduardo ; Hönig, Florian ; Zhang, Yue et al. / Assessing the prosody of non-native speakers of English : Measures and feature sets. Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. editor / Nicoletta Calzolari ; Khalid Choukri ; Helene Mazo ; Asuncion Moreno ; Thierry Declerck ; Sara Goggi ; Marko Grobelnik ; Jan Odijk ; Stelios Piperidis ; Bente Maegaard ; Joseph Mariani. European Language Resources Association (ELRA), 2016. pp. 1328-1332 (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016).

@inproceedings{2db21b05a0c5462d93ee29184b04d53a,

title = "Assessing the prosody of non-native speakers of English: Measures and feature sets",

abstract = "In this paper, we describe a new database with audio recordings of non-native (L2) speakers of English, and the perceptual evaluation experiment conducted with native English speakers for assessing the prosody of each recording. These annotations are then used to compute the gold standard using different methods, and a series of regression experiments is conducted to evaluate their impact on the performance of a regression model predicting the degree of naturalness of L2 speech. Further, we compare the relevance of different feature groups modelling prosody in general (without speech tempo), speech rate and pauses modelling speech tempo (fluency), voice quality, and a variety of spectral features. We also discuss the impact of various fusion strategies on performance. Overall, our results demonstrate that the prosody of non-native speakers of English as L2 can be reliably assessed using supra-segmental audio features; prosodic features seem to be the most important ones.",

keywords = "Feature evaluation, Index terms: non-native speech, Prosody",

author = "Eduardo Coutinho and Florian H{\"o}nig and Yue Zhang and Simone Hantke and Anton Batliner and Elmar N{\"o}th and Bj{\"o}rn Schuller",

year = "2016",

language = "English",

series = "Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016",

publisher = "European Language Resources Association (ELRA)",

pages = "1328--1332",

editor = "Nicoletta Calzolari and Khalid Choukri and Helene Mazo and Asuncion Moreno and Thierry Declerck and Sara Goggi and Marko Grobelnik and Jan Odijk and Stelios Piperidis and Bente Maegaard and Joseph Mariani",

booktitle = "Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016",

note = "10th International Conference on Language Resources and Evaluation, LREC 2016 ; Conference date: 23-05-2016 Through 28-05-2016",

}

Coutinho, E, Hönig, F, Zhang, Y, Hantke, S, Batliner, A, Nöth, E & Schuller, B 2016, Assessing the prosody of non-native speakers of English: Measures and feature sets. in N Calzolari, K Choukri, H Mazo, A Moreno, T Declerck, S Goggi, M Grobelnik, J Odijk, S Piperidis, B Maegaard & J Mariani (eds), Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016, European Language Resources Association (ELRA), pp. 1328-1332, 10th International Conference on Language Resources and Evaluation, LREC 2016, Portoroz, Slovenia, 23/05/16.

Assessing the prosody of non-native speakers of English: Measures and feature sets. / Coutinho, Eduardo; Hönig, Florian; Zhang, Yue et al.
Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. ed. / Nicoletta Calzolari; Khalid Choukri; Helene Mazo; Asuncion Moreno; Thierry Declerck; Sara Goggi; Marko Grobelnik; Jan Odijk; Stelios Piperidis; Bente Maegaard; Joseph Mariani. European Language Resources Association (ELRA), 2016. p. 1328-1332 (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Assessing the prosody of non-native speakers of English

T2 - 10th International Conference on Language Resources and Evaluation, LREC 2016

AU - Coutinho, Eduardo

AU - Hönig, Florian

AU - Zhang, Yue

AU - Hantke, Simone

AU - Batliner, Anton

AU - Nöth, Elmar

AU - Schuller, Björn

PY - 2016

Y1 - 2016

N2 - In this paper, we describe a new database with audio recordings of non-native (L2) speakers of English, and the perceptual evaluation experiment conducted with native English speakers for assessing the prosody of each recording. These annotations are then used to compute the gold standard using different methods, and a series of regression experiments is conducted to evaluate their impact on the performance of a regression model predicting the degree of naturalness of L2 speech. Further, we compare the relevance of different feature groups modelling prosody in general (without speech tempo), speech rate and pauses modelling speech tempo (fluency), voice quality, and a variety of spectral features. We also discuss the impact of various fusion strategies on performance. Overall, our results demonstrate that the prosody of non-native speakers of English as L2 can be reliably assessed using supra-segmental audio features; prosodic features seem to be the most important ones.

AB - In this paper, we describe a new database with audio recordings of non-native (L2) speakers of English, and the perceptual evaluation experiment conducted with native English speakers for assessing the prosody of each recording. These annotations are then used to compute the gold standard using different methods, and a series of regression experiments is conducted to evaluate their impact on the performance of a regression model predicting the degree of naturalness of L2 speech. Further, we compare the relevance of different feature groups modelling prosody in general (without speech tempo), speech rate and pauses modelling speech tempo (fluency), voice quality, and a variety of spectral features. We also discuss the impact of various fusion strategies on performance. Overall, our results demonstrate that the prosody of non-native speakers of English as L2 can be reliably assessed using supra-segmental audio features; prosodic features seem to be the most important ones.

KW - Feature evaluation

KW - Index terms: non-native speech

KW - Prosody

UR - https://www.scopus.com/pages/publications/85016645446

M3 - Conference contribution

AN - SCOPUS:85016645446

T3 - Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

SP - 1328

EP - 1332

BT - Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016

A2 - Calzolari, Nicoletta

A2 - Choukri, Khalid

A2 - Mazo, Helene

A2 - Moreno, Asuncion

A2 - Declerck, Thierry

A2 - Goggi, Sara

A2 - Grobelnik, Marko

A2 - Odijk, Jan

A2 - Piperidis, Stelios

A2 - Maegaard, Bente

A2 - Mariani, Joseph

PB - European Language Resources Association (ELRA)

Y2 - 23 May 2016 through 28 May 2016

ER -

Coutinho E, Hönig F, Zhang Y, Hantke S, Batliner A, Nöth E et al. Assessing the prosody of non-native speakers of English: Measures and feature sets. In Calzolari N, Choukri K, Mazo H, Moreno A, Declerck T, Goggi S, Grobelnik M, Odijk J, Piperidis S, Maegaard B, Mariani J, editors, Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. European Language Resources Association (ELRA). 2016. p. 1328-1332. (Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016).

Assessing the prosody of non-native speakers of English: Measures and feature sets

Abstract

Publication series

Conference

Keywords

Other files and links

Fingerprint

Cite this