Name
Title | ||
|---|---|---|
Sequential classifiers for the prediction of user judgments about spoken dialog systems |
Abstract | ||
|---|---|---|
So far, predictions of user quality judgments in response to spoken dialog systems have been achieved on the basis of interaction parameters describing the dialog, e.g. in the PARADISE framework. These parameters do not take into account the temporal position of events happening in the dialog. It seems promising to apply sequence classification algorithms to the raw annotations of the data, instead of interaction parameters describing the overall dialog. As dialogs can be of very different length, Hidden Markov Models (HMM) and Markov Chains (MC) are handy, because they describe the likelihood of traversing to a state given only the previous state and the transition probability, thus they can be trained and applied to sequences of different lengths. This paper analyzes the feasibility of predicting user judgments with HMMs and MCs. In order to test the models, we acquire data with different types of users, forcing users to do as similar interactions as possible, and asking for user judgments after each turn. This allows comparing predicted distributions of judgments to the distributions measured empirically. We also apply the models to less rich corpora and compare them with results from Linear Regression models as used in the PARADISE framework. |
Year | DOI | Venue |
|---|---|---|
2010 | 10.1016/j.specom.2010.06.004 | Speech Communication |
Keywords | Field | DocType |
hidden markov models,spoken dialog system,user quality judgment,paradise,user judgment,paradise framework,prediction model,sequential classifier,usability,different type,overall dialog,evaluation,dialog system,interaction parameter,different length,markov chains,transition probability,linear regression model,predictive distribution,hidden markov model,markov chain | Dialog box,Spoken dialog systems,Linear model,Computer science,Regression analysis,Markov chain,Usability,Speech recognition,Artificial intelligence,Statistical classification,Hidden Markov model,Machine learning | Journal |
Volume | Issue | ISSN |
52 | 10 | Speech Communication |
Citations | PageRank | References |
2 | 0.38 | 14 |
Authors | ||
2 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Klaus-Peter Engelbrecht | 1 | 98 | 10.88 |
| Sebastian Möller | 2 | 877 | 141.17 |