A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment + Has identifier
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment + Ksl tr id
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment + Number
| KSL-94-38 |
Bibtype
techreport
Has publishing details
1994
Has title
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment
Has where published
KSL-94-38
Has year
1994
Title
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment
Year
1994
Abstract
Objective: Evaluate the performance of a … Objective: Evaluate the performance of a continuous-speech interface to a decision-support system.Design: We performed a prospective evaluation of a speech interface that matches unconstrained utterances of physicians with controlled-vocabulary terms from Quick Medical Reference (QMR). We assessed the performance of the speech interface in two stages: in the real-time experiment, physician-subjects viewed audio-visual stimuli intended to evoke clinical findings, spoke a description of each finding into the speech interface, and then chose from a list generated by the interface the QMR term that most closely matched the finding. Subjects believed that the speech recognizer decoded their utterances; in reality, a hidden experimenter typed utterances into the interface (Wizard-of-Oz experimental design). Later, we replayed the same utterances through the speech recognizer and measured how accurately utterances matched with appropriate QMR terms using the results of the real-time experiment as the gold standard.Measurements: We measured how accurately the speech-recognition system converted input utterances to text strings (recognition accuracy) and how accurately the speech interface matched input utterances to appropriate QMR terms (semantic accuracy). Results: Overall recognition accuracy was less than 50%. However, using language-processing techniques that match keywords in recognized utterances to keywords in QMR terms, the semantic accuracy of the system was 81%. Conclusions: We found that reasonable semantic accuracy can be attained when language-processing techniques are used to accommodate for speech misrecognition. We also found that the Wizard-of-Oz experimental design offered many advantages for this evaluation and believe that this technique may be useful to future evaluators of speech-input systems. future evaluators of speech-input systems.
Note
Updated February 1995.
Author
William M. Detmer and Smadar Shiffman and Jeremy C. Wyatt and Charles P. Friedman and Christopher D. Lane and Lawrence M. Fagan +
Has author
William M. Detmer and Smadar Shiffman and Jeremy C. Wyatt and Charles P. Friedman and Christopher D. Lane and Lawrence M. Fagan +
Has identifier
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment +
Institution
Knowledge Systems, AI Laboratory +
Ksl tr id
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment +
Number
A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment +
Process note
YES +
redirect page A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment +
Categories KSL Technical Report +, Publication +, Technical Report +
|
