]> 2012-02-11T08:20:12-05:00 A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment Objective: Evaluate the performance of a continuous-speech interface to a decision-support system.Design: We performed a prospective evaluation of a speech interface that matches unconstrained utterances of physicians with controlled-vocabulary terms from Quick Medical Reference (QMR). We assessed the performance of the speech interface in two stages: in the real-time experiment, physician-subjects viewed audio-visual stimuli intended to evoke clinical findings, spoke a description of each finding into the speech interface, and then chose from a list generated by the interface the QMR term that most closely matched the finding. Subjects believed that the speech recognizer decoded their utterances; in reality, a hidden experimenter typed utterances into the interface (Wizard-of-Oz experimental design). Later, we replayed the same utterances through the speech recognizer and measured how accurately utterances matched with appropriate QMR terms using the results of the real-time experiment as the gold standard.Measurements: We measured how accurately the speech-recognition system converted input utterances to text strings (recognition accuracy) and how accurately the speech interface matched input utterances to appropriate QMR terms (semantic accuracy). Results: Overall recognition accuracy was less than 50%. However, using language-processing techniques that match keywords in recognized utterances to keywords in QMR terms, the semantic accuracy of the system was 81%. Conclusions: We found that reasonable semantic accuracy can be attained when language-processing techniques are used to accommodate for speech misrecognition. We also found that the Wizard-of-Oz experimental design offered many advantages for this evaluation and believe that this technique may be useful to future evaluators of speech-input systems. techreport 1994 A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment KSL-94-38 1994 Updated February 1995. A Continuous-Speech Interface to a Decision-Support System: II. An Evaluation Using a Wizard-of-Oz Experiment 1994 Year Title YES Process note Number Note Ksl tr id Knowledge Systems, AI Laboratory Institution Has year Has where published Has title Has publishing details Has identifier Has author Bibtype William M. Detmer and Smadar Shiffman and Jeremy C. Wyatt and Charles P. Friedman and Christopher D. Lane and Lawrence M. Fagan Author Abstract KSL Technical Report Publication Technical Report