Name
Title | ||
|---|---|---|
Designing a virtual patient dialogue system based on terminology-rich resources: Challenges and evaluation. |
Abstract | ||
|---|---|---|
Virtual patient software allows health professionals to practise their skills by interacting with tools simulating clinical scenarios. A natural language dialogue system can provide natural interaction for medical history-taking. However, the large number of concepts and terms in the medical domain makes the creation of such a system a demanding task. We designed a dialogue system that stands out from current research by its ability to handle a wide variety of medical specialties and clinical cases. To address the task, we designed a patient record model, a knowledge model for the task and a termino-ontological model that hosts structured thesauri with linguistic, terminological and ontological knowledge. We used a frame- and rule-based approach and terminology-rich resources to handle the medical dialogue. This work focuses on the termino-ontological model, the challenges involved and how the system manages resources for the French language. We adopted a comprehensive approach to collect terms and ontological knowledge, and dictionaries of affixes, synonyms and derivational variants. Resources include domain lists containing over 161,000 terms, and dictionaries with over 959,000 word/concept entries. We assessed our approach by having 71 participants (39 medical doctors and 32 non-medical evaluators) interact with the system and use 35 cases from 18 specialities. We conducted a quantitative evaluation of all components by analysing interaction logs (11,834 turns). Natural language understanding achieved an F-measure of 95.8%. Dialogue management provided on average 74.3 (+/- 9.5)% of correct answers. We performed a qualitative evaluation by collecting 171 five-point Likert scale questionnaires. All evaluated aspects obtained mean scores above the Likert mid-scale point. We analysed the vocabulary coverage with regard to unseen cases: the system covered 97.8% of their terms. Evaluations showed that the system achieved high vocabulary coverage on unseen cases and was assessed as relevant for the task. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1017/S1351324919000329 | NATURAL LANGUAGE ENGINEERING |
Keywords | DocType | Volume |
Dialogue system,Virtual patient,Terminology,Language resources | Journal | 26 |
Issue | ISSN | Citations |
2.0 | 1351-3249 | 1 |
PageRank | References | Authors |
0.38 | 0 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| leonardo campillos llanos | 1 | 9 | 8.39 |
| Catherine Thomas | 2 | 1 | 0.38 |
| Eric Bilinski | 3 | 57 | 9.39 |
| Pierre Zweigenbaum | 4 | 773 | 85.43 |
| Sophie Rosset | 5 | 393 | 61.66 |