Name
Abstract | ||
|---|---|---|
This paper describes the QUality and Experience Metric (QUEM), a method for estimating the skill level of a knowledge-based system based on the quality of the solutions it produces. It allows one to assess how many years of experience the system would be judged to have if it were a human by providing a quantitative measure of the system's overall competence. QUEM can be viewed as a type of achievement or job-placement test administered to knowledge-based systems to help system designers determine how the system should be used and by what level of user. To apply QUEM, a set of subjects, experienced judges, and problems must be identified. The subjects should have a broad range of experience levels. Subjects and the knowledge-based system are asked to solve the problems; and judges are asked to rank order all solutions驴from worst quality to best. The data from the subjects is used to construct a skill-function relating experience to solution quality, and confidence bands showing the variability in performance. The system's quality ranking is then plugged into the skill function to produce an estimate of the system's experience level. QUEM can be used to gauge the experience level of an individual system, to compare two systems, or to compare a system to its intended users. This represents an important advance in providing quantitative measures of overall performance that can be applied to a broad range of systems. |
Year | DOI | Venue |
|---|---|---|
1997 | 10.1109/69.649311 | IEEE Trans. Knowl. Data Eng. |
Keywords | Field | DocType |
knowledge-based system,individual system,achievement test,skill level,broad range,quality ranking,quantitative measure,worst quality,knowledge-based systems,solution quality,system designer,experience level,software metrics,knowledge based systems,system design,knowledge based system,knowledge engineering,confidence band,software quality | Data mining,Ranking,Computer science,Expert system,Knowledge-based systems,Achievement test,Knowledge engineering,Artificial intelligence,Knowledge base,Software metric,Software quality,Machine learning | Journal |
Volume | Issue | ISSN |
9 | 6 | 1041-4347 |
Citations | PageRank | References |
5 | 18.06 | 1 |
Authors | ||
2 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Caroline C. Hayes | 1 | 110 | 39.81 |
| Michael I. Parzen | 2 | 5 | 18.73 |