Name
Abstract | ||
|---|---|---|
We propose and demonstrate a simple method to explain the figure of merit (FoM) of a music information retrieval (MIR) system evaluated in a dataset, specifically, whether the FoM comes from the system using characteristics confounded with the “ground truth” of the dataset. Akin to the controlled experiments designed to test the supposed mathematical ability of the famous horse “Clever Hans,” we perform two experiments to show how three state-of-the-art MIR systems produce excellent FoM in spite of not using musical knowledge. This provides avenues for improving MIR systems, as well as their evaluation. We make available a reproducible research package so that others can apply the same method to evaluating other MIR systems. |
Year | DOI | Venue |
|---|---|---|
2014 | 10.1109/TMM.2014.2330697 | Multimedia, IEEE Transactions |
Keywords | Field | DocType |
information retrieval systems,music,Clever Hans,FoM,MIR evaluation,MIR system,dataset ground truth,figure-of-merit,music information retrieval system,musical knowledge,2-WORK system performance,5-CONT content description and annotation,5-SEAR multimedia search and retrieval | Music information retrieval,Human–computer information retrieval,Information retrieval,Musical,Computer science,Figure of merit,Ground truth,Visual Word | Journal |
Volume | Issue | ISSN |
16 | 6 | 1520-9210 |
Citations | PageRank | References |
24 | 1.23 | 25 |
Authors | ||
1 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Bob L. Sturm | 1 | 241 | 29.88 |