Paper Info
Title
Three scenarios of ranking inconsistencies involving search tasks.
Abstract
Our previous work on assessment of digital breast tomosynthesis (DBT) image quality revealed inconsistencies in ranking the reconstruction algorithms' performances for a location-known-exactly (LKE) detection and a location-unknown searching task. Such results made us wonder that ranking inconsistencies may not be rare phenomena at all. In this work, we conducted a small literature review that involved three publications (He, Samuelson, Zeng and Sahiner SPIE 2016; Park, Kupinski, Clarkson and Barrett, IPMI 2003 and JOSA 2005). These publications compared the LKE and search performance for a variety of observers using the AUC value as the performance criterion (human observers, CHOs for detection, scanning CHOs for search, and the Markov Chain Monte Carlo ideal observer for detection and search). We categorized the experimental findings into three types of ranking inconsistencies: 1) Ranking inconsistencies in LKE and search tasks; 2) human/ideal observer ranking inconsistencies; and 3) LKE/search ranking inconsistencies in the presence of signal variability. The empirical evidence presented in this work suggested that ranking inconsistencies for imaging systems existed, but these inconsistencies often do not draw enough attention in the literature.
Year
DOI
Venue
2016
10.1117/12.2217617
Proceedings of SPIE
Keywords
Field
DocType
search,effective set size,ROC analysis,signal variability,intrinsic uncertainty,falsifiability
Data mining,Computer vision,Information retrieval,Ranking,Markov chain Monte Carlo,Computer science,Image quality,Artificial intelligence,Digital Breast Tomosynthesis,Search ranking,Observer (quantum physics)
Conference
Volume
ISSN
Citations 
9787
0277-786X
0
PageRank 
References 
Authors
0.34
0
4
Name
Order
Citations
PageRank
Xin He101.35
Frank W. Samuelson202.37
Rongping Zeng34211.15
Berkman Sahiner422466.72