Paper Info
Title
Quantitative and qualitative methods for efficient evaluation of multiple 3D organ segmentations.
Abstract
Quantitative comparison of automatic results for multi-organ segmentation by means of Dice scores often does not yield satisfactory results. It is especially challenging, when reference contours may be prone to errors. We developed a novel approach that analyzes regions of high mismatch between automatic and reference segmentations. We extract various metrics characterizing these mismatch clusters and compare them to other metrics derived from volume overlap and surface distance histograms by correlating them with qualitative ratings from clinical experts. We show that some novel features based on the mismatch sets or surface distance histograms performed better than the Dice score. We also show how the mismatch clusters can be used to generate visualizations which may reduce the workload for visual inspection of segmentation results. The visualizations directly compare reference to automatic results at locations of high mismatch in orthogonal 2D views and 3D scenes zoomed to the appropriate positions. This can make it easier to detect systematic problems of an algorithm or to compare recurrent error patterns for different variants of segmentation algorithms, such as differently parameterized or trained CNN models.
Year
DOI
Venue
2019
10.1117/12.2512750
Proceedings of SPIE
Keywords
Field
DocType
deep learning,segmentation,radiotherapy,computational anatomy,pelvis,validation,evaluation
Computer science,Artificial intelligence,Qualitative research,Machine learning
Conference
Volume
ISSN
Citations 
10949
0277-786X
0
PageRank 
References 
Authors
0.34
0
7
Name
Order
Citations
PageRank
Volker Dicken121819.02
Annika Hänsch201.01
Jan Hendrik Moltz3808.85
Benjamin Haas401.01
Thomas Coradi500.34
Tomasz Morgas601.01
Jan Klein79510.94