Top as in TopCoW

Assessment aim(s)

The focus of this challenge is on topology-aware segmentations of CoW vessels. The assessment of algorithms are divided into two categories following our prior work [Shit et al. 2021]:

1. Volumetric metrics: Dice coefficient and centerline-Dice [Shit et al. 2021] of the CoW vessels
2. Topology-based metrics: errors of Betti numbers such as connected components and circular holes

The segmentation results will be evaluated only for the CoW region of interest (ROI). (Please refer to the annotation page for more info.) We will not assess the segmentation performance on the peripheral and further downstream vessels outside the CoW ROI. Participants should focus on segmenting the CoW vessel components necessary to characterize the CoW angio-architecture.

Metrics

Three evaluation metrics with equal weights for binary (CoW vessel vs background) segmentation task:

1. Dice similarity coefficient
2. cl-Dice
3. Betti number 0 errors

Three evaluation metrics with equal weights for multi-class (CoW anatomical vessels) segmentation task:

1. Class-average Dice similarity coefficient
2. Binary-merged cl-Dice
3. Class-average Betti number 0 errors

NOTE: Participants can choose to only tackle the multi-class segmentation task, as submissions to multi-class segmentation tasks are automatically evaluated by us for binary segmentation performance.

Reason behind our metrics

Dice similarity coefficient and centerline-Dice [Shit et al. 2021] measure the voxel overlap between the ground truth and the segmentation prediction. In particular, we highlight the centerline-Dice metric (clDice), which is suitable for evaluating voxel-wise overlap for tubular and curvilinear structures such as CoW vessels [Maier-Hein et al. 2022]. clDice extends the traditional Dice by also measuring how much of the vessels are covered (coverage of vessel network).

Betti numbers measure the topological properties such as connected components and circular holes.

The above metrics are largely based on the evaluations performed in our prior work, clDice [Shit et al. 2021] and Betti-Matching [Stucki et al. 2023].

References

• Shit, Suprosanna, et al. "clDice-a novel topology-preserving loss function for tubular structure segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
• Stucki, Nico, et al. "Topologically faithful image segmentation via induced matching of persistence barcodes." International Conference on Machine Learning. PMLR, 2023.
• Maier-Hein, Lena, et al. "Metrics reloaded: Pitfalls and recommendations for image analysis validation." arXiv preprint arXiv:2206.01653 (2022).