Task 2: CMR image segmentation with respiratory motion artifacts

Background

The automated CMR image segmentation model is prone to fail in face of unseen extreme images. In this task, we prepare an extreme dataset mimicking different levels of image degradation due to respiratory motion in clinical practice. For the images with diagnostic quality, we expect the challenge participants to develop a segmentation model robust to the respiratory motion artifacts.

Mission

The aim is to develop robust cardiac segmentation model in face of implications of respiratory motion.

Data

Scanner: Siemens 3T MRI scanner (MAGNETOM Vida).

CMR acquisition: We follow the recommendations of CMR scans reported in the previous publication (doi: 10.1007/s43657-02100018x). We use the clinical sequence ‘TrueFISP’ for CINE imaging. For this challenge, we provide the SA images at the end-diastole (ED) and end-systole (ES) frames. Typical scan parameters: spatial resolution 2.0×2.0 mm2, slice thickness 8.0 mm, and slice gap 4.0 mm. The SOP requires volunteers to adhere to breath-holds instructions. For the 45 volunteers, each participant undertakes a 4-stage scan in a single visit: 1) adhere to the breath-hold instructions; 2) halve the breath-hold period; 3) breathe freely; 4) breathe intensively.

Pre-processing: The CMR images are anoymized and exported to NIFTI files from the DICOM files. We recommend participating teams to do resampling and normalization themselves.

Annotation

All images with diagnostic quality in training, validation and test set are segmented by an experienced radiologist in 3D Slicer (www.slicer.org), including contours for the left (LV) and right ventricle (RV) blood pools, as well as for the left ventricular myocardium (MYO). Labels are: 1 (LV), 2 (MYO) and 3 (RV). This challenge follows the manual segmentation practice in previous challenges (M&M challenge, MICCAI 20). The radiologist studied the ground-truth annotations in the M&M challenge and followed the annotation protocol.

  • Training, validation, and test cases include a 3D short-axis CMR image and its manual label of left ventricle, left ventricle myocardium and right ventricle.
  • 160 training cases (20 volunteers*4 scans*2 frames), 40 validation cases (5 volunteers*4 scans*2 frames), and 160 test cases (20 volunteers*4 scans*2 frames). Images with severe motion artefacts are discarded from this task and are not segmented.
  • Publicly available data is allowed. But the source of data must be provided.

Metrics & Ranking

  • Metrics: Dice score and 95% Hausdorff distance.
  • Ranking method: Both the Dice scores and Hausdorff distances are used to compute the rankings. For each case, we will compute the 3 Dice scores and 3 95% Hausdorff distance measures between the ground truth and the submitted segmentation of LV, MYO, and RV, respectively. For X number of cases included in the test set, each participant has X*6 rankings. The final ranking score is the average of all these rankings normalized by the number of teams. It is noted that images with bad quality are excluded from the evaluation of segmentation performance.
  • Participating teams are required to process all the cases in the test set. For the cases without valid output, we set the ranks for the corresponding metrics to the maximum.
  • To assess whether the performance difference is significant, we will use paired and unpaired rank-based and t-test statistics for errors compared with permutation-generated one-sided null distributions.

Submission

The submission instructions will be released on the Synapse platform.