Task 1 : Image quality assessment of respiratory motion artefacts
Background
The CMR image quality is affected by respiratory motion. Images with severe respiratory motion artifacts are not eligible for diagnostics and should be reacquired if possible. It is useful to develop an automated image quality assement model to recognize images with bad quality. In this task, we expect the challenge participants to develop an image quality assessment model under respiratory motion artefacts.
Data
Scanner: Siemens 3T MRI scanner (MAGNETOM Vida).
CMR acquisition: We follow the recommendations of CMR scans reported in the previous publication (doi: 10.1007/s43657-02100018x). We use the clinical sequence ‘TrueFISP’ for CINE imaging. For this challenge, we provide the SA images at the end-diastole (ED) and end-systole (ES) frames. Typical scan parameters: spatial resolution 2.0×2.0 mm2, slice thickness 8.0 mm, and slice gap 4.0 mm. The SOP requires volunteers to adhere to breath-holds instructions. For the 45 volunteers, each participant undertakes a 4-stage scan in a single visit: 1) adhere to the breath-hold instructions; 2) halve the breath-hold period; 3) breathe freely; 4) breathe intensively.
Pre-processing: The CMR images are anoymized and exported to NIFTI files from the DICOM files. We recommend participating teams to do resampling and normalization themselves.
Annotation
All images are viewed in 3D Slicer (www.slicer.org) and the image quality is scored by radiologists. The standard 5-point Likert scale is used as follows: excellent diagnostic quality (5), more than adequate for diagnosis (4), adequate for diagnosis (3), questionable for diagnosis (2), and non-diagnostic (1).
| 5-point score | Grade | Details |
| 5 | Excellent | No artifacts present |
| 4 | More than adequate for diagnosis | Minor artifacts present but image quality somewhat reduced |
| 3 | Adequate for diagnosis | Minor artifacts present and image quality somewhat reduced but still sufficient for diagnosis |
| 2 | Questionable for diagnosis | Image quality impaired by artifacts so diagnostic value of images is questionable |
| 1 | Non-diagnostic | Image quality heavily impaired by artifacts and readers not able to assess |
For better reproducibility, 3 levels of motion artefacts are defined based on the original 5-point scores. Images with quality scores 4-5 are labeled as mild motion artefacts, images with quality score 3 are labeled as intermediate motion artefacts, and images with quality score 1-2 are labeled as severe motion artefacts.
The task is to predict the level of motion artefacts based on the CRM images:
- Label 1: mild motion; Label 2: intermediate motion; Label 3: severe motion.
- Training, validation, and test cases both include a 3D short-axis CMR image and its label of motion artefacts.
- 160 training cases (20 volunteers*4 scans*2 frames), 40 validation cases (5 volunteers*4 scans*2 frames), and 160 test cases (20 volunteers*4 scans*2 frames).
- Publicly available data is allowed. But the source of data must be provided.
Metrics & Ranking
- Metrics: Cohen’s kappa implemented in scikit .
- Ranking method: For each case in the validation and test set, participating teams should submit a label from {1, 2, 3}, corresponding to mild/intermediate/severe motion artefacts.
- We calculate the Cohen’s kappa between the submission and the ground-truth labels provided by the radiologist.
Submission
The submission instructions will be released on the Synapse platform.