Evaluation Metrics

We graded the pulmonary artery blood vessels into multiple levels according to the radius of blood vessels, and evaluated the accuracy of trunk and branch respectively. In the final testing phase, we apply a cubic box in the ground truth of each test case to contain the trunk of pulmonary artery manually,  then cut  the output of participant's algorithmn using a cubic box with the same position, eventually, calculate the Dice and HD95 coefficients of the truck structure. metrics of branch structure are calcluated similarly.

The segmentation performance is evaluated in four aspects following:

1.  Multi-level Dice Similarity Coefficient (Dice): Dice is used to evaluate the area-based overlap index.
2.  Multi-level Hausdorff distance (HD95): HD95 is used to evaluate the coincidence of the surface for stable and
   sensitive to outliers.
3.  Running time: Low time-comsuming is prefer for good algorithm.
4.   Maximum used GPU memory: Low memory-comsuming indicates the good application potential in more widely
   used computing platform.

Ranking Method

The ranking scheme includes the following steps:

1. Calculate the Dice, HD95, Maximum used GPU memory, and Running time for all cases.
2. Rank the Dice, HD95, Maximum used GPU memory, and Running time separately.
3. Average these rankings.
4. Tie if the rankings are equal.