Image Segmentation

For a quantitative imaging course at ETH Zurich, I had the opportunity to work on an ultrasound segmentation project. The goal was to extract a region of interest (ROI) from an ultrasound image of a tendon. The tendons were either from the bicep or the supraspinatus in the upper arm, and were used to compare degradation in tendon quality among wheelchair users and control subjects. My portion of the project was to extract the ROI from the two types of tendon and compare them with a small training set we had developed.

I experimented with many different types of filters, clustering and segmentation techniques and the best results were found using an active contour model after thresholding the image.

This image shows the actual tendon ultrasound as well as reference region. The white portion of the reference is the desired area. For the bicep tendon there are two metal bars placed on the skin that leave an imprint on the resulting image. These two bars can be faintly seen on the left side of the image near the top and can be extracted by taking the mean of the top few rows of the image, and then finding the resulting peaks. There should be two peaks at the x locations of the bars in the image.


For the bicep tendon the start of the ROI was a set distance from the middle of the bars, and we knew from the protocol that the ROI had a set x length. By Gaussian filtering and thresholding the image we found the brightest regions on the image, which mostly corresponded to the desired ROI. I used this thresholded image and the known start x to set an initial ROI based on the brightness.

This initial ROI was then used to initialize an active contour and extract a rough estimate of the desired ROI. In the image the blue is the active contour and the red rectangle is the initial extracted ROI. The reference image is superimposed underneath as the light gray region.


For the supraspinatus tendon, unlike the bicep, we were not able to assume a constant offset from the metal bars due to the protocol. To counter this variability, I set the x distance extracted from the metal bars as a suggested search region, and extracted the initial ROI based on the threshold in the region around the estimated area. This is shown as the green frame in the middle of the thresholded image. The reference image is again superimposed underneath as the light gray region.


Using the active contour boundaries shown in blue, the final rectangular ROI for the comparison statistics could be extracted. This final ROI is set to be the maximum rectangle that fits within the region as shown here. Due to lighting differences on the image leading to threshold variability, the final ROI might not be a perfect representation of the region. I was not able to find a suitable alternative, and this variability is part of the selected technique.


The active contour model for segmentation is highly dependent on having a good initialization around the desired region. This sensitivity led to some fairly high variability in the extracted final ROI compared to the training image. The worst example of this variability can be seen in this supraspinatus tendon image where the initialization failed to converge to the optimal location leading to a suboptimal result.


For comparison between the images, I found the precision, recall, and harmonic mean (F1 Score) based on the reference image. Keep in mind that the final ROI rectangle will not be a perfect fit due to shape differences between the reference area and the desired area. This means that F1 score will not be high even if it involves a large rectangular region that skews the recall numbers.

But by comparing all three values we can estimate if the extracted ROI is suitable. For this specific case the precision is the most valuable metric as it indicates how much of the ROI actually falls into the reference region. For the statistics used in the protocol being inside the actual tendon area is more important that extracting the largest possible region that might include the tendon edges or muscle. Image 1 is the best case, and image 2 is the worst case for the bicep tendon.


Precision: 0.99  | 0.96

Recall: 0.55 | 0.32

F1: 0.71 | 0.4

The variability in the region can be seen in the final ROI. The first ROI shown is the same as above where the ROI is highly skewed due to lighting and noise issues. The second image shows a good supraspinatus tendon ROI extraction. Unfortunately due to issues relating to the unevenness in the expected ROI, the final regions are not as accurate. Image 1 is the worst case, and image 2 is the best case for the supraspinatus tendon.


Precision: 0.73  | 0.90

Recall: 0.29 | 0.60

F1: 0.42 | 0.72

Based on the calculated F1, Precision, and Recall scores shown above we should weight the analysis in favor of the Precision, followed by the F1 score as balance. The Recall is a less useful metric due to the shape differences between the desired ROI and the reference region.

On a non-training data set the results look good. The extracted ROI and tendon width are comparable to the training data set for both the supraspinatus and the bicep tendon. Here is a test image for the supraspinatus tendon and extracted ROI.


The full ipython notebooks and example images can be found on my github. The code was written in python and used the skimage library.