Soybean Leaf Segmentation

For a semester project at the ETH Zürich as part of my masters in 2018, I worked on soybean leaf segmentation for the crop science group. Crop scientists are interested in using machine learning techniques to improve crop phenotyping by making automated systems to classify growth of plants in real-world conditions. Crop phenotyping is used to compare different cultivars (specific plant versions) of plants to determine which version produces the strongest harvest in specific conditions.  Current phenotyping methods require manual work to collect data to enable this comparison and does not scale.

The soybean images themselves are from a research facility run by the ETH Crop Science group. The research fields have large suspended cameras and sensor systems mounted over the fields that take high resolution images of the crops. For this project we used RGB images 0f early growth soybeans for the analysis.

I used a combination of Amazon Mechanical Turk and LabelMe annotation software to develop a set of ground truth segmentation masks for the training images. I then used various augmentation techniques to expand the dataset to allow better training.


For the models I used a simple color based thresholding model as the control, a random forest method, and a deep learning network called DeepLabV3+ implemented in tensorflow. For the evaluation I used the mean Intersection over the Union to determine how well the predicted mask aligned with the ground truth mask. The color based thresholding actually performed the best in correctly segmenting about 87% of the target leaf area. The DeepLab achieved 78% and the random forest only getting 51% of the target area.

This image shows a few example validation images with corresponding mask and predicted results.


This surprising result that the color thresholding is better than the deep learning model is partially caused by the small size of our dataset with regards to the deep learning model. The good results of the color thresholding was due to the uniform sizing of the image objects. The leaves in the images were all roughly the same size, color, and shape which allows a fine-tuned manual extraction involving specific Gaussian blur and color space conversion methods. This method would not work as reliably for a wide diversity of input images.

The results were published and presented in the British Machine Vision Conference in September in Newcastle, UK. All the code was written in python and used tensorflow and the ski-learn package.

The full  soybean segmentation report is available for all the details.