machine learning – Corbeau Innovation

Sep 30 2021

Cassava virus infection detected with handheld multispectral imager

Healthy cassava leaves (image courtesy of Pixabay)

One of the most important sources of carbohydrate energy in equatorial countries comes from the cassava plant. It looks like a trendy house plant but the roots are large tubers that provide valuable nutrition. Tubers can be boiled and mashed or dried, ground and turned into flour. Cassava is a robust crop but suffers from two virus diseases that can ruin an entire crop with very little warning until the tubers are dug up at harvest. Rotten tubers cannot be used for food and signs of viral infection are not obvious on the leaves or stems until it is too late to plant a replacement crop. Propagation of cuttings for the following year’s crop is also affected by virus so potentially two years worth of food could be lost in a single infection.

Cassava mosaic virus (CMV) and cassava brown streak disease (CBSD) are the two main viral diseases responsible for crop loss. There are now cassava varieties that are resistant to CMV but CBSD is still problematic. There are various biochemical diagnostic tests for the diseases but the most reliable, a PCR test, is expensive, invasive and requires a relatively high level of viral load. Prompted by the need for a better, quicker test a team of researchers from University of Manchester, North Carolina State University, Rutgers University and the International Institute of Tropical Agriculture have developed a handheld multispectral leaf imager that detects the presence of CBSD before signs are obvious to the human eye.

Cassava tubers ready for processing (image courtesy of PixaBay)

In a pre-print paper last month, Hujun Yin and colleagues reported how a compact 14 wavelength multispectral leaf imaging device utilising machine learning successfully classified diseased plants and control plants. Photos of the device are shown below.

Multispectral leaf imager (a) and sample chamber window with grid to hold leaf flat and LED ring for illumination (b) (image courtesy of Creative Commons, Research Square and the authors)

The Manchester study comprised three trials, each containing cassava plants naturally immune to CMV to minimise the chance of random viral infection from another source. All three trials had three treatment groups: controls; CBSD inoculated; and E. coli inoculated. The last treatment groups were used to test the susceptibility of the inoculation method itself, E. coli should have no effect on the health of the cassava plants. Plants were measured at days 7, 14, 21, 28, 52, 59 and 88 days post inoculation (dpi). Leaf images were recorded using 14 different LED light wavelengths (395, 415, 470, 528, 532, 550, 570, 585, 590, 610, 625, 640, 660, 700 and 880 nm). At each time point plant leaves were also given scores from 1 to 4 based on how they appeared to the eye. Typical leaves are shown below.

Cassava leaf scores (1-4) at days post inoculation

In trial 2, PCR tests were performed on leaves at each time point and visual scores were documented. The visual scores showed a progression with time for the inoculated leaves as expected (see below).

Cassava leaf scores as trial 2 progressed

To verify that the virus was indeed successfully inoculated into the plants, PCR tests confirmed the virus present in some plant leaves after day 52, with the highest levels at the end of the time course.

Analysis of the leaf image spectral hypercubes (14 wavelengths x 12 random groups of leaf pixels) produced metadata consisting of six vegetation indices (VIs); average spectral intensities (ASIs) and texture (a measure of the variation of the intensities from pixel group to pixel group on a leaf). Interestingly, even some simple VIs were capable of distinguishing some diseased plants from healthy ones (more than 60% successful classification at day 52, comparable to the PCR testing).

Metadata was used to create a classifier based on measurements taken from plants with positive PCR test results. Rather than use a convolutional neural network (CNN) approach to classify the images or metadata, the team used a Support Vector Machine (SVM). SVMs have the advantage that they typically do not require high computing power and they can be intuitively quantitative using simple regression to find the best dividing lines between image categories. SVM produced a marked improvement in classification with better than 80% success as a result of using positional information in addition to VI. An introductory reference is given at the end of this post.

The group made one further improvement to their model and this was to combine subsets of classifiers produced by the SVM. They called this approach Decision Fusion (DF) and Probabilistic Decision Fusion (PDF), which is basically saying if one classifier doesn’t work, combine it with another and see if the performance improves. Finally they achieved sophisticated classification of diseased and healthy plants at day 53 and 80-90% success.

It will be very interesting to see multispectral imaging applied to more diseases and to the classification of differing diseases.

The pre-print paper can be found here, courtesy of Research Square and the authors Yao Peng, Mary Dallas, José T. Ascencio-Ibáñez, Steen Hoyer, James Legg, Linda Hanley-Bowdoin, Bruce Grieve, Hujun Yin.

A nice introduction to Support Vector Machines from MonkeyLearn can be found here.

Information on our own BACO multispectral leaf imager can be found here.

user 0 Comments

Aug 29 2021

Counting grapes with machine learning

bunches of grapes — Bunches of grapes in a full canopy of leaves and stems

As the grapes swell in the late summer sun and rain, vignerons start thinking about the harvest. What is the yield going to be? how much sugar will there be in the grapes and finally how many bottles of wine can be made?

It would be really useful to have a way to predict the yield of each vine, sector and vineyard. Reliability is important because securing the wrong amount of vineyard labour, vat capacity or bottling can lead to either higher costs or loss of production. Hand counting is reliable but extremely time-consuming if a good representative portion of the vineyard is counted by visual inspection. Wouldn’t it be great if you could automate the inspection process by mounting a camera on a drone or quad-bike and then use AI (artificial intelligence) to pick out the grape bunches from the images of leaves, stems and bunches as the camera races up and down the rows of vines?

Well, this has been done by a number of developers and Corbeau reviewed one clever solution recently. The big challenge is to overcome the confusion of bunches partially obscured by leaves and stems.

Machine learning (ML) examples often use images of salads, with bright red tomato slices, green lettuce and orange carrot sticks. These can be easily sorted or counted using colour filters. In the vineyard you encounter green leaves, grapes, stems and tendrils. Each of them growing at different angles and with a range of different shades of green. Counting grape bunches is a much more difficult problem to solve than counting tomatoes.

Undaunted, Corbeau decided it was time to investigate Open Source AI as a way to count grape bunches in vine images. There are a wide range of Open Source tools available, some of which require programming expertise and some which do not. The main drawback of using Open Source tools is that the documentation is typically either very limited or written in a strange English dialect spoken only by Microsoft employees.

Corbeau started with a set of 30 colour vine images taken with a smartphone. Each image was taken in full sun and contained random combinations of grapes, leaves, stems and tendrils. Images typically covered a vine area of around 1 square metre. The first task was to create sets of annotations to identify features of interest. This was done manually with the Microsoft API, drawing labelling boxes around grape bunches and stems in the images. An example annotation is shown below.

Bunch label (yellow) and stem labels (green)

A CNN (convolutional neural network) was chosen to create a machine learning model. Convolution methods are useful ways to emphasise shapes in images. They work by trying to make the image look like a particular pattern, for example parallel vertical lines or circles. After convolving a reference labelled box with a pattern, the simplified new image is assessed: is it more ‘like’ the pattern than something else? When many different patterns are convolved with an image, a set of patterns will tend to characterise one image more than another one. Each of the pattern ‘like’-ness tests can be made a decision point in making an image classification. In this sense the decision-point network is an intelligent classifier. The CNN requires some ground truth, defined by human definitions of what different bunches of grapes look like. It becomes intelligent by learning from human definitions of different bunches and stems.

The set of convolutions which classify a bunch and a stem are different and require verification and testing before they can be used as a machine learning classification model. New vine images were taken to verify the performance of the classification model. Bunch classification was more successful than stem classification because a much higher proportion of bunches were annotated in the learning set of images than stems. After a few hours developing better classification models, the model produced a 0.77 confidence level for bunch identification and 0.12 confidence level for stem identification. When presented with similar images (size, resolution, range) the model should be capable of identifying grape bunches and counting them. The model was deployed as a TensorFlow classification task, ready for counting grape bunches in additional grapevine images.

To test how this ML approach could be used in the vineyard, a burst series of smartphone images were recorded as the phone was carried parallel to a test vine. Running the ML model produced a list of potential image boxes found by the classifier, shown below.

Smartphone burst image of vine with annotation labels

Two bunches were identified with 1.000 (right hand bunch) and 0.928 (top left bunch) confidence levels. The next image box (at the bottom left side of the picture above) was not a bunch and had a 0.019 confidence level.

These results are an encouraging start to the machine learning bunch counting project. If you would like to take part in the project, please get in touch with Pierre Graves by completing the Contact form!

user 0 Comments