Deep Siamese Networks for Plant Disease Detection

. Crop losses are a major threat to the wellbeing of rural families, to the economy and governments, and to food security worldwide. The goal of our research is to develop a multi-functional platform to help the farming community to tilt against plant diseases. In our previous works, we reported about the creation of a special database of healthy and diseased plants’ leaves consisting of ﬁve sets of grapes images and proposed a special classiﬁcation model based on a deep siamese network followed by k -nearest neighbors (KNN) classiﬁer. Then we extended our database to ﬁve sets of images for grape, corn, and wheat – 611 images in total. Since after this extension the classiﬁcation accuracy decreased to 86%, we propose in this paper a novel architecture with a deep siamese network as feature extractor and a single-layer perceptron as a classiﬁer that results in a signiﬁcant gain of accuracy, up to 96%.


Introduction
Globally, about 16 % of all crops are lost due to plant diseases each year so it is crucial to prevent the spread of the crop disease at its initial stage. The increasing number of smartphones and advances in the deep learning field opens new horizons in the plant disease detection. We started out research in early 2018. By that time, the only real application that could detect diseases by photos was Plantix (http://plantix.net). Plantix has very wide functionality and can recognize more than 300 diseases. Unfortunately, the access to the Plantix image database is restricted and we could not find any information about the methods used for the disease detection. We made a special study to measure the quality of the Plantix application which consisted of feeding up 70 grapes image from our self-collected database as input to the Plantix application for further identification of the diseases. Plantix identification of the plant type was rather good: 60 out of 70 images (87 %) were recognized as grapes. At the same time, the disease detection ability was rather limited: less than 10 % of images had the right disease at the top of suggestions, so we decided to develop our own application. Our main idea is to create a plant disease detection platform (PDDP) that will use modern organization and deep learning technologies to provide a new level of service to the farmer's community. To do it, we have to collect in a special database diseased and healthy plant leaves (http://pdd.jinr.ru/crops.php). Nowadays we have three crop types and fifteen classes of diseases in our database. We provide open e-mail: kaliostrogoblin3@gmail.com e-mail: auzhinskiy@jinr.ru access to our database and share the programming code and the trained models. As part of the PDDP, we developed a mobile application allowing users to send photos and text descriptions of sick plants and get answers concerning of the illness diagnostics.

Previous studies
The most popular way to deal with image classification problems in a vast majority of domains is to use a deep neural network trained on a big dataset with further fine-tuning deep classifier on your own data -such approach is broadly used and named transfer learning.
There are many deep learning based methods used to identify plant diseases [1][2][3][4]. Some of them report about a high detection level, exceeding 90 %. Generally, researches use a transfer learning approach on open-access databases or self-collected databases and concentrate on selecting the best base network and tuning of a model. Unfortunately, we could not get access to any of the selfcollected databases to reproduce their experience. However being impressed by the 99.7 % accuracy in the classification achieved in [1] with the help of transfer learning on a held-out test set, we made our own comparative study [5] of transfer learning models that are available in open access. We have found out that, although the ResNet50 [6] architecture reached 99.4 % classification accuracy on a test subset of the PlantVillage data, but it is stuck on our self-collected data with an unsatisfactory 48 % accuracy. We have found that the difficulty stems from the type of the used images. PlantVillage photos were collected and processed under special controlled conditions, so they are rather synthetic and differ from real-life images. It gave us the idea of creating our own database.
We collected our database from different open sources (guides, encyclopedias, forums). We reduced the image sizes and extracted only meaningful parts. At the very beginning, the PDDP database has only grapes leaves photos -130 healthy leaves, and 30-70 images with Esca, Chlorosis and Black Rot diseases, one can see examples of images in Fig. 1. But a set of 313 images is too small even for transfer learning and leads to overfitting the classifier. One of the way to train a deep neural network on a small data is the one-shot learning, in particular the siamese networks [7]. A siamese network consists of twin networks joined by a similarity layer with the energy function at the top. Weights of twins are tied (the same), thus the result is invariant and in addition guarantees that very similar images cannot be in very different locations in the features space. The similarity layer determines some distance metric between so-called embeddings, i.e. high-level features representations of input pairs of images. Training on pairs is more beneficial since it produces more data to train the model on, making it more difficult to overfit. As we could see from the papers [1-4] mentioned above and especially from the comprehensive survey [8], siamese networks were not applied before for the plant disease detection from leave images.
The classification accuracy measured by our new siamese network on a test subset of grapes images reached 95 % using all 5 classes.

PDDP dataset expansion
We have significantly expanded the PDDP dataset since an article with previous results was published. We have added two other crops -wheat and corn. Each of the added crops is represented by 5 sets of images: Corn 4 diseases -Downy mildew, Eyespot, Northern leaf blight, Southern rust, healthy; Wheat: 4 diseases -Black chaff, Brown rust, Powdery mildew, Yellow rust, healthy.
The final version of the dataset includes 15 classes with 611 leaf photos in total. Grape -healthy, Esca, Chlorosis, Powdery mildew and Black Rot; Corn -Downy mildew, Eyespot, Northern leaf blight, Southern rust, healthy; Wheat -Black chaff, Brown rust, Powdery mildew, Yellow rust, healthy. After training on all 15 classes even for 150 epochs the best version of the model from Sect. 2 obtained 86 % test classification accuracy.
Probably, such a poor accuracy value may be caused by using KNN as a classifier. It turns out that KNN could suffer from hubs when working with high-dimensional data [9]. A hub is a node which tends to have much more in-going edges than the other nodes. To deal with hubs one can reweight all distances using special scaling parameters, or simply replace KNN with another classifier as discussed in the next section.

Softmax regression with single-layer perceptron
To improve the test classification accuracy we made a special comparative study of different types of estimators including logistic regression, support vector machines with cosine similarity as a kernel, decision tree, random forest, gradient boosting and a simple perceptron with one input and one output layer ending with softmax activation.
The single-layer perceptron was trained for 100 epochs with the Adam [10] optimizer on 85 %/15 % training-testing set distribution. That allowed us to obtain a high classification accuracy of 95.71 % on the test subset of images. In the Figure 2 the confusion matrix is illustrated. We added on the top of our trained feature extraction the fully connected neural network layer with the softmax activation and trained the whole model with frozen weights of the feature extractor. It is very similar to the transfer learning method, but in our case, the base network was trained directly on our domain data.

Conclusion
We expanded the dataset with two additional crops -wheat and corn and developed a special siamese transfer learning method. This modification improved the accuracy of the model trained on 15 classes (611 images total) from 86 % to 95.7 %. The next step will be to explore other types of siamese loss functions (triplet loss) and optimization of the existing deep neural network architecture or using transfer learning to initialize siamese twin for further fine tuning in siamese mode. One can find our free mobile application at Google play -PDDApp. The other option to upload an image of diseased plant and get a prediction is our web-portal -pdd.jinr.ru