Introduction

The tomato plant (Solanum lycopersicum L.) is a species native to South America and was one of the most important vegetables in terms of global production, with approximately 186 million tons in 2022 (FAO, 2024). It is significant as a model organism for economic, nutritional, and biotechnological research, as results obtained with tomatoes are applicable to other vegetable species as well (Hiwasa-Tanase, 2016).

Tomato fruit classification is important in various contexts, including cultivar selection, production standardization, commercialization, and genetic improvement research (Causse et al., 2006).

Over the years, tomatoes have been selected to improve characteristics such as shape, color, size, and taste. In the last 30 years, new cultivars and hybrids, such as Santa Cruz and Italian, with specific traits, have been released (Razifard et al., 2020).

Visual classification of tomatoes for breeding programs faces challenges due to variety diversity, uneven ripening, external damages, and evaluator subjectivity. Recent advances in computational resources, such as image phenotyping, have allowed for fast and precise pre- and post-harvest evaluations. These technologies offer advantages such as low cost, speed, accuracy, and automation, the reduction of labor costs, and the elimination of variability due to human subjectivity in classification (Chandra et al., 2020).

The integration of innovations, such as the use of images together with computational intelligence, stands out as a promising approach in plant phenotyping (Sambasivam and Opiyo, 2021; Chandra et al., 2020). Deep learning models, Convolutional Neural Networks (CNNs) are widely recognized and employed for image classification. These algorithms can identify various visual elements, including faces, individuals, street signs, fruits, animals, and other attributes present in visual data (Li et al., 2022). The architecture of a CNN determines how the network processes input data, extracts feature, reduces dimensionality, and ultimately performs the task of classification, detection, or segmentation. The choice of architecture is crucial to achieving a balance between accuracy, computational efficiency, and generalization capability (Liu et al., 2020). Thus, this research aimed to verify the efficiency of CNNs in automating tomato fruit classification and compare different CNN architectures in terms of their performance of this task.

Materials and Methods

The experiment was conducted in Montes Claros, Minas Gerais state, Brazil (16°40’58.16" S, 43°50’20.15" W, altitude 661 m).

Experimental setup and evaluation

Five parental lines and their ten F1 hybrid combinations were evaluated, originating from a balanced diallel cross without reciprocals of five commercial lines: San Marzano and Santa Clara (Isla company), Santa Cruz Kada Giant and Improved Gaucho New Selection (Feltrin company), and Gaucho (Top Seed company).

Seeding was carried out in 128-cell polystyrene trays filled with the commercial vegetable substrate (Bioplant) and continued until transplanting, approximately 35 days after sowing.

After transplanting to the field, tomato plants were staked and trained with two stems. Each stem was managed so as to form six to seven fruit clusters. No fruit thinning was undertaken during cultivation, and the number of fruits varied accordingly based on the genotypes’ characteristics. Fertilization was carried out according to soil analysis and recommendations for tomato cultivation by Furlani and Bataglia (2018). Irrigation, performed daily, was conducted through drip irrigation. Phytosanitary control, whenever necessary, was carried out using products registered for the crop.

The experiment was conducted in a randomized complete block design (RCBD) with four replications, 15 treatments (ten hybrids and five parental lines), and five plants per plot, making a total of 300 plants. There were ten planting rows (beds), with plant spacing within plots of 0.70 m, 1 m between plots, and 2 m between rows.

Manual harvesting began 90 days after transplanting. Tomato fruits were evaluated over twelve harvests. Fruits at the ripe stage, with a red color, were harvested twice a week. Variables pertaining to tomato fruit were analyzed: group (Italian, Saladette, Cherry, and Santa Cruz), shape (oblong or round), color (striped, pink, green mature, red, and red mature), and defects (severe, mild, and no defects). Evaluations were conducted according to the descriptors and classifications of MAPA (2002), as exemplified in Figure 1A-E.

Image acquisition and processing

Images were acquired in a portable mini studio with dimensions of 60 × 60 cm at the base and a height of 60 cm (Figure 2A). A smartphone (Xiaomi company, Poco M3 model) was used to capture images with a fluorescent lamp under artificial lighting. The smartphone was attached to a stand at the top of the mini studio to standardize image acquisition, so as to ensure that all images were captured from the same height, 60 cm. The fruits were placed on a checkered blue background, each in a square (Figure 2B), allowing up to 20 fruits to be photographed simultaneously (Figure 2C). The ExpImage package in the R software program (R Core Team, version 4.4.1) was used to isolate one fruit per image (Figure 2D).

The images were categorized based on their classifications of fruit shape (Italian, Saladette, Salad, and Santa Cruz), fruit color (red, ripe red, blush, blotched, and ripe green), groups (oblong and round), and defects (mild, severe, and no defects) through visual analysis (Figure 3A-D).

Visual classification was carried out after obtaining images of each ripe fruit, resulting in the numbers presented in Table 1 (original numbers). Due to the natural and unrestricted development of the plants, the number of fruits for each classification differed. A small number of fruits was found for the "Italian" and "Persimmon" classifications for the "group" characteristic and the "Ripe Green" and "Ripe Red" classifications for the color characteristic. This work disregarded these classifications due to the low number of images, making it impossible to train the networks.

For the composition of the image database for training and validation of the networks, 1,500 fruits were considered for each classification. To expand the dataset in training, each image was replicated with three different rotations (90°, 180°, and 270°). Thus, for each classification, there were 4,000 images in training and 2,000 in validation (1,500 + 3 × 1,500). The images destined for the training and validation database were randomly selected. Python language was used on the Google Colab platform to train the networks. The Keras library was used, considering the VGG16, Inceptionv3, ResNet50, InceptionResNetV2, EfficientNetB3 architectures. A maximum of 100 iterations was considered, with early stopping tolerance set at ten iterations.

This work used CNNs for image training and classification. Each classifier then determined the probability of an instance belonging to a specific subclass, selecting the class with the highest probability as the final output. To evaluate the performance of convolutional neural networks, confusion matrices were constructed, comparing the classifications predicted by different network architectures with those obtained through visual analysis. The metrics Recall (Eq. 1), Accuracy (Eq. 2), Precision (Eq. 3), F-measure (Eq. 4), and Specificity (Eq. 5) were used to assess the efficiency of the network.

where TP = true positives: instances that were correctly predicted as belonging to the positive class; FP = false positives: instances that were erroneously predicted as belonging to the positive class when, in fact, they belonged to the negative class; TN = true negatives: instances that were correctly predicted as belonging to the negative class; FN = false negatives: instances that were erroneously predicted as belonging to the negative class when, in fact, they belonged to the positive class.

Results

Each architecture revealed distinct epochs and time required for training (Table 2). For shape and group, the Inceptionv3 architecture showed the shortest classification times, and the lowest number of epochs needed for training. However, for color and defect classification, the InceptionResNetV2 architecture had the shortest time. Except for the group variable, the EfficientNetB3 architecture required the longest time and the highest number of epochs, showing lower efficiency (Table 2).

The EfficientNetB3 architecture had higher TP classification numbers for groups. However, it did not identify any TN and obtained a high FP value, indicating inefficiency in correctly identifying instances that were classified as positive. The InceptionResNetV2 architecture achieved the highest efficiency, with greater accuracy in correctly classifying positive and negative instances and high values for TP and TN for all evaluated variables (Table 3). The number of positive instances (TP), cases where the model correctly recognized a pattern, and negative instances (TN), where the model correctly rejected an area that does not have an object of interest, represent the success of the architecture in classification.

The effectiveness of CNN architectures in classifying tomato fruit regarding shape, groups, color, and defects was evaluated using precision, recall, F-measure accuracy, and specificity values (Table 4). The InceptionResNetV2 architecture was the most successful of these metrics. It achieved higher precision, better accuracy, and specificity for most analyzed variables, with values above 84 %. This architecture enabled 100 % precision for color and defect classifications, with the latter achieving 100 % for most evaluation metrics, except for recall, which was 99 %. The EfficientNetB3 architecture achieved 100 % recall for shape and group, but did not have good values for other metrics, showing its low efficiency in classification. Overall, the VGG16 architecture showed the worst results, with lower estimates for the quality of fit evaluators.

Discussions

Appearance characteristics such as color, texture, size, shape, and various defects constitute important attributes of external sensory quality in fruits and vegetables. Computer vision systems have widely replaced visual and manual classification in the assessment of quality of food and agricultural products (Fracarolli et al., 2020). Compared to traditional computer vision methods, CNNs achieve superior performance, with faster inference times and higher detection rates, as evidenced by previous studies (Paymode and Malode, 2022). The networks learn through iterative processes of adjustments to synaptic weights known as training. Effective learning occurs when the neural network reaches a generalized solution to a specific problem (Razifard et al., 2020). The different architectures tested in this study showed that it is possible to detect and classify not just one but several different classes of tomato fruit efficiently and quickly. Training of CNNs consists of stages called epochs or cycles. Each of these stages represents the number of iterations of the process, during which all training input data are applied to the network, aiming to adjust it to reduce the mean error (Guimarães et al., 2008). However, as the number of iterations during training increases, networks tend to memorize the data more, resulting in the non-generalized nature of the system, known as "overfitting". This problem and performance arise as training data decreases. Its performance improves as the number of training data increases (Mamat et al., 2023). Therefore, it is essential to determine an optimal number of iterations for the analyzed datasets, and this can be accomplished through the application of a strategy known as early stopping (Fernandes et al., 2023). For this work, according to the results, the dataset size and early stopping were sufficient to adjust all layers of the InceptionResNetV2 architectures efficiently, suggesting that the model is robust and generalizes well to test data.

Each type of architecture, as well as each variable used, revealed different numbers of epochs. Notably, the InceptionV3 and InceptionResNetV2 architectures demonstrated the lowest numbers of epochs and, therefore, the shortest classification times. These results point to a time-saving in obtaining results (Ni et al., 2020), which does not necessarily lead to a good fit. On the other hand, the VGG16, EfficientNetB3, and ResNet50 architectures showed lower efficiency in classifying the analyzed dataset, requiring the highest number of epochs and a longer time to obtain results.

The performance of the CNN model for classification tasks is generally evaluated considering various criteria such as precision, recall, F-measure, accuracy, and specificity. When comparing the precision of the architectures, InceptionResNetV2 showed the best result. Choosing an architecture that allows for more efficient analysis qualifies characteristics in large datasets with little need for labor. This can help breeders evaluate experiments more effectively, leading to the identification of potential new cultivars in a shorter timeframe (Haque et al., 2021).

In the present study, the InceptionResNetV2 architecture demonstrates higher precision, better accuracy, and specificity for most of the variables analyzed. The precision for this network was superior to 93 % in all variables. The EfficientNetB3, although it registered higher TP values for groups, exhibited lower overall efficiency and greater computational demand. On the other hand, VGG16 showed lower effectiveness, especially in detecting shape, color, and defects, suggesting possible limitations in these contexts.

Another critical factor is that the challenge of automatic identification of defects and surface damage has always represented a difficulty in the classification of agricultural products (Sugawara et al., 2018). In this study, the InceptionResNetV2 architecture overcame this difficulty by achieving precision and accuracy greater than 98 %, thus contributing to the success in the classification of this architecture. These high degrees of precision of the architectures used in classifying tomato fruits can further improve the decision-making process in agricultural practices (Vasconez et al., 2020).

Strategies aimed at image acquisition and data analysis in agricultural environments can enhance practices related to tomato breeding, including fruit counting, yield estimation, detection of pathogens and diseases, and crop maturity classification.

The effective classification of tomato fruits through computational analysis, highlighting the superiority of the InceptionResNetV2 architecture, demonstrates the ability of CNNs to classify fruits based on specific characteristics. The sharpness in fruit coloration influences the metrics, as defective regions may present distinct color patterns, enhancing classification accuracy and providing additional information about fruit quality (Haque et al., 2021). Thus, the ease of distinction in the RGB (Red, Green, and Blue) matrix of the image concerning objects and their parts is a crucial element for success in detecting the evaluated classes (Jeong et al., 2020). These findings emphasize the importance of selecting an appropriate CNN architecture adapted to the specific task of classification. The insights gained from this study advance our understanding of tomato classification and provide valuable guidance for optimizing CNNs in broader agricultural product quality assessment applications. Future research may further explore fine-tuning strategies, and ensemble approaches to increase the efficiency and accuracy of tomato classification models. This indicates the potential for developing applications and devices for tomato classification, providing valuable support to breeders in various research endeavors.

This work concluded that there are differences between CNN architectures in terms of training time, number of epochs, and performance in different classification variables. The InceptionResNetV2 architecture stood out in accurately classifying tomato fruits as regards shape, group, color, and defects, showing high estimates for quality assessment parameters.

This work highlighted how image analysis combined with deep learning is a valuable alternative for enhancing the quality of tomato fruits, reducing subjectivity in analyses and decreasing the phenotyping time of the crop. With more precise and efficient phenotyping of tomato fruits, this can effectively prevent errors and increase efficiency in utilizing relevant data for future tomato breeding and other crop improvement efforts.

Data availability statement

The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.

References

Edited by

Publication Dates

History