An objective grading method using a Computer Vision System (CVS) for mangoes is proposed. Red peel was selected using two types of color space values at chroma = 22 and hue angle = 52
°. Eighteen out of 25 fully-ripened fruits were graded as “excellent,” determined by the share of red area per fruit being in the range of 80% - 100%. In contrast, all green-mature fruits were graded as “fair,” where the share of red area per fruit was <30%. If the threshold for the share of the red area on the peel is set between 10% (maximal green-mature fruits) and 18% (minimal fully-ripened fruits), automatic removal of green-mature fruits on a grading line is feasible. CVS was effective for nondestructively assuming anthocyanin concentration. A linear relationship between the natural logarithm of the concentration and hue angle was observed (y =
- 0.0542x + 7.83), with a correlation coefficient accuracy of 0.94 and root mean square error of 1.31 mg
·kg
-1. This result may be effective for the visualization of anthocyanin distribution on mango skin. The threshold for red peel can be in the range of 131 - 186 mg
·kg
-1. This suggests that the pigment concentration is usable as a universal threshold. This value is unaffected by conditions for image acquisition or color measurement (e.g., light source, sensor, filter, and optical geometry), unlike color space values as hue angle.
<i>Mangifera indica</i> L. Nondestructive Analysis Digital Camera Anthocyanin1. Introduction
Mango is a popular fruit worldwide; the total production of mangoes, mangosteens, and guavas was 43 Mt in 2013, which has linearly increased from approximately 1 Mt/y since 1993 [1] .
Many researches have attempted nondestructive evaluation of the internal qualities of mangoes using light sensing, for example, Soluble Solid Content (SSC) [2] - [8] , dry matter [6] - [9] , and other contents [2] [4] [5] .
External fruit quality, as determined by skin color, is as important as internal factors because it is the first aspect sensed by consumers. However, there are few studies on the evaluation of the external quality of mangoes. Malevski et al. [10] associated HunterLab color values with the ripening velocity of “Haden” mangoes. Jha et al. [11] associated SSC with color space values of “Dashehari” mangoes, measured using HunterLab colorimeter and by Multiple Linear Regression (MLR), Principal Components Regression (PCR), and Partial Least Squares Regression (PLSR) analyses. Kang et al. [12] monitored the change in maturity of “B74” mangoes on the basis of hue angle measured using a Computer Vision System (CVS). Moreover, Makino et al. [13] predicted anthocyanin concentration in mango peels using hyperspectral imaging combined with PLSR analysis.
Kikuchi et al. [14] reported that mangoes can be graded on the basis of the share of red area on the surface of the fruit; such a type of grading can be done with the naked eye. However, subjective methods may be unreliable as the threshold depends on an individual’s evaluation; this may cause human errors. Therefore, evaluating external fruit quality objectively is desirable.
In this study, an objective determination of the share of red area on the surface of mangoes (i.e., peel) was attempted using CVS in combination with a digital still camera. Some thresholds were proposed on the basis of color space values and pigment concentration in a Region of Interest (ROI) on the surface of the fruit, and the share of red area was calculated on the basis of the proposed thresholds using a mathematical procedure. CVS was found to be usable for determining anthocyanin concentration in mango peels. Anthocyanin is responsible for the red color of this fruit [15] . A scientifically-based calculation method to select the red area of mango peel is proposed in this study.
2. Materials and Methods2.1. Samples
Twenty-five fully-ripened and 15 green-mature mangoes (Mangifera indica L., cv. “Irwin”) were harvested on July 14, 2014 at a farmland in Yaese (Okinawa Prefecture, Japan; 26˚09'43.5"N, 127˚42'53.1"E). This cultivar show brilliant red color on the peel around ripe stage. Typical appearance of fully-ripened and green-mature fruits were shown in Figure 1. After harvest, the samples were transported to a laboratory within 1 d at ambient temperatures. Samples consisting of 40 fruits were stored at a constant temperature and humidity [27˚C, 90% (RH)] for 1 d for subsequent experiments.
2.2. Shooting Samples Using CVS
CVS was used for shooting the samples. A USB camera (FMVU-13S2C-CS, Serial #: 13088045, Point Grey Research Inc., Richmond, British Columbia, Canada) with lens (13FM06IR, Tamron Co., Ltd., Saitama, Japan) was fixed to the central pole of a color viewer (PIAS IS-500, Sugiura Laboratory Inc., Tokyo, Japan). The lower
Typical appearance of (left) green-mature and (right) fully-ripened mango fruits used in the present study.
side of the lens was 187 mm above the top of the sample. This color viewer was equipped with six 10 W fluorescent lamps with a color temperature of 6500 K (D65) as the light source. The angle between the camera lens and lighting source axis was approximately 45˚, the same as that used by Mendoza et al. [16] .
To determine the thresholds to select red peel, the following experiment was conducted. sRGB images of a mango were acquired before and after ROI sampling. The images were acquired in bitmap format by FlyCapture2 ver. 2.4.3.11 (Point Grey Research Inc.). The shutter was set at 30 ms and the white balance (red and blue) was 530. By comparing both images shot before and after ROI sampling, the mean sRGB value of the ROI (a square piece of peel measuring 30 × 30 mm) was calculated using MATLAB ver. 8.3.0.532 (MathWorks Inc., Natick, Massachusetts, USA). This value was used for subsequent mathematical analyses. The ROIs were subjectively classified into three red, yellow, and green. Then, a square piece of the peel equivalent to the ROI sampled was used for measuring anthocyanin concentration.
2.3. Calculation of Color Space Values of the ROI
The conversion from sRGB to device-independent L*a*b* cannot be directly done using a standard formula [17] . Therefore, an equation for the calibration of the camera was made. When sRGB values are transformed to XYZ values, the following formula [16] can be used:
where sR, sG, and sB are the intensities of light on red, green, and blue colors acquired using CVS; X0, Y0 and Z0 are values of the Commission Internationale de l’Éclairage (CIE) XYZ color space before being calibrated using a standard color chart.
ColorChecker Classic (X-rite, Inc., Grand Rapids, Michigan, USA) was captured using the same camera. X0Y0Z0 values were calibrated to the correct XYZ values supplied by the maker of the standard color chart using quadratic equations, referred in the literature [17] .
The values for elements in the matrix M were calculated by MLR using the Unscrambler X (ver. 10.3.39870.111, CAMO Software AS, Oslo, Norway). CIE L*a*b* values were calculated from the calibrated XYZ values using the same equations as those used by Papadakis et al. [18] . Chroma and hue angle were calculated using a* and b*, according to the study conducted by Kang et al. [12] . Then, hue angle (radian) was transformed to degree (˚) by multiplying 180/π.
2.4. Calculating the Share of Red Area on the Peel
The share of red area on the peel was calculated using the Image Processing Toolbox complemented in MATLAB. First, background was removed from the image of a fruit. The number of pixels (A) on the fruit was then counted. The image without the background was also used for counting the number of pixels on the red area (B) using the thresholds. The thresholds to define the red area were determined by plotting color space values of the ROI on chroma-hue angle coordinates. The share of red area [S (%)] on the fruit was calculated using the following equation:
2.5. Calculating the Share of Red Area on the Peel
Anthocyanin concentration in the sampled peel was measured according to the method proposed by Nagata et al. [19] . Formic acid (5%) was used as the solvent for anthocyanin extraction. The absorbance of the extract at 530 nm was converted to the mass of delphinidin (molar extinction coefficient: 2.154 × 104 L∙mol−1∙cm−1), a type of anthocyanin. This value was expressed in fresh weight basis. The relationship between anthocyanin concentrations and color space values was investigated.
3. Results
The matrix M to convert X0Y0Z0 to XYZ used in Equation (2) was determined by MLR as follows:
Color space values of the ROI plotted on chroma-hue angle coordinates are shown in Figure 2; ranges of red coloring were determined using the thresholds at chroma = 22 and hue angle = 52˚. According to the results, two color space values were needed to select the red color. Hue angle was effective for determining the difference between red and yellow. However, the value of green peel was in a wide range and chroma was thus needed for selecting green.
Typical selection results of red color pixels are shown in Figure 3 as binary images. The shares of red pixels in a whole fruit were calculated by dividing the number of pixels for red peel areas in a binary image by the number of pixels for the complete area in a binary image [Equation (3)]. The images expressed as Ew and Er were of a fully-ripened fruit, and those as Fw and Fr were of a green-mature fruit. The difference in the share of red areas between the two fruits was apparent, according to the calculated results.
In this study, 40 fruits were graded on the basis of the share of red area reported by Kikuchi et al. (2011) calculated using thresholds determined on chroma-hue angle coordinates (Table 1). Eighteen fully-ripened fruits of 25 were graded as “excellent.” In contrast, all green-mature fruits were graded as “fair”. The minimum value
Color space values of the region of interest on the surface of mangoes. Circles, triangles, and squares denote red, yellow, and green regions, respectively. Ranges of red were determined using the thresholds at chroma = 22 and at hue angle = 52˚
Calculation of share by the red area on the peel of a mango using color space values measured using a computer vision system. Ew and Fw denote binary images expressing complete areas of fully-ripened and mature green fruits, respectively. Er and Fr denote binary images expressing areas of red peel selected on the basis of the thresholds at chroma ≥22 and hue angle ≤52˚. The shares of red areas were calculated by dividing the number of pixels for red peel areas in a binary image by the number of pixels for the complete area in a binary image: (Er/Ew) × 100 = 91% (Excellent); (Fr/Fw) × 100 = 10% (Fair)
Grading of mangoes on the basis of the share of red area on the peel. Twenty-five fully-ripened and 15 green-ma- ture mangoes were graded using thresholds on the basis of a chroma-hue angle coordinate. “Excellent”, “good”, and “fair” were used to denote the external quality of the fruit in which the shares of red area were in the ranges of 100% - 70%, 69% - 30%, and 29% - 0%, respectively
Fully-ripened
Green-mature
Excellent
Good
Fair
Minimum value (%)a
Excellent
Good
Fair
Maximum value (%)a
18
5
2
18
0
0
15
10
aMinimum and maximum values (%) of the share of red area on the peel of fully-ripened and green-mature mangoes, respectively.
for fully-ripened fruits was 18, which was higher than the maximum value of 10 for green-mature fruits. This suggests that the share value of the red area is useful for removing green-mature fruits on a grading line.
The relationship between hue angle and anthocyanin concentration in mango peels (ROI) is shown in Figure 4. A linear relationship was observed between both variables. This suggests that CVS is effective for nondestructively assuming anthocyanin concentration. The ROI of the green area, however, did not fit the relationship. The distribution of the concentration of the pigment on the surface of a fully-ripened fruit was visualized as a two-dimensional map (Figure 5). This image may be useful for objective confirmation of the distribution of the strength of redness on the fruit surface.
4. Discussion
A method to objectively grade mangoes using images acquired by CVS has been proposed in the present study. Some thresholds to select pixels equivalent to red peel are proposed. When the surface color of mangoes was classified into red, yellow, and green, two types of variable were needed to determine the range for the red color. Hue angle was ineffective for determining the difference between red and green peels because the green ones were in a wide range (Figure 2). This value is decreased with the color change from green to red; therefore, the value for red peels may be low. However, when the peel of a green-mature fruit includes high anthocyanin concentrations, the color is close to purple and hue angle is low. Chroma was also needed to determine the difference
Relationship between hue angle and anthocyanin concentration in mango peels (region of interest). Circles and triangles denote the measured values for red and yellow peels, respectively. The solid line (y = ?0.0542x + 7.83) denotes the linear regression line at a correlation coefficient accuracy of 0.94, root mean square error of 1.31 mg∙kg<sup>−1</sup>, and p value of <0.0001. Anthocyanin concentration was expressed in fresh weight basis
Distribution of anthocyanin on mango peels estimated using a computer vision system. The scale bar denotes anthocyanin concentration (mg∙kg<sup>−1</sup>) expressed in fresh weight basis
between red and green peels. Chlorophyll a and b absorb light at 427, 429, and 660 nm [20] . CIE designated that red (sR) and blue (sB) are determined using reflectance at 700 nm and 435.8 nm, respectively. These findings suggest that chroma as the strength of a color was decreased by reducing the strength of reflection from a sample including chlorophyll as green-mature fruits. This phenomenon may be the reason that chroma was effective for determining the difference between red and green peels.
According to the graded results, a gap was observed between the minimum values of the share for fully- ripened fruits and the maximum values for green-mature ones (Table 1). This suggests that the threshold set between the minimum and maximum values is effective to remove green-matured ones that may have slipped into fully-ripened ones.
Figure 4 shows that CVS is used for determining anthocyanin concentrations in the peel of fully-ripened- mangoes. Makino et al. [13] reported that a hyperspectral camera can be used for the same objective at a correlation coefficient accuracy of 0.94 and root mean square error of 1.31 mg∙kg−1. In the present study, the concentration of anthocyanin was more precisely assumed using CVS than using the hyperspectral camera. The accuracy, however, may be affected by system devices and optical geometry. The distance between the lens and a sample in the present study was shorter by approximately 110 mm than that reported in a previous study by Makino et al. [13] . The data accuracy improves with the strength of the ray. Reduction in distance in the present study might improve accuracy for assuming anthocyanin concentrations. According to the study conducted by Makino et al. [13] , prediction accuracy for anthocyanin concentrations in the peel of green-mature fruits using a hyperspectral camera was affected by chlorophyll a concentration on the basis of the analytical results obtained on using PLSR analysis. This may be caused by the chlorophyll a compound seriously affecting the intensity of the ray reflected from the fruit. This suggests the reason why the ROI data of green-mature fruits did not fit the equation presented in Figure 4.
Grading results for fully-ripened fruits using 151 mg∙kg−1 of anthocyanin concentration (=52˚ of hue angle) agree with the results presented in Table 1. According to the result in Figure 4, the threshold for red peel may be in the range of 131 - 186 mg∙kg−1. This may represent scientific evidence of a threshold for determining red peel and suggests that the pigment concentration is usable as a universal threshold. This value is unaffected by the shooting conditions of light source, sensor, filter, or optical geometry, unlike color space values as hue angle.
In the present study, the thresholds were subjectively determined; however, the share of red area was objectively calculated. In the conventional method, all the processes to determine the grade of a fruit are subjectively treated [14] . This suggests that the method proposed here progresses the grading method toward an automatic one. The thresholds and linear equation to estimate anthocyanin concentration may also be usable for grading further varieties of red-colored mangoes other than “Irwin,” such as “Tommy Atkins” and “Heidi” [15] .
The time required to acquire an image and the cost to purchase a device for CVS are 1/4000th and 1/100th of a hyperspectral camera, respectively. This contributes to high efficiency of fruit selection on grading lines, indicating that CVS can be easily and practically used at grading lines.
Acknowledgements
Financial support was provided to our research group by the Japan Society for the Promotion of Science Grant-in-Aid for Scientific Research (A) 25252045.
Cite this paper
Yoshio Makino,Kenjiro Goto,Seiichi Oshita,Akari Sato,Masato Tsukada, (2016) A Grading Method for Mangoes on the Basis of Peel Color Measurement Using a Computer Vision System. Agricultural Sciences,07,327-334. doi: 10.4236/as.2016.76033
NOTESReferencesFAO (2015) FAO Stat. http://faostat3.fao.org/home/ETanabe, T., Akinaga, T., Kohda, Y., Kouno, Y., Maeda, H. and Mizuno, T. (1995) Internal Quality Measurement of Tropical Fruits by Near Infrared Spectroscopic Technique. Journal of the Society of Agricultural Structures, Japan, 26, 17-24.Tanabe, T., Akinaga, T., Kohda, Y., Kawasaki, S., Kouno, Y., Maeda, H., Mizuno, T. and Aoki, H. (1996) Internal Quality Measurement of Tropical Fruits by Near Infrared Spectroscopic Technique (Part 2)—Using the Specific NIR Absorption Wavelength on Water, Cellulose and Sugar. Journal of the Society of Agricultural Structures, Japan, 27, 29-34.Schmilovitch, Z., Mizrach, A., Hoffman, A., Egozi, H. and Fuchs, Y. (2000) Determination of Mango Physiological Indices by Near-Infrared Spectrometry. Postharvest Biology and Technology, 19, 245-252.
http://dx.doi.org/10.1016/S0925-5214(00)00102-2Hasbullah, R., Tanabe, T., Tanaka, M. and Akinaga, T. (2002) Non-Destructive Prediction of Internal Quality of Heat-Treated “Irwin” Mango by Near Infrared Spectroscopy. Journal of the Society of Agricultural Structures, Japan, 33, 155-166.Saranwong, S., Sornsrivichai, J. and Kawano, S. (2004) Prediction of Ripe-Stage Eating Quality of Mango Fruit from Its Harvest Quality Measured Nondestructively by Near Infrared Spectroscopy. Postharvest Biology and Technology, 31, 137-145. http://dx.doi.org/10.1016/j.postharvbio.2003.08.007Subedi, P.P., Walsh, K.B. and Owens, G. (2007) Prediction of Mango Eating Quality at Harvest Using Short-Wave Near Infrared Spectroscopy. Postharvest Biology and Technology, 43, 326-334.
http://dx.doi.org/10.1016/j.postharvbio.2006.09.012Subedi, P.P. and Walsh, K.B. (2011) Assessment of Sugar and Starch in Intact Banana and Mango Fruit by SWNIR Spectroscopy. Postharvest Biology and Technology, 62, 238-245. http://dx.doi.org/10.1016/j.postharvbio.2011.06.014Guthrie, J. and Walsh, K.B. (1997) Non-Invasive Assessment of Pineapple and Mango Fruit Quality Using Near Infrared Spectroscopy. Australian Journal of Experimental Agriculture, 37, 253-263. http://dx.doi.org/10.1071/EA96026Malevski, Y., Gómez, B.L., Peleg, M. and Silberg, M. (1977) External Color as Maturity Index of Mango. Journal of Food Science, 42, 1316-1318. http://dx.doi.org/10.1111/j.1365-2621.1977.tb14486.xJha, S.N., Chopra, S. and Kingsly, A.R.P. (2007) Modeling of Color Values for Nondestructive Evaluation of Maturity of Mango. Journal of Food Engineering, 78, 22-26. http://dx.doi.org/10.1016/j.jfoodeng.2005.08.048Kang, S.P., East, A.R. and Trujillo, F.J. (2008) Colour Vision System Evaluation of Bicolour Fruit: A Case Study with “B74” Mango. Postharvest Biology and Technology, 49, 77-85. http://dx.doi.org/10.1016/j.postharvbio.2007.12.011Makino, Y., Isami, A., Suhara, T., Goto, K., Oshita, S., Kawagoe, Y., Kuroki, S., Purwanto, Y.A., Ahmad, U. and Sutrisno (2015) Nondestructive Evaluation of Anthocyanin Concentration and Soluble Solid Content at the Vine and Blossom Ends of Green Mature Mangoes during Storage by Hyperspectral Spectroscopy. Food Science and Technology Research, 21, 59-65. http://dx.doi.org/10.3136/fstr.21.59Kikuchi, K., Nakamura, T. and Taira, E. (2011) Problems and Prospects Associated with the Mango Producing District in Okinawa—Towards Tropical Fruit Tree Branding. Agriculture & Forestry Statistics Publishing Inc., Tokyo.Berardini, N., Fezer, R., Conrad, J., Beifuss, U., Carle, R. and Schieber, A. (2005) Screening of Mango (Mangifera indica L.) Cultivars for Their Contents of Flavonol O- and Xanthone C-Glycosides, Anthocyanins, and Pectin. Journal of Agriculture and Food Chemistry, 53, 1563-1570. http://dx.doi.org/10.1021/jf0484069Mendoza, F., Dejmek, P. and Aguilera, J.M. (2006) Calibrated Color Measurements of Agricultural Foods Using Image Analysis. Postharvest Biology and Technology, 41, 285-295. http://dx.doi.org/10.1016/j.postharvbio.2006.04.004León, K., Mery, D., Pedreschi, F. and León, J. (2006) Color Measurement in L*a*b* Units from RGB Digital Images. Food Research International, 39, 1084-1091. http://dx.doi.org/10.1016/j.foodres.2006.03.006Papadakis, S., Abdul-Malek, S., Kamdem, R.E. and Jam, K.L. (2000) A Versatile and Inexpensive Technique for Measuring Color of Foods. Food Technology, 54, 48-51.Nagata, M., Kobayashi, T., Tallada, J., Toyoda, H. and Goto, Y. (2006) Study on Anthocyanin Pigment Distribution Estimation for Fresh Fruits and Vegetables Using Hyperspectral Imaging. Journal of Science and High Technology in Agriculture, 18, 42-49. http://dx.doi.org/10.2525/shita.18.42O’Neil, M.J. (2006) The Merck Index. 14th Edition, Merck & Co. Inc., Whitehouse Station.