Abstract

Automated image systems to characterize aquatic organisms improve research and enable fast response to environmental risk situations. In November 2015, a dam in Mariana City-MG (Brazil) collapsed and led to the disposal of mud tailings from the mining process to the Doce River. The accident resulted in several casualties and incalculable damage to surrounding communities and the environment. The mud increased water turbidity, an essential condition to the functioning of the image analysis systems, and directly affected the characterization of the organisms, making it impossible to distinguish copepods in the mud, due to the blurred outline. To get a quick response evaluating environmental situations, this work aimed to develop and test different algorithms characterizing and classifying copepods by their size (length and area) using in situ images acquired by the Lightframe On-Sight Keyspecies Investigation device. Field tests were carried out under different turbidity levels throughout the gradient observed in the coastal zone adjacent to the Doce River. The best algorithm reduced nearly 50% of the noise in some images when compared with manual treatment and led to 96% accuracy in measurement and counting. Semi-automated devices that perform post-processing corrections are suitable for fast environmental evaluation under high turbidity scenarios.

Keywords

Imaging, Zooplankton, Doce River, LOKI, Copepod

Share and Cite:

1. Introduction

The zooplankton assemblage comprises a diverse group of organisms that inhabit aquatic ecosystems around the world [1]. They play a key role in the organic matter transfer, from primary producers to higher trophic levels and, consequently, in the global carbon cycle [1]. Copepods are usually the dominant group in plankton assemblages and thus a proxy for the ecosystem status [2]. Most studies addressing copepod diversity and abundance still depend on the classical WP-2 cylindrical nets used for zooplankton sampling, a device that integrates the whole plankton assemblage through depth and is inaccurate to address vertical differences in the distribution of copepods in the water column. Additionally, the time required to analyze many samples under microscopy imposes constraints on long-term studies or large geographical areas. For this reason, other tools have been improved, such as several environmental imaging systems, which provide fast and accurate in situ measurements of the physical and biological properties of the ecosystem [3]. In the last three decades, a great number of technologies capable of recording zooplankton images have been developed. This includes portable laboratory equipment, such as ZooScan [4] and Flow Cytometer and Microscope (FlowCAM) [5], as well as in situ systems, such as “Underwater Vision Profiler” (UVP) [6], “Zooplankton Visualization System” (ZOOVIS) [3], and the In Situ Ichthyoplankton Imaging System (ISIIS) [7].

These instruments have been developed and used for automated particle analysis through the detection (binarization and segmentation), counting, and measurement of individual particles and planktonic organisms. The automatic methods, with their high acquisition potential, gather a great number of images, intensifying the analysis and treatment of image issues. The majority of these devices, however, are less suitable for the acquisition of images of plankton under high turbidity conditions since the fuzziness tends to increase. According to [3]-[8], the processing of images obtained from muddy waters, with high concentrations of debris, remains a great challenge. Highly turbid waters reduce light penetration and add noise from light refraction and attenuation, preventing the acquisition of high-quality images needed for accurate analysis. Second [10] in nearshore areas, a turbidity around 6.2 NTU yielded blurry images. The background on the images was much brighter, therefore making it harder to distinguish the particle of interest. When turbidity rose to 10.2 NTU, images were recorded. This turbid condition imposes a challenge during the cropping of regions of interest (ROIs) that should contain only a single and distinguishable organism at once, without losing any part of the target organisms. Hence, the number of particles in each image results in an inaccurate estimation of ROIs and numerous erroneously segmented objects. For instance, gelatinous plankton is particularly susceptible to fragmentation in multiple pseudo-organisms. This happens due to the uncertainty as to whether one marginal pixel belongs to the organism or actually to the background, as suspended solids in front of organisms represent a noise in the image.

An effective method to accurately identify and account for marine organisms, particularly copepods, would allow the reprocessing of samples stored in plankton collections, enabling new large-scale spatial and temporal projects to be carried out [3]. The development of copepod enumeration and classification algorithms would improve the data analysis process and, therefore, would enable long-term environmental monitoring of the plankton dynamics. Considering the advantages of the use of imaging systems as complementary to the traditional techniques, this study aimed to develop a procedure to reduce noise in images obtained in waters with a high quantity of suspended solids. On top of that, the main aim is to develop and test algorithms that automatically enumerate and measure copepods from images obtained in situ under low or high turbid waters.

2. Material and Methods

2.1. Study Site

A set of 1000 images produced in two sites with different suspended sediment concentrations was selected to test the noise-reduction algorithms. The images were obtained on November 27, 2015, two days after the collapse of the dam located in Mariana (MG), when the waste with a high concentration of suspended sediment reached the outfall of the Doce River. During the operations of the R/V H39 “Vital de Oliveira”, from the Brazilian Navy, images were recorded in situ through a Light Frame On-Sight Key Species Investigation (LOKI, Isitech, Bremerhaven, Germany). The hauls were vertical from near the bottom with a mesh of 200 μm and an acquisition rate of 20 frames per second. The hauls were carried out at two distinct sites along the plume of sediments in the coastal zone, throughout a gradient of turbidity. The first site (1) was located as close as possible to the outfall of the Doce River, under high turbidity and shallow copepod assemblage (~20 meters deep). The second site (2) was located far from the Doce River opening with less turbid waters and deeper copepod assemblage (~100 meters).

The vertical profile of turbidity (NTU) and suspended particle concentration (CPS, particles·L⁻¹) in the two sites were simultaneously addressed using a turbidity sensor attached to a Multiparameter Datalogger (Horiba) and a Moving Vessel Profiler (MVP) equipped with a Laser Optical Plankton Counter (LOPC). The turbidity sensor was down from the deck up to 20 meters on both sites 1 and 2, and the water turbidity was annotated. The MVP + LOPC was deployed from site 1 to site 2 with seven sequential launches to address the suspended particle concentration in the plume.

2.2. The Light Frame On-Sight Key Species Investigation (LOKI)

The LOKI is a system with 5 main parts: 1) A concentration net, with a mouth opening of 0.28 m² and a mesh aperture of 200 μm, 2) a computer with a solid-state driver to store the images, 3) a CTD with temperature, pressure, dissolved oxygen, and fluorescence sensors, 4) a Prosilica GC 1380H (AVT-Allied Vision Technologies, Canada) with the Pentax 2514-M lens and 5) a set of battery. The raw image has 1360 1024 pixels with a final resolution of 23 μm·pixel⁻¹. It has a high-power LED unit, synchronized with the camera’s exposure-shooting signal, which allows a fast shut-off time (55 μs) avoiding motion blurring that causes image distortion. In combination, it has an image channel 4 mm high (length = 31.3 mm, width = 20.75 mm, volume = 2.6 cm³), causing all the particles of the image to stay in focus. All images are stored in the LOKI solid-state drive, where they can be accessed for further analysis. The recording starts instantly after the power is on, resulting in many waste images gathered either before or after the actual haul. From all images gathered since the equipment started functioning on the ship deck, only those generated during the haul from near the bottom to the surface represent a valid set of images of organisms in the water column and, therefore, were selected for characterization and quantification of copepod populations. The distinction and separation of these images of interest were performed automatically according to the actual depth measured by the pressure sensor. Hence, the onset of the haul was defined as the instant of maximum pressure (Figure 1). The end of the haul, in contrast, includes safety procedures to bring the LOKI back to the ship deck. These procedures produce an oscillation in the equipment due to the waves at the surface that are captured by the pressure sensors and might bias the density estimation near the surface (Figure 1). All images produced during this oscillation, therefore, were removed to avoid counting the same particle twice.

Figure 1. Example of vertical profile of temperature (˚C) and pressure (mbar) during a LOKI haul. The red dotted square highlights the valid set of images obtained during the vertical haul from near the bottom back to the surface.

After sorting the valid set of raw images, we performed 1) automatic recognition of the region of interest (ROI) and cropping, 2) background removal, and 3) counting and measurement of organisms (Figure 2). A vignette is a smaller image cut from the original raw image and containing a prominent particle. To check if the counting and measurement steps are less accurate in places with high turbidity, thus biasing the estimation of copepod size and abundance, simultaneous manual and semi-automated procedures were compared. To perform the automated measurements of organisms surrounded by distinct amounts of suspended sediments, a subset of 1000 vignettes were randomly selected, after which all vignettes with a single copepod (singlets) were manually counted and sorted into a different folder. This subset of copepods was processed in the 16 different binarization algorithms available in the ImageJ software (2.3.0) [9]. The accuracy of each algorithm was calculated by the comparison of the counting made manually by the taxonomist and automatically by the ImageJ contour detection (function “Analyze particle”, ImageJ). The same approach was adopted for morphometric measurements of the copepod’s body (e.g., area, perimeter, and fitted ellipses). The abundance was calculated by the percentage of singlets correctly identified by the algorithms.

Figure 2. Steps of the treatment of the images. (A) Original vignette, a singlet of a copepod automatically cropped by the LOKI system; (B) Resulting binarization of the vignette exemplified by the “Default” algorithm (ImageJ); (C) Resulting contour detection algorithm used in the copepod counting and measurement; (D) Resulting fragmentation of copepod’s body after binarization by distinct algorithms, compared to the actual copepod contour (manual). The numbers in red represent the particles detected and counted.

To improve the accuracy of copepod enumeration and measurement, three additional algorithms based on convolution matrices (herein named algorithms 2 to 4) were elaborated with different sequential steps of noise reduction done before the binarization. To evaluate the constancy and linearity of the results, the algorithms were tested with 3 subsets of 1000 randomly selected images from each site. Algorithm 2 includes an increase in contrast before the binarization step to accentuate details in the target ROI by applying a kernel to replace each pixel with a weighted average of the 3 × 3 neighborhood:

$\begin{array}{ccc}-1& -1& -1\\ -1& 12& -1\\ -1& -1& -1\end{array}$

The third algorithm, in contrast, reduced the exposition of the vignettes to check if noise could be removed before the binarization step to let only those pixels from a target ROI be left in the vignette. The reduction of exposition was achieved by reducing each pixel value by 25 (10% of the full 255 grayscale). Pixel values between 0 and 24 were set to zero. The fourth algorithm applies the “Statistical Region Merging” algorithm (Nock and Nielsen 2004) to highlight the copepod from the surrounding suspended particles (“noise”).

3. Results

3.1. Selecting a Valid Set of Raw Images in Vertical Haul, Recognition of ROI and Cropping

Each vertical haul lasts for nearly two minutes, starting approximately five minutes after the LOKI is powered on (Figure 3, red dotted rectangle). In site 1 (Figure 3A), the equipment was towed over nearly 20 meters (pressure ~260 mbar) for 2 minutes and a half, with an oscillating minute in the surface (8' - 9'). In contrast, the net in site 2 was quickly towed for 100 meters (5× longer) and only 2 minutes, with no oscillation near the surface. The tow speed of the first haul was estimated as 0.12 meter·sec⁻¹, while the second haul reached 0.9 meter·sec⁻¹.

Figure 3. Vertical profiles of temperature (˚C) and pressure (mbar) during the two LOKI hauls at the point with high turbidity (A), site 1) and low turbidity (B), site 2). The red dotted rectangles highlight the accepted vertical haul from which images were analyzed.

As expected, the water turbidity was extremely different between sites, mainly near the bottom where the plume of suspended sediments was more intense (Table 1). Therefore, the number of particles in suspension near site 1 (>5000) was 5 times higher than that at site 2 (<1000 CPS). In total, 40,375 vignettes were produced under high turbidity conditions, contrasting to only 21,664 under low turbidity conditions. By considering the different depths and operating times at each point, more vignettes were obtained in shallower and highly turbid waters (2018 vignettes·m⁻¹) than in deeper ones (217 vignettes·m⁻¹).

Table 1. Turbidity and particle concentration at the two sites.

The performance of the LOKI system to automatically recognize an ROI in a raw image and crop the corresponding vignettes was significantly affected by the turbidity and concentration of suspended particles (Table 2). From 1000 randomly selected images obtained under high turbid waters (Figures 4A-J), the majority (96%) were either badly segmented or non-segmented ROI (Figure 4B) that needed additional processing steps. In contrast, under less turbid waters (site 2), nearly 99% of the stored images were correctly segmented singlets (Figures 4K-R).

Table 2. Comparison of the performance of the LOKI segmentation algorithm in cropping vignettes (ROI) under high and low turbidity.

Figure 4. Examples of vignettes obtained under high turbid waters (A-J) and low turbid waters (K-R). All vignettes are on the same scale (scale bar = 1 mm).

The size of the ROI cropped reflected the smaller particles dominating site 1 with high turbidity. Nearly 60% of the random set represents vignettes with 1 to 10 kb (Figure 5A), while those from site 2 predominate in the 10 - 50 kb (Figure 5B). Full images resulting from failed segmentation have 1.2 Mb and were more frequent under high turbidity.

Figure 5. File size distribution (histogram) of the 1000 vignettes sorted from high turbidity (A) and low turbidity (B) areas.

The presence of suspended sediments can be seen in vignettes from site 1 as white dots around the copepods (Figure 6A). In contrast, copepods from site 2 with less suspended sediments have less “noise” and the resulting histogram of grayscale values is less skewed to the right (Figure 6B). Nevertheless, the presence of “noise” in one vignette is revealed by the differences between modes, rather than the mean pixel value (Table 3). Both pictures have very similar means (50 and 56), while the vignette with more suspended particles exhibits a surprisingly darker mode (17). In both cases, the standard deviation was higher than the mean.

Figure 6. Examples of in situ photography (grayscale) and the corresponding histogram showing more suspended sediments (“noise”) around the copepod in site 1 (A) than in the copepod in site 2 (B).

Table 3. Statistics of the above grayscale images (vignettes) from site 1 (with high “noise”) and site 2 (with low “noise”).

3.2. Background Removal and Binarization Performance during Counting and Measuring of Copepods

The resulting number of copepods according to the four automatic approaches revealed percentages of accuracy ranging between 90% and 507% (Table 4). A total of 678 and 1038 copepods were registered from a set of 1000 randomly selected images, respectively from high and low turbid waters. The abundance was either highly overestimated (>300%) or underestimated in turbid waters due to the presence of sediments when either the contrast is increased, or the adjacent mean pixel value (Statistical Region Merging) is used as pre-treatment. Despite underestimation, the most efficient method for counting organisms under highly turbid water was just binarization using “the first higher average” algorithm (or Default in ImageJ) without pre-treatment or by performing a slight reduction of exposition to remove some noise. The abundance under high turbidity was 5% to 10% underestimated (90% to 95%), while under low turbidity it was 13% overestimated (113%). Such results are observed in (Table 4) and the binarization evident in (Figure 7).

Table 4. Comparison of distinct pre-treatment performances in estimating the number of copepods (N) and the corresponding percentage (%) done automatically by the computer and manually by an expert (visual counting, 100%) on two sets of 1000 images obtained under low and high turbid waters. Note that some images can have more than one copepod, while others can have no copepods but other groups.

Figure 7. Results of pre-treatments before binarization on two vignettes (original image) from high and low turbidity waters.

The use of distinct binarization steps without pre-treatment to reduce the noise in turbid waters also resulted in different estimates of abundance according to the algorithm used. The total abundance was again underestimated in turbid waters while under less turbid ones it was occasionally overestimated (Table 5). The three best algorithms for high turbid waters (Huang, Intermodes, and Li) estimated nearly 88% of the actual abundance. As expected, higher efficiency (>95%) occurs when addressing the abundance of copepods from low turbid waters. There was no coincidence in algorithms with the best performance under low and high turbid waters, except for the “Intermodes”.

Table 5. Percentual of the actual abundance of copepods automatically recovered by the 16 algorithms, under high and low turbidity.

The surrounding particles that might be miscounted as a copepod were evidenced in vignettes with increasing contrast (Figures 8E-H and Figures 8M-P) but smoothed in vignettes with high degree of erosion (Figures 8A-D and Figures 8I-K). An apparent good contour of copepod, with full antennae preserved and proportionally less surrounding particles was achieved after the “Huang” binarization algorithm (Figure 8B).

From a dorsal view, the average area of a copepod (prosome + urosome + antennae) as manually estimated by visual analysis was equal to 312,639 μm² (mean = 591 pixels² ± 369 pixels²). Most of the automatic algorithms tested herein resulted in a good approximation of the actual size, as demonstrated by the high linear correlation (R² > 80%) (Table 6). The “Renyi Entropy” algorithm had the highest correlation (R² = 0.895, Figure 9M).

Figure 8. The 16 binarization methods ((A)-Default, (B)-Huang, (C)-Intermodes, (D)-IsoData, (E)-Li, (F)-MaxEntropy, (G)-Mean, (H)-MinError, (I)-Minimum, (J)-Moments, (K)-Otsu, (L)-Percentile, (M)-RenyiEntropy, (N)-Shanbhag, (O)-Triangle, (P)-Yen) available on ImageJ, compared to the manual method.

Table 6. Linear regression between the copepod’s area automatically estimated by the 16 algorithms and manually by an expert.

Figure 9. Relationship between the copepod’s body area as estimated from 16 automatic algorithms and the actual area manually measured. The linear regression equation and the correlation coefficient (R²) for each algorithm are shown inside the scatterplot.

4. Discussion

Considering the factors of operation time, local depth, and turbidity, the results point to water turbidity and haul speed as highly effective factors in the generation of multiple ROIs. To avoid the bias produced by operator procedures, the speed of a vertical haul, although subjected to sea conditions and the winch quality, should be kept as close as possible to 1 meter·sec⁻¹ regardless of local depth. Our results highlighted the potential use of pressure sensors while towing the net as an additional step in validating the haul.

The set of images used to address the effect of turbidity on the in situ analyses of the abundance and size of copepods indicates that sediments and suspended particles attenuate and scatter light, reducing image quality and limiting the performance of the equipment. Additionally, we found that increasing the number of particles in the background reduces the effectiveness of the histogram as a tool to set the threshold during binarization. The large amount of suspended sediments was expected to displace the mode toward the white value (255), but our results revealed the opposite. The presence of sediments attenuates the detection of the organism’s contour and leads to an increased surrounding margin of most vignettes obtained under high turbidity. A similar problem occurs when the target organisms are gelatinous or include translucent [10], parts in which the image quality is strongly affected and thus the recognition performance [11].

The presence of scattered material that is not the target object, like sediments and particles in the water under analysis is thought to impose additional challenges for automatic segmentation [3]. The higher variability in the resulting accuracy under high turbid waters compared with less turbid ones imposes additional steps in the quality checking. Some approaches tend to underestimate 10%, while others overestimate 500%. The overestimation in counting organisms was evident in algorithms affecting the image contrast when either a large amount of noise (sediment) is highlighted, or the target organism was eroded. The bad erosion of the thinnest parts of the body, like the antennae, led to multiple “chimaera particles” counted independently as one entire organism and resulted in 500% overestimation of the abundance obtained in their study a loss of the smaller and more translucent copepods [12].

In our result vignettes, the background is toned toward the black (0) and both the organism’s body and sediments to the white (255), creating a confusing contrast. The presence of chitin in the exoskeleton of copepods usually culminates in a well-identifiable body contour, but the appendages and antennae are generally not clear enough. Thus, more noise can be added to the automatic recognition of a copepod’s silhouette by performing an increase in the contrast, as observed in both the Statistical Region Merging. Differently, low contrast images usually result in underestimation of size and counting [13].

This additional noise as white dots derived from the sediments in the surrounding background was thus compensated by the large margin full of black pixels in vignettes from highly turbid waters. Our results reveal no significant difference between means and modes of vignettes from high and low turbid conditions. Despite more white dots, the large margin with more black pixels sets the mean and mode pixels back to the “natural” value. Thus, by using the smoothed histogram instead of the intra-class variance, the Intermodes algorithm resulted in the best estimation of copepod abundance in general. The widely used algorithm that searches for a minimum intra-class variance (Otsu) was significantly affected by the sediments (noise) in the background and seems to be not suitable for high turbid waters. Similarly to the use of a smoothed histogram, the search for a maximum entropy among the two classes—white and black—resulted in good estimations of organisms’ contour, particularly after combining the maximum entropy with the correlation entropy used in the Renyi Entropy algorithm [1]. The use of entropy as a measure of fuzziness is largely adopted mostly due to the high correspondence between binary and original image [14] [15] and our results revealed that entropy (Renyi, MaxEntropy, and Li) is better than the iterative intermeans algorithm under high turbid waters (Default, IsoData, and Intermodes). Exceptions to this rule resulted from Huang and Otsu algorithms, reinforcing the use of either local thresholding or iterative triclass approaches. Our results suggest that the use of the Otsu algorithm should be avoided on images from high turbid waters due to the imbalance of variances between classes [16] [17].

Automated and semi-automated zooplankton imaging systems have long been sought as part of a modern approach to monitoring the marine environment. The need for sensors capable of providing abundance and biomass data in high resolution over an extent of time has generated a growing effort to bridge the gap between different contemporary sampling methods in marine sciences [18]. The results acquired in this research highlighted the future trend in the improvement of the available plankton image processing technologies, particularly regarding better algorithms for counting and measurement.

Funding

This project has received funding from the European Union’s Horizon 2020 research and innovation program under Grant Agreement No 862428 (MISSION ATLANTIC). This output reflects only the author’s view and the Research Executive Agency (REA) cannot be held responsible for any use that may be made of the information contained therein.

Acknowledgements

The authors would like to express their gratitude to the Sea Studies Institute Admiral Paulo Moreira.

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References