One of the major challenges in the field of contemporary ecology is the documentation of ecosystem change over time. Among coastal marine biota, coral reefs are home to a unique hotspot of biodiversity. In the last decades, coral reefs are undergoing a severe decline worldwide1,2,3 due to a combination of ocean acidification4,5, and seawater warming6,7, their adverse impacts intensified by anthropogenic eutrophication and pollution8. These bring about both the decline in live reef cover and a decrease in coral species diversity1. Hence it is of paramount importance to monitor and document the rates of reef decline and identify the relative importance of stressors in each reef. An additional benefit of automated analysis of reef images is its potential as a tool to evaluate the long-term success of bioremediation projects of damaged coral reefs9 and reef protection measures.
The use of artificial intelligence (AI) to solve the time-consuming, tedious manual classification of coral species and determination of their abundance in real-time, is a Herculean task by itself due to the immense numbers of necessary images and their examination. Automated Deep learning (DL), a branch of AI, has the potential of solving this problem efficiently, and by far exceeds in terms of reliability and accuracy human reef documentation and monitoring. Like the human brain, the more data the computer learns under the DL mechanism, the better it performs at distinguishing among classes of coral species in the present application.
The highly-diverse coral reefs of the Gulf of Eilat (Aqaba) are of special scientific interest not only for being the most Northern reefs, but for benefiting the economy of neighboring communities in both Israel and Jordan. Although highly-diverse10, these reefs have suffered from a sequence of disasters, including a rare low tide in 197011, the recovery from which was impeded by repeated oil pollution following the closure of the Suez Canal between 1967 and 197512. The subsequent recovery of the gulf’s reefs was slowed down again by the episode of the cooling of the Gulf’s waters due to the eruption of Mount Pinatubo in 1991. That event caused erosion of the Gulf’s thermocline and led to deep mixing of its waters, enriching surface waters with nutrients. The resulting proliferation and subsequent decomposition of seaweeds smothered some 25% of the juvenile corals13. The anthropogenic eutrophication of the Gulf due to the increase in fish farming until the farm closing14, also reduced the transparency of its waters by increasing the concentration of phytoplankton15. These events, as well as the forthcoming Red-Dead Canal, call for frequent, detailed monitoring of any changes in the situation of the coral reefs of the Gulf, as are evident in the reduction of live coral cover and species biodiversity.
Since its early development, DL has been used in human facial discrimination16, handwriting recognition17, and forensic applications, such as fingerprint identification18 and voice analysis19.
DL has also been applied to various coral reef studies, in which it was used to discriminate among benthos types: sand, urchins, and three types of branched corals: brain coral, massive favids, and dead coral20, as well as to distinguish between healthy and bleached corals (see, e.g.,21,22). Shihavuddin et al.20 demonstrated the capability of DL to identify five coral genera from large assemblages of underwater images. In a recent study identification among branched and brain corals, was reported23.
In their research, Gómez Ríos et al.23 also distinguished among favids, brain coral, and three branched coral types: I, II, and III (an urchin, dead corals, and pavements based on an image mosaic).
Among DL studies at species level discrimination the following datasets are noteworthy:
The first dataset is the Pacific Labelled Corals (PLC) dataset, which contains 5,090 images from four locations: 1. Mo’orea (French Polynesia), 2. Northern Line Islands, 3. Nanwan Bay (Taiwan), and 4. Heron Reef (Australia). The PLC dataset contains: 251,988 annotations from these four locations made by a coral reef expert using a random point tool. In addition, six experts cross-annotated 200 images from each location21.
The second dataset is the Mo’orea Labeled Corals (MLC) dataset that includes five coral classes: Acropora, Pavona, Montipora, Pocillopora, and Porites, and four non-coral classes: crustose coralline algae, turf algae, macroalgae and sand. The MLC dataset contains over 400,000 human expert annotations of 2055 Mo’orea island survey images (https://vision.ucsd.edu/datasetsAll).
To apply DL to a coral reef dataset, there is no need to sample small fragments of corals for the subsequent tedious identification in the laboratory. DL enables direct classification of a large amount of photographs, in minimum time.
The novelty of the present study is the application of a fully-automated DL-based methodology to an imaging dataset, for the classification of 11 common coral types from the set of species reported in the Gulf of Eilat11. We applied DL to over 5,000 underwater images taken specifically by us from a shallow reef in the Gulf of Eilat, with the aim of documenting the distribution of the test types in the sample reef.
We demonstrated the power of DL, using shallow coral reefs in the Gulf of Eilat for comparison with its efficiency in other reef sites. That site allowed us to compare our DL data with the detailed manual coral surveys previously conducted on these reefs.
Corals and reefs
The nature and global importance of corals and the rapid destructive impact of Global Climate Change call for extensive and fast indexing and monitoring. Coral reefs cover less than 1% of the total area of the oceans and seas, yet they are the main repository of oceanic biodiversity (25% of all marine species)24. Extant hermatypic (reef-building) coral species are estimated at 3,23524, of which 100 were recorded in Eilat, Gulf of Aqaba, Northern Red Sea10. Hexacorals, based on six fold symmetry, or scleractinian corals are the most important hermatypic organisms25.
The decline of reefs leads to the collapse of their entire complex ecosystem depending on the calcium carbonate skeletons of the corals intricate reef structures, for food and shelter. Hermatypic corals are home to symbiotic algae living within their cells in specialized organelles, the symbiosomes. Called zooxanthellae by their first reporter, Brandt26, these greenish microalgae limited to sunlit shallow waters (~ 0–120 m), provide the corals with energy through their photosynthesis27,28, which also stimulates calcification29.
In most corals, the tentacles are retracted by day and spread out at night30 to catch plankton and other small organisms, while avoiding diurnal coral-feeding predators. This behaviour also optimizes the supply of oxygen for nocturnal respiration.
Unlike in shallow water, corals satisfy their energy needs in the deep water and dim light by zooplankton consumption, as an energy supplement to the algal light-limited photosynthetic products (see review by Dubinsky and Iluz28).
The photosynthetic activity of the zooxanthellae, raises the internal pH of the coral facilitating the skeletal calcification by “light enhanced calcification”31,32, a paradigm recently challenged by Cohen et al.33. Conversely, ocean acidification makes coral calcification more difficult.
The future of coral reefs
Coral reefs are exposed to many dangers because of global climate-change effects34,35, blast and cyanide fishing36, coral collection by the marine coral aquarium trade37, sunscreen use38, and light pollution interference with lunar cycle reproduction timing39. SCUBA diving pressure40. Anthropogenic eutrophication, acts synergistically with all the above listed detrimental factors, stimulating fast seaweed growth, that easily outcompete the slowly growing corals. The ensuing algal blooms, smother the coral colonies and prevent the settlement of juveniles41. Kaneohe Bay, a coral reef ecosystem at Oahu, Hawaii, illustrates the sensitivity of coral reefs to nutrient enrichment resulting from treated sewage disposal, leading to the reversible proliferation of seaweeds42. Fish cage farming released nutrients that affected the coral reefs in Eilat by causing deterioration in water quality due to eutrophication and by promoting seaweed growth and phytoplankton proliferation reducing the Gulf’s water transparency, thus reducing light necessary for symbiont photosynthesis, interfering with reproduction, increasing bio-erosion and epizootic infestation14.
Coral species differ in their tolerance to climate change and coral bleaching43. Corals experience bleaching as water temperature increases and causes loss of the zooxanthellae, and subsequently of live coral tissue, resulting in wide spread coral mortality followed by reef destruction. Unless the algal population recovers within weeks, the bleaching results in widespread reef mortality44. The ongoing increase in atmospheric carbon dioxide since the industrial revolution leads to ocean acidification or lowering of ocean pH, and affects corals negatively by shifting the balance from skeletal aragonite deposition toward its dissolution4. In addition, light pollution by artificial light, even at the weakest intensities45,46, can cause the disruption of coral reproduction that is controlled by lunar periodicity47,48. The planned Red Sea–Dead Sea Conveyance49 will cause a change in the regime of the Gulf currents50. Such a change could reduce the supply of larvae of corals and other reef organisms, and have a far-reaching deleterious impact on reef systems.
The real-time characteristics of DL tools are crucial for the rapid detection of reef damage allowing implementation of bioremediation measures. The DL characteristics are valuable tools assuring the health and long-term survival of the coral reefs in the Gulf of Eilat and worldwide.
Deep learning
The efficiency of the methodology of DL-based classification of coral species consists of efficient algorithms that reveal and extract common-patterns and features from large image datasets. Two popular schemes applied to coral reef data are the convolutional neural network (CNN)21 and deep belief net (DBN)21. A generic structure of CNN is a multi-layer, feed-forward, supervised neural network that recognizes objects from spatial-based images with little or no pre-processing. It consists of: (1) feature extraction (convolution layer); (2) distortion invariance (sub-sampling layer); and (3) classification (output layer). A DBN, consists of probabilistic models composed of multiple layers of random variables51.
Any coral-reef classification should consist of five main steps:
- 1.
Taking sufficient high quality underwater images.
- 2.
Detecting the chosen coral and cropping its image.
- 3.
Downscaling the cropped images to 200 × 200 pixels.
- 4.
Preprocessing the images to compensate for different imperfections (e.g., blurring, colour change, sunlight wave patterns, sky colour, nekton scattering effects etc.).
- 5.
Labelling each one of the coral species is labelled.
In case of an automatic model, Steps 3 and 4 may not be required.
Traditional machine learning methods need extensive domain expertise, human intervention, and are only capable of what they were originally designed for.
Additional works on growth modelling and quantification of morphological variation in coral types are due to, Kruszyński et al.52 and Chindapol et al.53. The former focused on the analysis of three-dimensional (3D) coral images scanned by X-ray tomography, and the latter, modelled the effects of flow on colony growth and shape, using analyzed advection–diffusion equations.
The increased interest in DL has also been recently reflected in the analysis of previously published coral datasets. Specifically, recent work22 has demonstrated the efficiency of neural networks and DL in distinguishing among various marine benthos components such as bare ground, seagrass meadows, algal cover, sponges, and identified some coral species. Additional recent work has shown the capability of neural networks and DL to distinguish among coral species and live corals from bleached colonies (see, e.g.,21,22).
Mahmood et al.22 combined CNN representations with manually obtained colony parameters. Their algorithms, based on image information, extract CNN images obtained from the deep VGGnet network with a 2-layer multilayer perceptron (MLP) classifier (trained on the MLC dataset). They achieved 77.9% accuracy.
Mahmood et al.54, reviewed the power of DL for machine monitoring of coral reefs.
Mahmood et al. (2016a)55 reported a decrease trend in coral density and species numbers in the reefs of Abrolhos Islands. Their analysis was based on CNN images obtained from VGGnet. They proved the reliability of their classifier on unlabelled coral image mosaics.
Mahmood et al. (2018)56 used CNN-based features and ResFeats to annotate corals and demonstrated the temporal changes in their association. They applied generic features from VGGnet and ResNet to classify corals and non-corals. They analysed unlabeled coral mosaics of three Abrolhos Island sites generating maps for the aforementioned mosaics.
Mahmood et al. (2020)57 applied computerized DL characterization of annotated kelp species. They presented an automatic hierarchical classification method to classify kelps in collected images. Their study summarises the considerable advantages of using deep residual networks (ResNets) over traditional, manual classifications of the same reefs. They showed that the sibling hierarchical training approach outperforms the traditional parallel multi-class classifications by a significant margin (90.0% vs. 57.6% and 77.2% vs. 59.0%) on Benthoz15 and Rottnest datasets, respectively. They used an application to study the changes in kelp cover over time for annually repeated AUV surveys.
Mahmood et al. (2020)58 evaluated how well features extracted from deep neural networks transfer to underwater image classification. They investigated the effectiveness of transfer learning of the ResFeats. They proposed applying new image features (called ResFeats) extracted from the different convolutional layers of a deep residual network pre-trained on ImageNet to the MLC, Benthoz15, EILAT and RSMAS datasets.
Gómez-Ríos et al.23 included more corals than previous studies by applying three CNNs: Inception v359, ResNet60, and DenseNet61 (see Supplementary Table 1).
Two datasets were analysed: Both the EILAT and RSMAS were analysed. These datasets comprise patches of coral images discriminating branched and massive colonies. The EILAT dataset contains 1123 images of eight classes (sand, urchin, branched type I, II, and III corals, brain coral, favid coral, and dead coral) and the RSMAS dataset contains 776 images of 14 classes, including 9 classes of the following scleractinian coral species: Acropora cervicornis, Acropora palmata, Diploria strigosa, Montastraea cavernosa, Meandrina meandrites, Montipora spp., Siderastrea sidereal, Colpophyllia natans (a boulder brain coral), and the colonial fire coral Millepora alcicornis (a species of hydrozoa with a calcareous skeleton). The other five classes included in the RAMAS dataset are non-coral species: Diadema antillarum is a sea urchin, Gorgonians are a genus of soft corals in the family Gorgoniidae, Palythoas palythoa is a genus of anthozoans in the order Zoantharia, and sponge fungus and tunicates are marine invertebrates of the subphylum Tunicata.
CoralNet (https://coralnet.ucsd/edu), conceived by Beijbom et al.9,62, uses deep neural networks for fully- or semi-automated image annotation. It also serves as a convenient, user-friendly collaboration platform. In early 2019, Williams et al.63 in a large study showed that the automated annotations for CoralNet Beta, produced benthic cover estimates comparable to controls gathered by human annotation.
Hoegh-Guldberg states that “CoralNet will allow the world’s scientists to quickly assess the health of endangered coral reefs at scales never dreamed of before”, in (https://blogs.nvidia.com/blog/2016/06/22/deep-learning-save-coral-reefs/).
The BenthoBox image labelling system for ecologists allows storing images of the dataset.
The software uses learning algorithms to recognise ‘tagged’ seabed features such as sand, algae, sponges and corals.
History of coral classification in the Gulf of Eilat
Traditional methods have been used in Gulf of Eilat research studies for coral classification since the pioneering work by Loya and Slobodkin10. Some 100 coral species were listed in their study.
Whenever confronted with doubt concerning the species of a certain coral underlying a transect, a small piece was sampled and manually identified by a taxonomist11, a tedious and destructive practice based on limited sample size.
These surveys were based on colour photographs taken by a camera with a flash attachment. Close-ups were taken by a Rolleiflex camera. A measuring tape was spread over the reef, and the divers recorded the projected length of all the organisms and substrates underneath the line transect to a resolution of 1 cm. Photographs were taken at 1 m intervals along the transect. This study was based on permanent transects photographed over a period of 20 months that yielded about 3,000 photographs of corals belonging to Loya’s11 list of approximately 100 species. However, the author noted that many cryptic species do not show up in the photographs.
Similar additional surveys were conducted following various disturbances that affected the coral reefs of the Gulf: the 1970 low tide64, the repeated oil spills65, the Pinatubo eruption of 199166, and the fish farming episode of 1995–200814.
Diver-based methods for classifying corals are almost impossible underwater, and require time-consuming expertise. Furthermore, coral pigmentation and morphology are plastic changes in response to environmental forcing functions such as light and current, eliciting wide phenotypic variability28,67. Ever since the National Monitoring Program (https://www.iuieilat.ac.il/Research/NMPmeteodata.aspx) of the Eilat reefs was initiated (2003), annual surveys by divers have been conducted.
The images are taken at a fixed area at six reef sites, namely the North Beach, the Dekel Beach, the Eilat Ashkelon Pipeline Co. Ltd. (EAPC), the coral reserve, the Interuniversity Institute for Marine Sciences in Eilat (IUI) marine laboratory, and Taba. Each site has fixed camera brackets for five cameras, and each of these takes four images. In this way, 20 pictures are taken at each site, and 120 pictures are taken for quantitative analysis of the changes at the various sites. Monitoring is done once a year in early summer. Corals were identified as far as possible at the species level, and were also classified according to functional groups. The results are presented graphically following statistical processing. Due the disintegration of the rock to which the cameras were attached, some new sites had to be added68.
Automated DL seeks to avoid these difficulties, profiting from the latest advances in computerized handling of large quantities of visual images9. Indeed, these novel developments have been increasingly applied to the survey and analysis of coral reefs in the studies listed in Supplementary Table 2. Since all previous surveys, as well as those of the current monitoring program of the Gulf of Eilat reefs, were based on the manual and visual analysis of large numbers of photographs, we present here a first example at using automated machine-based analysis for the red sea coral reef.
Source: Ecology - nature.com
