sensors Article A Flexible Autonomous Robotic Observatory Infrastructure for Bentho-Pelagic Monitoring † Jacopo Aguzzi 1,2 , Jan Albiez 3 , Sascha Flögel 4 , Olav Rune Godø 5,6, *, Endre Grimsbø 7,8 , Simone Marini 9 , Olaf Pfannkuche 10 , Erik Rodriguez 11 , Laurenz Thomsen 10,12 , Terje Torkelsen 5 , Javier Valencia 13 , Vanesa López-Vázquez 11 , Henning Wehde 7 and Guosong Zhang 7 1 Instituto de Ciencias del Mar (ICM-CSIC), E-08003 Barcelona, Spain; jaguzzi@icm.csic.es 2 Stazione Zoologica Anton Dohrn, 80122 Naples, Italy 3 Kraken Robotik, 28197 Bremen, Germany; jalbiez@krakenrobotik.de 4 GEOMAR, Helmholtz Centre for Ocean Research Kiel, 24148 Kiel, Germany; sfloegel@geomar.de 5 Metas AS, N-5106 Bergen, Norway; terjet@metas.no 6 Norwegian Research Centre NORCE, N-5838 Bergen, Norway 7 Institute of Marine Research, N-5817 Bergen, Norway; endreg@hi.no (E.G.); henningw@hi.no (H.W.); guosong.zhang@hi.no (G.Z.) 8 The Arctic University of Norway, N-9037 Tromsø, Norway 9 CNR-ISMAR, 19032 La Spezia, Italy; simone.marini@sp.ismar.cnr.it 10 iSeaMC, 28759 Bremen, Germany; o.pfannkuche@iseamc.com (O.P.); l.thomsen@iseamc.com (L.T.) 11 DS LABS, 01015 Vitoria-Gasteiz, Spain; erodriguez@deustosistemas.net (E.R.); vlopez@deustosistemas.net (V.L.-V.) 12 Jacobs University, 28759 Bremen, Germany 13 Deusto Sistemas, E-01015 Vitoria-Gasteiz, Spain; javi.valencia.m@gmail.com * Correspondence: olavr@metas.no; Tel.: +47-41479176 † This paper is an extended version of Wehde, Thomsen, Pfannkuche, Albiez, Godø, Torkelsen, Valencia, Rodriguez, Lopez, Marini, Grimsbø, Zhang, and Aguzzi, 2019 IMEKO TC-19 International Workshop on Metrology for the Sea Genoa, Italy, 3–5 October 2019.  Received: 15 February 2020; Accepted: 10 March 2020; Published: 13 March 2020  Abstract: This paper presents the technological developments and the policy contexts for the project “Autonomous Robotic Sea-Floor Infrastructure for Bentho-Pelagic Monitoring” (ARIM). The development is based on the national experience with robotic component technologies that are combined and merged into a new product for autonomous and integrated ecological deep-sea monitoring. Traditional monitoring is often vessel-based and thus resource demanding. It is economically unviable to fulfill the current policy for ecosystem monitoring with traditional approaches. Thus, this project developed platforms for bentho-pelagic monitoring using an arrangement of crawler and stationary platforms at the Lofoten-Vesterålen (LoVe) observatory network (Norway). Visual and acoustic imaging along with standard oceanographic sensors have been combined to support advanced and continuous spatial-temporal monitoring near cold water coral mounds. Just as important is the automatic processing techniques under development that have been implemented to allow species (or categories of species) quantification (i.e., tracking and classification). At the same time, real-time outboard processed three-dimensional (3D) laser scanning has been implemented to increase mission autonomy capability, delivering quantifiable information on habitat features (i.e., for seascape approaches). The first version of platform autonomy has already been tested under controlled conditions with a tethered crawler exploring the vicinity of a cabled stationary instrumented garage. Our vision is that elimination of the tether in combination with inductive battery recharge trough fuel cell technology will facilitate self-sustained long-term autonomous operations over large areas, serving not only the needs of science, but also sub-sea industries like subsea oil and gas, and mining. Sensors 2020, 20, 1614; doi:10.3390/s20061614 www.mdpi.com/journal/sensors Sensors 2020, 20, 1614 2 of 17 Keywords: benthic and pelagic monitoring; image processing; acoustics; crawler; docking station; cabled observatories; fuel cells; ecosystem component classification 1. Introduction New technology has always been the driver for breakthroughs in marine ecology. When ecosystems exploration and monitoring are enforced in the deep-sea realms, spectacular development in robotics emerges [1]. Now, mechatronics and the Internet of Things paradigm are setting the basis for the development of improved platform mobility and autonomy in navigation and sampling [2,3]. In this scenario, cabled observatories and their infrastructures are integrated with spatial mobile networks where the fixed nodes coordinate the functioning of movable platforms (e.g., docked Remotely Operated Vehicles (ROVs) and Autonomous Underwater Vehicles (AUVs)) in-between [4]. Such development will promote the spatially extensive and temporally intensive monitoring of ecosystem biodiversity and functioning in relation to, for example, the rate and quantity of energy exchanged within the food web and resulting biomass production with special reference to management services for exploited fish stocks. We also note that the turnover of marine life in deep-sea benthic communities is slow with sometimes extremely long recovery time [5]. Any sampling tool and associated sampling design must therefore be planned to minimize the damaging risk on the fragile parts of the ecosystem. The overall goal is the reduction of vessel-assisted operations, which have very high operating costs, by enhancing platforms operability and autonomy in situ. In this framework, optoacoustic imaging technologies play a central role in determining richness and the relative abundance of species within a wide range of ecological sizes (e.g., [6]), delivering at the same time life history trait data relevant to management and conservation (e.g., phenology such as activity rhythms, growth and reproduction cycles, ecological interactions, bioturbation) [7]. In this paper, we present the technological solutions implemented to enable remote, autonomous, and long-lasting monitoring of the deep-sea ecosystem. Demonstration and development are done in a Norwegian coastal deep-sea canyon with cold water coral (CWC) at approximately 250 m depth within the framework of the project, Autonomous Robotic Sea-Floor Infrastructure for Bentho-Pelagic Monitoring (ARIM). This project aims to deliver new advanced robotic products for deep-sea bentho-pelagic monitoring on representative ecological scales (i.e., from a few meters around cabled nodes to hundreds of meters around). A new prototype of a mobile crawler and cabled observatory with an integrated docking station, from which it can be operated, has been designed. At the same time, the autonomy of the system with novel fuel cell technology is under development. Our vision is that fuel cell technology will expand the use of crawlers in self-sustained long-term autonomous operations, for science as well as for industry purposes, when the umbilical is eventually removed. The first step in development is centered on advancing crawler autonomy under controlled conditions (i.e., with a tether to explore the neighborhood of the stationary instrumented garage). A combination of visual and acoustic sensing along with standard oceanographic sensors has been conceived to support remote, high-frequency, and continuous multiparametric biological and environmental data collection and on-board data processing. A core activity is therefore to develop artificial intelligence (AI) algorithms that facilitate processing of the images required for automated adaptive driving as well as for the identification of biological and habitat characteristics (i.e., counted individuals and species local abundances, coral growth). This development aim at supporting science and monitoring services without backing from expensive ship time. The objectives of this paper are to describe the technology and ecological basis for the development of ARIM as well as demonstrate its applicability for research and industry through generated results; the novelty of our work is linking the various technologies together in an interdisciplinary fashion. Sensors 2020, 20, 1614 3 of 17 2. Materials and Methods Cabled marine observatories are developed worldwide and will eventually become an important component of global marine monitoring networks [1,8–10]. In Section 2.1, we present the Norwegian deployment area and already existing cabled observatory infrastructures, within which ARIM is being developed. Then, we describe the technology components and the associated tasks (Sections 2.2 and 2.3). This includes the design of the ARIM concept, and how all components are tailored for the project purpose and turned into a coherent integrated product for research and monitoring. 2.1. The Hosting Infrastructure and the Development Area The Norwegian continental shelf hosts some of the richest CWC reefs in the world, which act as shelters and nurseries for a number of species of economic interest [11]. To better understand their biological and physical settings, cabled observatory nodes were installed in the Hola trough (Figure 1), a deep-sea area with CWC mounds surrounded by complex oceanographic conditions, off the Norwegian coast [8,12]. This was stage one of the development of the Lofoten-Vesterålen (LoVe) ocean observatory [13], which has been in operation since 2013 [8]. Sensors 2020, 20, 1614 4 of 18 Figure 1.1. Overview Overviewofofthe study the study area areawhere where thethe Lofoten-Vesterålen Lofoten-Vesterålen (LoVe) observatory (LoVe) observatoryis located. (A) is located. The bathymetric map of the canyon area with the extended cabled transect with the (A) The bathymetric map of the canyon area with the extended cabled transect with the nodes (black dots) nodes (black dots) numbered 1–7 1–7 and and connected connectedby bya atelecommunication telecommunication cable cable (continuous (continuous line). line). Node Node 1 is1located is located in in the the deeper deeper partpart of the oftrough the trough (marked (marked with southernmost with southernmost blue arrow). blue arrow). This is This is a central a central locationlocation for for Lophelia Lophelia reef mounds. reef mounds. The new The new Autonomous Autonomous Robotic Infrastructure Robotic Sea-Floor Sea-Floor Infrastructure for Bentho-Pelagic for Bentho-Pelagic Monitoring (ARIM) test (platform Monitoring ARIM) test platform will be locatedwillatbe located node at node 2 (northernmost 2 (northernmost blue arrow).blue (B) Aarrow). (B) A three- three-dimensional (3D) detailed(3D) dimensional representation of the area around detailed representation of thenode 1, wherenode area around the satellite 1, whereX-frame with X-frame the satellite video camera with is encircled video camera (inisred). Here (in encircled we red). collected Herethewefootage collectedused thetofootage establishused the artificial intelligence to establish (AI) the artificial procedures for intelligence (AI)later transfer into procedures forthe crawler later for on-board transfer into the image crawler autonomous for on-board processing. (C) Enlarged image autonomous view of the (C) processing. areas surrounding Enlarged view ofthe thenode where areas Lophelia reefs surrounding the nodeare schematized. where Lophelia reefs are schematized. Thean At full LoVestage, early infrastructure consists the involved of a high scientists voltage and (3 kV)realized engineers backbone thatpower theircable withgoals original fiber-optic were communication and several 240V, 50Hz connection points including subsea distribution unobtainable by only one of the partners. This became the basis for the wide international units (SDUs); see Section 3.2. partnership, The fullfrom as apparent system consists the list of several of authors nodes connected and institutions to in specialized therobotic infrastructure, with developments instrumentation and sensor technology for measuring environmental and ecology. Our motivation for this approach is based on three main principles: parameters and processes related to oceanography, geology, and biology. Measuring equipment and nodes are supplied by several 1. Collaboration manufacturers, between including science Metas. and industry The X-frame, creates part of the Metassubstantial research X-net [14] (Figure andalso 1, see development Section 3.2), (R&D) gains; has for several years been the basic central unit collecting data either from sensors installed on the 2. No or X-frame research or industry from sensors mounted organization alone at the satellite holds the platforms competence (Figure and knowledge 1). The X-frame is directly needed connectedto to thedevelop SDUs for usable power robotic tools for sustainable management of the marine environment and its and communication. resources; 3. Multidisciplinary and multi-nationality cooperation fosters new ideas and solutions that are applicable to an extended market of industrial maritime users. In the future, ecological monitoring will be requested from these industries to demonstrate the sustainability of their activity. Sensors 2020, 20, 1614 4 of 17 LoVe is constructed with a robust and flexible infrastructure that can accommodate additional observation/sampling platforms and sensors that meet different ecosystem monitoring requirements. The observatory network is a good example of joint science-industry-based ecosystem monitoring efforts, of relevance for many industries (i.e., petroleum, fisheries), although the objectives might be potentially conflicting. The observatory is based on technical requirements that meet the North Sea standard defined for the petroleum industry. This represents higher initial investments but lower operational costs in the long run. The experience so far has been positive with relatively stable operation of multiple sensor platforms including echosounders, oceanographic sensors, and high-definition (HD) cameras. Node 1 is located at a depth of ∼260 m, 20 km off the coast north of the Lofoten Islands (Norway) in the Hola trough (see Figure 1). This glacially-created trough substantially increases the continental shelf depth in a north-west to south-east direction. The location of node 1 is enclosed by two 100-m deep banks: Vesterålsgrunnen in the northeast and Eggagrunnen in the southwest. The trough has a diverse topography with sand wave fields of up to 7 m high, 10–35 m high ridges, and approximately 20 m high CWC mounds [15]. The CWC mounds are predominantly found in the south-eastern part of the trough at a depth of ∼260 m, just south of the Vesterålsgrunnen bank, and are mostly constituted by CWC Lophelia pertusa. A satellite with a camera (see Figure 1) provided data for the AI training during the first year of ARIM (see Section 2.2). The LoVe observatory is now under expansion and 7 nodes will cover the continental shelf and the shelf break, with depths ranging from about 100 m down to 2500 m (see Figure 1). Nodes 1, 3, 4, and 7 are connected to the backbone cable; node 5 and 6 are autonomous with the option for acoustic communication to backbone nodes. All nodes are now implemented, except node 2 and 6, but some have incomplete instrumentation and functionality. Data from this expansion will enhance understanding of the interaction between offshore Atlantic water and inshore coastal water. Further, transect data support in understanding of the impact of water circulation on the ecosystem in general and on harvested fish stocks, in particular, that spawn in this area [8]. The new ARIM platform with the crawler will be tested connected to node 2, west of the original observatory (see Figure 1). At an early stage, the involved scientists and engineers realized that their original goals were unobtainable by only one of the partners. This became the basis for the wide international partnership, as apparent from the list of authors and institutions specialized in robotic developments and ecology. Our motivation for this approach is based on three main principles: 1. Collaboration between science and industry creates substantial research and development (R&D) gains; 2. No research or industry organization alone holds the competence and knowledge needed to develop usable robotic tools for sustainable management of the marine environment and its resources; 3. Multidisciplinary and multi-nationality cooperation fosters new ideas and solutions that are applicable to an extended market of industrial maritime users. In the future, ecological monitoring will be requested from these industries to demonstrate the sustainability of their activity. 2.2. AI Developments for Image Processing While autonomy in space [16] and terrestrial robots have experienced significant increases in research and applied technologies, robots in the underwater domain are still mostly tele-operated, although this scenario is closest to space research [17]. Tethered crawler technology operated remotely through an Internet-based connection, has already been used in association with cabled observatory infrastructure [18,19]. The crawler is belt driven, which gives access to a great variety of habitats. When the crawler and its sensors are available via Ethernet communication, the piloting of the remotely operated vehicle is supported by online camera information. This combination facilitates an important fundament for the development of autonomy. Our new autonomous crawler solution now under testing, is equipped with camera and standard oceanographic sensors and is designed based on 10 years Sensors 2020, 20, 1614 5 of 17 of experience. While tethered crawlers have so far mainly been tele-operated, new monitoring needs are pushing toward fully self-sustained independent navigation and data collection. The fundament for this development is on-board data collection and processing of visual and bathymetry information in near real-time. This requires advanced navigation algorithms, which are now under development. Our solution includes a stepwise approach. The first step was to create supervised autonomous navigation, that allows the crawler to do more complex data gathering missions on its own, while its activities are monitored though cable communication. The next step is to operate the crawler fully autonomously without a cable. At the same time, autonomy in biological data collection is pursued to efficiently monitor megafauna (i.e., animals larger than 2 cm in size). In full operation, the system not only collects data but also extracts biological information on-board and presents results for the user. A major part of our development includes processing images in order to track and classify motile species of fishes, cephalopods, echinoderms, and crustaceans [20]; and in the case of long-term monitoring, establish information on activity patterns at diel and seasonal scales, as well as growth patterns of CWCs [4,7,18,21]. Below we describe the development from interactive to fully autonomous operation via a data-cabled connection for training the navigation and animals counting software. The specific technology and operational tasks include: • Create a docking interface for ROV free maintenance with an intelligent mobile seafloor monitoring system (crawler; [22] to provide independence from an infrastructure cable; • Drastically increase the operational area and capability of the crawler by using an extended fiber-optic cable and thus provide the option to operate without umbilical; • Construct a deep-sea fuel cell to provide energy for long-term self-sustained operations; • Improve observation capacities using state-of-the-art active acoustics and 3D camera technologies; • Implement routines for automatic organism classification and tracking and integrate this information with the crawler sensor data; • Improve integration of hardware/software components to provide an efficient user interface; • Demonstrate performance and develop training facilities for new users and markets. 2.3. The Targeted Ecosystem Functioning Components and Monitoring Approach Ecosystem functioning is the mode and rate of circulation of organic and inorganic matter among species within a community, and such studies require sensors deployed along benthic and pelagic environmental gradients [4,23]. Behavior and its spatiotemporal modulation cover an important role in that transference. Thus, when populations migrate at tidal, inertial, or day-night temporal scales, then they are active carriers of organic carbon into 3D water column seabed scenarios [24,25]. Their intermittent presence into different continental margins and abyssal depth strata influences the rate of their energy exchanges beyond classic benthopelagic and upwelling phenomena in a still poorly understood fashion. When high-frequency and multiannual image acquisition are coupled with concomitant environmental monitoring, solid cause–effect relationships can be established measuring the behavior of individuals in response to environmental cycles (i.e., the activity rhythms) and stochastic perturbations [10,21]. When that monitoring is replicated over observational platforms into a network, the validity of local observations can be extrapolated at larger spatial and ecological scales [1]. In this context, automatic image and data processing would fundamentally improve monitoring through the analysis of biological/environmental interrelated data complexly [1]. The envisaged ARIM development aims to target megafauna with imaging and acoustic technologies to measure behavioral rhythms of community components, including composition changes at both smaller (i.e., daily) and larger (i.e., seasonal, interannual) temporal scales. We aim to identify the best monitoring strategies and data processing routines to extrapolate platform data at larger spatial scales of the network, choosing the best multivariate statistic strategies to link the observed variability in animal abundances to variation in environmental variables. Sensors 2020, 20, 1614 6 of 17 3. Results: Progress of System Assembly and Data Production This section describes the basic technology on which ARIM was developed and explains in detail the hardware (Sections 3.1–3.3) and software (Sections 3.4 and 3.5) development done in the project and illustrates the outcome of the technology in terms of example results (Section 3.5). So far, we have experience with the various technologies separately. We now aim to merge cable-based observatory, mobile robotic seafloor technology images, and acoustic processing with modeling methods into one operational autonomous product (Figure 2). This required work along five avenues that in the end were combined into a coherent subsea monitoring tool. Sensors 2020, 20, 1614 7 of 18 Figure Figure 2. 2. TheThe ARIMbenthic ARIM benthicplatform platform(X-Net (X-Net® ® , Metas Metas AS, AS, Bergen, Bergen,Norway) Norway)withwiththethebattery battery driven driven Rossia Rossia crawler.(a) crawler. (a)and and (b) (b) show showRossia Rossiaequipped equipped with with cameras camerasandand oceanographic oceanographic sensors testedtested sensors on- board R/V on-board R/VAlkor Alkor in November in November 20192019 (a) and (a)inand laboratory (b). (c) shows in laboratory theshows (b). (c) bottomthe platform bottom (X-Frame platform and X-Node) (X-Frame with garage. and X-Node) with(see Figure(See garage. 3 for details3 about Figure X-Net® for details (Metas about AS,® Bergen, X-Net (Metas Norway)). AS, Bergen, During the field tests Rossia was equipped with the basic payload and remotely controlled Norway)). During the field tests Rossia was equipped with the basic payload and remotely controlled at 10 m depth through a surface buoy. at 10 m depth through a surface buoy. 3.1. TheThe garage Crawler has several operational cameras (see Table 1) and light installed for monitoring System docking and cable winch during operation. Under autonomous operation (without umbilical), the An early version of the crawler has been in operation in the Neptune observatory of Barkley pin-less connector can provide data communication between the infrastructure and the crawler when canyon (off Victoria, connected. B.C., Data, such Canada;plans, as mission fromcan2010 be[26,27]. Theto transferred robustness the crawlerand stability before demonstrate the mission start, andthat thisdata unit is readybytothe collected becrawler used incan routine monitoring be uploaded to the systems. Based infrastructure onthe after thecrawler Neptunehas experience, docked afteran improved mission completion. During training of the autonomous piloting software, the operator ARIM crawler version called “Rossia” (Figure 2) with a functionality tailored to the can, viaproject, the was constructed. communicate infrastructure, Rossia has a plug-and-play with the crawler interface throughfacilitating an optical easy fiber implementation of newwinch cable from a dedicated sensors according connecting to the functiontoand the crawler usage ofARIM the garage. the system. In the first version aims at autonomous operationsof including ARIM theeasy crawler launchwas connected to the garage through an Ethernet cable without power. Navigation and recovery of the full system. An X-Frame/X-Node combination with automatic subsea interface with the cable behind thewhere vehicle without remotely entangling operated it in habitat wet mate-able obstacles connectors is a critical provide power andelement in the functionality. communication to the systemWe has beentherefore envisaged developed.twoThe connecting/disconnecting options: (i) installing a winch process on theofcrawler, the X-Node canout to pay be cable done as with thea crawler ROV or easier moves. and heavier A cable cheaper than with water the tailored is left Launch and Recovery on the bottom Tool until the (X-LRT; crawler seemoving starts Figure 3). Thus, along “home” the instrument platform with crawler can be launched and recovered without removing the exact same path. The winch will then start a tension-based retrieval of the cable. (ii) An alternativethe permanent installed solution is abottom X-Node infrastructure. garage-based winch and a positive buoyant cable. Alternative (i) was considered safer Table 1. The main environmental sensors assets used for ecological monitoring as pursued by ARIM. Sensors Type/Model Crawler X-Frame Current profiler long range Acoustic current meter/Nortek X (m/s and °) Signature 250 Oxygen (µ mol/l)) AADI optode 4531 X Turbidity (FTU) Seapoint turbidity meter X * Chlorophyll (μg/l) Seapoint fluorometer X High-definition (HD) X imaging Sensors 2020, 20, 1614 7 of 17 3 Metas web-cameras (1 UWC- Operational cameras X 960, 2 UWC-210) but the winch consumes power and reduces operation duration autonomy. Alternative (ii) with a Simrad WBT-mini with ES-70 Acoustic floating cable imaging increases risk for cable abrasion and entanglement to the bottom habitat but X extends CD and ES-200 CD transducers operation in time and space. The key crawler sensor is the imaging system collecting information about Ambient noise Ocean Sonics SB-35 ETH X the surveyed benthic habitat. Repeated surveys enable analysis of the development of the benthic Laser Scanning Kraken SeaVision® X marine life over time, where changes detected in the acquired data are associated to changes in the *Optional sensors that can be installed in the interface container lid. marine environment following monitoring protocols already developed in other studies [28–32]. The The X-LRT caterpillar trackshas the capacity create to carry a footprint on thea payload seafloor of 60.35 m2 with tonnes supported by of a weight the≈wire 2 , thus to the 10 g/cm ship.very The unit is maneuverable through thrusters and the coupling–uncoupling and locking–unlocking little compaction and disturbance of sediments can be expected. Throughout typical operations during processes environmentalare monitored monitoringwith multiplestays the crawler cameras, ensuring correct on predetermined positioning tracks during theastransect well asanalyses. correct mating The of theuses operator connectors for power the cameras and theand communication. manipulator Thisand to record project probehas bolstered sediments inthe thedevelopment vicinity. This of the new is procedure design used toof avoid a garage module that unnecessary fits into damage of the fragile existingenvironment X-Net® (Metas [26].AS, Bergen, Norway) infrastructure (see Figure 2). Figure 3. The bottom unit X-Net® (Metas AS, Bergen, Norway) includes a bottom-based permanent cable connected docking station X-Node (green; see also Figure 2) and an instrumented exchangeable top unit (X-Frame; yellow) that can be launched and recovered with the Launch Recovery Tool (X-LRT). Note that the ARIM X-Frame includes the garage for the crawler (see Figure 2). This allows the user to operate and maintain the ARIM system without the assistance of expensive ROV vessel time. Sensors 2020, 20, 1614 8 of 17 3.2. The Bottom-Based System The cabled observatory node system X-Net® (Metas AS, Bergen, Norway) is composed of three main units: X-Node, X-Frame, and X-LRT (Launch and Recovery Tool; Figure 3). The X-Node is the permanent installed docking station connected directly to the infrastructure cable on the seabed. The X-Node is designed to receive an instrument platform with a garage module for the crawler. The instrument platform (X-Frame) is in principle the same as that used at LoVe but has been reconstructed and expanded to accommodate the garage module developed for Rossia. The front of the garage supports a ramp for the crawler to enter the garage from the seabed. The ramp also includes guiding for the crawler during on and off driving. During launch and recovery operations, the ramp is folded up, closing and protecting the crawler inside the garage. The garage holds a pin-less connector providing charging power for the crawler supported by the infrastructure when the crawler is docked in the garage. In case of power limitation from the infrastructure, the connector can provide charging power from a battery that is charged whenever power is available. The garage has several operational cameras (see Table 1) and light installed for monitoring docking and cable winch during operation. Under autonomous operation (without umbilical), the pin-less connector can provide data communication between the infrastructure and the crawler when connected. Data, such as mission plans, can be transferred to the crawler before the mission start, and data collected by the crawler can be uploaded to the infrastructure after the crawler has docked after mission completion. During training of the autonomous piloting software, the operator can, via the infrastructure, communicate with the crawler through an optical fiber cable from a dedicated winch connecting the crawler to the garage. ARIM aims at autonomous operations including easy launch and recovery of the full system. An X-Frame/X-Node combination with automatic subsea interface where remotely operated wet mate-able connectors provide power and communication to the system has been developed. The connecting/disconnecting process of the X-Node can be done with a ROV or easier and cheaper with the tailored Launch and Recovery Tool (X-LRT; see Figure 3). Thus, the instrument platform with crawler can be launched and recovered without removing the permanent installed bottom X-Node infrastructure. Table 1. The main environmental sensors assets used for ecological monitoring as pursued by ARIM. Sensors Type/Model Crawler X-Frame Acoustic current meter/Nortek Current profiler long range (m/s and ◦ ) X Signature 250 Acoustic current meter/Nortek Current profiler short range (m/s and ◦ ) X X Aquadop Pressure (dbar) Xylem AADI 4117D * * Temperature (◦ C) Xylem AADI 4117D * * Salinity (◦ /oo ) Xylem AADI 4319A * * CTD (◦ C, dBar, ◦ /oo ) ADM CTD X Oxygen (µmol/L)) AADI optode 4531 X Turbidity (FTU) Seapoint turbidity meter X * Chlorophyll (µg/L) Seapoint fluorometer X High-definition (HD) imaging X 3 Metas web-cameras Operational cameras X (1 UWC-960, 2 UWC-210) Simrad WBT-mini with ES-70 Acoustic imaging X CD and ES-200 CD transducers Ambient noise Ocean Sonics SB-35 ETH X Laser Scanning Kraken SeaVision® X * Optional sensors that can be installed in the interface container lid. The X-LRT has the capacity to carry a payload of 6 tonnes supported by the wire to the ship. The unit is maneuverable through thrusters and the coupling–uncoupling and locking–unlocking processes Figure 3. The bottom unit X-Net® (Metas AS, Bergen, Norway) includes a bottom-based permanent cable connected docking station X-Node (green; see also Figure 2) and an instrumented exchangeable top unit (X-Frame; yellow) that can be launched and recovered with the Launch Recovery Tool (X- LRT). Note that the ARIM X-Frame includes the garage for the crawler (see Figure 2). This allows the Sensors user 20,operate 2020,to 1614 and maintain the ARIM system without the assistance of expensive ROV vessel time. 9 of 17 3.3. Self-Sustained Operation with Fuel Cells are monitored with multiple cameras, ensuring correct positioning as well as correct mating of the The flexibility connectors for powerand and modularity communication. of ourThissystem projectishas supported bolsteredby theeither power supply development and of the new communication at sites of cabled observatories (first ® step) or stand-alone operation design of a garage module that fits into the existing X-Net (Metas AS, Bergen, Norway) infrastructure using new fuel cellFigure (see (FC) technology 2). to power long-term operations (second step). To ensure efficient stand-alone operation, the energy system must carry enough power capacity to keep the sensors and the crawler 3.3. Self-Sustained in operation overOperation an adequate withtime Fuel period. Cells Further, the overall operation of the system must be safe and The easy flexibility enough to and ensure efficient launch modularity of ourand recovery. system is supported by either power supply and communication at sites of cabled observatories (first step)for FC and rechargeable batteries are key technologies or 21st century operation stand-alone marine research using newand monitoring systems. The long-term supply of energy in deep-sea research is currently fuel cell (FC) technology to power long-term operations (second step). To ensure efficient stand-alone facilitated via primary batteries operation, the energy with associated system limitations must carry enoughandpower is not sustainable. capacity Thissensors to keep the sparked andthe theidea for the crawler in development of a deep-sea FC system for long-term monitoring approaches. Within operation over an adequate time period. Further, the overall operation of the system must be safe and this autonomous infrastructure, easy we built enough to ensure an FC launch efficient system and that recovery. provides about 160 kWh of energy and includes a 4 kWh storage system using lithium polymer (LiPo) FC and rechargeable batteries are key technologies battery technology for 21st century(Figure 4). The marine fuel cell research andpermanently monitoring delivers ~100–200 W. systems. The long-term supply of energy in deep-sea research is currently facilitated via primary Setup batteries withand operation associated of the and limitations FC isare notpresently sustainable. supported by ROV This sparked operations, the idea but ROV-free for the development of operation will be possible in the future by using the new X-LRT illustrated a deep-sea FC system for long-term monitoring approaches. Within this autonomous infrastructure,in Figure 3. Using the LRT we givesanscientists built FC system and industrial that providesoperators about 160morekWhflexibility of energyto move and the complete includes a 4 kWhsystem storage(X-Frame with system using crawlerpolymer lithium and the(LiPo) fuel cell power battery bank) to(Figure technology new locations 4). The fuelwithout very expensive cell permanently specialized delivers ~100–200 ROV W. vessels. Figure4.4. Long-term Figure Long-termoperation operationwith witha afuel fuelcell celland andpower powerstorage storagesystem systemprovides providesthe thebasis basisfor for advanced marine research infrastructures. The frame on the left (a) shows the lander system advanced marine research infrastructures. The frame on the left (a) shows the lander system that will that will carrythe carry thefuel fuelcell cellcomponents componentsininthe thecenter centerwith withthe thegas gasbottles bottlesoccupying occupyingthethesurrounding surroundingslots. slots.On On the right (b) is the membrane system of the fuel the right (b) is the membrane system of the fuel cell. cell. 3.4. Setup Navigation and Piloting and operation of the FC are presently supported by ROV operations, but ROV-free operation will be possible in the future by using the new X-LRT illustrated in Figure 3. Using the LRT gives Autonomous navigation and processing of images collected by the crawler is a core task in scientists and industrial operators more flexibility to move the complete system (X-Frame with crawler ARIM. To avoid fatal operational accidents due to failures in the autonomous navigation software, and the fuel cell power bank) to new locations without very expensive specialized ROV vessels. 3.4. Navigation and Piloting Autonomous navigation and processing of images collected by the crawler is a core task in ARIM. To avoid fatal operational accidents due to failures in the autonomous navigation software, we used Sensors 2020, 20, 1614 10 of 17 Sensors a 2020, 20, tethered 1614 during our first stage testing to navigate safely back to base, until the next10fully version of 18 autonomous tether-free generation vehicle is available. The autonomous control system is based on we used two majora tethered version during our first stage testing to navigate safely back to base, until the next components: fully autonomous tether-free generation vehicle is available. The autonomous control system is based 1. on two The core components: major element for autonomy is the simultaneous localization and mapping (SLAM) navigation algorithm, that creates the 2.5D bathymetry on the fly, using the SeaVision laser scanner plus the 1. The core element for autonomy is the simultaneous localization and mapping (SLAM) same map to localize itself (see Section 3.5); navigation algorithm, that creates the 2.5D bathymetry on the fly, using the SeaVision laser 2. A flexible mission execution layer based on Behavior Tree (BT) [33], that provides the functionality scanner plus the same map to localize itself (see Section 3.5); to act in response to events (e.g., payload sensor data) during the mission instead of following a 2. A flexible mission execution layer based on Behavior Tree (BT) [33], that provides the fixed waypoint plan. functionality to act in response to events (e.g., payload sensor data) during the mission instead of following SLAM a fixedusing algorithms waypoint pointplan. clouds from 3D laser scanners are quite common in terrestrial systems SLAM (seealgorithms e.g., [34]), but these using algorithms point cannot clouds from 3Dbelaser transferred scannersto the are marine environment quite common directly. in terrestrial The main reason is the limited range of underwater scanning (attenuation, visibility, etc.), systems (see e.g., [34]), but these algorithms cannot be transferred to the marine environment directly. which results in Thecomparably main reason smallis point clouds.range the limited Theseofinunderwater turn lead to limited scanningoverlapping areas (attenuation, between etc.), visibility, consecutive which scans. This alone makes point cloud fusion problematic. The SLAM solution we results in comparably small point clouds. These in turn lead to limited overlapping areas between propose uses several modalities consecutivetoscans. pre-align Thisthe point alone clouds, makes pointtherefore allowing cloud fusion a faster fitThe problematic. withSLAM small solution overlap. we Thepropose system uses uses the odometry several of thetocrawler modalities pre-alignas the the first pointmotion clouds,estimation thereforeand then fuses allowing it with a faster a visual fit with smallodometry overlap. from the SeaVision cameras using a Kalman filter. That estimation is The system uses the odometry of the crawler as the first motion estimation and then fuses then used as an input for a it with a multi-hypothesis (e.g., Bayes or particle filter) point cloud fusion estimator. In visual odometry from the SeaVision cameras using a Kalman filter. That estimation is then used as a second step the point cloud is projected an input into a 2.5D bathymetry for a multi-hypothesis (e.g., Bayes map. Figure 5 shows the first tests with real world data on or particle the fusion between crawler odometry and multiple static point clouds. Figure 5. Figure 5. First First results results of of aa point point cloud cloud stably stably merging merging with with the the crawler crawler odometry. odometry. Multiple Multiple scans scans were were taken with taken with aa very very slow slow stop-scan-move stop-scan-move mission mission profile. profile. The The colorization colorization shows shows the the reflectivity reflectivity of the of the seafloor to the laser line (remission). The data were acquired during a cruise on FS Alkor in November seafloor to the laser line (remission). The data were acquired during a cruise on FS Alkor in November 2019, with 2019, with the the SeaVision SeaVision system system mounted mounted onon Rossia’s Rossia’s predecessor predecessor VIATOR VIATORfromfromGEOMAR. GEOMAR. The mission execution layer used in trials during November 2019, was based on “Behavior Tree.CPP”,aafree Tree.CPP”, freeand andpublicly publiclyavailable availableimplementation implementation of of BTBT [35]. [35]. TheThe BTBT kernel kernel waswas attached attached to to the the action action state state automaton automaton of the crawler of the crawler and to and allowed allowed to run The run missions. missions. The next step next will step will be be integration of integration sensor of sensor information information into the systeminto the system to allow to allow for reaction for reaction during during a mission. a mission. 3.5. Automatic Data 3.5. Automatic Data Acquisition Acquisition and and Processing Processing for for Ecosystem Ecosystem Functioning Functioning The key sensor The key sensor (see (see Table Table 1) 1) of of the the benthic benthic platform platform includes cameras as includes cameras as well well as as aa 3D 3D SeaVision SeaVision scanner scanner and andbioacoustics bioacousticspackage. package.Imaging Imaging assets at LoVe assets observatory at LoVe (i.e.,(i.e., observatory a camera mounted a camera on node mounted on node 1; see Figure 1) are used to track faunal changes at a coral reef habitat (Figure 6). By keeping the camera position fixed, changes in visual counts for individuals of different species over time can be efficiently obtained and thus uncover dynamics at all temporal scales (from hours to years). The Sensors 2020, 20, 1614 11 of 17 1; see Figure 1) are used to track faunal changes at a coral reef habitat (Figure 6). By keeping the camera Sensors 2020, fixed, position 20, 1614changes 11 of 18 in visual counts for individuals of different species over time can be efficiently obtained and thus uncover dynamics at all temporal scales (from hours to years). The images and the images and the experiences experiences from this projectfrom this used are now project are now usedofinthe in development development ARIM imageofanalysis. the ARIM image analysis. Figure 6. 6. Figure Coral reef Coral with reef visible with visiblepolyps. Their polyps. activity Their patterns activity in in patterns opening and opening closing and inform closing inform about about feeding feedingdynamics. dynamics. Further, activity Further, patterns activity areare patterns also associated also associated with human with human impacts impacts(e.g., (e.g.,from from industrial activities; industrial Photo: activities; S. S. Photo: Flögel). Flögel). AtAtthe thesame sametime, time,ananacoustic acousticpackage packageincluding includingEK-80 EK-80broadband broadbandscientific scientificechosounder echosounder (KongsbergASA) (Kongsberg ASA) with with twotwomultiplexed multiplexed 70 kHz transducers 70 kHz (see Table transducers (see 1)Table provide1) continuous monitoring provide continuous of the water monitoring column of the water(Figure column 7).(Figure While the vertically 7). While pointing transducer the vertically is a reference pointing transducer is a for the vertical reference for thedistribution and migration vertical distribution and of organisms migration up to 1000 m of organisms upabove (dependent to 1000 from frequency), m above (dependent from the horizontal frequency), themoveable transducer horizontal moveable enables a pelagic transducer habitata search, enables pelagiccovering habitat asearch, sector from the bottom covering a sectorto from the surface the with range bottom to theup to 1000 surface m. range with Backscattering up to 1000 from m. planktonic organisms Backscattering to whales is from planktonic recorded to organisms at whales any time is interval recorded from seconds at any time to seasonsfrom interval [7,36]. Autonomous seconds operation, to seasons including feedback [7,36]. Autonomous from the operation, acoustic including backscatter feedback fromtothe the acoustic operationbackscatter software, enables detection of to the operation behavioral-related software, dynamics enables detection of organisms. of behavioral- With an related adequateofping dynamics rate the acoustic organisms. With anmonitoring can resolve adequate ping rate theindividual behavior to acan acoustic monitoring scale of mm resolve and seconds [7]. This includes tracking the behavior of individual organisms individual behavior to a scale of mm and seconds [7]. This includes tracking the behavior of as well as the crawler. In parallel individual withas organisms the acoustic well as the monitoring, crawler. the automated video tracking and classification have provided information on the dynamics of benthic megafauna in space and time. The detailed dynamics obtained through the coral reef images (see Figure 6) have helped in understanding temporal dynamics of polyps while the acoustic data have provided a better understanding of timing of ecosystem processes as well the impact of human activities ([37]; Figure 6). Such studies are important for the development of automatic image processing software for categorization and identification of species as well as of processes. Figure 7. Seasonal dynamics of biomass in the water column (vertical axis) as observed in the acoustic records from October 2013 to June 2014 (horizontal time axis). The acoustic system also observes with seconds and cm resolution giving access to, for example, individual fish behavior information by depth (vertical axis). High biomass densities are defined by red color and biomass is gradually bottom to the surface with range up to 1000 m. Backscattering from planktonic organisms to whales is recorded at any time interval from seconds to seasons [7,36]. Autonomous operation, including feedback from the acoustic backscatter to the operation software, enables detection of behavioral- related dynamics of organisms. With an adequate ping rate the acoustic monitoring can resolve individual behavior to a scale of mm and seconds [7]. This includes tracking the behavior of Sensors 2020, 20, 1614 12 of 17 individual organisms as well as the crawler. Sensors 2020, 20, 1614 12 of 18 decreasing when the color goes towards blue. No biomass is recorded when the echogram is white (processed from LoVe data by E. Johnsen, Institute of Marine Research, Bergen). In parallel with the acoustic monitoring, the automated video tracking and classification have provided Figureinformation Figure 7. Seasonal 7. on the of Seasonal dynamics dynamics dynamics of biomass in biomass inofthebenthic the water column water megafauna column (verticalin (vertical space axis) axis) and time. as observed as observed theThe in the in detailed acoustic acoustic dynamics records recordsobtained from fromOctoberthrough October 2013 2013tothe toJune Junecoral 2014reef 2014 images (horizontal (horizontal time(see time Figure axis). axis). The 6) have The acoustic acoustic helped system system alsoin also understanding observes observes with with temporal dynamics seconds secondsand andcm of polyps cmresolution resolution while giving giving the accessacoustic accessto, for data to, example, have for example, provided individual a better fish behavior individual understanding information fish behavior of by depth information timing by of ecosystem (vertical processes axis). depth (vertical High axis). as well biomass High the impact densities biomass of are are defined densities human by redactivities defined color red ([37]; by and biomass Figure color and 6). Such is gradually biomass studies are isdecreasing gradually important when the forcolor thegoes development towards blue.ofNo automatic image processing biomass is recorded software when the echogram for categorization is white (processed from and identification LoVe data of byspecies E. Johnsen,as well as ofofprocesses. Institute Marine Research, Bergen). One of the core ideas of our approach is to establish a real-time data processing and One of thesystem interpretation core ideasthatofenables our approach is to establish categorization and, whena real-time data species possible, processing and interpretation identification in near system real time both from the imaging and the acoustic sensors. An image processingtime that enables categorization and, when possible, species identification in near real both from pipeline was the imaging and the acoustic sensors. An image processing pipeline developed based on computer vision and machine learning tools, for detecting, segmenting, and was developed based on computer vision and machine classifying underwater learning tools, animals for detecting, from the datasetsegmenting, collected by and theclassifying underwater LoVe observatory animals (Figure from 8). First, the dataset these images collected by the are resized LoVe observatory to speed up the process; (Figure then 8). First,techniques several these images are are resized applied to speed the to improve up the process; then several techniques are applied to improve the contrast contrast between the background and the species to be detected. Afterwards, the background is between the background and the species followed subtracted, to be detected. Afterwards,ofthe by the application background filters to segment isthe subtracted, contours followed correctly. by the application Finally, once these of filters to segment the contours correctly. Finally, once these contours are extracted, different global characteristics are extracted from each cropped image contours are extracted, different and global characteristics are extracted from each cropped image and combined, combined, and ultimately classified. Temporal variation in images at LoVe is demonstrated in the and ultimately classified. Temporal variation inofimages video presentation imageatdata LoVeset is from demonstrated in the video presentation LoVe (Supplementary Video S1).ofWith imagestereodata set from camera LoVe images we were able to establish 3D images of objects and this supports the identification of (Supplementary Video S1). With stereo camera images we were able to establish 3D images of objects component and this supports the characteristics identification of (Supplementary component Video S2). characteristics (Supplementary Video S2). Figure 8. Designed automatic automatic image imageprocessing processingpipeline pipelinefor forHD HDimaging imagingthat that could could also also bebe applied applied to to acoustic acoustic outputs. outputs. Fortunately, Fortunately, processing processingsoftware softwarefor forimage image interpretation interpretationimproves improves continuously continuously [38,39], andand [38,39], we are we now are calibrating our analysis now calibrating our from images’ analysis fromdatabases images’containing databases identified objects. containing A computationally identified objects. A demanding function computationally of an underwater demanding function video monitoring system of an underwater is the capability video monitoring system to automatically track is the capability to and classify animals automatically within track and different classify species animals as pre-established within categories, different species based on pattern as pre-established recognition categories, based methods. on patternOnce patternmethods. recognition recognitionOnce has taken recognition pattern place, it is necessary has takento develop place, different methods it is necessary of to develop supervised classification different methods based on of supervised training setsbased classification (Figure on 9) to associate training them with sets (Figure 9) to their corresponding associate them with species and validate the their corresponding performances species and validateof the thedifferent routinesofaccording performances to the the different image quality routines according(Figure 7; to the see alsoquality image Supplementary (Figure 7; Video S2). see also Supplementary Video S2). This pipeline is now being adapted for crawler images. This adjustment is needed as the environment will become more challenging. The advancing crawler creates a moving background and reduces visibility due to the turbidity caused by the sand cloud in the wake of the crawler, as well as by normal sediments/rocky background. SensorsSensors 20, 1614 2020, 2020, 20, 1614 13 of 17 13 of 18 Figure Figure 9. (A) 9. (A) A set A set of images of images froma alarger from larger training training library libraryasasananexample exampleof supervised tracking of supervised and and tracking classification of Rockfish (Sebastes sp.) within a constant field of view from archived videos classification of Rockfish (Sebastes sp.) within a constant field of view from archived videos at LoVe. at LoVe. (B) Results (B) Results of automated of automated trackingand tracking andcounting counting of of individuals individualspassing passingby,by, as evidenced in a video as evidenced by by in a video the recognition boundary boxes (see Supplementary Video S1). the recognition boundary boxes (see Supplementary Video S1). ThisAbove we described the development of various components and tailoring of a complete novel pipeline is now being adapted for crawler images. This adjustment is needed as the monitoring system that is now under testing and completion. The status, challenges, and expected environment will become more challenging. The advancing crawler creates a moving background and time for realistic full-scale testing is summarized in Table 2. The crawler will then perform transect reduces visibility due to the turbidity caused by the sand cloud in the wake of the crawler, as well as analyses along the seafloor using its full set of sensor technologies including the SeaVision 3D camera, by normal which sediments/rocky quantifies volume background. changes of habitats forming megafauna. Similar to the bioacoustics sensor Above weon operations described the X-Net, thethe development of various crawler operations components will then drasticallyand tailoring expand of a complete novel the study/monitoring monitoring site fromsystem m to km 2 that 2 . is now under testing and completion. The status, challenges, and expected time for realistic full-scale testing is summarized in Table 2. The crawler will then perform transect Table 2. analyses along theStatus, expected seafloor challenges using its fullduring set of completion, and expectedincluding sensor technologies time of firstthe full-scale testing.3D camera, SeaVision which quantifies volume changes of habitats forming megafauna. Expected Similar to the bioacoustics sensor Element operations Statusthe crawler Challenges on the X-Net, operations will then drastically expand the Comment study/monitoring site 2 2 Completion from m to km . Similar crawlers have been Table 2. Status, expected challenges during completion, and expected time Garage–crawler of first produced andfull-scale testing. given valuable Crawler Tested at First test mid- symbiosis and cable Expectedexperience for the completion of Element system Status shallow water Challenges 2020 Completion Comment operation the ARIM crawler. Similar crawlers have been produced Garage–crawler symbiosis Crawler system Tested at shallow water First test mid-2020 and given valuable experience for the and cable operation completion of the ARIM crawler. Integrating garage Integrating garage with The optical fibre cable on the winch is Platform tested at LoVe and charging and cable winch for First field tests with The bottom-based used for training the navigation at a commercial oil with charging the crawler.and Combine these complete system system software prior to full autonomy production site functions and X-Frame as a mid-2020 The Platform tested cable winchsensor for the carrier First field tests The optical fibre operations cable on the Design and laboratory bottom- Self-sustained fuel at LoVe and at a version completed. crawler. Combine Integration with complete of fuel cell within winchThis First field tests with is used moduleforistraining presently the an add-on to the ARIM system. Establish complete system the ARIM platform. We expect a full cell Construction of frame and based commercial oil these functions and X- operational routinessystem mid- mid-2020navigation softwareintegrationprior tofuture in the full recovery system finished system production site test. First Frame as a sensor 2020 Tests under autonomy operations under This software will be Mainly laboratory Navigation and Deep water operation in realistic operational continuous development based on field test in shallow water in piloting unknown location carrier conditions experience and on development in a 2019 mid-2020 fast-developing software field Changing habitat conditions like light Large amount of data from Evaluation of system and turbidity affect visibility. This is a Design and Integration of fuel cell changing Tests under First field tests This module ischallenge presently an add- Automatic data LoVe used to train the capacity in handle general in marine imaging Self-and realistic operational acquisition system. All routines are visibility and routines that requires attention. The complete laboratory within the ARIM with conditions on to the ARIM platform. We complete processing operated according to capability to work at processing pipeline as well as the sustained mid-2020 specifications different habitat conditions associated problems and future version system. Establish system mid- expect a full integration challenges in the is detailed in [40] fuel cell completed. operational routines 2020 future 4. Discussion In this paper, we merged technologies and competence with the overarching goal to meet future requirements in marine research and monitoring. We experienced that multidisciplinary engineering and ecological expertise can efficiently respond to challenges that would be unsolvable individually [22]. Sensors 2020, 20, 1614 14 of 17 With our technological advances we demonstrate that the hardware and software components from various partners can be merged, and preliminary results demonstrate the technology’s potential. The next step is to carry out pilot monitoring activities where all technological components (i.e., node, crawler, and fuel cell) operate together under full autonomy. Our technology collects key information of ecosystem components and their interaction and dynamics. The urgent need for such key information from the marine environment, and associated tools for filling critical knowledge and information gaps, is obvious from the United Nations (UN) 2017 proclamation of “An International Decade of Ocean Science for Sustainable Development” [41], based on recommendation from the Intergovernmental Oceanographic Commission (IOC) [42]. The basic need for adequate time–space information of key ecosystem components cannot be satisfied through conventional methodologies due to cost and capacity limitations associated with using available research vessel capacity [7]. Further, few available approaches allow a combination of pelagic and demersal sampling simultaneously. We think that the ARIM development scenario is a promising approach to making adequate sampling available to scientists and might thus be an important tool for fulfilment of the goals launched by the UN. ARIM’s performance is totally dependent on a cross-disciplinary team of scientists and engineers working together towards a common goal. The team aims at continuing the development and utilizing the experiences of partners in ongoing projects at existing observatories (Neptune and LoVe). A combination of acoustics with oceanography as done by Engeland and colleagues [43] demonstrates the strength of coordinated multiparametric sampling in time and space with various sensors, with imaging at the center of that development for ecological monitoring (e.g., [44]). The unique contribution of ARIM is to establish technology and routines for combining imaging techniques of the benthic habitat and acoustic sampling of the pelagic habitat. This is a key factor for understanding marine ecosystem variability in time and space. Massive recurring population movement rhythms into the water column and across seabed depth ranges facilitate the exchange of energy among different marine compartments at rates faster than mere oceanographic conditionings, thus making animals efficient carbon carriers both as predators and preys, as well as when defecating [6]. Synchronization of biological activity in deep-sea benthos may also occur in relation to day-night cycles in an indirect fashion; i.e., mediated by the intermittent presence and absence of deep-scattering layers of predators and preys, rhythmically appearing within the benthic boundary layer over a 24-h period (e.g., [44]). Such rhythmic dynamics may also be accompanied by not yet quantified changes in background illumination at the seabed caused by bioluminescence of the scattering layers [45]. If such rhythms influence species relative abundances and community composition towards the continental margins where observatory networks are deployed, a temporal variability in measured ecological indicators (species abundance, biomass as well as biodiversity) must be expected [4]. We expect that an entirely new class of moving robotic platforms, spanning from tethered and untethered crawlers with a suite of advanced sensors, will support a dramatic improvement in deep-sea monitoring [5]. Such a technological development will favor a cross-over with other monitoring networks of ecological relevance, merging Eulerian and Lagrangian approaches [4]. Cabled observatories and their docked crawlers will increasingly integrate the capability for communication with other technological assets, such as neutrino telescopes with their vertically-extended moored profilers, ARGO floats, and even animal-borne sensors. Envisaged autonomous robotic exploration and monitoring technologies are being developed using space analogs on Earth [46,47]. Marine observatory deployment scenarios are considered operative analogues of extraterrestrial oceans exploration, testing for increased reliability in robot autonomy in self-assembling, -repair, and -energetic tasks [17]. These aspects are of broad relevance for planetary surface explorations [48], where robotic platforms will subsidize humans in space exploration over the next decades [49]. Lagrangian exploration approaches, such as the one ARIM action envisages, should be used as a test bed for the tailoring of solutions adapted to explore exo-oceans on, for example, the Enceladus moon of Saturn [34]. Sensors 2020, 20, 1614 15 of 17 Automatic processing and interpretation of both images and acoustics are still under development and will require substantial input in the years to come to produce reliable results [20]. Although the technique is promising, we still see major challenges in establishing fully automatic systems that can operate unattended over extended periods. Energy will restrict sampling density, and hence, temporal gaps may obscure processes that otherwise would have been detected. The energy limitations are continuously improved through better batteries and more power efficient fuel cells and sensing systems. Reliable imaging techniques require long-term operation with cable collections through an online observatory to ensure that data are properly validated and that recognition algorithms are updated accordingly [10]. Supplementary Materials: The following are available online at http://www.mdpi.com/1424-8220/20/6/1614/s1, Video S1. Demonstrates a typical data set from the LoVe coral reef that is used to train the AI identification algorithms. Video S2. Establishing 3D images from stereo images. Author Contributions: Conceptualization, S.F., O.R.G., O.P., L.T. and T.T.; Data curation, J.A. (Jacopo Aguzzi) and G.Z.; Formal analysis, E.R. and V.L.-V.; Funding acquisition, O.R.G., O.P., L.T. and T.T.; Investigation, S.F., E.G., L.T. and T.T; Methodology, J.A. (Jacopo Aguzzi), J.A. (Jan Albiez), S.F., S.M., E.R., T.T., J.V. and G.Z.; Project administration, E.G. and H.W.; Resources, E.R. and H.W.; Software, J.A. (Jan Albiez); Writing—original draft, O.R.G.; Writing—review & editing, J.A. (Jacopo Aguzzi), J.A. (Jan Albiez), S.F., O.R.G., E.G., S.M., O.P., E.R., L.T., T.T., J.V., G.Z. and V.L.-V. Please turn to the CRediT taxonomy for the term explanation. Authorship must be limited to those who have contributed substantially to the work reported. All authors have read and agreed to the published version of the manuscript. Funding: This project is funded by The Norwegian Research Council, Federal Ministry for Economic Affairs and Energy of Germany (03SX464C) and the HGF-project MOSES, Spanish Centre for the Development of Industrial Technology (EXP 00108707/SERA-20181020), and co-funded by European Union’s Horizon 2020 research and innovation program under the framework of ERA-NET Cofund MarTERA (Maritime and Marine Technologies for a new Era). Acknowledgments: We are grateful to our funding sources for supporting our consortium and our efforts. J.A. is a member of the Tecnoterra CSIC-UPC Associated Research Unit. Conflicts of Interest: The authors declare no conflicts of interest. References 1. Danovaro, R.; Aguzzi, J.; Fanelli, E.; Billet, D.; Gjerde, K.; Jamieson, A.; Ramirez-Llodra, E.; Smith, C.R.; Snelgrove, P.V.R.; Thomsen, L.; et al. A new international ecosystem-based strategy for the global deep ocean. Science 2017, 355, 452–454. [CrossRef] 2. Del Río, J.; Toma, D.M.; Martínez, E.; O’Reilly, T.C.; Delory, E.; Pearlman, J.S.; Waldmann, C.; Jirka, S. A Sensor Web Architecture for integrating smart oceanographic sensors into the semantic sensor Web. IEEE J. Ocean. Eng. 2018, 43, 830–842. [CrossRef] 3. Buck, J.J.; Bainbridge, S.J.; Burger, E.F.; Kraberg, A.; Casari, M.; Darroch, L.; Del Rio Fernandes, J.; Metfies, K.; Delroy, E.; Fischer, P.; et al. Ocean data product integration through innovation-The next level of data interoperability. Front. Mar. Sci. 2019, 6, 32/1–32/19. [CrossRef] 4. Aguzzi, J.; Chatzievangelou, D.; Marini, S.; Fanelli, E.; Danovaro, R.; Flögel, S.; Lebris, N.; Juanes, F.; De Leo, F.; Del Rio, J.; et al. New high-tech interactive and flexible networks for the future monitoring of deep-sea ecosystems. Environ. Sci. Technol. 2020, 53, 6616–6631. [CrossRef] 5. Danovaro, R.; Fanelli, E.; Aguzzi, J.; Billett, D.; Carugati, L.; Corinaldesi, C.; Dell’Anno, A.; Gjerde, K.; Jamieson, A.J.; Kark, S.; et al. Ecological indicators for an integrated global deep-ocean strategy. Nat. Ecol. Evol. 2020, 4, 181–192. [CrossRef] 6. Rountree, R.; Aguzzi, J.; Marini, S.; Fanelli, E.; De Leo, C.F.; Del Rio, J.; Juanes, F. Towards an optimal design for ecosystem-level ocean observatories. Ocean. Mar. Biol. Ann. Rev. 2020, 50, 58. (In Press) 7. Godø, O.R.; Handegard, N.O.; Browman, H.; Macaulay, G.J.; Kaartvedt, S.; Giske, J.; Ona, E.; Huse, G.; Johnsen, E. Marine ecosystem acoustics(MEA): Quantifying processes in the sea at the spatio-temporal scales on which they occur. ICES J. Mar. Sci. 2014, 71, 2357–2369. [CrossRef] 8. Godø, O.R.; Torkelsen, T.; Johnsen, S. The LoVe Ocean Observatory in Operation. Mar. Technol. Soc. J. 2014, 48, 24–29. [CrossRef] Sensors 2020, 20, 1614 16 of 17 9. Aguzzi, J.; Sbragaglia, V.; Santamaría, G.; Del Río, J.; Sardà, F.; Nogueras, M.; Manuel, A. Daily activity rhythms in temperate coastal fishes: Insights from cabled observatory video monitoring. Mar. Ecol. Prog. Ser. 2013, 486, 223–236. [CrossRef] 10. Aguzzi, J.; Doya, C.; Tecchio, S.; De Leo, F.L.; Azzurro, E.; Costa, C.; Sbragaglia, V.; Del Río, J.; Navarro, J.; Ruhl, H.A.; et al. Coastal observatories for monitoring of fish behaviour and their responses to environmental changes. Rev. Fish Biol. Fish. 2015, 25, 463–483. [CrossRef] 11. Roberts, J.M.; Wheeler, A.J.; Freiwald, A. Reefs of the Deep: The biology and geology of Cold-Water Coral Ecosystems. Science 2006, 312, 543–547. [CrossRef] [PubMed] 12. Godø, O.R.; Tenningen, E.; Ostrowski, M.; Kubilus, R.; Kutti, T.; Korneliussen, R.; Fosså, J.H. The Hermes Lander Project—The technology, the data, and an evaluation of concept and results. Fisken Og Havet 2012, 3, 1–65. 13. The Lofoten Vesterålen Ocean Observatrory website. Available online: https://love.equinor.com/ (accessed on 12 March 2020). 14. Metas AS website. Available online: https://metas.no/portfolio/x-net-system/ (accessed on 12 March 2020). 15. Bøe, R.; Bellec, V.K.; Dolan, M.F.J.; Buhl-Mortensen, P.; Buhl-Mortensen, L.; Slagstad, D.; Rise, L. Giant sandwaves in the Hola glacial trough off Vesterålen, North Norway. Mar. Geol. 2009, 267, 36–54. [CrossRef] 16. Woods, M.; Shaw, A.; Tidey, E.; Van Pham, B.; Simon, L.; Mukherji, R.; Maddison, B. Seeker—Autonomous Long-range Rover. Navigation for Remote Exploration. J. Field Robot. 2014, 31, 940–968. [CrossRef] 17. Aguzzi, J.; Flexas, M.M.; Flögel, S.; Lo Iacono, C.; Tangherlini, M.; Costa, C.; Marini, S.; Bahamon, N.; Martini, S.; Fanelli, E.; et al. Exo-ocean exploration with deep-sea sensor and platform technologies. Astrobiology 2020, 20. (In Press) [CrossRef] 18. Thomsen, L.; Purser, A.; Schwendner, J.; Duda, A.; Flögel, S.; Kwasnitschka, T.; Pfannkuche, O.; Wilde, D.; Rosta, R. Temporal and spatial benthic data collection via mobile robots: Present and future applications. In Proceedings of the OCEANS-Genova, Genova, Italy, 18–21 May 2015; pp. 1–5. 19. Matabos, M.; Bui, A.O.; Mihály, S.; Aguzzi, J.; Juniper, D.S.K.; Ajayamohan, R.S. High-frequency study of benthic megafaunal community dynamics in Barkley canyon: A multidisciplinary approach using the NEPTUNE Canada network. J. Mar. Syst. 2014, 130, 56–68. [CrossRef] 20. Marini, S.; Fanelli, E.; Sbragaglia, V.; Azzurro, E.; Del Rio, J.; Aguzzi, J. Tracking fish abundance by underwater image recognition. Sci. Rep. 2018, 8, 13748. [CrossRef] 21. Aguzzi, J.; Company, J.B.; Costa, C.; Matabos, M.; Azzurro, E.; Mànuel, A.; Menesatti, P.; Sardà, F.; Canals, M.; Delory, E.; et al. Challenges to assessment of benthic populations and biodiversity as a result of rhythmic behaviour: Video solutions from cabled observatories. Oceanogr. Mar. Biol. Annu. Rev. 2012, 50, 235–286. 22. iSeaMC website. Available online: https://www.iseamc.com/crawler-systems/advanced-crawler-systems/ (accessed on 12 March 2020). 23. Bahamon, N.J.; Bernardello, R.; Ahumada, M.-A.; Puigdefabregas, J.; Muñoz, E.; Velásquez, Z.; Cruzado, A. The new pelagic Operational Observatory of the Catalan Sea (OOCS) for the multisensor coordinated measurement of atmospheric and oceanographic conditions. Sensors 2011, 11, 11251–11272. [CrossRef] 24. Hays, G.C.; Ferreira, L.C.; Sequeira, A.M.; Meekan, M.G.; Duarte, C.M.; Bailey, H.; Eguíluz, V.M. Key questions in marine megafauna movement ecology. Trends Ecol. Evol. 2016, 31, 463–475. [CrossRef] 25. Irigoien, X.; Klevjer, T.A.; Røstad, A.; Martinez, U.; Boyra, G.; Acuña, J.L.; Agusti, S. Large mesopelagic fishes biomass and trophic efficiency in the open ocean. Nature 2014, 5, 3271. [CrossRef] [PubMed] 26. Purser, A.; Thomsen, L.; Barnes, C.; Best, M.; Chapman, R.; Hofbauer, M.; Wagner, H. Temporal and spatial benthic data collection via an internet operated Deep Sea Crawler. Methods Oceanogr. 2013, 5, 1–18. [CrossRef] 27. The Ocean Network website. Available online: https://www.oceannetworks.ca/introduction-barkley-canyon (accessed on 12 March 2020). 28. Doya, C.; Chatzievangelou, D.; Bahamon, N.; Purser, A.; De Leo, F.; Juniper, K.; Thomsen, L.; Aguzzi, J. Seasonal monitoring of deep-sea cold-seep benthic communities using an Internet Operated Vehicle (IOV). PLoS ONE 2017, 12, e0176917. [CrossRef] 29. Chatzievangelou, D.; Suarez, A.; Aguzzi, J.; Bigham, K.; Thomsen, L. Optimization of surveys with Internet Operated Deep-sea Crawlers, as an integrated tool for ocean cabled observatories: Monitoring the benthic community of a methane hydrates site in Barkley Canyon (BC, Canada). In Proceedings of the AGU Fall Meeting, New Olrleans, LA, USA, 11–15 December 2017. Sensors 2020, 20, 1614 17 of 17 30. Chatzievangelou, D.; Doya, C.; Mihály, S.; Sastri, A.R.; Thomsen, L.; Aguzzi, J. High-frequency patterns in the abundance of benthic species near a cold-seep: An Internet Operated Vehicle application. PLoS ONE 2016, 11, e0163808. [CrossRef] [PubMed] 31. Chatzievangelou, D.; Aguzzi, J.; Thomsen, L. Quality control and pre analysis treatment of 5-year long environmental datasets collected by an Internet Operated Deep sea Crawler. In Proceedings of the 2019 IMEKO TC-19 International Workshop on Metrology for the Sea (METROSea), Genoa, Italy, 3–5 October 2019. 32. Thomsen, L.; Aguzzi, J.; Costa, C.; De Leo, F.; Ogston, A.; Purser, A. The oceanic biological pump: Rapid carbon transfer to the Deep Sea during winter. Sci. Rep. 2017, 7, 10763. [CrossRef] [PubMed] 33. Colledanchise, M.; Ögren, P. Behavior Trees in Robotics and AI: An Introduction; Artificial Intelligence and Robotics Series; Chapman & Hall/CRC: London, UK, 2018; 192p, ISBN 9781138593732. 34. Carrio, J.H. Adaptive Localization and Mapping for Planetary Rovers. Ph.D. Thesis, University of Bremen, Bremen, Germany, 2018. Available online: https://elib.suub.uni-bremen.de/edocs/00106623-1.pdf (accessed on 12 March 2020). 35. Behavior Tree website. Available online: https://www.behaviortree.dev/ (accessed on 12 March 2020). 36. Simmonds, J.; MacLennan, D. Fisheries acoustics: Theory and practice. In Fish and Aquatic Resources; Blackwell Science: London/Oxford, UK, 2005. 37. Buhl-Mortensen, P.; Tenningen, E.; Tysseland, A.B.S. Effects of water flow and drilling waste exposure on polyp behaviour in Lophelia pertusa. Mar. Biol. Res. 2015, 11, 725–737. [CrossRef] 38. Aguzzi, J.; Costa, C.; Robert, K.; Matabos, M.; Antonucci, F.; Juniper, K.; Menesatti, P. Automated image analysis for the detection of benthic crustaceans and bacterial mat coverage using the VENUS undersea cabled network. Sensors 2011, 11, 10534–10556. [CrossRef] [PubMed] 39. Nattkemper, T.; Osterloff, J.; Nilssen, I.; Van Engeland, T.; Buhl-Mortensen, P.; Järnegren, J. Computer vision enables short- and long-term analysis of Lophelia pertusa polyp behaviour and colour from an underwater observatory. Sci. Rep. 2019, 9, 6578. [CrossRef] 40. Lopez-Vazquez, V.; Lopez-Guede, J.M.; Marini, S.; Fanelli, E.; Johnsen, E.; Aguzzi, J. Video Image Enhancement and Machine Learning Pipeline for Underwater Animal Detection and Classification at Cabled Observatories. Sensors 2020, 20, 726. [CrossRef] 41. UN. General Assembly of the United Nations. Seventy-Second Session. Agenda Item 77 (a). Oceans and the Law of the Sea. Resolution A/72/L.18. 2017. Available online: https://undocs.org/en/A/72/L.18 (accessed on 12 March 2020). 42. IOC. Intergovernmental Oceanographic Commission, Decision IOC-XXIX/2 Prov; IOC: Paris, France, 2017. 43. Engeland, T.; Godø, O.R.; Johnsen, E.; Duineveld, G.C.A.; van Oevelen, D. Cabled ocean observatory data reveal food supply mechanisms to a cold-water coral reef. Prog. Oceanogr. 2019, 172, 51–64. [CrossRef] 44. Aguzzi, J.; Company, J.B.; Costa, C.; Menesatti, P.; Garcia, J.A.; Bahamon, N.; Puig, P.; Sardà, F. Activity rhythms in the deep-sea: A chronobiological approach. Front. Biosci.-Landmark 2011, 16, 131–150. [CrossRef] [PubMed] 45. Aguzzi, J.; Fanelli, E.; Ciuffardi, T.; Schirone, A.; Craig, G.; The NEMO Consortium. Inertial bioluminescence rhythms at the Central Mediterranean KM3NeT deep-sea neutrino telescope. Sci. Rep. 2017, 7, 44938. [CrossRef] [PubMed] 46. Forrest, A.L.; Trembanis, A.C.; Todd, W.L. Ocean floor mapping as precursor for space exploration. J. Ocean Technol. 2012, 7, 70–86. 47. Trembanis, A.C.; Forrest, A.L.; Miller, D.C.; Lim, D.S.; Darlene, S.S.; Gernhardt, M.L.; Todd, W.L. Multiplatform ocean exploration: Insights from the NEEMO space analog mission. Mar. Technol. Soc. J. 2012, 46, 7–19. [CrossRef] 48. Howe, A. Cassette Factories and Robotic Bricks: A Roadmap for Establishing Deep Space Infrastructures. SAE Tech. Pap. 2005, 114, 330–353. [CrossRef] 49. Pedersen, L.; Smith, T.; Lee, S.Y.; Cabrol, N. Planetary LakeLander: A robotic sentinel to monitor remote lakes. J. Field Robot. 2016, 32, 860–879. [CrossRef] © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).