Research in marine robotics requires frequent use of simulators due to the complexity of trials which usually happen in the water. During OCEANS 2023 Limerick, we presented our high-fidelity simulator MARUS that offers advanced capabilities of generating realistic maritime environments allowing for closer-to-reality V&V (Verification and Validation) of applications developed for maritime vehicles. The simulator offers synthetic dataset generation with perfect annotations for various sensors (cameras, LiDAR, sonar) and allows for interaction with the environment for closed loop simulation. This simulator is highly applicable for the researchers in the field of marine robotics in industry and academia and it has been used in a myriad of applications such as diver-robot interaction, unmanned ships etc. ^{13 , 14} MARUS simulator is built in Unity Engine as it provides tools for generating realistic environments and easy to use interface for quick setup which makes it a perfect tool for development of a simulator. The public received a deep dive tutorial into MARUS by its creators - the Laboratory for Underwater Systems and Technologies (LABUST) from the University of Zagreb Faculty of Electrical Engineering and Computing, one of the MONUSEN partners. The tutorial introduced all the tools used for the development and characterization of the simulator. Features include many different types of sensors, thruster simulation, tools for data annotation, connection with the Robotic Operating System (ROS) 1 or 2 middleware ^{15 , 16} simple controls etc. During the practical part of the tutorial the participants assembled a vehicle in a pre-built marine environment and equipped it with different sensors such as cameras, LiDAR, Inertial Measurement Unit (IMU), GPS, etc. In addition, the audience was introduced with ROS and in particular with the sensor data available in ROS topics. The second practical part involved the use of annotation tools that can be useful for solving situational awareness tasks. We showed annotation tools for different sensors developed in the simulator. Participants also performed a Citizen Science activity where they completed a task in the simulator environment.

In addition to the tutorial, the MONUSEN booth included a PC with the simulator installed and ready to be easily used by any by-passer. Visitors were briefly introduced to the task needed to be completed and asked to participate in the experiment. In addition, after completing the activity they were invited to complete the survey online. Unfortunately, collecting good data in these conditions was not always easy as attendees have a limited time to visit the exhibition area and might not stay until the end of the experiment and/or not all participants in the experiment actually filled out the survey. These are limitations that affected the number of completed surveys and experiments. Nevertheless, performing this experiment at OCEANS 2023 Limerick allowed us to explain the need of efficient autonomous navigation to the attendees and to show our work to a very broad international audience, which gave us a good qualitative feedback. Informed consent was sought and obtained for each individual that participated in the OCEANS 2023 Limerick activities.

In the citizen science experiment, participants engaged in a Unity game where they controlled the MARUS boat, navigating through various maritime scenarios to avoid collisions with other ships. The main ship is the MARUS vessel, i.e., the vessel controlled by the autonomous collision avoidance algorithm. The term target ship refers to any surrounding vessel involved in a potential encounter with the main ship. Depending on the scenario, the encountered vessel acted either as the stand-on vessel or the give-way vessel according to the International Regulations for Preventing Collisions at Sea (COLREGs).

The game featured five levels, each progressively more challenging than the last. Initially, participants were briefed on the International Regulations for Preventing Collisions at Sea (COLREGs) to ensure they understood the fundamental rules of maritime navigation. ¹⁷ Potential collision situations are separated into five scenarios: Overtaking, Head on, Right crossing, Left crossing and Head on - Right crossing. In Level 1, players faced an overtaking scenario, ¹⁸ where they needed to overtake a slower boat directly in front of them to reach their goal. Level 2 introduced the head-on scenario, ¹⁹ requiring players to avoid a boat moving towards them. Level 3 focused on right crossing, ^{20 , 21} a straightforward scenario similar to earlier levels, but requiring precise manoeuvring. Level 4 was more complex, involving a left crossing ²⁰ where the target ship did not adhere to COLREGs, ²² challenging players to navigate safely despite the non-yielding ship. The final level combined the challenges of head-on and right crossing scenarios, testing players’ ability to handle multiple potential collision situations simultaneously. This experiment aimed to gather data on how well participants could apply COLREGs in simulated environments and improve their understanding of maritime navigation through an interactive gameplay. The same collision situations are tested with an autonomous ship and autonomous collision avoidance algorithms. The additional goal is to compare the efficiency of citizens’ choices with those generated by the autonomous collision avoidance algorithm, where the algorithm computes trajectories that ensure collision avoidance while minimizing fuel consumption. This comparison aims to inform citizens of the importance of making navigation decisions that are both safe and fuel-efficient, thereby contributing to energy savings and reducing the environmental impact of maritime operations.

The trajectory generated by the autonomous navigation algorithm is considered optimal. Participants were instructed to navigate the vessel while following this path as closely as possible, while still applying their own judgment and experience regarding safe maneuvering practices. The “best” and “worst” performing players are identified based on their deviation from the reference trajectory, rather than solely on proximity to other vessels or collision avoidance outcomes. Nonetheless, here we present quantitative results based on 12 attendees. Figure 1 shows the paths of the main ship, for the 5 different scenarios, when controlled by an autonomous collision avoidance algorithm.

Figure 2 presents the corresponding distances between the main ship and the target ships for the case when the main ship is controlled by the autonomous collision avoidance algorithm. Figure 3, Figure 4 and Figure 5 indicates the best and the worst performing player results, as well as normalized RMSE (Root Mean Square Error) values and deviations from the optimal path calculated by the algorithm. An example of the Head on scenario from the citizen science activity is shown in Figure 6. The use of citizen science in this context provided valuable insights into public awareness and adherence to maritime safety rules. Looking at the average of normalized RMSE, the values range between 0.15125 and 0.27796, with the lowest value in Level 1 and the highest value in Level 5, which was to be expected due to the increased level of difficulty. The RMSE is computed on the 2D position error between each participant’s traversed path and the reference (optimal) path generated by the autonomous path planner. Specifically, for each pair of paths, both are resampled to a common number of points by normalized path progress (fraction of trajectory completed) and the RMSE is calculated as the root-mean-square of the point-wise Euclidean distance between them. This correspondence is established by path progress rather than by elapsed time, since paths of different duration cannot be directly time-aligned. This as a limitation of the current metric, as it does not capture timing differences between participants and the reference trajectory.

Through a survey we gathered the participants’ feedback on the experiment. We also fostered discussions during the tutorial and welcomed qualitative feedback during the interactions with visitors of the booth. As in previous similar user experiences, ²³ the survey included a first part with background questions regarding the previous experience of attendees with boat driving followed by a second part dedicated to task load evaluation. It is important to understand if the simulator can be used as a training tool for potential boat captains. Thus, the NASA Task Load Index (TLX) developed by Hart and Staveland ²⁴ was used to evaluate the subjective mental workload with indexes of mental, physical and temporal demand, performance, and difficulty levels. The five components were rated from 1 to 10. Finally, the third and fourth parts of the survey addressed qualitative feedback regarding the simulator and the experiment itself and asked for suggestions on future developments of the simulator.

Twenty participants provided complete answers to the questionnaires of which 16 attendees of the tutorial and 4 booth visitors. 45% of the participants were between 18 and 30 years old. 25% of the respondents were between 31 and 40 years old. Out of the 20, 10 did not have any previous boat driving experience, 5 occasionally engaged in boat driving, 2 had a medium level of experience, 2 an advanced level (e.g. part of their jobs) and one did not answer. Only a quarter of the attendees (5 people) had experience of developing simulations or games and the average first-person-view gaming skills was 5.65 out of 10.

When it comes to the NASA Task Load Index (TLX) estimation, as mentioned, 5 indexes were used: mental effort (ME), physical effort (PE), time pressure (TP), performance (PF) and task difficulty (TD). On a scale of 1 to 10 (0 no effort, 10 impossible), ME average was 4.26 while PE was only 2.74. This is expected as pressing buttons is not physically demanding, but the mental effort required (especially for non-specialists) is higher than the physical effort. In terms of time performance (amount of pressure felt due to the rate at which the task elements occurred), the average value was 4.11 (still below half of the scale). On the other hand, participants rated their performance successful on an average of 5.11 (above half ) with a standard deviation of 2.81. This large standard deviation and half-success rate can be explained by the fact that the sample was not broad enough (20 participants from different backgrounds) and all missions were considered together when answering this question (both the more advanced levels and the easier ones). This is corroborated by the fact that the average value for task difficulty was 4.53 (10 = difficult, 1 = easy).

Looking into the feedback about the simulator itself, the median value of realism was 7 (from 1 to 10 as well). 70% of the players classified the 5th level as the hardest as expected as this was the level involving more boats, while 88% declared that their longest trajectory was on the 5th level. Attesting to the lack of experience, 50% of the attendees was not aware of COLREG rules at all before the game. 20% instead knew all the COLREG rules involved and 30% knew just a few. The median value for the potential of using this simulator for boat driving training was a high 8 (from 1 to 10) with 55% of the players choosing 8 or above, which shows the goodness of the MARUS simulator for this training task. Similarly, 65% of the attendees chose 8 or above (out of 10) to rate the potential of using situational awareness modules to help humans follow COLREG rules in Marine Traffic. When asked about the biggest drawback of the simulator, several attendees identified the lack of a wide field of view/lateral vision, while final comments were positive and encouraging, highly appreciating the simulator.

Students and teachers from High School ‘Mladost’ in Tivat and Gymnasium Kotor - Montenegro - participated in the demo workshop as ‘citizen scientists.’ Project members presented their research in the area of underwater sensor networks during the workshop. The session included an explanation of the construction and functionality of the autonomous surface vehicle SWAMP (Shallow Water Autonomous Multipurpose Platform). ²⁶ Students collected data by piloting SWAMP remotely. Such data trained the controller for automatic vehicle control. The students also received insights into various methods for automatic vehicle control, methodologies for collecting and visualizing data, and performance evaluation of the vehicle in task completion. To promote research activities in the field of underwater sensor networks, promotional materials such as t-shirts, notebooks, and flyers were distributed among the students, along with Certificates of Participation. The CSI MONUSEN workshop aimed to highlight the importance of new technology in addressing environmental challenges along the coast and sea. Through active involvement in data collection, participants gained knowledge of state-of-the-art methods for controlling autonomous surface vehicles. Twenty-five high school students participated in data collection as “citizen scientists” by manually controlling the autonomous surface vehicle. The purpose of involving high school students is to promote science and technology, encourage creative thinking, support young talents, and contribute to developing a knowledge-based society.

For activities involving underage students, parental or guardian consent was obtained prior to any photo or video recording conducted during the CSI–MONUSEN events. The consent forms explicitly authorized the use of such materials for pedagogical purposes (e.g., documentation of student activities and achievements, presentation of work by students, associates, and faculty, and professional development) and for promotional purposes related to the Faculty of Electrical Engineering of the University of Montenegro and the MONUSEN project (e.g., billboards, electronic and printed materials, academic publications, the faculty website, social media profiles, and press releases). As the activities were limited to outreach and educational actions within the CSI–MONUSEN project, no separate ethical approval number was issued by an institutional ethics committee.

The main objective was to demonstrate to participants the principle of controller design for Unmanned Surface Vehicles (USV) based on the imitation learning paradigm. ²⁷ In contrast to conventional controller design methods that rely on knowledge of the vehicle model, the imitation learning paradigm bases controller design on collected data. Typically, the robot/vehicle is manually controlled to complete a specific task, and during the experiment, all data is collected. Furthermore, based on the collected data, the neural-network (NN) controller with a predefined architecture is trained to imitate human actions. Specifically, in Kumbor, the imitation learning paradigm is employed to control the SWAMP platform shown in Figure 7. Four buoys were positioned at predetermined locations in the sea, in front of the coast of Hotel Carine in Kumbor. The SWAMP was developed and assembled at the Institute of Marine Engineering (INM), National Research Council (CNR) laboratories located in Genoa, Italy. The sea trial was structured to actively involve citizens in data collection by allowing them to steer the vehicle remotely, thus contributing to the research efforts. The vehicle weighs approximately 40 kg and has dimensions of 1.25 m in both length and width, with a height of 0.5 m. Its draft is 0.12 m. The design of SWAMP incorporates a soft structure made from Polyethylene soft foam, which, combined with its propulsion system based on flush-integrated Pump-Jet Thrusters, ²⁶ ensures safe operation even in densely populated environments, such as areas filled with swimming tourists. The use of PE foam minimizes the risk of injury upon contact, while the flush design of the propulsion unit eliminates the exposure of moving parts, further enhancing safety. These features not only protect individuals coming across SWAMP, but also foster public trust and acceptance of robotic technology in recreational or leisure environments, where safety is often the highest priority. As a result, the SWAMP robot is well-suited for deployment in sensitive areas, such as beaches and swimming pools, helping to bridge the gap between advanced technology and public safety concerns.

In order to implement an imitation learning control scheme, each task demonstration from the human controller has been recorded at discrete time steps. Specifically, the variables recorded at each time step k are x ( k), y ( k), ϕ ( k), joythrust ( k) and joyangle ( k), i.e., the position and angle of the robot and the corresponding given joystick inputs. ²⁶

From each trial, the datasets u ⁱ  = { x( k), y ( k), ϕ ( k)} and y ⁱ  = {joythrust ( k), joyangle ( k)} for k = 0, K, K ^j are collected, where j denotes ordinal number human operator and is the total number of steps composing the trajectory. The collected data is used to train a neural network controller, with u ^j serving as the input to the controller and y ^j as the desired output. In particular, due to the dynamic nature of the application, a recurrent neural network having the form of an Echo State Network (ESN) has been employed. For a detailed description of the neural controller and the training procedure, the reader is referred to the manuscript Odetti et al. 2020. ²⁶ During the training process, the coefficients of the neural network are optimized to approximate the relationship between joystick inputs and the position and angle of the robot, as illustrated in Figure 8. Following the training phase, the acquired neural network is employed to automatically control the SWAMP based on real measurements. In other words, the NN controller uses actual measurements of the robot’s position and angle to generate outputs that correspond to joystick inputs, enabling the USV to perform tasks for which the neural network has been trained ( Figure 8).

Following a brief demonstration and instructional session, students were tasked with manually operating SWAMP. Specifically, students used a Thrustmaster USB Joystick to pilot the SWAMP through a course signposted by two gates, formed by four buoys, creating an S-shaped path ( Figure 9).

The vehicle’s trajectories and control inputs were recorded and used to train a neural network using the imitation learning paradigm. The objective of this training was to train an AI (Artificial Intelligence) controller to autonomously drive SWAMP. Students proactively engaged with remotely controlling the robot, for some the experiment has been perceived as playing a video game.