In a bid to optimize maritime training, researchers have turned to machine learning to predict student performance in simulator exercises. The study, led by Ziaul Haque Munim from the Faculty of Technology, Natural and Maritime Sciences at the University of South-Eastern Norway, explores how different data sampling frequencies impact the accuracy of machine learning models in predicting student outcomes during simulator training. The findings, published in the journal ‘Array’ (translated to English), offer valuable insights for the maritime industry, particularly in terms of resource optimization and training efficiency.
The research focuses on simulator log data from navigation students performing a Williamson Turn in both ballast and loaded ship conditions. The team examined data frequencies ranging from 1 to 9-second intervals, evaluating the models using various metrics such as Area Under the Curve (AUC), Accuracy, Log Loss, Precision, Recall, and F1 Scores. The machine learning models used included eXtreme Gradient Boosted Trees, variants of Keras Residual Neural Network, and Light Gradient Boosted Trees.
The results were clear: the best accuracy measurement scores were achieved with 1-second frequency intervals in both ballast and loaded conditions. Munim noted, “The 1-s frequency intervals models are also the fastest and require less Random Access Memory (RAM).” However, as the data frequency intervals increased, the model evaluation metrics deteriorated. This indicates that while higher frequency data yields better predictions, it also demands more computational resources.
For the maritime industry, these findings present both challenges and opportunities. On one hand, the need for high-frequency data to achieve the best predictive accuracy could mean increased data storage and handling costs. On the other hand, the study highlights the potential for significant resource savings if acceptable prediction accuracy can be achieved with lower frequency data.
Munim explained, “If acceptable prediction accuracy can be achieved by using lower frequency data with larger time intervals between recorded data points, valuable resources in terms of data storage, handling, and computational cost, can be potentially saved.” This could be particularly beneficial for maritime training institutions looking to optimize their resources and improve training efficiency.
The study also underscores the importance of machine learning in predictive learning analytics, a field that is gaining traction in various sectors, including maritime education and training. By leveraging machine learning models, maritime training programs can potentially enhance their performance assessment processes, leading to better-trained and more competent seafarers.
In summary, the research by Munim and his team offers a glimpse into the future of maritime training, where data-driven approaches and machine learning models play a pivotal role in optimizing resources and improving training outcomes. As the maritime industry continues to evolve, such studies will be instrumental in shaping the future of maritime education and training.