In the world of machine learning, optimization algorithms are the unsung heroes, guiding the training process through complex, high-dimensional landscapes. One such algorithm, stochastic gradient descent with momentum (SGDM), is widely used for its ability to speed up convergence. However, it’s not without its limitations, especially in challenging optimization landscapes. Enter dual enhanced stochastic gradient descent (DESGD), a new method developed by Mohamed A. Mokhtar from the Basic and Applied Science Department at the College of Engineering and Technology, Arab Academy for Science, Technology and Maritime Transport. Published in Scientific Reports, this research could have significant implications for the maritime sector, where machine learning is increasingly being used for predictive maintenance, route optimization, and more.
DESGD addresses the limitations of SGDM by dynamically adapting both momentum and step size, all while using the same update rules. In simple terms, it’s like having a more intelligent and adaptable compass for navigating those complex landscapes. The results speak for themselves. On two optimization test functions, the Rosenbrock and Sum Square functions, DESGD outperformed both SGDM and Adam, another popular optimization algorithm. It achieved comparable errors with up to 81–95% fewer iterations and 66–91% less CPU time than SGDM, and 67–78% fewer iterations with 62–70% quicker runtimes than Adam.
But what does this mean for the maritime sector? Well, consider predictive maintenance. Ships are complex machines with countless components that can fail. Predictive maintenance uses machine learning to predict when these components might fail, allowing for timely interventions and reducing downtime. The more accurate the model, the better the predictions. DESGD could potentially improve the accuracy of these models, leading to more reliable predictions and significant cost savings.
Route optimization is another area where DESGD could make a difference. Shipping companies are always looking for the most efficient routes to save on fuel and reduce emissions. Machine learning models can help with this, and again, the more accurate the model, the better the results. DESGD could potentially improve the accuracy of these models, leading to more efficient routes and significant cost savings.
In the words of the lead author, Mohamed A. Mokhtar, “Although SGDM remained the fastest per-step optimizer, our method’s computational cost is aligned with that of other adaptive optimizers like Adam. This marginal increase in per-iteration overhead is decisively justified by the substantial gains in model accuracy and reduction in training loss, demonstrating a favorable cost-to-performance ratio.”
So, while DESGD might not be the fastest optimizer per step, the benefits it brings in terms of accuracy and reduced training loss make it a worthwhile investment. As machine learning continues to play a bigger role in the maritime sector, optimizers like DESGD could become increasingly important. They could help improve the accuracy of models, leading to better predictions, more efficient routes, and significant cost savings. It’s an exciting development that maritime professionals should keep an eye on.