Accurate weather forecasting has, by and large, been situated squarely in the domain of high-performance computing – just this week, the UK announced a nearly $1.6 billion investment in the world’s largest supercomputer for weather and climate. Now, researchers at Johannes Gutenberg University Mainz and Università della Svizzera italiana are aiming to challenge that status quo with a new algorithm that allows PCs to run tasks that used to require supercomputers.

The algorithm is based on a concept called scalable probabilistic approximation, or SPA, and took many years to develop. The SPA algorithm is able to take just a few dozen components of a system and analyze those elements to predict future behavior with strong accuracy. “For example, using the SPA algorithm we could make a data-based forecast of surface temperatures in Europe for the day ahead and have a prediction error of only 0.75 degrees Celsius,” said Susanne Gerber, co-author of the research and a bioinformatics specialist at Johannes Gutenberg University Mainz.

The researchers specifically designed the algorithm to be interpretable, in contrast to existing machine learning methods. “Many machine learning methods, such as the very popular deep learning, are very successful, but work like a black box, which means that we don’t know exactly what is going on,” Gerber said. “We wanted to understand how artificial intelligence works and gain a better understanding of the connections involved.”

The real advantage of the algorithm, of course, is its performance requirements. “This method enables us to carry out tasks on a standard PC that previously would have required a supercomputer,” said Illia Horenko, another co-author and a computer expert at Università della Svizzera italiana. For example, in Gerber’s weather prediction example, the algorithm produces a result with an error rate 40% better than the computer systems used by many weather services – all while running on a conventional PC at a cost that is lower by five to six orders of magnitude.

The algorithm has applications in a wide range of sectors, ranging from weather to breast cancer diagnosis to neuroscience. For biological applications, the algorithm is broadly useful in situations where large numbers of cells need to be sorted. “What is particularly useful about the result is that we can then get an understanding of what characteristics were used to sort the cells,” said Gerber.

“The SPA algorithm can be applied in a number of fields, from the Lorenz model to the molecular dynamics of amino acids in water,” said Horenko. “The process is easier and cheaper and the results are also better compared to those produced by the current state-of-the-art supercomputers.”

About the research

The research discussed in this article was published as “Low-cost scalable discretization, prediction, and feature selection for complex systems” in the January 2020 issue of Science Advances. It was written by Susanne Gerber, L. Pospisil, M. Navandar and Illia Horenko and can be accessed at this link.

To read the release from Johannes Gutenberg University Mainz discussing this research, click here.