New numerical software tools to give AI a boost
Artificial intelligence (AI) algorithms are, in essence, computational problem-solvers. The basis of AI and machine learning (ML) often involves solving classic numerical problems such as optimisation, linear algebra and differential equations. Researchers in AI and ML commonly turn to existing numerical software tools as they are widely available. Yet most numerical algorithms are decades old and were built for a very different purpose, such as applications in physics, economics or scientific simulations, and were not designed specifically for AI. These generic algorithms assembled in such collections tend to be inefficient on any specific task, and can be unsafe when used incorrectly on problems they were not designed for. The PANAMA project, funded by the European Research Council, developed a framework to update numerical methods for AI algorithms. The overall goal was to update and integrate numerical computation more seamlessly within AI research, and ultimately make AI faster and easier to use. “A central aim of our project PANAMA was to offer better interfaces for AI and ML methods to ‘plug into’,” says Philipp Hennig, professor of ML methods at the University of Tübingen.
Parallels in computation
A numerical method is a program that uses computer chips to estimate the solution to a mathematical problem by calculating a series of ‘helpful numbers’ – for example, the values of terms in an equation. It then uses them to estimate the solution in another equation. Similarly, an ML algorithm estimates unknown quantities from data, which is also a collection of ‘helpful numbers’. Measurements of the trajectory of an astronomical object, for example, can be used to estimate its future trajectory. There is therefore a close semantic similarity between ML and numerical computation as both rely on collecting information in the form of numbers. The new methods developed in PANAMA explicitly express these steps in the same mathematical language.
Building a new framework for numerical computation
The project developed a software framework to allow numerical methods for AI to be developed in an increasingly automated way. To do this, the team used a computational technique known as automatic differentiation, a powerful mathematical concept. PANAMA’s software is available online, and the team created a series of practical implementations of the new tools. The project also contributed low-level routines for ML, all of which can be directly used by engineers and scientific practitioners. Another main result is the publication of a full textbook, published by Cambridge University Press and available online. “This is the first book that directly covers computation from the perspective of inference,” Hennig explains.
Enhancing AI in widespread applications
PANAMA’s tools could be used by AI practitioners in a broad range of fields, including neuroscience, public health and geosciences. The methods have applications in simulating biological neural networks, adaptively tracking and controlling pandemics, and simulating complex geological processes with high uncertainty. PANAMA’s results will lead to cleaner paradigms for writing and realising AI systems, making them more powerful, efficient and safer to use. “This kind of theoretical work, thankfully, doesn’t proceed at the breakneck speed of AI applications,” adds Hennig. “But it can have a more lasting long-term effect, making AI systems more powerful and efficient, but also easier and safer to use.”
Keywords
PANAMA, artificial intelligence, machine learning, probabilistic, applications, numerical, solving, problems