CASCB Talk: Swarm-based gradient descent methods for non-convex optimization by Eitan Tadmor
Time
Monday, 1. July 2024
12:00 - 13:00
Location
ZT 702 and online
Organizer
CASCB
Speaker:
Prof. Eitan Tadmor, University of Maryland, College Park
Eitan Tadmor will discuss a novel class of swarm-based gradient descent (SBGD) methods for non-convex optimization. The swarm consists of agents, each is identified with position, x, and mass, m. There are three key ingredients to the SBGD dynamics.
(i) persistent transition of mass from agents at high to lower ground;
(ii) mass-dependent marching in directions randomly aligned with gradient descent; and
(iii) time stepping protocol which decreases with m.
The interplay between positions and masses leads to dynamic distinction between 'leaders' and 'explorers': heavier agents lead the swarm near local minima with small time steps; lighter agents use larger time steps to explore the landscape in search of improved global minimum, by reducing the overall ‘loss’ of the swarm.
Convergence analysis and numerical simulations demonstrate the effectiveness of SBGD method as a global optimizer.
Eitan Tadmor is a Distinguished University Professor at the University of Maryland, College Park, where he served as the Director of the Center for Scientific Computation and Mathematical Modeling (CSCAMM), 2002-2016. He held professorship positions at Tel-Aviv University, 1983-1995, and at UCLA, 1995-2002, where he was the founding co-director of the NSF Institute for Pure and Applied Mathematics (IPAM), 1999-2001. Tadmor was the PI for NSF Kinetic Research network (Ki-Net), 2012-2020.
In 2016-2017 he was a Senior Fellow at the Institute for Theoretical Studies at ETH-Zurich (ETH-ITS), and is the current FSMP Chair of excellence, hosted by the Laboratoire Jacques Louis Lions, Sorbonne University.
Tadmor gave an ICM invited address in 2002, a SIAM invited JMM address in 2014, a 2019 plenary ICIAM lecture, and the 2022 American Math Society (AMS) Gibbs Lecture. Tadmor was awarded the SIAM-ETH Henrici prize in 2015, and the AMS-SIAM Norbert Wiener prize in Applied Mathematics in 2022. He is an AMS and SIAM Fellow, and a member of Academia Europaea and the European Academy of Science.
