The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks 

Abstract

Modern neural networks must be able both to learn general patterns from data and to remember specific facts or exceptions, yet these two capabilities are often studied separately in existing theory. In this talk, Prof. Zdeborová will introduce the Rules-and-Facts (RAF) model, a simple framework in which part of the data follows an underlying rule, while the remaining examples consist of isolated facts that must be memorized. Using this framework, she will discuss when a learner can successfully generalize while also recalling exceptions. The results show that the central issue is not simply whether a model has sufficient capacity, but how that capacity is organized and used. In particular, the talk will explore how regularization and the geometry of learned representations, such as kernels or feature maps, determine whether a model can allocate resources to memorization without disrupting rule learning. The RAF model provides a precise theoretical perspective on how neural networks balance abstraction and memory, and on how architectural and algorithmic choices shape this trade-off. 

Our Speaker

Lenka Zdeborová is a Professor of Physics and Computer Science at École Polytechnique Fédérale de Lausanne, where she leads the Statistical Physics of Computation Laboratory. She received a PhD in physics from the University of Paris-Sud and Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director's Postdoctoral Fellow. Between 2010 and 2020, she was a researcher at CNRS, working in the Institute of Theoretical Physics in CEA Saclay, France. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize, in 2021 the Gibbs lectureship of AMS and the Neuron Fund award, in 2025 she received an ERC Advanced Grant. Lenka's expertise is in the application of concepts from statistical physics, such as advanced mean field methods, the replica method, and related message-passing algorithms, to problems in machine learning, signal processing, inference, and optimization. Currently, she focuses on statistical physics of learning, developing solvable models and theoretical principles that explain how modern AI systems generalize, memorize, and scale. She enjoys erasing the boundaries between theoretical physics, mathematics and computer science.