The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks 

Abstract

Modern neural networks can both learn general patterns from data and memorize specific facts or exceptions. While both abilities are essential in practice, existing theory largely treats them separately. In this talk, I will introduce the Rules-and-Facts (RAF) model, a simple framework where part of the data follows an underlying rule, while the rest consists of isolated facts that must be memorized. This setting allows us to study when a learner can simultaneously generalize and recall exceptions. Our results show that the key question is not whether a model has enough capacity, but how this capacity is organized and used. In particular, we identify how regularization and the geometry of the learned representation (e.g., kernel or feature map) determine whether a model can allocate resources to memorize facts without interfering with rule learning. The RAF model thus provides a precise lens on how modern neural networks balance abstraction and memory, and how architectural and algorithmic choices control this trade-off. 

Our Speaker

Lenka Zdeborová is a Professor of Physics and Computer Science at École Polytechnique Fédérale de Lausanne, where she leads the Statistical Physics of Computation Laboratory. She received a PhD in physics from the University of Paris-Sud and Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director's Postdoctoral Fellow. Between 2010 and 2020, she was a researcher at CNRS, working in the Institute of Theoretical Physics in CEA Saclay, France. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize, in 2021 the Gibbs lectureship of AMS and the Neuron Fund award, in 2025 she received an ERC Advanced Grant. Lenka's expertise is in the application of concepts from statistical physics, such as advanced mean field methods, the replica method, and related message-passing algorithms, to problems in machine learning, signal processing, inference, and optimization. Currently, she focuses on statistical physics of learning, developing solvable models and theoretical principles that explain how modern AI systems generalize, memorize, and scale. She enjoys erasing the boundaries between theoretical physics, mathematics and computer science.