TOPML Workshop 2021
April 20-21, 2021
Virtual event
Organizing Committee
Demba Ba, Harvard University
Richard Baraniuk, Rice University
Mikhail Belkin, UC San Diego
Yehuda Dar, Rice University
Vidya Muthukumar, Georgia Tech
Ryan Tibshirani, Carnegie Mellon U.
Invited Speakers
Peter Bartlett, UC Berkeley
Florent Krzakala, EPFL
Gitta Kutyniok, LMU Munich
Michael Mahoney, UC Berkeley
Robert Nowak, U. of Wisconsin-Madison
Tomaso Poggio, MIT
Matthieu Wyart, EPFL
2021 Schedule
Tuesday, April 20, 2021
| Time (PT) | |
| 8:00am-8:10 | Opening remarks |
| 8:10-9:10 | Invited talk Florent Krzakala, EPFL Generalization in Machine Learning: Insights from Simple Models Abstract |
| 9:10-9:35 | Contributed talk Nicole Muecke, Technical University Berlin The Influence of Overparameterization and Regularization on Distributed Learning Abstract |
| 9:35-9:45 | Break |
| 9:45-10:45 | Invited talk Peter Bartlett, UC Berkeley Benign Overfitting Abstract |
| 10:45-11:10 | Contributed talk Xiangyu Chang, UC Riverside Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Abstract |
| 11:10-11:20 | Break |
| 11:20-12:20 | Invited talk Michael Mahoney, UC Berkeley Practical Theory and Neural Network Models Abstract |
| 12:20-1:20 | Lightning talk session #1 |
| 1:20-2:00 | Break |
| 2:00-3:00 | Invited talk Robert Nowak, University of Wisconsin-Madison Banach Space Representer Theorems for Neural Networks Abstract |
| 3:00-4:00 | Lightning talk session #2 |
Wednesday, April 21, 2021
| Time (PT) | |
| 8:00am-9:00 | Invited talk Matthieu Wyart, EPFL A Phase Diagram for Deep Learning Unifying Jamming, Feature Learning and Lazy Training Abstract |
| 9:00-9:25 | Contributed talk Jeremy Bernstein, Caltech Computing the Typical Information Content of Infinitely Wide Neural Networks Abstract |
| 9:25-9:30 | Break |
| 9:30-10:30 | Invited talk Gitta Kutyniok, LMU Munich Graph Convolutional Neural Networks: The Mystery of Generalization Abstract |
| 10:30-10:55 | Contributed talk Raaz Dwivedi, UC Berkeley Revisiting Complexity and the Bias-Variance Tradeoff Abstract |
| 10:55-11:10 | Break |
| 11:10-12:10 | Invited talk Tomaso Poggio, MIT Deep Puzzles Abstract |
| 12:10-1:05 | Lightning talk session #3 |
| 1:05-2:00 | Lightning talk session #4 |
| 2:00-3:00 | Panel discussion Panelists: Misha Belkin, Ryan Tibshirani, Peter Bartlett, Tomaso Poggio, Florent Krzakala, Michael Mahoney. Moderators: Vidya Muthukumar and Yehuda Dar |
| 3:00-3:10 | Closing remarks |
Lightning Talks
Session #1
Exact Expressions for Double Descent and Implicit Regularization via Surrogate Random Design
Michal Derezinski (UC Berkeley); Feynman Liang (UC Berkeley); Michael Mahoney (UC Berkeley)
A Data-dependent Theory of Overparameterization: Phase Transition, Double Descent, and Beyond
Zhenyu Liao (UC Berkeley); Romain Couillet (Central Supélec); Michael Mahoney (UC Berkeley)
For interpolating kernel machines, minimizing the norm of the ERM solution optimizes stability
Akshay Rangamani (MIT); Lorenzo Rosasco (Università degli Studi di Genova); Tomaso Poggio (MIT)
Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models
Zitong Yang (UC Berkeley); Yu Bai (Salesforce Research); Song Mei (UC Berkeley)
Lower Bounds on the Generalization Error of Deep Neural Networks
Inbar Seroussi (Weizmann Institute of Science); Ofer Zeitouni (Weizmann Institute of Science)
Towards Sample-Efficient Overparameterized Meta-Learning
Yue Sun (University of Washington); Ibrahim Gulluk (Bogazici University); Adhyyan Narang (University of Washington); Samet Oymak (UC Riverside); Maryam Fazel (University of Washington)
On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces
Zhuoran Yang (Princeton University); Chi Jin (Princeton University); Zhaoran Wang (Northwestern University); Mengdi Wang (Princeton University/DeepMind); Michael Jordan (UC Berkeley)
Asymptotic Risk of Overparameterized Likelihood Models with Application to Double Descent on Deep Neural Networks
Ryumei Nakada (Rutgers University / University of Tokyo); Masaaki Imaizumi (The University of Tokyo / RIKEN AIP)
Session #2
Benign Overfitting in Binary Classification of Gaussian Mixtures
Ke Wang (UC Santa Barbara); Christos Thrampoulidis (University of British Columbia)
Label-Imbalanced and Group-Sensitive Classification under Overparameterization
Ganesh Ramachandra Kini (UC Santa Barbara); Orestis Paraskevas (UC Santa Barbara); Samet Oymak (UC Riverside); Christos Thrampoulidis (University of British Columbia)
Margin Distribution: Are All Data Equal?
Andrzej Banburski (MIT); Fernanda De La Torre Romo (MIT); Ishana Shastri (MIT); Nishka Pant (MIT); Tomaso Poggio (MIT)
Deep Learning Generalization, Extrapolation, and Over-parameterization
Roozbeh Yousefzadeh (Yale University)
Characterizing High Dimensional Representation Learning in Overparameterized Neural Networks
Arna Ghosh (McGill University, Mila); Kumar Krishna Agrawal (UC Berkeley); Blake Richards (McGill University)
Mitigating Deep Double Descent by Concatenating Inputs
John Chen (Rice University); Qihan Wang (Rice University)*; Anastasios Kyrillidis (Rice University)
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?
Gavin Brown (Boston University); Mark Bun (Boston University); Vitaly Feldman (Apple); Adam Smith (Boston University); Kunal Talwar (Apple)
Understanding Implicit Regularization in Over-Parameterized Nonlinear Statistical Model
Mengxin Yu (Princeton University); Zhuoran Yang (Princeton University); Jianqing Fan (Princeton University)
Session #3
Why Do Overparameterized CNNs Outperform Overparameterized FCNs?
Alon Brutzkus (Tel Aviv University); Amir Globerson (Tel Aviv University, Google)
Overparametrized Regression Under L2 Adversarial Attacks
Antônio H. Ribeiro (Uppsala University); Thomas B. Schön (Uppsala University)
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise
Spencer Frei (UCLA); Yuan Cao (UCLA); Quanquan Gu (UCLA)
Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition
Yaodong Yu (UC Berkeley); Zitong Yang (UC Berkeley); Edgar Dobriban (University of Pennsylvania); Jacob Steinhardt (UC Berkeley); Yi Ma (UC Berkeley)
Overparameterized Linear Subspaces and Generative Adversarial Networks: The Effects of Supervision and Orthonormality Constraints
Lorenzo Luzi (Rice University); Yehuda Dar (Rice University); Richard Baraniuk (Rice University)
On the Computational and Statistical Complexity of Over-Parameterized Matrix Sensing
Jiacheng Zhuo (University of Texas at Austin); Jeongyeol Kwon (University of Texas at Austin); Nhat Ho (University of Texas at Austin); Constantine Caramanis (University of Texas)
Recovery and Generalization in Over-Realized Dictionary Learning
Jeremias Sulam (Johns Hopkins University); Chong You (UC Berkeley); Zhihui Zhu (University of Denver)
Session #4
Towards Understanding Learning in Neural Networks with Linear Teachers
Roei Sarussi (Tel Aviv University); Alon Brutzkus (Tel Aviv University); Amir Globerson (Tel Aviv University, Google)
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network
Mo Zhou (Duke University); Rong Ge (Duke University); Chi Jin (Princeton University)
Achieving Zero Training Error Does Not Guarantee Anything About Generalization Performance in Over-Parameterized Regimes
Nicole Muecke (Technical University Berlin); Ingo Steinwart (University of Stuttgart)
On Random Kernels of Residual Architectures
Etai Littwin; Tomer Galanti (Tel Aviv University); Lior Wolf (Tel Aviv University)
Learning and Generalization in Overparameterized Normalizing Flows
Kulin Shah (Microsoft Research); Amit Deshpande (Microsoft Research); Navin Goyal (Microsoft Research India)
The Role of Overparameterization and Optimization in Deep CNN Denoisers
Julián Tachella (University of Edinburgh); Junqi Tang (University of Edinburgh); Mike Davies (University of Edinburgh)
Learning and Generalization in RNNs
Abhishek Panigrahi (Princeton University); Navin Goyal (Microsoft Research India)