Speaker: Jeffrey Pennington, Google Research
Talk title: Covariate Shift in High-Dimensional Random Feature Regression
Time: Tuesday, April 5, 2:00pm-3:00pm (ET)
Abstract:
Recent empirical observations have demonstrated that overparameterized neural networks exhibit good generalization properties, and they do so not just in distribution, but out of distribution as well. In this talk, I will analyze this effect in the context of high-dimensional random feature regression under covariate shift, and present a precise characterization of the generalization error in this setting. I will also introduce a natural partial order over covariate shifts that provides a sufficient condition for determining when the shift will harm (or even help) test performance. The analysis reveals an exact linear relationship between in-distribution and out-of-distribution generalization performance, offering an explanation for this surprising empirical phenomenon.
Return to workshop schedule