Download Overfitting - Tsinghua University

Overfitting
Study Guide for ES205
Yu-Chi Ho
Jonathan T. Lee
Jan. 9, 2001
Outline
Simple Example
 Fitting
 Cause of Overfitting
 Resolution

2
Simple Example

Overfitting: fitted model is too
complex
x
x
x
N
3
Simple Example (cont.)

Overfitting  poor predictive power
x
x
x
x
N
4
Fitting

A common problem – fit a model to
existing data:
•
•
•
•
•
N
Regression
Training Neural Networks
Approximation
Data Mining
Learning
5
Cause of Ovefitting

Noise in the system:
•

N
Greater variability in data
Complex model  many parameters
 higher degree of freedom 
greater variability
6
Resolution
More data points:
average out the noise
 Multiple models

N
7
References:
• Estienne, F., web-site,
http://minf.vub.ac.be/~fabi/calibration/multi/pcr/ch
aps/chap13.html
• Stokes, L and J. Ghosh, web-site,
http://lans.ece.utexas.edu/course/ee380/1999fall/p
apers/eminer/.
8