Download Office hrs

Tuesday February 19, 2002
92.6180-01 DATA MINING AND KNOWLEDGE DISCOVERY
Instructor:
Office hrs:
Class Time:
Book:
Prof. Mark J. Embrechts (x 4009 or 371-4562)
CII 5217 Tuesday 10:00-12:00
Monday/Thursday 10:00-11:20 am (Low 3112)
Michael J. A. Berry, and Gordon Linoff, Data Mining Techniques: For
Marketing, Sales, and Customer Support, John Wiley (1997). ISBN 0-47117980-9
Lectures #10&11: Partial Least Squares Regression
Partial Least Squares Regression was invented by the Swedish statistician Herman Wold for
econometrics applications. In the meanwhile PLS has become a popular and powerful tool in
chemometrics, but has been partially ignored in mainstream statistics. Svante Wold (the son of
Herman Wold) popularized PLS for drug design applications (i.e., QSAR = quantum structural
activity relationships). PLS can be viewed as a better principal components, where the data are
first transformed into a different orthogonal basis, just as in PCA, and only a few (the most
important) PLS components (or latent variables) are considered for building a regression model
(just as in PCA). The difference between PLS and PCA is that the new orthogonal set of basis
vectors (similar to the eigenvectors of XTX in PCA) is not a set of successive orthogonal
directions that explain the largest variance in the data, but consider the answer as well while
building a set of basis vectors. Just like in PCA the basis vectors are peeled off from the data
matrix X successively in the NIPALS algorithm (Nonlinear Iterative Partial Least Squares,
introduced by Herman Wold by approximating X as X = TPT (similar to X = TB in PCA). This
week’s lectures are based on the recent paper PLS-regression by Svante Wold. PLS regression is
one of the most powerful data mining tools for large data sets with an overabundance of
descriptive features. The NIPALS implementation of PLS is elegant and fast. What makes PLS
especially interesting for data mining applications is a recent extension to nonlinear PLS or
kernel PLS which is almost equivalent to support vector machines which are very popular in
machine learning. Nonlinear Kernel PLS is in certain cases fully equivalent with neural networks
(i.e., perceptrons and radial basis functions). This week’s lectures will be illustrated with several
case studies using the StripMiner code. Special emphasis will be placed on feature selection with
PLS and using the StripMiner code for nonlinear kernel PLS.
Handouts:
1.
Svante Wold, Michael Sjölström, Lennart Erikson, "PLS-regression: a basic tool
of chemometrics," Chemometrics and Intelligent Laboratory Systems, Vol 58,
pp. 109-130 (2001).
2.
Lecture Slides.
1
Quiz
A 20 minute quiz on PLS based In the paper by Svante Wold will be polled on
March 4.
Deadlines:
January 24
January 28
February 31
February 19
March 4
March 7
March 18
April 4
April 29/May 2
HW#0 (Web browsing).
Project Proposal
HW #1
HW #2
Quiz #1 on PLS paper by Svante Wold et al.
No Class
Progress Report
No Class
Final Presentations
2