9.520 2003春季課程：統計學習理論及應用(Statistical Learning Theory and Applications, Spring 2003)

We introduce and motivate the main theme of the course, setting the problem of learning from examples as the problem of approximating a multivariate function from sparse data. We present an overview of the theoretical part of the course and sketch the connection between classical Regularization Theory and its algorithms -- including Support Vector Machines -- and Learning Theory, the two cornerstones of the course. We mention theoretical developments during the last few months that provide a new perspective on the foundations of the theory. We briefly describe several different applications ranging from vision to computer graphics, to finance and neuroscience.

18.02, 9.641, 6.893 or permission of instructor. In practice, a substantial level of mathematical maturity is necessary. Familiarity with probability and functional analysis will be very helpful. We try to keep the mathematical prerequisites to a minimum, but we will introduce complicated material at a fast pace.

There will be two problem sets, a MATLAB® assignment, and a final project. To receive credit, you must attend regularly, and put in effort on all problem sets and the project.

Problem Sets

See the assignments page for the two problem sets.

Projects

Some of the most promising projects:

Hypothesis testing with small sets
Connection between MED and regularization
Feature selection for SVMs theory and experiments
Bayes classification rule and SVMs
IOHMMs evaluation of HMMs for classification vs. direct classification
Reusing the test set datamining bounds
Large-scale nonlinear least square regularization
Viewbased classification
Local vs. global classifiers experiments and theory
RKHS invariance to measure historical math
Concentration experiments (dot product vs. square distance)
Decorrelating classifiers: experiments about generalization using a tree of stumps
Kernel synthesis and selection
Bayesian interpretation of regularization and in particular of SVMs
History of induction from Kant to Popper and current state
Bayesian Priorhood

Resources

The Center for Biological and Computational Learning (CBCL) at MIT was founded with the belief that learning is at the very core of the problem of intelligence, both biological and artificial, and is the gateway to understanding how the human brain works and to making intelligent machines. CBCL studies the problem of learning within a multidisciplinary approach. Its main goal is to nurture serious research on the mathematics, the engineering and the neuroscience of learning. CBCL is based in the Department of Brain and Cognitive Sciences at MIT and is associated with the McGovern Institute for Brain Research and with the Artificial Intelligence Laboratory at MIT.

MATLAB® is a trademark of The MathWorks, Inc.

Problem Sets

Problem Set 1: Kernel Hilbert Spaces (PDF)
Problem Set 2: RBF Interpolation Schemes (PDF)

Project (PDF)

For the final project, students select from one of the following suggested topics, and solve the problem that is described. If students prefer, they can bring their own project ideas to the professor or TAs for approval.

Topics:

Hypothesis testing with small sets
Connection between MED and regularization
Feature selection for SVMs theory and experiments
Bayes classification rule and SVMs
IOHMMs evaluation of HMMs for classification vs. direct classification
Reusing the test set datamining bounds
Large-scale nonlinear least square regularization
Viewbased classification
Local vs. global classifiers experiments and theory
RKHS invariance to measure historical math
Concentration experiments (dot product vs. square distance)
Decorrelating classifiers: experiments about generalization using a tree of stumps
Kernel synthesis and selection
Bayesian interpretation of regularization and in particular of SVMs
History of induction from Kant to Popper and current state
Bayesian Priorhood

為了離線或窄頻使用者，「麻省理工開放式課程」免費提供了完整的課程檔案下載。「麻省理工開放式課程」永遠是免費開放的電子出版品，你的捐款將可以讓我們維持更高的出版品質，及提供免費下載.ZIP檔案。請閱讀本頁以了解如何在經濟上支援我們。「麻省理工開放式課程」檔案下載的文件內容與課程網頁版本相同，讓你在本機上就可以瀏覽OCW課程材料。

課程下載

9-520Spring-2003.zip

下載課程的方法

尋找並使用課程內容

開啟.ZIP並解壓縮後，就可使用瀏覽器開啟「麻省理工開放式課程」的html網頁。解壓後根目錄下的 Welcome.htm 檔案會將你導向該課程首頁。

.ZIP檔案中的課程內容與「麻省理工開放式課程」所出版的材料一樣，必需依照創作共享理念授權同意書規範。

常見問答集