9.520 2003春季課程:統計學習理論及應用(Statistical Learning Theory and Applications, Spring 2003)
翻譯:周科
編輯:馬景文 劉慕華

Designing and building a system that will function the same way as a human visual system, but without getting bored, and with a greater degree of accuracy. (Image courtesy of Poggio Laboratory, MIT Department of Brain and Cognitive Sciences.)
課程重點
Support vector machines have proven to be very useful in classification networks. These SVMs are now being used by drivers for pedestrian avoidance. This is one of the first truly universal applications of this technology.
本課程是為了計畫在計算神經科學領域工作的高年級研究生開設。作業集中在一些使電腦更有效解決問題的功能。可供學生選擇的專題題目是基於這領域仍未解決的問題。課程結束後,學生應當可以解決這些問題的一二,也能對其他問題架構解決方法。
This course is for upper-level graduate students who are planning careers in computational neuroscience. The assignments focus on some of the functions needed to make problem-solving more efficient for computer systems. The project topics students can choose from are based on unsolved problems in the field today. By the conclusion of this course, students should be able to solve one or two of these problems, and should be able to frame an approach to the rest of them.
課程描述
Focuses on the problem of supervised learning from the perspective of modern statistical learning theory starting with the theory of multivariate function approximation from sparse data. Develops basic tools such as Regularization including Support Vector Machines for regression and classification. Derives generalization bounds using both stability and VC theory. Discusses topics such as boosting and feature selection. Examines applications in several areas: computer vision, computer graphics, text classification and bioinformatics. Final projects and hands-on applications and exercises are planned, paralleling the rapidly increasing practical uses of the techniques described in the subject.
師資
Tomaso Poggio 教授
Sayan Mukherjee 博士
Ryan Rifkin 博士
Alex Rakhlin
上課時數
每週2節
每節1.5小時
程度
其他資源
|
回應
告訴我們您對本課程或「開放式課程網頁」的建議。
聲明
麻省理工學院開放式課程認可開放式課程計畫(OOPS)的翻譯計畫,開放式課程計畫(OOPS)乃是運用其獨立團隊、獨立資源、獨立流程進行翻譯計畫之團隊。
所有麻省理工學院開放式課程之材料皆以麻省理工學院開放式課程創作共享授權發佈,所有之翻譯資料皆由開放式課程計畫(OOPS)所提供,並由其負翻譯品質之責任。
此處麻省理工學院開放式課程之資料乃由 開放式課程計畫(OOPS) 譯為正體中文。麻省理工學院開放式課程在此聲明,不論是否遭遇或發現相關議題,麻省理工學院開放式課程、麻省理工學院教師、麻省理工學院校方並不對翻譯正確度及完整性作保證。上述單位並對翻譯後之資料不作明示或默許對任一特定目的之適合性之保證、非侵權之保證、或永不出錯之保證。麻省理工學院校方、麻省理工學院開放式課程對翻譯上之不正確不負任何責任。由翻譯所引發任何關於此等資料之不正確或其他瑕疵,皆由開放式課程計畫(OOPS)負全責,而非麻省理工學院開放式課程之責。
原文聲明
兩個問題集參見作業。
一些最有希望的專題:
小資料集的假設檢驗
MED和正則化的聯繫
支援向量機理論和實驗的特徵提取
貝葉斯分類規則和支援向量機
用於分類輸入輸出隱馬可夫模型與直接分類的比較
重用測試集資料挖掘邊界
大規模非線性最小平方正則化
基於視角的分類
局部與全局分類器的比較:實驗和理論
再生核希爾伯特空間不變數衡量古代數學
集中實驗(點乘和平方距離的比較)
解分類:樹結構的泛化實驗
核的合成與選擇
正則化的貝葉斯解釋以及在具體支援向量機中的應用
歸納法的歷史:從Kant到Popper以及現狀
貝葉斯先驗
資源
麻省理工學院的生物和計算學習中心(CBCL) ,自成立以來就堅信學習是生物和人工智慧中的智慧問題核心,並且是理解人類大腦如何工作和製造智慧型機器的大門。CBCL(生物和計算學習中心)以跨學科方法研究學習問題,主要目標是孕育數學、工程學和神經科學有關學習的嚴肅研究。CBCL(生物和計算學習中心)屬於麻省理工學院的大腦與認知科學系 ,並與McGovern大腦研究所 和麻省理工學院的 人工智慧實驗室 有聯繫。
See the assignments page for the two problem sets.
Some of the most promising projects:
Hypothesis testing with small sets
Connection between MED and regularization
Feature selection for SVMs theory and experiments
Bayes classification rule and SVMs
IOHMMs evaluation of HMMs for classification vs. direct classification
Reusing the test set datamining bounds
Large-scale nonlinear least square regularization
Viewbased classification
Local vs. global classifiers experiments and theory
RKHS invariance to measure historical math
Concentration experiments (dot product vs. square distance)
Decorrelating classifiers: experiments about generalization using a tree of stumps
Kernel synthesis and selection
Bayesian interpretation of regularization and in particular of SVMs
History of induction from Kant to Popper and current state
Bayesian Priorhood
Resources
The Center for Biological and Computational Learning (CBCL) at MIT was founded with the belief that learning is at the very core of the problem of intelligence, both biological and artificial, and is the gateway to understanding how the human brain works and to making intelligent machines. CBCL studies the problem of learning within a multidisciplinary approach. Its main goal is to nurture serious research on the mathematics, the engineering and the neuroscience of learning. CBCL is based in the Department of Brain and Cognitive Sciences at MIT and is associated with the McGovern Institute for Brain Research and with the Artificial Intelligence Laboratory at MIT.
課程表底部有三堂課:兩個數學營和一個額外題目。如學生需要這背景以理解下一系列的講座和問題時,這〔三堂課〕才會給出。
There are three sessions, two Math Camps and an extra topic, at the bottom of the calendar. These will be given when students require the background needed to understand the next series of lectures and problems.
|
本課程沒有教科書。所有必需資訊會在每節課以幻燈片發表。以下列出的書本和論文是十分有用的一般參考資料,特別是理論角度。額外閱讀材料列 課堂講稿PDF文件。
There is no textbook for this course. All the required information will be presented in the slides associated with each class. The books and articles listed below are useful general reference reading, especially from the theoretical viewpoint. Additional readings are listed in the lecture note PDF files.
Cristianini, N.和J. Shawe-Taylor. 《支持向量機導論》 Cambridge, 2000
Cristianini, N., and J. Shawe-Taylor. Introduction To Support Vector Machines. Cambridge, 2000.
Cucker, F.和S. Smale.〈關於學習的數學基礎〉 《美國數學學會通訊》 2002
Cucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society. 2002.
Devroye, L., L. Gyorfi和G. Lugosi.《模式識別的機率理論》Springer, 1997.
Devroye, L., L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer, 1997.
Evgeniou, T., M. Pontil和T. Poggio.〈正則化網路和支援向量機〉《計算數學的進展》2000.
Evgeniou, T., M. Pontil, and T. Poggio. "Regularization Networks and Support Vector Machines." Advances in Computational Mathematics. 2000.
Poggio, T.和S. Smale.〈學習的數學:處理數據〉《美國數學協會公告》2003
Poggio, T., and S. Smale. "The Mathematics of Learning: Dealing with Data." Notices of the AMS. 2003.
Vapnik, V. N.《統計學習理論的本質》Springer,1995
Vapnik, V. N. The Nature of Statistical Learning Theory. Springer, 1995.
———.《統計學習理論》Wiley,1998
———. Statistical Learning Theory. Wiley, 1998.
問題集1:核希爾伯特空間 (PDF)
問題集2:徑向基函數插值方法 (PDF)
專題 (PDF)
就期末專題,學生從以下的建議題目自選一題,解決所描述的問題。如果學生願意,可以請教授或助教批準自己提出的專題意見。
題目
小資料集的假設檢驗
MED和正則化的聯繫
支援向量機理論和實驗的特徵提取
貝葉斯分類規則和支援向量機
用於分類輸入輸出隱馬可夫模型與直接分類的比較
重用測試集資料挖掘邊界
大規模非線性最小平方正則化
基於視角的分類
局部與全局分類器的比較:實驗和理論
再生核希爾伯特空間不變數衡量古代數學
集中實驗(點乘和平方距離的比較)
解分類:樹結構的泛化實驗
核的合成與選擇
正則化的貝葉斯解釋以及在具體支援向量機中的應用
歸納的歷史從Kant到Popper以及現狀
貝葉斯先驗
Problem Set 1: Kernel Hilbert Spaces (PDF)
Problem Set 2: RBF Interpolation Schemes (PDF)
Project (PDF)
For the final project, students select from one of the following suggested topics, and solve the problem that is described. If students prefer, they can bring their own project ideas to the professor or TAs for approval.
Topics:
Hypothesis testing with small sets
Connection between MED and regularization
Feature selection for SVMs theory and experiments
Bayes classification rule and SVMs
IOHMMs evaluation of HMMs for classification vs. direct classification
Reusing the test set datamining bounds
Large-scale nonlinear least square regularization
Viewbased classification
Local vs. global classifiers experiments and theory
RKHS invariance to measure historical math
Concentration experiments (dot product vs. square distance)
Decorrelating classifiers: experiments about generalization using a tree of stumps
Kernel synthesis and selection
Bayesian interpretation of regularization and in particular of SVMs
History of induction from Kant to Popper and current state
Bayesian Priorhood
|
為了離線或窄頻使用者,「麻省理工開放式課程」免費提供了完整的課程檔案下載。「麻省理工開放式課程」永遠是免費開放的電子出版品,你的捐款將可以讓我們維持更高的出版品質,及提供免費下載.ZIP檔案。請閱讀本頁以了解如何在經濟上支援我們。「麻省理工開放式課程」檔案下載的文件內容與課程網頁版本相同,讓你在本機上就可以瀏覽OCW課程材料。
課程下載
下載課程的方法
尋找並使用課程內容
開啟.ZIP並解壓縮後,就可使用瀏覽器開啟「麻省理工開放式課程」的html網頁。解壓後根目錄下的 Welcome.htm 檔案會將你導向該課程首頁。
.ZIP檔案中的課程內容與「麻省理工開放式課程」所出版的材料一樣,必需依照創作共享理念授權同意書規範。
常見問答集
麻省理工開放式課程的.ZIP檔案大小約介於 1MB 到 100MB間,多為 25MB-30MB。下載.ZIP檔案時可能佔用您一些時間,所需時間依您網路的連線速度而定。
為什麼.ZIP檔案裏少了某些課程材料,像是影片課程或模擬媒體?
像是影片、Java Applet等材料,以及一些沒有直接放在「麻省理工開放式課程」伺服主機上的特殊內容,在.ZIP裏亦不直接收錄,而是以連結的形式提供。你可以瀏覽內容網頁來下載這些影音檔案,請閱讀開放式系統說明網頁的我能夠把RealPlayer的影像檔案儲存到我的硬碟嗎?
如果下載課程時發生問題,我該找誰?
請寄信到 意見反應信箱或ocw@mit.edu。
留下您對本課程的評論 |
標籤 現有標籤:1 |
有關本課程的討論
課程討論
影片,教材無法下載
影片不能下載zip