Machine Learning
Izhak Shafran

News   Course Information   Lectures   Textbooks   Schedule   Links


Course Information

This course aims to provide theoretical foundations and practical experience in statistical learning methods or machine learning. The techniques covered in this course have wide application including but not limited to bioinformatics, speech and image processing, robotic planning and control, diagnostic systems, complex system modeling, and iterative optimization. Students will gain practical experience implementing and evaluating systems applied to pattern recognition, prediction, and optimization problems. Suggested prerequisites: Some experience with multi-variate calculus and linear algebra, at least one high-level programming language, and an elementary undergraduate course in probability and statistics. It should also enable students to read and discuss technical papers in statistical learning.

There will be no final exam. Instead, the course requires a final project of interest to student, chosen in consultation with the instructor. The project requires a written report and a final presentation. In most cases, the data, software toolkit, and key components for the project will be made available. The students will also get an opportunity to present papers related to the topics covered under the syllabus and related to their project.


Overview Administrivia, Learning -- Supervised, Unsupervised, Other
Introduction Bishop Ch.1 Background: Probability & Statistics, Gaussians, Detection Theory, Information Theory
Linear Models for Regression Bishop Ch.3 Linear basis function models; bias-variance; Bayesian linear regresson
Linear Models for Classification Bishop Ch.4 Discriminant functions; Probabilistic Generative Models; Probabilistic Discriminative Models
Decision Trees Mitchell Ch.3 Decision tree learning algorithm; bias issues
Neural Networks Bishop Ch.5 Feed-forward networks; Error back propagation; Regularization
Large Margin Machines Bishop Ch.7 Maximum margin machines; support vector machines; relevance vector machines
Perceptron Algorithm Article Perceptron algorithm; application to sequence processing (tagging); feature selection
Kernel Methods Bishop Ch.6 Dual representations; constructing kernels; radial basis functions
Feature Selection/Reduction Bishop Ch.12 Features transformations; PCA; kernel PCA; ICA; other feature selection strategies
Reinforcement Learning Mitchell Ch.13 Learning task; MDPs; Q learning more slides
Graphical Models Bishop Ch.8 Probabilisitic graphical models

Textbook and Other Useful Books

Note: For recently developed techniques, we will rely on selected papers, which will be provided in required readings.


Lectures Tue/Thu 1130 - 1300 hrs
Venue (West Campus, in-person)Central 123
Office hours Central 123, Thu 2:30pm -- 3:30pm with prior email confirmation


Relevant Software Tools & Resources
Journals, Conference Proceedings & Related Online Lectures
This page is maintained by Zak Shafran. Last updated on Jan 1, 2009.