1. Statistics 695A: Machine Learning, Fall 2004
Machine Learning Student Responsibilities
Machine learning consists of theory, methods, algo- Seminar Presentation
rithms, models, and software for enabling a computer to
learn from data and to improve learning performance as Each student should present a journal or conference pa-
the amount of data increases. The area has been developed per, or a small set of papers on the same topic. Students
by people in many fields such as computer science, statis- can work in small groups, where the maximum group size
tics, engineering, the biological sciences, and the physical will be determined by the class size, but each student in
sciences. But applications of machine learning can be car- the group should carry out a portion of the presentation.
ried out in any field in which it is necessary to learn from Students are free to select topics as long as they are within
data. the scope of the course. The choices should be submitted
to the TA by Sept. 30, and the papers conveyed as pdf
files. The complete schedule of the presentations will be
Prerequisites and Questions announced on October 7. Each presentation will be re-
hearsed with the TA a week prior to the class presentation.
Permission of the instructor is required. Each student is expected to read the papers of each of the
No previous course in machine learning is expected. Pre- other presentations before it is given. At the end of each
requisites are a basic knowledge of presentation, there will be a discussion session.
probability Students are encouraged to discuss plans with the TA
mathematics through multi-variable calculus and and instructor who will be happy to make comments and
linear algebra suggestions about topics.
least-squares fitting of parametric functions to data
(Gauss’s machine learning tool)
Project Paper
The level of the course will be comparable to that in the
Each student should conduct a study, preferably in the
book Machine Learning by Tom M. Mitchell.
area of their presentation topic, testing new ideas for tools
Please send questions about the course or permission to or using current tools to learn from a set of data. Students
attend to the instructor at wsc@stat.purdue.edu. can work in small groups, where the maximum group size
will be determined by the class size. Students should
report on this work in a short paper of about 8 pages.
Course Orientation and Objectives There should be a discussion of the tasks carried out along
While the prerequisites do not require previous knowl- with the interpretation of the results. To write the pa-
edge of machine learning, the course nevertheless has a re- pers, students should find the most current Web page of a
search orientation. The objectives are to provide students leading machine learning conference and use the template
with the opportunity and guidelines of the conference to prepare the paper, but
to understand in depth selected areas of machine keeping it to about 8 pages. The paper should have the
learning quality of presentation, at least, of those appearing in such
a conference. The paper should be submitted to the TA
to review research in machine learning
electronically as pdf by November 30. The papers will be
to experience either the development of machine reviewed by the TA and comments returned on December
learning tools or their use to learn from a set of data 7. Students should revise the paper, if necessary, based on
to experience giving a research talk these comments, and send to the instructor by December
to experience writing a research paper. 15.
1

2. Students are encouraged to discuss plans with the TA such tools, and to evaluate the performance of the tool in
and instructor who will be happy to make comments and learning from the data.
suggestions about topics or available data. Robust Learning: What is often overlooked is that in ap-
plications a very small fraction of the data can dramati-
cally distort the learning output, forcing the results to fol-
Proposed Instructor Lecture Topics low their aberrant behavior. Visualization can often reveal
N.B. This list could change based on the make-up and such distortion, but robust learning methods try to prevent
interests of the class. such distortion in the first place.
Perceptrons and Artificial Neural Nets: Classical tools, Learning Theory: There will be a sprinkling of “theory”,
one of the first ones that became a part of what we think of not mathematical derivations, but rather epistemological
today as machine learning. The ANN structure resembles foundations such as why it helps to think of learning from
biological neural networks, and is made up of multilayer data as an updating of knowledge, and why it is impor-
networks of perceptrons. tant to learn not just the pattern in a set of data, but the
Local Learning: This computer intensive approach departures of the data from the pattern.
works locally in a multidimensional space, which makes
it amenable to parallel computation. Despite this new-
age usage, basic ideas got started in the 19th and early Reading about Instructor Lecture Topics
20th Centuries by brilliant actuaries learning about death
Various writings on the topics will be available on the
and sickness rates as a function of age. In the 1970s
course Web page.
statisticians began what would become a big industry in
statistics research that is often called “nonparametric re-
gression”. Both loess (locally weighted regression) and
projection pursuit regression became widely used tools. Course Instructor
Machine learning researchers picked up on this work and William S. Cleveland has been a Professor of Statistics
made many advances in beautiful applications, for exam- and Computer Science at Purdue University since January
ple, to robotics. 2004. Previous to this he was a Distinguished Member of
Bayesian Learning: Bayesian learning, single-handedly Technical Staff in the Statistics and Data Mining Research
intellectually revived by Jimmie Savage in the 1950s and Department at Bell Labs, Murray Hill.
1960s, is now advancing at a furious pace due to compu- His areas of research include machine learning, data
tational breakthroughs over the past two decades. mining, data visualization, statistical methods and mod-
Bayesian Networks: Models with a network structure els, and computer networking.
and computational efficiencies based on certain assump- Cleveland has introduced tools for local machine learn-
tions. There are a number of marvelous applications to ing, as well as many visualization tools, that are widely
software systems, for example, developed at Microsoft. used in engineering, science, medicine, and business. He
An L.A. Times article quotes Bill Gates in an inter- has participated in the design and implementation of soft-
view: ”Microsoft’s competitive advantage is its expertise ware for these tools that is now a part of many commercial
in Bayesian networks.” systems. He has been involved in many projects apply-
Visualization: It cannot be emphasized too strongly how ing machine learning and visualization tools to data from
important it is to use visualization tools in any application several fields including environmental science, customer
of learning tools to real data. Visualization helps guide the opinion polling, visual perception, and computer network-
choice of tools and the estimation of parameters in such ing.
2