COMPUTATIONAL METHODS IN STATISTICS II

Electronic Version of the Syllabus Syllabus.pdf

Instructor: Dr. Anuj Srivastava (Room OSB 106D, 644-8832) anuj@stat.fsu.edu

Office hours: Tuesday and Thursday 10:00 to 11:00am or by appointment.

CLass Location: Room 108 OSB, Tuesday and Thursday 12:30 to 1:45pm

Class Website: http://stat.fsu.edu/~anuj/classes/5107-s-10.php

Course Objective:

• To gain an understanding of advanced techniques and ideas used in implementing mathematical/statistical formulations on computers, with a focus on common statistical approaches. Some applications including image analysis will be studied. 

Prerequisites: Probability theory, linear algebra, advanced calculus, and Computational Statistics I.

Reference Texts: None of these texts are required. I will use material from these books.

1. Monte Carlo Statistical Methods by C. P. Robert and G. Cassella (Springer Text in Statistics, Springer Verlag, 1999).
2. A First Course in Stochastic Processes by Karlin and Taylor (Academic Press, 1975).
3. Sequential Monte Carlo Methods in Practice Edited by Doucet, de Freitas, and Gordon (Springer, 2001).
4. An Introduction to the Bootstrap by B. Efron and R. J. Tibhsirani (Chapman and Hall, 1993).

Topics Covered:

1. Markov chain Monte Carlo methods:
1. Introduction to stochastic processes, examples: Poisson counting process, random walk, Markov processes, stationarity, homogeneous Markov chains
2. Finite-state case: Markov chain, transition matrix, Perron-Frobenious theorem, irreducibility, aperiodicity, uniqueness of stationary measure
3. Countable case: recurrence, basic limit theorem of Markov chains, positive recurrence, ergodicity
4. Metropolis-Hastings algorithm: general algorithm, detailed balance conditions, independent M-H, random walk M-H.
5. Gibbs sampler: general algorithm, bivariate Gibbs sampler, completion Gibbs sampler, slice sampling.
6. Hybrid methods, convergence diagnosis.
2. Sequential Monte Carlo Methods: hidden Markov models, Kalman filter in linear Gaussian systems, importance sampling, resampling.
3. Perfect Sampling:
4. Stochastic Optimization: stochastic gradient, simulated annealing.
5. Bootstrap Methods: introduction, plug-in estimator, bootstrap estimate of standard error, confidence intervals based on bootstrap.
6. Markov Random Fields: potts model, ising model, image modeling.
7. Simulation of some specific processes: random walk on a circle, sphere, a homogeneous Poisson process in a plane
8. Image Modeling: spectral decomposition, component analysis, statistical models for components.
9. Statistics on nonlinear spaces: computations of means and variances on circle and sphere.

Grading Policy:

• 50% homework, 20% mid-term, 20% final, 5% class presentation, and 5% class participation. Both the mid-term and the final will be take-home projects. You will be asked to write reports on the projects. These projects are graded for both the solutions and the presentations. (grading will be relative) 

Homework:

• It will be assigned every Tuesday and will be due the next Tuesday. Late assignments will not be accepted. Please write clear and detailed answers to the homework problems. If a problem involves writing a program, submit a copy of the code with the solution. It is important to provide illustrative outputs of your programs to accompany the homework solutions. For instance, the graphs should be labeled and placed close to the associated written part. There will be separate assignments for undergraduates and graduates.

Attendance Policy:

• It is strictly required to attend all the classes. It is the student's responsibility to make up for the material covered in the class during any absence.

Academic Honor System:

"The Academic Honor System of The Florida State University is based on the premise that each student has the responsibility to: 1) Uphold the highest standards of academic integrity in the student’s work, 2) Refuse to tolerate violations of academic integrity in the academic community, and 3) Foster a high sense of integrity and social responsibility on the part of University community."

Please note that violations of this Academic Honor System will not be tolerated in this class. Specifically, incidents of plagiarism of any type or referring to any unauthorized material during examinations will be rigorously pursued by this instructor. Before submitting any work for this class, please read the "Academic Honor System" in its entirety (as found in the FSU General Bulletin and in the FSU Student HandBook) and ask the instructor to clarify any of its expectations that you do not understand).

Students with disabilities needing academic accommodation should: (1) register with and provide documentation to the Student Disability Resource Center; (2) bring a letter to the instructor indicating the need for accommodation and what type. This should be done during the first week of class.

PRINTED NOTES

GO TO TOP OF PAGE

An incomplete set of notes for part I and II can be downloaded from here (PDF format)

• Class Notes (01/01/10)

HOME ASSIGNMENTS

GO TO TOP OF PAGE