Instructor: Dr. Anuj Srivastava (Room OSB 106D) anuj@stat.fsu.edu

Office hours: Tuesday and Thursday 10:30 – 11:30am or by appointment.

Class Website: http://stat.fsu.edu/~anuj/classes/5106-f-09.php

Grader: Mr. Muffasir Badshah muffasir@stat.fsu.edu

Course Objective:

• To gain an understanding of the techniques and ideas used in implementing mathematical/statistical formulations on computers, with a focus on common statistical approaches. Students will also obtain practice in using Matlab.

Prerequisites: Probability theory (discrete and continuous random variables), linear algebra, and advanced calculus.

Reference Texts: There is no specific textbook requires for this class. Instead, it may be useful to purchase Matlab software (student version, approx. $100 in FSU bookstore). I will use material from these books to prepare class notes.

1. Computational Statistics G. H. Gavins and J. A. Hoeting (Wiley Series 2005)
2. Matrix Computations, G. Golub and R. VanLoan (Johns Hopkins University Press, 96).
3. Simulation by Sheldon Ross, Second Edition, (Academic Press, 1997).
4. Random Number Generation and Monte Carlo Methods by James Gentle (Springer Verlag, 98).
5. Pattern Classification by R. O. Duda, P. E. Hart, and D. G. Sork (Wiley-Interscience, Second Edition, 2001)
6. Dynamic programming and Optimal Control volume 1 and 2, Second Edition by D. P. Bertsekas (Athena Scientific, 2000)

Topics Covered:

1. Linear Methods for Regression Analysis : Floating point arithmetic and error analysis, Multiple regression analysis, orthogonalization by Householder transformations; singular value decomposition; linear projection methods for dimension reductions: principal component analysis (PCA), PCA and linear regression.
2. Nonlinear Methods for Maximum Likelihood Estimation: numerical optimization, scaled-iterations, Newton-Raphson method, maximum likelihood estimation, missing data problems, expected-maximization (EM) algorithm, mixture of Gaussians.
3. Elementary Pattern Recognition
   1. Clustering: distances, data normalization, hierarchical clustering, partitional clustering, k-means clustering, Fisher's discriminant analysis, PCA versus FDA.
   2. Classification: Bayesian classification, minimax criterion, k-nearest neighbor classification.
4. Simulation of Random Variables: Uniform random number generators, modular arithmetic, combination generators, discrete and continuous random variables; inverse transform method, acceptance-rejection method, mixture methods.
5. Monte-Carlo methods for Integration:
   1. General MC formulation: sample mean and variance
   2. Importance sampling (ex: Cauchy), optimal choice of sampling density
   3. Variance reduction techniques: antithetic variables, control variates, variance reduction by conditioning, importance sampling via twisted simulations.
6. Dynamic Programming: shortest path determination, deterministic problem, string matching.
7. Special Topics
   1. Density Estimation Techniques: one-dimensional estimation problems, histograms, kernel-based methods, smoothing.
   2. Directional Statistics: statistics on a circle, computation of mean, variance, von Mises density

   Tentative Schedule : Linear Methods (2 weeks), Nonlinear Methods (2 weeks), Pattern Recognition (2 weeks), Simulation of Variables (2 weeks), Monte-Carlo Methods (2 weeks), Dynamic Programming (2 weeks), Special Topics (2 weeks).

   Tentative topics for II: Markov chain Monte Carlo methods: (properties of Markov chains, Gibb’s sampler, Metropilis-Hastings, hybrid methods). Kalman Filtering and Sequential Monte Carlo method. Perfect sampling, Simulated annealing and stochastic optimization. Estimator performance analysis: bootstrap and jackknife. Classification: boosting. Markov Random Fields in image analysis: smoothing, de-noising. edge detection. Simulation of stochastic processes.