ECE 6254, Spring 2016, Home

Home

Instructor
Justin Romberg
Email: jrom@ece.gatech.edu
Office: Centergy 5227
Phone: 404-894-3930

Description
This course will explore a range of modern “data driven” approaches to signal processing. In contrast to most traditional approaches to signal processing, in this course we will focus on how to learn effective models from data and how to apply these models to practical signal processing problems.

We will approach these problems from the perspective of statistical inference. We will study both practical algorithms for statistical inference and theoretical aspects of how to reason about and work with probabilistic models. We will consider a variety of applications, including classification, prediction, regression, clustering, modeling, and data exploration/visualization.

Download the syllabus (pdf)

Outline (preliminary and subject to change)

I. Supervised Learning
– The Bayes classifier and the likelihood ratio test
– Nearest neighbor classification
– Linear classifiers
plugin classifiers (LDA, logistic regression, Naïve Bayes)
the perceptron learning algorithm
maximum margin principle, separating hyperplanes, and SVMs
– From linear to nonlinear: feature maps and the kernel trick
– Kernel-based SVMs
– Generalization Theory I
concentration inequalities
VC dimension
VC generalization
– Neural networks
– Linear regression
least-squares
regularization
the LASSO
empirical risk minimization
– Generalization Theory II
overfitting
bias-variance trade-off
error estimation and validation

II. Unsupervised Learning
– Density estimation
– Linear dimensionality reduction, PCA
– Clustering
$k$-means
GMMs and the EM algorithm
spectral clustering
Euclidean embeddings
mutltidimensional scaling
manifold learning
– Latent variables and matrix factorization
dictionary learning
non-negative matrix factorization
blind source separation
– Feature selection