Signal compression is generally considered as a signal processing or communications technique. In general it consists of the reduction of an analog or high rate digital signal to a relatively low rate digital representation for high speed transmission or efficient storage while preserving the best possible reproduction quality. Statistical classification and regression are usually thought of as information extraction techniques, the mapping of a chunk of input data into a guess of the value of an unobserved random variable.  The mapping is considered as statistical classification or detection or segmentation if the guessed quantity is discrete, for a example a label of a group of image pixels as representing a text, photo, background. or other. The mapping is considered as statistical regression or estimation if the unknown quantity is continuous, as in the estimation of a local covariance matrix or a prediction of the mean of an adjacent pixel block.  The two fields of compression and classification/regression have had largely disjoint histories, literature, and audiences. In recent years, however, their common themes and methods have been more recognized and exploited for both theory and applications.

This tutorial is aimed at surveying fundamental ideas, theory, algorithms, and properties for compression and classification/regression systems from a unified viewpoint which takes into consideration related issues in density estimation and modeling based on observed training or learning data. The goals are to demonstrate the intimate connections between compression and classification, how each type of signal processing can cooperate with the other, and how the two can be combined. Many of the ideas will be exemplified through an interpretation of LPC/CELP speech coding and possible applications to image compression and segmentation.

Outline:
1. Introduction and outline
2. Information sources: real and models
 A. Random vectors and processes
 B. Mixtures and composite sources
 C. Speech, images, video, ...
 D. Gauss models and Gauss mixture models
 E. Waveform and model distortion measures
     squared error, input and output-weighted quadratic distortion,
     Bayes risk, minimum discrimination information distortion and
     its relation to ``maximum entropy'' methods

3. Quantizers, source codes, and classifiers
 A. Vector quantization --- Shannon's source coding model
 B. Classification, regression, and density estimation:
    maximum a posteriori, maximum likelihood,
    partition and nearest neighbor methods,
    kernel methods, minimum discrimination information (relative
    entropy) methods, other related methods
    classification as quantization and vice versa
 C. Classification/regression based on quantized data
 D. Joint quantization and classification/regression
 E. Optimality properties and clustering
 F. Asymptotic theory:
    Bennett/Zador high rate theory
    Shannon large dimension theory
 G. Robust coding: the Gaussian model as worst case

4. Code Structures for Quantization and Classification
 A. Model quantization and density estimation
 B. Classified and universal quantization
 C. Classification and quantization for mixture and composite sources
 D. LPC/CELP speech coding as minimum discrimination information
    quantization plus mixture quantization, possible extensions to image
    coding and segmentation