Handbook of Face Recognition

Editors: Stan Z. Li and Anil K. Jain

Springer. New York. ISBN# 0-387-40595-x.

16 Chapters. 400 pages. Hardcover.

To Order from Bookstore: in USD, in GBP, in EUR, in JPY

Table of Contents

1.        Introduction (Stan Li, Chinese Academy of Sciences  & Anil Jain, Michigan State University)

1.1        Face Recognition Processing

1.2        Analysis in Face Subspaces

1.3        Technical Challenges

1.4        Technical Solutions

1.5        Current Technology Maturity

2.        Face Detection (Stan Li, Microsoft Research Asia)

2.1        Appearance and Learning Based Approach

2.2        Preprocessing

2.2.1  Filtering Using Skin Color

2.2.2  Image Normalization

2.2.3  Multi-Gaussian Clustering

2.3        Neural and Kernel Methods

2.4        Boosting Based Methods

2.4.1  Haar-Like Features

2.4.2  Learning Feature Selection

2.4.3  Learning Weak Classifiers

2.4.4  Boosted Strong Classifier

2.4.5  FloatBoost Learning

2.4.6  Cascade of Strong Classifiers

2.5        Dealing with Head Rotations

2.6        Post-Processing

2.7        Performance Evaluation

3.        Modeling Facial Shape and Appearance (Tim Cootes & Chris Taylor, Haizhuang Kang & Vladimir Petrovic, University of Manchester)

3.1        Background

3.2        Statistical Models of Appearance

3.2.1  Statistical Shape Models

3.2.1.1    Aligning sets of shapes

3.2.1.2    Statistical Models of Variation

3.2.1.3    Examples

3.2.2  Statistical Texture Models

3.2.3  Combined Models of Appearance

3.3        Active Shape Models

3.3.1  Modeling Local Structure

3.3.2  ASM Search Algorithm

3.3.3  Examples of Search

3.4        Active Appearance Models

3.4.1  Quality of Match

3.4.2  Iterative Model Refinement

3.4.3  Examples of AAM Search

3.4.4  Alternative Strategies

3.5        Discussion

4.        Parametric Face Modeling and Tracking (Jorgen Ahlberg, Swedish Defense Research Agency & Fadi Dornaika, Linkoping University)

4.1        Introduction

4.2        Previous Work

4.3        Parametric Face Modeling

4.3.1  Facial Parameterizations

4.3.1.1    Facial Action Coding System

4.3.1.2    MPEG-4 Facial Animation

4.3.2  Face Models

4.3.3  Adapting a model to an image

4.4        The Tracking Problem

4.4.1  Appearance-based or Featureless Tracking

4.4.2  Feature-based tracking

4.5        Tracker examples

4.5.1  A feature-based tracker using EKF and SfM (Str�m)

4.5.2  Appearance-based trackers (Ahlberg, LaCascia)

4.5.3  Combining appearance and feature-based tracking (Dornaika)

4.6         Discussion

5.        Illumination Modeling Illumination Modeling for Face Recognition (Ronen Basri, The Weizmann Institute of Science & David Jacobs, University of Maryland)

5.1        Empirical Motivations for Linear Lighting Models

5.2        Linear lighting models: without shadows

5.3        Attached shadows: non-linear constraints

5.4        Spherical Harmonics for lighting modeling

5.4.1  Linear models of attached shadows

5.4.2  Non-linear constraints

5.5        Specularity and cast shadows

5.6        Reconstruction

6.        Facial Skin Color Modeling (Birgitta Martinkauppi & Matti Pietik�inen,  University of Oulu)

6.1        Introduction

6.2        Skin Colors in Different Color Spaces

6.3        Skin Color Models

6.4        Skin Color Correction

6.5        Comparison of Different Models

6.6        Discussion

7.        Face Recognition in Subspaces (Gregory Shakhnarovich, Massachusetts Institute of Technology Baback Moghaddam, MERL Cambridge Research)

7.1        Dimensionality

7.1.1  Representation, Oversampling (Nyquist and beyond)

7.1.2  "Imagespace": Intrinsic Degrees-of-Freedom

7.1.3  "Facespace": Pixels vs. Basis Functions

7.1.4  Linear Algebraic Concepts

7.1.4.1    KLT, SVD and Eigenvector Decompositions

7.1.4.2    Rank, DOF and the Eigenspectrum

7.2        Linear Subspaces

7.2.1  Neural Network formulations

7.2.2  Karhunen-Loeve, PCA and "Eigenfaces"

7.2.3  Linear Discriminants: "Fisherfaces"

7.2.4  "Dual Eigenspace" (Bayesian) Methods

7.2.5  Factor Analysis, ICA & Source Separation

7.2.6  Multidimensional SVD: "Tensorfaces"

7.2.7  Local Feature Analysis [optional]

7.3        Nonlinear Subspaces

7.3.1  Auto-Encoder MLPs

7.3.2  Principal Curves/Surfaces (NLPCA & Regression)

7.3.3  Kernel-PCA and Kernel-Fisher Methods

7.3.4  IsoMap, LLE and variants [optional]

7.4        Methodology and Usage [brief discussion + pointers]

7.4.1  Appearance-Based Representations

7.4.2  Multiple View-Based Approach for Pose

7.4.3  Fisher & "Illumination Cones" for Lighting

7.4.4  2D/3D Shape-Texture Models

8.        Face Tracking and Recognition from Video (Rama Chellappa, ShaoHua Zhou, University of Maryland)

8.1        Review

8.2        Simultaneous Tracking and Recognition from Video

8.2.1  A Time Series State Space Model for Recognition

8.2.2  The Posterior Probability of Identity Variable

8.2.3  Sequential Importance Sampling Algorithm

8.2.4  Experimental Results

8.3        Enhancing Tracking and Recognition Accuracy

8.3.1  Modeling Inter-frame Appearance Changes

8.3.2  Modeling Appearance Changes between Frames and Gallery Images

8.3.3  Experimental Results

8.4        Issues and Discussions References

9.        Face Recognition across Pose and Illumination (Ralph Gross, Simon Baker, Iain Matthews, Takeo Kanade, Carnegie Mellon University)

9.1        Review

9.2        The CMU Pose, Illumination, and Expression (PIE) Database

9.3        Eigen Light-Fields for Face Recognition Across Pose

9.4        Normalizing for Illumination

9.5        Modeling Pose and Illumination

10.   Morphable Models of Faces (Sami Romdhani, University of Basel & Volker Blanz, Max-Planck-Institut fur Informatik & Curzio Basso & Thomas Vetter, University of Basel)

10.1    Morphable Model for Face Analysis

10.1.1      Three dimensional representation

10.1.2      Correspondence based representation

10.1.3      Face Statistics

10.2    3D Morphable Model Construction

10.2.1      Dense correspondences computed by optical flow

10.2.2      Face Space

10.3    A Morphable Model to Synthesize Images

10.3.1      Shape Projection

10.3.2      Inverse Shape Projection

10.3.3      Illumination and Color Transformation

10.3.4      Image Synthesis

10.4    Image Analysis with a 3D Morphable Model

10.4.1      Stochastic Newton Descend

10.4.2      Inverse Image Compositional Alignment

10.5    Identification

10.5.1      Face Images Databases

10.5.2      Pose Variation

10.5.3      Pose and Illumination Variations

10.5.4      Identification Confidence depends on Fitting Accuracy

10.5.5      Virtual views as an aid to standard face recognition algorithms

11.   Facial Expression Analysis (Ying-li Tian, IBM Watson Research Center & Takeo Kanade, Carnegie Mellon University & Jeffrey Cohn, University of Pittsburgh)

11.1    Introduction

11.2    Problem Space for Face Expression Analysis

11.2.1      Level of Description

11.2.2      Individual Differences in Subjects

11.2.3      Degrees of Facial Expression

11.2.4      Databases

11.2.5      Reliability of Ground truth

11.2.6      Lighting Changes

11.2.7      Head Orientation and Scene Complexity

11.2.8      Relation to Other Facial Behavior or Non-facial Behavior

11.3     Automatic Facial Expression Analysis

11.3.1      Face Detection or Head Detection

11.3.2      Facial Data Extraction

11.3.3      Feature Based Methods

11.3.4      Template Based Methods

11.3.5      Facial Expression Recognition

11.3.6      Facial Expression Recognition from Static Image

11.3.7      Facial Expression Recognition from Image Sequences

11.4    Discussion

12.   Face Synthesis (ZiCheng Liu, Bai-Ning Guo, Microsoft Research)

12.1    Review

12.2    Face Modeling

12.2.1      Face modeling from an image sequence

12.2.2      Face modeling from two views

12.2.3      Face modeling from a single view

12.3    Face relighting

12.4    Facial animation capturing

12.5    Speech driven facial animation

12.6    Facial expression mapping

12.7    Expression morphing

12.8    Expression detail synthesis

13.   Face Databases (Ralph Gross, Carnegie Mellon University)

13.1    Databases for Face Detection

13.1.1      MIT-CMU Test Set

13.1.2      CMU Test Set II

13.1.3      Others

13.2    Databases for Face Recognition

13.2.1      FERET

13.2.2      Yale Face Database B

13.2.3      AR Database

13.2.4      PIE

13.2.5      Notre Dame HumanID database

13.2.6      Asian Face Database

13.2.7      Others

13.3    Databases for Expression Analysis

13.3.1      JAFFE Dataset

13.3.2      Cohn-Kanade DB

13.3.3      Others

13.4    Other Modalities

13.4.1      3D Faces

13.4.2      Hyperspectral Images

14.   Performance Evaluation (P. Jonathan Phillips, Patrick Grother, Ross Micheals,  National Institute of Standards and Technology)

14.1    Introduction

14.2    Performance Measures

14.2.1      Watch List

14.2.2      Verification

14.2.3      Identification

14.3    FRVT 2002 Protocol

14.3.1      Similarity Scores

14.3.2      Virtual Galleries and Probe Sets

14.3.3      Normalization

14.4    Variability and Demographics

14.5    Advanced Statistical Techniques

14.6    Conclusion

15.   Psychological and Neural Perspectives (Alice J. O'Toole, University of Texas)

15.1    Characteristics of Human Face Recognition

15.1.1      Extracting information from the human face

15.1.1.1             Identity

15.1.1.2             Visual categories: sex, age, race

15.1.1.3             Movement and social signals

15.1.1.4             Facial expressions and emotions

15.1.2      Factors that affect performance

15.1.2.1             Stimulus factors

15.1.2.2             Subject factors

15.1.2.3             Photometric factors

15.2    The Neural Systems Underlying Human Face Recognition

15.2.1      The Multiple Systems Model

15.2.1.1             Fusiform face area: identification and categorization

15.2.1.2             Superior temporal sulcus - Movement and social signals

15.2.1.3             Neural processing of emotion from facial expression

16.   Face Recognition Applications (Thomas Huang, Ziyou Xiong, ZhenQiu Zhang, University of Illinois)

16.1    Introduction

16.2    Face ID

16.3    Access Control

16.4    Security

16.5    Surveillance

16.6    Smart Cards

16.7    Law Enforcement

16.8    Face Databases Applications

16.9    Multimedia Management

16.10        Human Computer Interaction

16.11        Other Applications

16.12        Limitations of Current Face Recognition Applications   

Back to Handbook of Face Recognition