Brief Development History

The earliest work on face recognition can be traced back at least to the 1950s in psychology (4) and to the 1960s in the engineering literature (5). Some of the earliest studies include work on facial expression of emotions by Darwin (6) [see also Ekman (7) and on facial profile-based biometrics by Galton (8)].

But research on automatic machine recognition of faces really started in the 1970s after the seminal work of Kanade (9) and Kelly (10). Over the past 30 years, extensive research has been conducted by psychophysicists, neuroscientists, and engineers on various aspects of face recognition by humans and machines.

Psychophysicists and neuroscientists have been concerned with issues such as whether face perception is a dedicated process [this issue is still being debated in the psychology community(11,12)], and whether it is done holistically or by local feature analysis. With the help of powerful engineering tools such as functional MRI, new theories continue to emerge (13).

Many of the hypotheses and theories put forward by researchers in these disciplines have been based on rather small sets of images. Nevertheless, many of the findings have important consequences for engineers who design algorithms and systems for the machine recognition of human faces.

Until recently, most of the existing work formulates the recognition problem as recognizing 3-D objects from 2-D images. As a result, earlier approaches treated it as a 2-D pattern recognition problem. During the early and middle 1970s, typical pattern classification techniques were used that measured attributes of features (e.g., the distances between important points) in faces or face profiles (5,9,10).

During the 1980s, work on face recognition remained largely dormant. Since the early 1990s, research interest in FRT has grown significantly. One can attribute this growth to several reasons: the increase in interest in commercial opportunities, the availability of real-time hardware, and the emergence of surveillance-related applications.

Over the past 18 years, research has focused on how to make face recognition systems fully automatic by tackling problems such as localization of a face in a given image or a video clip and by extracting features such as eyes, mouth, and so on. Meanwhile, significant advances have been made in the design of classifiers for successful face recognition. Among appearance-based holistic approaches, eigenfaces (14,15) and Fisherfaces (16-18) have proved to be effective in experiments with large databases.

Feature-based graph matching approaches (19) have also been successful. Compared with holistic approaches, feature-based methods are less sensitive to variations in illumination and viewpoint and to inaccuracy in face localization. However, the feature extraction techniques needed for this type of approach are still not sufficiently reliable or accurate.

During the past 8-15 years, much research has been concentrated on video-based face recognition. The still image problem has several inherent advantages and disadvantages. For applications such as airport surveillance, the automatic location and segmentation of a face could pose serious challenges to any segmentation algorithm if only a static picture of a large, crowded area is available.

On the other hand, if a video sequence is available, segmentation of a moving person can be accomplished more easily using motion as a cue. In addition, a sequence of images might help to boost the recognition performance if we can use all these images effectively. But the small size and low image quality of faces captured from video can increase significantly the difficulty in recognition.

More recently, significant advances have been made on 3-D based face recognition. Although it is known that face recognition using 3-D images has many advantages than face recognition using a single or sequence of 2-D images, no serious effort was made for 3-D face recognition until recently. This delay was mainly caused by the feasibility, complexity, and computational cost to acquire 3-D data in real-time. Now, the availability of cheap, real-time 3-D sensors (21) makes it much easier to apply 3-D face recognition.

Recognizing a 3-D object from its 2-D images poses many challenges. The illumination and pose problems are two prominent issues for appearance-based or image-based approaches (22). Many approaches have been proposed to handle these issues, and the key here is to model the 3-D geometry and reflectance properties of a face. For example, 3-D textured models can be built from given 2-D images, and the images can then be used to synthesize images under various poses and illumination conditions for recognition or animation.

By restricting the image-based 3-D object modeling to the domain of human faces, fairly good reconstruction results can be obtained using the state-of-the-art algorithms. Other potential applications in which modeling is crucial includes computerized aging, where an appropriate model needs to be built first and then a set of model parameters are used to create images that simulate the aging process.