Commercial Application and Advanced Research Topics

Face recognition is a fascinating research topic. On one hand, many algorithms and systems have reached a certain level of maturity after 35 years of research. On the other hand, the success of these systems is limited by the conditions imposed by many real applications.

For example, automatic focus/exposure based on face detection has been built into digital cameras. However, recognition of face images acquired in an outdoor environment with changes in illumination and/or pose remains a largely unsolved problem. In other words, current systems are still far away from the capability of the human perception system.

Commercial Applications of FRT. In recent years, we have seen significant advances in automatic face detection under various conditions. Consequently, many commercial applications have emerged. For example, face detection has been employed for automatic exposure/focus in digital cameras, such as Powershot SD800 IS from Canon and FinePix F31fd from Fuji.

These smart cameras can zero in automatically on faces, and photos will be properly exposed. In general, face detection technology is integrated into the camera’s processor for increased speed. For example, FinePix F31fd can identify faces and can optimize image settings in as little as 0.05 seconds.

One interesting application of face detection technology is the passive driver monitor system installed on 2008 Lexus LS 600hL. The system uses a camera on the steering column to keep an eye on the driver. If he or she should be looking away from the road ahead and the pre-collision system detects something beyond the car (through stereo vision and radar), the system will sound a buzzer, flash a light, and even apply a sharp tap on the brakes.

Finally, the popular application of face detection technology is an image-based search of Internet or photo albums. Many companies (Adobe, Google, Microsoft, and many start-ups) have been working on various prototypes. Often, the bottle-neck for such commercial applications is the difficulty to recognize and to detected faces. Despite the advances, today’s recognition systems have limitations. Many factors exist that could defeat these systems: facial expression, aging, glasses, and shaving.

Nevertheless, face detection/recognition has been critical for the success of intelligent robots that may provide important services in the future. Prototypes of intelligent robots have been built, including Honda’s ASIMO and Sony’s QRIO.

Advanced Research Topics. To build a machine perception system someday that is close to or even better than the human perception system, researchers need to look at both aspects of this challenging problem: (1) the fundamental aspect of how human perception system works and (2) the systematic aspect of how to improve system performance based on best theories and technologies available.

To illustrate the fascinating characteristics of the human perception system and how it is different from currently available machine perception systems, we plot the negative and upside-down photos of a person in Fig. 9. It is well known that negative or upside- down photos make human perception of faces more difficult (59)). Also we know that no difference exists in terms of information (bits used to encode images) between a digitized normal photo and a digitized negative or upside-down photo (except the sign and orientation information).

Figure 9. Typical limitation of human perception system with negative and upside-down photos: which makes it difficult or takes much longer for us to recognize people from the photos. Interestingly, we can manage eventually to overcome this limitation when recognizing famous people (President Bill Clinton in this case)

From the system perspective, many research challenges remain. For example, recent system evaluations (60) suggested at least two major challenges: the illumination variation problem and the pose variation problem. Although many existing systems build in some sort of performance invariance by applying pre-processes, such as histogram equalization or pose learning, significant illumination or pose change can cause serious performance degradation.

In addition, face images could be partially occluded, or the system needs to recognize a person from an image in the database that was acquired some time ago. In an extreme scenario, for example, the search for missing children, the time interval could be up to 10 years. Such a scenario poses a significant challenge to build a robust system that can tolerate the variations in appearances across many years.

Real problems exist when face images are acquired under uncontrolled and uncooperative environments, for example, in surveillance applications. Although illumination and pose variations are well-defined and well-researched problems, other problems can be studied systematically, for example, through mathematical modeling.

Mathematical modeling allows us to describe physical entities mathematically and hence to transfer the physical phenomena into a series of numbers (31). The decomposition of a face image into a linear combination of eigenfaces is a classic. example of mathematical modeling. In addition, mathematical modeling can be applied to handle the issues of occlusion, low resolution, and aging.

As an application of image analysis and understanding, machine recognition of faces benefits tremendously from advances in many relevant disciplines. To conclude our article, we list these disciplines for further reading and mention their direct impact on face recognition briefly

- Pattern recognition. The ultimate goal of face recognition is recognition of personal ID based on facial patterns, including 2-D images, 3-D structures, and any pre-processed features that are finally fed into a classifier.

- Image processing. Given a single or a sequence of raw face images, it is important to normalize the image size, enhance the image quality, and to localize local features before recognition.

- Computer vision. The first step in face recognition involves the detection of face regions based on appearance, color, and motion. Computer vision techniques also make it possible to build a 3-D face model from a sequence of images by aligning them together. Finally, 3-D face modeling holds great promises for robust face recognition.

- Computer graphics. Traditionally, computer graphics are used to render human faces with increasingly realistic appearances. Combined with computer vision, it has been applied to build 3-D models from images.

- Learning. Learning plays a significant role to building a mathematical model. For example, given a training set (or bootstrap set by many researchers) of 2-D or 3-D images, a generative model can be learned and applied to other novel objects in the same class of face images.

- Neuroscience and Psychology. Study of the amazing capability of human perception of faces can shed some light on how to improve existing systems for machine perception of faces.

Bibliography:
1. W. Zhao, Tutorial on face recognition, European Conference on Computer Vision, 2004.
2. H. Rowley, S. Baluja, and T. Kanade, Neural network based face detection, IEEE Trans. Patt. Anal. Mach. Intell., 20: 39-51, 1998.

3. T. Cootes, C. Taylor, D. Cooper, and J. Graham, Active shape models-their training and application, Comp. Vis. Image Understand., 61: 18-23, 1995.
4. I. Bruner and R. Tagiuri, The perception of people, In G. Lindzey (ed.), Handbook of Social Psychology, Reading, MA: Addision-Wesley, 1954.
5. M. Bledsoe, The model method in facial recognition, Technical Report PRI 15, Palo Alto, CA: Panoramic Research Inc., 1964