Detecting Faces According to their Pose

Mohammad Mojtaba Keikhayfarzaneh^1,2*,Javad Khalatbari¹, B.Somayeh Mousavi³ and Saeedeh Motamed^1,4

¹Department of Psychology, Islamic Azad University, Tonekabon Branch, Tonekabon (Iran).

²Department of Psychology, Young Researcher Club, Zahedan Branch, Islamic Azad University, Zahedan (Iran).

³Department of Computer, Young Researcher Club, Zahedan Branch, Islamic Azad University, Zahedan (Iran).

⁴Department of Psychology, Young Researcher Club, Tonekabon Branch, Islamic Azad University, Tonekabon (Iran).

ABSTRACT:

Human face detection is one of the vital techniques in computer vision. It plays so important role in a wide variety of applications such as surveillance systems, video tracking applications and image database management. In this paper a new method to detect faces with different pose in colour images, is proposed. Skin colour, lip position, face shape information and statistical texture properties are the key parameters for developing fussy rule based classifiers to extract face candidate from an image. The algorithm consist of two main parts: detecting frontal face system and detecting profile face system. In first step, skin regions are identified in HSI colour space, using fuzzy system, applying distance of each pixel colour to skin colour cluster as input and producing a skin-likelihood image in output. The regions owning the most likelihood of belonging to skin are labelled and enter the frontal face detecting part . To extract frontal face regions ,fuzzy rule based system is used, applying face and lip position, lip area data and face shape. The detected faces are removed and remain areas are tested by the profile face finding algorithm. This algorithm utilizes statistical texture properties of ear to verify profile face detection. By this system, 98%, 90% and 83.33% detection rates are achieved, respectively for frontal, near frontal and profile faces.

KEYWORDS: Face detection; Lip detection; colour space; fuzzy rule based system; Morphological processing

Copy the following to cite this article: Keikhayfarzaneh M. M, Khalatbari J, Mousavi B. S, Motamed S. Detecting Faces According to their Pose. Orient. J. Comp. Sci. and Technol;4(1)

Copy the following to cite this URL: Keikhayfarzaneh M. M, Khalatbari J, Mousavi B. S, Motamed S. Detecting Faces According to their Pose. Orient. J. Comp. Sci. and Technol;4(1). Available from: http://www.computerscijournal.org/?p=2351

Introduction

Face detection, defined as detection and localisation of human faces in images, has been challenging topic in human computer interaction (HCI) researches. Human face detection application is observable mainly in video surveillance, face recognition and image databases management.

In general, single image detection methods can be classified in to four categories¹

1. Knowledge-based methods.
2. Feature invariant approaches.
3. Template matching methods.
4. Appearance-based methods.

Among methods which lie within the second category, using skin colour is one of the most useful approaches in many applications, from face detection to hand tracking. It has been proved that the differ of skin colour in different peoples is more due to intensity².

When building a system, utilizing skincolour as a feature for face detection, three main questions wouldbe arisen. First, whatcolour space to choose.

Second, how the skin colour distribution should be modelled and third, what will be the way of processing of colour segmentation results for face detection?

The choice of colour space is considered as the primary step in skin-colour classification. The RGB colour space is the default colour space for most available image formats. Any other colour spaces could be obtained from a linear or non-linear transformation from RGB.

Several colour spaces have been proposed for skin detection. The most widely used colour spaces are classified as follow²:

• Basic colour spaces (RGB³, normalised RGB⁴, CIE-XYZ).
• Perceptual colour spaces (HSI, HSV⁵, HSL, TSL).
• Orthogonal colour spaces (YCbCr⁶, YIQ, YUV, YEs).
• Perceptually uniform colour spaces (IE-Lab and CIE-Luv).
• Other colour spaces (Mixture spaces, colour ratio spaces).

After choosing suitable colour space, the decision rule to discriminate between skin and non-skin pixels should be built. To this aim, four group of skin modelling methods are utilized⁷.

1. Explicitly defined skin region
2. Nonparametric skin distribution modelling.
3. Parametric skin distribution.
4. Dynamic skin distribution models.

In this paper, a novel method to face detection, is proposed. Until now, most of works in face detection domain is dedicated to just frontal face detection or just profile one; steel, this new system have the advantage of detecting frontal, near frontal and profile faces. Moreover, applying fuzzy rule based classifiers is another priority of this algorithm.

Following sections describe how skin colour, lip colour and intensity information, beside texture analysis are utilized to make a robust algorithm for both frontal and profile faces.

Proposed Method

Skin Colour Segmentation

To skin colour segmentation in first step, the appropriate colour space should be selected. The transformation of RGB to perceptual colour spaces is invariant to high intensity at white lights, ambient light and surface orientations relative to the light source; consequently, they can be a so suitable choice for skin detection methods.

By following non-linear equations, RGB colour space is transform to HSI colour space, which has the advantage that intensity component is separated from chrominance components:

After transform input image to selected colour space, next step is finding skin pixels. To this goal, the Euclidean distance of the colour of each pixel to skin colour sub-space, is used as an input for a fuzzy system. Fuzzy system is a mamdani type, 1-input, 1-output system. To achieve crisp output centroid method is chosen, which is the most widely used one among seven defuzzification approaches [8]. The membership functions (MFs) of input and output is depicted in Fig. 1.

The fuzzy linguistic terms of input (d) are: {Face, Rather Face, Low Probability Face, Not Face}. Similarly, the terms set of output (p) is: {Face, Rather Face, Low Probability Face, Not Face}.

The result of apply such a system is the skin-likelihood image; that is, the gray-scale image whose gray values represent the likelihood of the pixel belonging to skin. The pixels with %89 likelihood or more are regarded as skin pixels. The binary image is formed by setting skin pixels to 1 and all other pixels to 0. After this, morphological processing, consist of filling holes and opening followed by closing⁹, is accomplished to acquire separate and connected regions. Each region is applied as an input fornext step.

Figure 1: (a) input and (b) output MFs for skin colour segmentation fuzzy classifier Click here to View figure

 

Figure 2: Input and (b) output MFs for lip colour segmentation fuzzy classifier Click here to View figure

 

Figure 3: (a), (b),(c) inputs and (d) output MFs for detecting frontal face candidate fuzzy system Click here to View figure

 

Lip Colour Segmentation

Lip colour segmentation part, searches whole skin region to find lip pixels. Finding lip pixels system describes as follow:

The target region is transformed to normalised RGB colour space. This colour space is achieved using “Equation

Figure 4: (a) and (b) input and (c) output MFs for ear texture searching system Click here to View figure

 

This colour space has been shown best result in finding lip cases⁴. To reduce false positive in lip detection, lip pixels are searched just in the lower half region. With this technique false positive, causing by redness of cheeks is ignored.

In normalised RGB colour space and in desired region, the r-g values are utilized as fuzzy system inputs. The properties of this system are same as the skin colour segmentation system, except the membership functions shapes. The MFs are shown in Fig. 2.

The input (r-g) is recognized by linguistic terms: {Lip, Rather Lip, Not Lip}; likewise, for the output (l) linguistic terms are: {Lip, Rather Lip, Not Lip}. Similar to skin detection system, the result is lip-likelihood image. 84% likelihood is chosen as threshold to make binary image of lip regions.

Fill the holes and erosion plus opening by reconstruction⁹, are the morphological processing done at the end of this stage.

Frontal Face Detecting System

In final step, the face blobs must be selected among skin regions. Deciding system for frontal face detection is a fuzzy system with three inputs and one output. The first input (A) is the ratio of lip area to face area. The second input (B) is the tangent of angle between maximum length line of face area and minimum length line of lip area. These two lines should be parallel, approximately. Angels less than 30° are acceptable. The third input (C) reveals the similarity of region shape to an ellipse. This ellipse estimates the face shape. Output (Y) expresses likelihood of the region being face. Region with more than 80% likelihood is introduced as a frontal face.

Figure 5: (a) main image, (b) skin region, (c) skin region after morphological processing, (d)  half region, (e) half region transformed to normalised RGB colour space, lip detected area, (g) approximated face shape, (h) detected face in image Click here to View figure

 

The inputs and output MFs, are illustrated in Fig. 3. Detected regions in this system are omitted from skin regions and remain blobs, if any, are used as input for the next part of algorithm. Profile Face Detecting System As is mentioned in the previous section, the input of this stage is the skin regions that the face candidates, detected by frontal face detecting system, are removed from them.

Figure 6: (a) main image, (b) skin region and nose tip position, (c) divide region to estimate ear area (two right square are searched to find ear texture), (d) detected face in image Click here to View figure

 

Figure 7: Examples of applying algorithm on images with various pose and background (images are from Bao image database) Click here to View figure

 

To search profile faces among these regions, the statistical properties of the intensity histogram of ear texture is used. First of all, the ear location is estimated by its relative position to nose tip.

The suggested process of finding nose tip in [10], is altered to be size independent and used here. The new process is as follows: the X values along each row which we first encounter a white pixel in the binary image is searched. With the median of these values ( X_median ) the approximation X value of the face contour is obtained. Then, the median of Y values in the range of X_median is calculated (Y_median ). In the last step, the X value of nose tip is introduced as the minimum X values in the range of:

where n is the number of Y values in the range of X_median . This method avoids the possibility of locating hair or chin as the nose tip.

Since determining X and Y values of nose tip, the ear location is estimated by use of the distance between nose tip and the last white pixel, its Y value equal to Y_nose . In the regions with high probability of ear existence (two right squares in Fig. 6(c)), the statistical properties of texture is checked.

After studying various statistical properties of ear texture, entropy and average intensity¹¹ are chosen to find ear area. Again, a fuzzy rule based system is applied, using these two properties as inputs, in the aim of testing texture of target regions. The MFs of this system is illustrated in Fig. 4.

Table 1: Experimental Results (DR: Detection Rate)

Head Pose	Frontal	Near-Frontal	Profile
Number of Images	100	20	30
DR(%)	98	90	83.33

 

If the system verifies the existence of ear in the skin blob, this blob is known as the profile face.

Results

To test the proposed approach, 150 images contain frontal, near frontal and profile faces, are selected. These images are gathered among Bao¹², Caltech¹³ image datasets and some samples used in other articles. The image sizes are between 125×93 to 296×448 pixels.

The detection results are summarized in TABLE I. Because of frontal face detection method dependency to lip detection, it could not be so useful for other head pose. However, the method shows nearly acceptable result for near frontal faces.

Fig. 5 and 6 show instances of applying proposed algorithm to frontal and profile face picture. The step-by-step results of different stages, described in previous sections, are observable. Fig. 7 illustrates some more obtained results. Whole described stages are converted to computer program using MATLAB software.

Conclusion

A new fuzzy rule based method to detect human faces in colour image was proposed and described.

They were four fuzzy systems, respectively to create skin-likelihood gray-scale image, lip-likelihood gray-scale image, select frontal face and profile face blobs among skin regions.

Beside colour information, the position and area of lip relative to face area and approximated face shape for frontal faces and texture properties for profile ones, are used to verify detected faces.

In experimental results, it can be seen that the algorithm works well for frontal faces. In faces in which the lip is not completely observable, such as near frontal faces, the detection rate is less. Considering the facts that profile face detection system works inside complete algorithm for detecting face in various pose, it can be said that 83.33% is acceptable result. The problem of this part is that the properties range, defined for ear texture, is depend on image intensity and might be different for every database or even every image.

Altering system to adapt itself with textures in different intensity range and using other specifications of profile shape to increase detection rate, are the next goals.

References

1. M.Yang, D.J.Kriegman, N.Ahuja, “Detecting Face in Image: A Survey”, IEEE Transaction on Pattern Analysis and Machine Intelligence,24(1): (2002).
2. P.Kakumanu, S.Makrogiannis, N.Bourbaki “A Survey of Skin-Color Modeling and Detection methods”, Pattern Recognition, pp. 1106-1122 (2007).
3. N.Eveno, A.Caplier, Coulon, “A New Transformation For Lips Segmentation”, Multimedia Signal Processing, 2001 IEEE Fourth Workshop, 3-8 (2001).
4. J.A.Dargham, A.Chekima, “Lips Detection in the Normalised RGB Color Scheme”, Information and Communication Technologies, 2006.ICTTA ’06.2nd,1: 1546-1551 (2006).
5. J-C.Terrillon, M.David, S.Akamatsu, “Detection of Human Faces in Complex Scene Image by Use of a Skin Color Model and of Invariant Fourier-mellin Moments”, Pattern Recognition, 1998. Proceedings. Fourteenth International Conference,2:1350-1355 (1998).
6. Rein-Lein Hsu, M.Abdel-Mottaleb, A.K.Jain, “Face Detection in Color Image”, IEEE Transaction on Pattern Analysis and Machine Intelligence,24: 696-706 (2002).
7. V.Vezhnevets, V.Sazonov, A.Andreeva, “A Survey on Pixel-Based Skin Color Detection Techniques”, Proc.Graphicon-2003, pp.85-92, Moscow, Russia, (2003).
8. S.N.Sivanandum, S.Sumathi, S.N.Deepa, “Introduction to Fuzzy logic using MATLAB”, springer-verlag Berlin Heidelberg, 2007.
9. R.Davies, “Machine Vision”, Morgan Kaufman (2004).
10. P.Yan, K.W.Boyer, “Biometric Recognition Using 3D Ear Shape”, IEEE Transaction on Pattern Analysis and Machine Intelligence,29(8): (2007).
11. M.Petrou, P.Garcia Sevilla, “Image Processing Dealing with Texture”, WILEY(2006).
12. http://www.facedetection.com
13. http://www.vision.caltech.edu

---

This work is licensed under a Creative Commons Attribution 4.0 International License.