Jawad H AlKhateeb

Computer Science Department Taibah University Madinah, KSA


Corresponding author Email: jkhateeb@taibahu.edu.sa

DOI : http://dx.doi.org/10.13005/ojcst/10.04.03


A new framework recognition system for identifying the missed pilgrims in Hajj and Umrah is presented in this paper. The framework recognition system is based on the Wavelet Probabilistic Neural Network (WPNN) classifier. The entire framework recognition system is capable of identifying the missed pilgrims. The framework recognition systems is developed as an application for mobile phones The proposed framework recognition system is applied to a huge database for pilgrims in the Ministry of Hajj Computer server for matching and identifying the missed pilgrim. The results are superior comparing to the existing systems


DCT; WPNN; Hajj; Umrah; Missed pilgrim

Download this article as: 

Copy the following to cite this article:

AlKhateeb J. H. A New Framework For Identifying the Missed Pilgrims In Hajj and Umrah. Orient.J. Comp. Sci. and Technol;10(4)

Copy the following to cite this URL:

AlKhateeb J. H. A New Framework For Identifying the Missed Pilgrims In Hajj and Umrah. Comp. Sci. and Technol;10(4). Available from: http://www.computerscijournal.org/?p=7026


Mainly, Islam has five main pillars to believe in. Hajj to Mecca is the fifth pillar. Hajj is an annual Islamic physical religious journey to Mecca which is obligatory once in life time for the Muslim believers who are physically and financially capable perform it. Furthermore, Umrah is another physical religious journey to Mecca which can be performed at any time of the year. In Kingdom of Saudi Arabia (KSA) there are two holy cities for Muslim believers. Mecca is first holy city Muslims believers where both Umrah and Hajj are performed. Madinah is the second holy city in KSA for Muslim believers. People who are performing both Hajj and Umrah are called pilgrims. pilgrim: pilgrim: pilgrim: According to the Saudi General Authority for Statistics (https://www.stats.gov.sa), in average there are 10 million people entered Mecca from inside and outside the KSA as pilgrims in order to perform Umrah or Hajj yearly. In average, there are five million pilgrims performing Umran in the holy month of Ramdhan, and three million pilgrims are performing Hajj in the holy month of Dhu al-HijjahThe other two million pilgrims are performing Umrah in the other ten months of the year. According to new vision of the Saudi Government, the number of pilgrims who are looking to perform Umrah or Hajj will be increased to be 20 million pilgrims yearly[1][2].

According to the above facts, A huge crowd will occur in both cities Mecca and Madinah because of the pilgrims who are performing Umrah or Hajj. Because of this crowd, there are several questions can be asked such as: How to identify the dead, lost pilgrims? Also, How to gather the missing items and distributes it? The answer for such questions is to develop a recognition system to identify and detect people individually. The system will use a registered database of faces for all local and foreign pilgrims. A solution for such crowd is to design a robust recognition system in order to identify the pilgrims. A robust recognition system needs a huge database to be built. Such a huge database gives the robustness for any recognition system  if well trained and tested. Working on an existing database for pilgrims is quiet difficult since the pilgrims are changing every year[1][2][3].

In order to perform Hajj or Umra, a foreign pilgrim who lives outside KSA needs a Hajj/Umrah visa to enter KSA. This visa takes place in all KSA embassies and consulates. On the other hand, local Umrah pilgrims can go to Mecca at any time of the year. But local Hajj pilgrims need a permission to perform Hajj by applying for it. The Mistry of Hajj in KSA checks about the eligibility for a pilgrim for the eligibility of a expatriate for Hajj. If the pilgrim is eligible for Hajj, he/she will be given a permit otherwise no permit will be issued. For local pilgrims, they are eligible to perform Hajj every five years [1][2][3].

Performing Hajj needs a registration via Umra/Hajj travel agent who deals with the KSA embassies and consulates all over the world for foreign pilgrims. Also, there are local Umra/Hajj travel agents who deal with the Ministry of Hajj [1][2].

Identifying the missed people framework system

In this paper, a new framework system for identifying the missed pilgrims in Hajj and Umrah is proposed. The system consists of two main phases. Firstly, Build the database is accomplished. Secondly, a recognition system is designed to identify the missed pilgrims. The recognition system is designed based or iris recognition. Preparing Database

Based on the fact, the pilgrims are not same pilgrims every year. Snicethe pilgrims are changing every year, a yearly huge database for all pilgrims is needed every year. Build such a database is done as follows:

  1. The local individual local pilgrims apply for Hajj online firstly. The registration is done via the Mistry of Hajj web site (www.localhaj.gov.sa). All the needed documents have to be loaded vis the site. Later on the Hajj permit will be issued to all local pilgrims [1][2].
  2. The foreign pilgrims apply via Hajj/Umrah agent offices. Those agents take all the needed documents to the KSA embassy or consulate in order to issue a Hajj Visa. It is noted that every year the each Muslim country has its own Hajj quota which it depends on its Muslim population [1][2].
  3. Finally, a Hajj Identification card (ID) will be issued to each pilgrim either local or foreign. The pilgrim personal information exists on the Hajj ID. The information contains full name, local phone number, the host phone number in Madinah and Mecca. An Example of Hajj ID is shown in figure 1 [1].
Fig. 1 Hajj ID Sample

Figure 1: Hajj ID Sample 

Click here to View figure


It is noted that the Hajj ID contains all the personal information for the pilgrim. This information is the pilgrim name, local address, both Madinah and Mecca residency, and the agent name.

Millions of pilgrims go to Mecca and Madinah in KSA to conduct Hajj or Umrah. Unfortunately, some of the pilgrims will miss their group member or they will die Iidentifying and finding the missed or dead pilgrims is a difficult [1][3].

There many application for identifying the missed pilgrims such as the e bracelet application. Special electronic (E) bracelets were issued for all pilgrims. The E bracelets contain all the details about the pilgrim wearer. Figure 2 shows a sample of the E bracelets. A drawback for such application is the there is a high probability to lose the E bracelets from the pilgrim wearer. The loss of the e bracelets may occur any time while washing the hands, or while taking a shower even though all the E bracelets are water resistant [1]. To overcome such a drawback, a new framework for identifying the missed pilgrims is proposed in this paper.

Fig. 2  The E Hajj  bracelet sample

Figure 2: The E Hajj bracelet sample 

Click here to View figure


Proposed system

In this paper, a new framework for identifying the missed pilgrims in Hajj and Umrah is proposed. Basically, this system uses the conventional phases for a recognition system. The phases are summarizes as, image acquisition for preparing the database, the pre-processing for noise removal and normalization, the feature extraction for converting the 2D image into 1D vector, and the classification Phase for identification process [4]. The proposed system is shown in figure 3.

Fig. 3 The Proposed System

Figure 3: The Proposed System 

Click here to View figure


Preparing Database

The database for all pilgrims has to be gathered. Here, collecting iris images for all the pilgrims is the main goal. This is done in previous research by acquiring both iris and Hajj ID images in order to construct a huge database for pilgrims. The database for all pilgrims is stored in a huge storage at the Ministry of Hajj. Each year this database is updated due to different pilgrims are come to perform Hajj. For example the database for 2105 differs from the database for 2016.All the pre-processing techniques were conducted to the database in order to make it ready for classification and recognition [5].

The first phase in constructing and building a robust recognition system database is to find suitable source of data. In this paper, collecting the iris image for all pilgrims is the main challenge. In our previous research, a database for identifying the missed pilgrims in both Hajj and Umrah was proposed [6]. The iris image for all the foreign pilgrims who are coming to KSA via all the borders will be captured and saved in the Ministry of Hajj computer server. Also, capturing the iris image for the local pilgrims will be taken and saved as well.

In this paper, the system is proposed to identify the missed pilgrims. The proposed system is designed based on iris recognition. The iris recognition system is a classical biometric recognition system, and it is considered as one the fastest and accurate recognition systems. Due to overcrowd either in Mecca or Madinah, the police officers will be notified for missed pilgrims. Basically, the missed pilgrims miss their group members for any reasons. The police officers role here is to capture an iris image for the missed pilgrim using their own mobiles. The mobile camera is supported with zoom lenses to have better quality image. Based on the online image capturing, the officer directly uses the Missed Hajj mobile application which is installed on his mobile. The application uses the captured image and search among the huge database at the ministry of Hajj in order to identify the missed pilgrim [2]. The mobile zoom lens is shown in figure 4.

Fig. 4 The Mobile zoom Lens

Figure 4: The Mobile zoom Lens 

Click here to View figure


Feature Extraction

An efficient representation by using a set of numerical values of the input image is the main goal the feature extraction phase. Furthermore, another goal for feature extraction is removing the redundant data from the input images. Later on, thesefeatures are mapped into a chosen classifier in order to identify and verify the input image. In this phase, the normalized grey iris image is used to extract the features based on the Discrete Cosine Transform (DCT).

The DCT coefficients are well known for reducing the redundant data. It focuses on computing the energy for the input image. The DCT features using the DCT coefficients are used in the proposed system. Extracting the DCT features is done firstly by finding the DCT of the input normalized grey iris image using equation (1) [3][5]


After applying DCT to the full normalized grey iris image, the DCT features are extracted in a vector sequence by applying the zigzag order to the DCT image of the normalized grey iris image. Basically, there are two well-known methods to get 1D vector from the 2D DCT coefficients. The zigzag order is illustrated in Figure 5, where Figure 3(a) is used the proposed system.

Fig. 5 The Two zigzag order Methods

Figure 5: The Two zigzag order Methods 

Click here to View figure


The feature matrices normalized into the range of  [-1, 1]. Each normalized grey iris image is represented by a feature vector of size 50 which is empirically tested [4][5].

Feature Extraction Algorithm


   for k=1 to number of images

     Img_in=Read the normalized grey iris image

        Find the 2D DCT for the Img_in

         Img_dct= DCT2(Img_in)

       Apply Zigzag order to obtain 1D vector of the Img_dct

     Choose the first 50 DCT coefficients of the 1D vector


Classification (Pattern Matching and Verification)

The Wavelet Probabilistic Neural Network (WPNN) classifier is applied to the proposed system as a classifier. The four layers of the WPNN architecture is shown in figure 6.

Fig. 6 The Wavelet Probabilistic Neural Network

Figure 6 The Wavelet Probabilistic Neural Network 

Click here to View figure


The four layers are classified as feature layer, wavelet layer, Gaussian layer and finally the decision layer.

Firstly, the feature layer, X1,…,XN are the input data which are the sets of feature vectors, where N is the dimension size of data.

Secondly, in the wavelet layer: this is a linear combination of several multidimensional wavelets. Each wavelet neuron is equal to a multidimensional wavelet. The wavelet is computed using equation 2.


Each wavelet is considered as a family function generated from the single function Φ(x). Both translation and scaling were used in computing the wavelet in order to be localized in time space and the frequency space respectively. The Φ(x) is known as the mother wavelet and both a, b parameters are known as scaling and translation factors respectively[7]. Finally, in Gaussian layer the probability density function of each Gaussian neuron is computed using equation 3.


where X is known as feature vector, p is known as the training set dimension size, n is known as the input data dimension size, j is known as the jth data set, Sij is known as the training set and σ is known as Gaussian function smoothing factor.

The entire scaling factor, the translation factor and the smoothing factor are randomly initialized at the beginning in order to be trained by The Particle Swarm Optimization (PSO). Based on the completion of the training, the WPNN architecture, all the parameters will be fixed for more and further verification.

Learning Algorithm

The PSO was developed by James Kennedy and Russell Eberhart in 1995. The PSO is used for training a single neuron for optimizing the model of the WPNN architecture. Basically, the PSO is considered as a new bio-inspired method in optimization [6]. The main algorithm looks for finding the best solution in the search space iteratively using the particle movement.For example, at any time unit t, the position of ith particle xi, i =1,2,…,M, where M is the number of particles, moves by summing the velocity vector vi. vi is the function of the best position pi found by that particle, and the best position g can be found among all particles of the swarm. Basically, the movement of the particle is formulated using equations 4 and 5 respectively. :


The w(t) is known as the inertia weight, c is known as the acceleration constants, and μÎ(0,1) is known as the uniformly distributed random numbers. Both the wavelet neuron and the Gaussian factors are encoded.   The scaling and translation factors are used to encode the wavelet neuron. In addition, the smoothing factor is used to encode the Gaussian factor. The PSO is used to search for the best set of factors in the multidimensional space [7].


Finally, there are five output probabilistic indicated values for iris features. Those probabilistic indicated values are located in In the last layer of WPNN, the decision layer. By finding the average of those five probabilistic indicated values which are called. In the face features, only one output probabilistic value is computed and it is called. The average value of both probabilistic values the and the is computed. By adjusting the threshold of the average value the false rejection ratios (FRR) and false accept ratios (FAR) are obtained.Figure 7 illustrates both FRR and FAR ratio [7].

Fig 7. The curve of  both FRR AND FAR

Figure 7: The curve of both FRR AND FAR 

Click here to View figure


Using multimodal biometrics recognition, the decision threshold is shown in the horizontal axis. Each false rate is shown in the vertical axis. From figure6, there are two main points are concluded. They are:

a) When the value of Pav for an unregistered sample is less than the decision threshold, a false accepted occurs. The FAR is computed by counting all the false acceptance trails.

b) When the value of Pav for an unregistered sample is greater than the decision threshold, the registered sample is wrong rejected. The FRR is computed by counting all the trials of false rejection

Mobile Application

Upon completeion of all the recognition phases, a mobile application is developed to be installed on he smart mobile phones supporting both systems the Andriod and the IOS.

Basically, when the proposed system runs on the smart mobile phones, an iris image will be taken for the pilgrim which will be sent to Ministry of Hajj server for further analysis. Here, the huge database will be accessed and the iris recognition is run using image search engine.

Experimental results

Obtaining a high recognition rate for any recognition system requires a huge database for both trainingg and testing. several experiments are performed on the iris database in order to evaluate the proposed system. The WPNN classifier is used to measure the recognition rate of the proposed system. The recognition rate of the proposed system is 86% which is acceptable.

Due to the fact that the iris last for twenty hours after death, the proposed system works well for identifying the dead pilgrims who passed away less than twenty four hours; however, the proposed system does not work for the dead pilgrims one day or more from their death.


A new framework recognition system for identifying the miss pilgrims in both Hajj and Umrah is proposed in this paper, The WPNN classifier is used in the proposed system based on DCT features. The features are extracted using the DCT coefficients. Finally, a mobile application for both Android and IOS system is proposed as well. The results of the proposed system and its performance based on WPNN classifier are very good comparing to the existing ones.


I would like to express my sincere appreciation and thanks owe thanks to TAIBAH University, College of Computer Science and Engineering (CCSE), Kingdom of Saudi Arabia for supporting this research.


  1. http://www.haj.gov.sa/English/pages/default.aspx
  2. https://www.stats.gov.sa
  3. Salah A. Aly, A Missing and Found Recognition System for Hajj and Umrah,arXiv.org.,[arXiv:1208.5365v1],2012. (https://arxiv.org/abs/1208.5365).
  4. Jawad H. AlKhateeb, Olivier Pauplin, Jinchang Ren, Jianmin Jiang, Performance of hidden Markov model and dynamic Bayesian network classifiers on handwritten Arabic word recognition, Knowledge-Based Systems, Vol 24, pp680-688,2011.
  5. Jawad H AlKhateeb, Jinchang Ren, Jianmin Jiang, Husni Al-Muhtaseb, Offline handwritten Arabic cursive text recognition using Hidden Markov Models and re-ranking, Pattern Recognition Letters, Vol 32,, pp 1081-1088, 2011.
  6. Jawad H. AlKhateeb , A Database For IdentifyingTthe Missed Pilgrims in Hajj and Umrah, accpted to the Interrnational Journal of Computer and Information Technology (IJCIT), October 2017.
  7. Ching-Han CHEN and Chia Te CHU, Fusion of Face and Iris Features for Multimodal Biometrics, Lecture Noootes in Computer Science, Jan. 2006.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License.

Share Knowledge: Share on LinkedInShare on FacebookTweet about this on TwitterShare on Google+Share on RedditEmail this to someone

Comments are closed.