CURVELET TRANSFORM AND HMM CLASSIFIER BASED SIGN LANGUAGE RECOGNITION SYSTEM

: A communication tool in the form of sign language is required for deaf and dump persons as there is no oral communication possible between them. They perform the conversion of sign languages into voice/text. Recently, many algorithms are developed for this purpose. An Indian Sign Language Recognition (ISLR) system is presented in this paper. It uses curvelet transform based entropy features for the recognition, and the transform is applied only to the segmented hand region. Then, the features of each sign of English alphabets are modelled by a classier network called Hidden Markov Models (HMM). The system gives an average accuracy of 82.95% using 3rd level features which can help to reduce the communication gap between deaf-dumb and normal people in the world.


I. INTRODUCTION
Sign language using hand gesture is one of the emerging technologies for hearing impaired community.It helps deaf and dump persons to understand the meaning of languages in the world by the use of a communication system using hand gesture images.Some of the previously available SLR systems are reviewed here.The sign language recognition based on K-Nearest Neighbour (KNN) search is discussed in [1].The input hand gesture images are preprocessed, and then edge detection is done to detect the shape of the alphabets, and the discontinuities present between the edges are joined.At last the fuzzy rule is applied to recognize the sign language.
HMM classifier model is most widely used in speech recognition and recently in the hand gesture recognition [2].Literally, it is perfect for visual recognition of hand gestures to identify the English alphabets.Then a method of HMM based system is implemented to recognize the sentences in American Sign Language (ASL).
A method of automatic Chinese SLR is discussed in [3].It uses the surface Electromyographic (sEMG) and portable Accelerometer (ACC) sensors for its operation.From the extracted features of sEMG and ACC data, the basic components like movement and hand shape orientation are modeled.These features are learned by component classifiers.The process of speech reading of British Sign Language (BSL) with a functional magnetic resonance imaging is used in [4].At the temporal-parietal-occipital junction, a higher activation is given at BSL processing.Distinct regions that are present between the temporal cortexes are not only sensitive but also differentially sensitive to the different articulators within the signed language.
A sequential data classification method is used for the glove based Arabic SLR system in [5].A preprocessing and feature extraction techniques are used in order to emphasize and capture the temporal dependence of the data.Then a Modified KNN (MKNN) is used for the classification process.The vision based sign language recognition system is explained in [6] for the translation of signs to English.It is an automated method in which the signs are breaking up into convenient subunits.A boosting algorithm is used to learn the weak classifiers subset that represents the combinations of subunits and the features for each sign.
An automatic system of identifying and training signs from continuous sentences of sign language is explained in [7].The multiple learning density matrix algorithms are used to extract the signs of noisy and weak supervision text translation from full sentences.The isolated samples are extracted in order to train the hand posture classifiers and the spatiotemporal gestures.Translator system of ISL using a Gesture recognition algorithm is explained in [8].The system translates the ISL gestures including alphabets, numbers, and phrases into English.Using the combinational and template matching algorithm, the system performs the data acquisitions followed by pre-processing for the recognition of the sign.
The pipeline process system that allows the users to attain the Italian language text translation in the virtual avatar signing animation is discussed in [9].It gives a brief explanation about the translation system architecture and the integration of resource languages in it.A model of similarity measurement for the vision and sign language semantics from the video content in Chinese sign language is explained in [10].The volume local binary patterns are defined as the distance between the vision components of the model.For measuring the shape and size, the contour is used whereas for measuring the movement, location and quantizing the trajectory is used in the sign language semantic.
The binary representation of UP & DOWN positions of a set of 32 signs of the five fingers is explained in [11].The pre-processing for the static images using the feature point method is done and 10 number of each sign images are used for training.For the identification of the finger tip position and the conversion of image into text the IP technique over the static images.For the extraction of the exemplars from signer independent hidden Markov models, affinity propagation is used in [12].The method is used to shape the adaptive training vocabulary without any human intervention.The grouping of iterative vector field smoothing is done from the collected sign gestures vocabulary, and is used to generate the maximum posterior.

II. RESEARCH FRAMEWORK
The framework of Indian sign language recognition system is shown in Figure 1.The Indian SLR system consists of the following modules; pre-processing by thresholding, feature extraction by curvelet transform and recognition by HMM.

A. Preprocessing
The Indian sign languages are recognized by using hand gesture images.At first, the input images are pre-processed by colour space conversion from RGB to gray scale.Then a simple thresholding approach is applied to separate the foreground from the background region.Hence, the hand region is separated from the background.

B. Curvelet Based Entropy Feature Extraction
From the segmented hand region, the entropy features are extracted after decomposing it by curvelet transform algorithm.The multi-scale representation of the image is obtained by Curvelets, a non-adaptive technique which can represent edges and curves singularities effectively.Curvelet captures point discontinuities into linear structures.From the decomposed sub-bands obtained from the curvelets, the entropy feature known as the log entropy is extracted.It characterizes the information's in an image which is uncertainty in nature.
As an image could be viewed as a combination of different blocks, the information's available in each block is described by its entropy.The intensity entropy of an image   y x f , with dimension M by N is defined as

C. Classification by HMM
It is the final stage of sign language recognition system by curvelets.After the features are extracted from the hand gesture image by using the curvelet transform, the classification is done to make the final decision by HMM classifier.HMM is applied in many pattern recognition approaches such as handwriting, speech, and hand gesture recognition.The first step for the classification is to divide the dataset for each class into two sets.Then train the HMM per class using one set.The remaining set is used for testing which is based on the likelihood of each model.They are related to the use of Markov process rather than independent of each other.

III. RESULTS AND DISCUSSION
The evaluation of the sign language recognition system using the curvelet based entropy features and HMM classifier is completed using hand gestures images which are acquired using 5 megapixels Logitech C170 digital camera.The vertical and horizontal distance between camera and person is fixed at 105 cm and 80 cm respectively.In order to differentiate the hand gesture from the background, all the persons are asked to wear a black jacket.100 hand gesture images per alphabet are captured from different individuals.Figure 2 shows hand gesture of English alphabets in the database and the outputs of preprocessing steps are shown in Figure 3.The performance of the system is measured by classification accuracy which is defined as the ratio between correctly classified hand gestures images and tested images of the same Alphabet.It is already stated that curvelet is a multi-scale representation.Hence, the system is analyzed from 1-5 scale of features.Figure 4 shows the average accuracies at each scale.

Fig. 4 Average Accuracies vs. Curvelet scale of decompositions
It is observed that the ISLR gives maximum average accuracy at 3rd scale of curvelet features with HMM classifier.Table 1 shows the accuracies of Indian SLR system for each English alphabet.

IV. CONCLUSION
In this paper, an approach for ISLR is proposed by curvelet based entropy features and the HMM classifier.Hand gesture images are used to find the sign of one of the 26 English alphabets.It is achieved by designing three modules; preprocessing by thresholding, feature extraction by curvelet transform and recognition by HMM.The evaluation of the system is carried on the collected hand gesture image samples for all the 26 alphabets.By using this system, a maximum Curvelet scale of decomposition of 82.95% overall accuracy is obtained from the hand gesture images while using 26 English alphabets with HMM classifier.

Fig. 3 (
Fig. 2 Sample images in the database