Multimodal Human verification with performance evaluation

Identity recognition systems are an important part of our every day life. Informationsystem/computer network security such as user authentication and access to databases isan important potential application area for biometrics. Biometric systems based on faceimages and/or speech signals have been shown to be quite effective. However, theirperformance easily degrades in the presence of a mismatch between training and testingconditions.A system which uses more than one biometric at the same time is known as amultimodal system. It often consists of several modality experts and a decision stage.Multimodal systems can be more robust and give higher recognition accuracy. One of thefactors important to the accuracy of a multimodal system is the choice of the techniquedeployed for data fusion. Another important issue is that of variations in the biometricdata. Such variations are reflected in the corresponding biometric scores, and thereby caninfluence the overall effectiveness ofmultimodal biometric recognition.In this thesis, a score fusion personal identification method using both face and speechis introduced to improve the rate of single biometric identification. For speakerrecognition, an effective and robust method is proposed to extract speech features, capableof operating in noisy environment. Based on the time-frequency multi-resolution propertyof wavelet transform, the input speech signal is decomposed into various frequencychannels. For capturing the characteristic of the signal, Mel-Frequency CepstralCoefficients (MFCCs) of the wavelet channels are calculated. Hidden Markov Models(HMMs) are used for the recognition stage as they give better recognition for the speaker’sfeatures than Dynamic Time Warping (DTW). Comparison of the proposed approach withthe MFCCs conventional feature extraction method shows that the proposed method notonly effectively reduces the influence of noise, but also improves recognition.For face recognition, the wavelet-only scheme is used in the feature extraction stage offace and nearest neighbour classifier is used in the recognition stage. Seeking the mostsuccessful subbands, it is noted that the highest recognition accuracy is obtained usingapproximations at level 3, followed by the horizontal details at level3. The vertical anddiagonal details give poor performance. Z-score is performed on the selected waveletsubband coefficients by subtracting the mean and dividing by the standard deviation.Histogram Equalization (HE) and Adaptive Histogram Equalization (AHE) are applied in.