In this book, an innovative approach for text-independent speaker recognition has been proposed. The Speech signal produced by time varying excitation (source) of time varying vocal tract mechanism (system) carries information pertaining to source as well as system which is a potential source for speaker-specific information ignored by most of the present day speaker recognition systems, which use vocal tract system information. Addressing the issues involved in developing a speaker recognition system using source feature from linear prediction residual is crucial. It is observed that hidden Markov models (HMM) perform better than GMM as the source information such as glottal vibrations and prosodic features such as intonation and duration can be captured effectively. Further it is observed that the performance of the source feature based speaker recognition using ergodic HMM out performed left-right HMM as the speaker specific time varying source feature do not follow a consistent pattern. The performance is evaluated on TIMIT speech data base.