Speaker recognition is the process of automatically recognizing the person on the basis of the information contained in speech signal of the person. The extraction of this information, called feature extraction and then, feature matching process are implemented right after the pre-processing of the signal. Mel-Frequency Cepstral Coefficients (MFCCs) are taken as features for modeling by the Gaussian Mixture Model (GMM) during the identification process. The GMM algorithm [1, 2] is one of the clustering analyses for the text-independent speaker recognition. One of the main shortcomings of the traditional fuzzy clustering algorithm is the inability of determining the correct number of clusters except by trial and error. In this paper, an algorithm to determine optimal number of clusters automatically is developed by adopting the idea of hierarchical clustering and GMM. Numerical experiments demonstrate that the proposed algorithm achieves better performance than the traditional GMM where the number of clusters is fixed.