In this paper, a new algorithm is presented for unsupervised learning of finite mixture models (FMMs) using data set with missing values. This algorithm overcomes the local optima problem of the Expectation-Maximization (EM) algorithm via integrating the EM algorithm with Particle Swarm Optimization (PSO). In addition, the proposed algorithm overcomes the problem of biased estimation due to overlapping clusters in estimating missing values in the input data set by integrating locally-tuned general regression neural networks with Optimal Completion Strategy (OCS). A comparison study shows the superiority of the proposed algorithm over other algorithms commonly used in the literature in unsupervised learning of FMM parameters that result in minimum mis-classification errors when used in clustering incomplete data set that is generated from overlapping clusters and these clusters are largely different in their sizes.