Unsupervised cluster analysis, the process of grouping sets of points according to one or more similarity criteria, plays an essential role in various fields. The two most popular algorithms for this process are the k-means and the Gaussian Mixture Models (GMM). The former assigns each point to a single cluster and uses Euclidean distance as similarity. The latter determines a probability matrix of points to belong to clusters, and the Mahalanobis distance is the underlying similarity. Apart from the difference in the assignment method - the so-called hard assignment for the former and soft assignment for the latter - the algorithms also differ concerning the cluster structure, or shape: the k-means considers spherical structures in the data; while the GMM considers ellipsoidal ones through the estimation of covariance matrices. In this work, a mathematical optimization problem that combines the hard assignment with the ellipsoidal cluster structure is detailed and formulated. Since the estimation of the covariance plays a major role in the behavior of ellipsoidal cluster structures, regularization techniques are explored. In this context, two meta-heuristic methods, a Random Swap perturbation and a hybrid genetic algorithm, are adapted, and their impact on the improvement of the performance of the methods is studied. The central objective is three-fold: to gain an understanding of the conditions in which ellipsoidal clustering structures are more beneficial than spherical ones; to determine the impact of covariance estimation with regularization methods; and to analyze the effect of global optimization meta-heuristics on unsupervised cluster analysis. Finally, in order to provide grounds for comparison of the present findings to future related works, a database was generated together with an extensive benchmark containing an analysis of the variations of different sizes, shapes, number of clusters, and separability and their impact on the results of different clustering algorithms. Furthermore, packages written in the Julia language have been made available with the algorithms studied throughout this work.