Predictive PolieDRO is a novel analytics framework for classification and regression that harnesses the power and flexibility of Data-Driven Distributionally Robust Optimization to circumvent the need of regularization hyperparameters. Recent literature shows that traditional machine learning methods such as SVM and (square-root) LASSO can be written as Warserstein-based DRO problems. We built on those results and propose a hyperparameter-free ambiguity set that explores the polyhedral structure of data-driven convex hulls. Given any convex loss function, the proposed model is computationally tractable since we develop a con-vex single-level equivalent reformulation. To cope with high-dimensional feature spaces, we propose an ensemble method that uses the predictive PolieDRO as base model, with randomly chosen subsets of features and samples. Numerical results based on 50 databases show that our method consistently outperforms their parametric counterparts.