The performance measurement system (PMS) is considered as a fundamental tool that collaborates directly in the evaluation and achievement of the strategic objectives and excellence of the supply chain management. In the scenario of constant changes in business environment, many companies has been directing efforts to develop new projects, reviews and adjustments of their PMSs, as well as in the efficient use of this management tool. However, many organizations are faced with unexpected difficulties during this task. Improving understanding of the factors that affect the success of PMS can enable practitioners to be better prepared and more responsive during their application, achieving more effective strategies and better resource allocation. This dissertation presents a multiple case study, from the analysis of three performance measurement initiatives applied to the petroleum and oil derivates supply chain of a single company. A complete lifecycle approach – design, implementation and use / review – is adopted to identify the critical factors that have affected the success of these PMS. As a result of the research, a list of critical success factors and barriers to the application of a PMS was obtained empirically and compared with previous findings from the literature. From this list, a practical guideline for evaluation of PMS has been proposed, which is recommended to be used by organizations for each PMS development or review, in order to gain a better understanding of the challenges to be overcome throughout the supply chain PMS life cycle. The outcome of the case study suggests that there has been organizational learning in dealing with the critical factors affecting the PMSs from each applied initiative.