Fatigue is one of the most common problems in asphalt pavements and it is characterized by the appearance and subsequent development of micro and macro cracks. Asphalt mixtures are composed of large aggregates and a matrix that surrounds these aggregates, composed of asphalt binder, small aggregates and filler, known as Fine Aggregate Mixture (FAM). Studies indicate that the phenomena that affect micro cracks starts in the FAM. Therefore, this work aims to investigate the fatigue behavior of the Asphalt mixtures, using as intermediate means an analysis of the FAM. To achieve this objective, three asphalt mixtures were selected. They are composed of the same binder, but with aggregates distributed according to three different aggregate gradation curves, varying their maximum nominal size. The FAM had gradation curves proportional to that applied in asphalt mixtures. Mixtures and MAFs followed similar experimental programming with linear viscoelastic characterization and dynamic uniaxial fatigue tests. In order to interpret the results it was used the Simplified Viscoelastic Continuum Damage (S-VECD) model, G(R) and D(R) failure criteria, which presented a good prediction of the fatigue behavior of the Mixtures and MAFs studied. The MAF s tolerance to damage was analyzed as well as the influence of the maximum nominal size variation. A relation between strains in the asphalt mixtures and the strain in the fine portion of the asphalt mixtures was studied. The asphalt mixtures with higher content of fine asphalt mixture showed better fatigue performance and the strain proportionality factors were higher for mixtures with larger nominal maximum sizes.