The field of image classification has been heavily explored for years, especially with the big advancements in deep neural networks seen in the last decade. However, most of the focus has been dedicated to cases with significant inter-class differences and minor intra-class differences. In this work we explore how well convolutional networks deal with cases with small inter-class differences and whose classification carries a degree of subjectivity, making non-obvious the relationship between visual features and classification and differentiating it from the traditional field of fine-grained classification. To do that, we approach a specific instance of this problem: Determining a character s narrative importance based solely on its image. We have evaluate the performance of different CNN models in our task, using a dataset we created for it, and we have analysed which patterns were found when it comes to the relationship between visual features and classification. We show that, for our specific task, CNNs are able to exceed human performance in pure accuracy and, more interestingly, mirror many of the patterns humans show when judging characters, even if some of those patterns are inaccurate. This means that this kind of model may be able to serve as a good surrogate for human evaluators when designing characters.