Remote sensing imagery enables accurate crop mapping and monitoring, supporting efficient and sustainable agricultural practices to ensure food security. However, accurate crop type identification and crop area estimation from remote sensing data in tropical regions are still challenging tasks. Compared to the characteristic conditions of temperate regions, the more favorable weather conditions in tropical regions permit higher flexibility in land use, planning, and management, which implies complex crop dynamics. Moreover, the frequent cloud cover prevents the use of optical data during large periods of the year, making SAR data an attractive alternative for crop mapping in tropical regions. To exploit both spatial and temporal contex, conditional random fields (CRFs) models have been used successfully in the classification of RS imagery. These approaches deliver high accuracies; however, they rely on features engineering manually designed based on domain-specific knowledge. In this context, deep learning methods such as convolutional neural networks (CNNs) proved to be a robust alternative for remote sensing image classification, as they can learn optimal features and classification parameters directly from raw data. This work introduces a novel end-to-end hybrid model based on deep learning and conditional random fields for crop recognition in areas characterized by complex spatio-temporal dynamics typical of tropical regions. The proposed framework consists of two modules: a CNN that models spatial and temporal contexts from the input data and a CRF that models temporal dynamics considering label dependencies between adjacent epochs. These dependencies can be learned or designed by an expert in local agricultural practices. Comparisons between data-driven and prior-knowledge temporal constraints are presented for two municipalities in Brazil, using multi-temporal SAR image sequences. The experiments showed significant improvements in per class F1 score of up to 30 percent and up to 12 percent in average F1 score against a baseline model that doesn t include temporal dependencies during the learning process.