This thesis investigates neuro-fuzzy hybrid models for automatic learning of actions taken by agents. The objective of these models is to provide an agent with intelligence, making it capable of acquiring and retaining knowledge and of reasoning (infer an action) by interacting with its environment. Learning in these models is performed by a non-supervised process, called Reinforcement Learning. These novel neuro-fuzzy models have the following characteristics: automatic learning of the model structure; auto-adjustment of parameters associated with the structure; capability of learning the action to be taken when the agent is on a given environment state; possibility of dealing with a larger number of inputs than those of traditional neuro-fuzzy systems; and the generation of hierarchical linguistic rules. This work comprised three main stages: bibliographic survey and study of learning models; definition and implementation of two new hierarchical neurofuzzy models based on Reinforcement Learning; and case studies. The bibliographic survey and the study of learning models considered learning models employed in agents (aiming to enhance the autonomous action) and in large and/or continuous state spaces. The definition of the two new neuro-fuzzy models was motivated by the importance of extending the autonomous capacity of agents through its intelligence, particularly the learning capacity. The models were conceived from the study of the existing limitations in current models, as well as the desirable characteristics for RL-based learning systems, particularly, when applied to continuous and/or high dimension environments. These environments present a characteristic called curse of dimensionality, which makes impracticable the direct application of the traditional RL- methods. Therefore, the decision of using a recursive partitioning methodology (already explored with excellent results in Souza, 1999), which significantly reduces the existing neuro-fuzzy systems limitations, was crucial to this work. The BSP (Binary Space Partitioning) and the Quadtree/Politree partitioning were then chosen, generating the RL-NFHB (Reinforcement Learning - Hierarchical Neuro- Fuzzy BSP) and RL-NFHP (Reinforcement Learning - Hierarchical Neuro-Fuzzy Politree) models. These two new models are derived from the hierarchical neuro-fuzzy models NFHB and NFHQ (Souza, 1999), which use supervised learning. By using these partitioning methods, together with the Reinforcement Learning methodology, a new class of Neuro-Fuzzy Systems (SNF) was obtained, which executes, in addition to structure learning, the autonomous learning of the actions to be taken by an agent. These characteristics represent an important differential when compared to the existing intelligent agents learning systems. In the case studies, the two models were tested in three benchmark applications and one application in robotics. The benchmark applications refer to 3 problems of control systems : the mountain cart problem, cart-centering problem, and the inverted pendulum. The application in robotics made use of the Khepera model. The RL-NFHB and RL-NFHP models were implemented using the Java language in Windows 2000 platform microcomputers. The experiments demonstrate that these new models are suitable for problems of control systems and robotics, presenting a good generalization and generating their own hierarchical structure of rules with linguistic interpretation. Moreover, the automatic environment learning endows the agent with intelligence (knowledge base, reasoning and learning). These are characteristics that increase the autonomous capacity of this agent. The hierarchical neuro-fuzzy systems field was also enhanced by the introduction of reinforcement learning, allowing the learning of hierarchical rules and actions to take place within the same process.