In this work, methods were developed and evaluated in order to improve and accelerate the learning process of Reinforcement Learning Neuro-Fuzzy Hierarchical Politree Model (RL-NFHP). This model is employed to provide an agent with intelligence, making it autonomous, due to the capacity of ratiocinate (infer actions) and learning, acquired knowledge through interaction with the environment by Reinforcement Learning process. The RL-NFHP model has the following features: automatic learning of structure of the model; self-adjustment of parameters associated with its structure, ability to learn the action to be taken when the agent is in a particular state of the environment; ability to handle a larger number of inputs than the traditional neuro-fuzzy systems; and generation of rules with linguistic interpretable hierarchy. With the aim to improve and accelerate the learning process of the model, six selection action policies were developed, one of them an innovation of this work (Q-DC-roulette); implemented the early stopping method for automatically determining the end of the training; developed a cumulative eligibility trace; created a method of pruning the structure, for removing unnecessary cells; in addition to rewriting the original computer code. The modified RL-NFHP model was evaluated in three applications: the simulated benchmark Car-Mountain problem, well known in the area of autonomous agents; a simulated application in robotics based on the Khepera robot; and an application in a real robot. The experiments show that this modified model fits well the problems of control systems and robotics, with a good generalization. Compared the modified RL-NFHP model with the original one, there was acceleration of learning process and smaller structures of the model trained.