As obras disponibilizadas nesta Biblioteca Digital foram publicadas sob expressa autorização dos respectivos autores, em conformidade com a Lei 9610/98.
A consulta aos textos, permitida por seus respectivos autores, é livre, bem como a impressão de trechos ou de um exemplar completo exclusivamente para uso próprio. Não são permitidas a impressão e a reprodução de obras completas com qualquer outra finalidade que não o uso próprio de quem imprime.
A reprodução de pequenos trechos, na forma de citações em trabalhos de terceiros que não o próprio autor do texto consultado,é permitida, na medida justificada para a compreeensão da citação e mediante a informação, junto à citação, do nome do autor do texto original, bem como da fonte da pesquisa.
A violação de direitos autorais é passível de sanções civis e penais.
Título:HIERARQUICAL NEURO-FUZZY MODELS BASED ON REINFORCEMENT LEARNING FOR INTELLIGENT AGENTS Instituição:PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO - PUC-RIO Autor(es):KARLA TEREZA FIGUEIREDO LEITE
This thesis investigates neuro-fuzzy hybrid models for
automatic learning of actions taken by agents. The
objective of these models is to provide an agent with
intelligence, making it capable of acquiring and retaining
knowledge and of reasoning (infer an action) by interacting
with its environment. Learning in these models is performed
by a non-supervised process, called Reinforcement Learning.
These novel neuro-fuzzy models have the following
characteristics: automatic learning of the model structure;
auto-adjustment of parameters associated with the
structure; capability of learning the action to be taken
when the agent is on a given environment state; possibility
of dealing with a larger number of inputs than those of
traditional neuro-fuzzy systems; and the generation of
hierarchical linguistic rules.
This work comprised three main stages: bibliographic survey
and study of learning models; definition and implementation
of two new hierarchical neurofuzzy models based on
Reinforcement Learning; and case studies.
The bibliographic survey and the study of learning models
considered learning models employed in agents (aiming to
enhance the autonomous action) and in large and/or
continuous state spaces.
The definition of the two new neuro-fuzzy models was
motivated by the importance of extending the autonomous
capacity of agents through its intelligence, particularly
the learning capacity. The models were conceived from
the study of the existing limitations in current models, as
well as the desirable characteristics for RL-based learning
systems, particularly, when applied to continuous and/or
high dimension environments. These environments present a
characteristic called curse of dimensionality, which makes
impracticable the direct application of the traditional RL-
methods. Therefore, the decision of using a recursive
partitioning methodology (already explored with excellent
results in Souza, 1999), which significantly reduces the
existing neuro-fuzzy systems limitations, was crucial to
this work. The BSP (Binary Space Partitioning) and the
Quadtree/Politree partitioning were then chosen, generating
the RL-NFHB (Reinforcement Learning - Hierarchical Neuro-
Fuzzy BSP) and RL-NFHP (Reinforcement Learning -
Hierarchical Neuro-Fuzzy Politree) models. These two
new models are derived from the hierarchical neuro-fuzzy
models NFHB and NFHQ (Souza, 1999), which use supervised
learning. By using these partitioning methods, together
with the Reinforcement Learning methodology, a new class of
Neuro-Fuzzy Systems (SNF) was obtained, which executes, in
addition to structure learning, the autonomous learning of
the actions to be taken by an agent.
These characteristics represent an important differential
when compared to the existing intelligent agents learning
In the case studies, the two models were tested in three
benchmark applications and one application in robotics. The
benchmark applications refer to 3 problems of control
systems : the mountain cart problem, cart-centering
problem, and the inverted pendulum. The application in
robotics made use of the
Khepera model. The RL-NFHB and RL-NFHP models were
implemented using the Java language in Windows 2000
The experiments demonstrate that these new models are
suitable for problems of control systems and robotics,
presenting a good generalization and generating their own
hierarchical structure of rules with linguistic
Moreover, the automatic environment learning endows the
agent with intelligence (knowledge base, reasoning and
learning). These are characteristics that increase
the autonomous capacity of this agent. The hierarchical
neuro-fuzzy systems field was also enhanced by the
introduction of reinforcement learning, allowing the
learning of hierarchical rules and actions to take place
within the same process.