The main issue when dealing with non-stationary processes is related to the requirement of fast adaptation while simultaneously preventing catastrophic damage to previously learned behavior. In this thesis, two on-line learning techniques, one for supervised and the other for unsupervised artificial neural networks, are proposed. A new supervised procedure to continuously adjust weights in a multi layered perceptrons neural networks is proposed. This methodology is tailored to be used in time varying (or non-stationary) models, eliminating the necessity of retraining. The main objective is to keep the error related to the latest income data within a pre established tolerance, while maximizing the information incorporated up to that point. By choosing a balance parameter, the designer is able to decide on the relevance that should be attributed to the new data. Non-linear programming techniques are used in order to properly solve this trade off optimization problem and on-line calculate the new weight set. Numerical results for real and non- real data are presented, illustrating the potentiality and properties of the proposed approach. A modular Fuzzy ART model for hierarchical categorization of data is proposed in such a manner that each module groups the prototypes of the categories developed in the previous module or layer. In this way, the hierarchical levels of successively higher layers learn a more general pattern from the input data while the lower layers learn more specific categories. Interesting properties of the component Fuzzy ART network also apply to this new hierarchical network model like, fast and stable learning of arbitrary sequences of analogical or binary input patterns and the dynamic creation of categories for new input patterns presentation. Some theoretical results related to properties of this new approach for clustering applications are presented, as well as some illustrative numerical results.