Publications related to learning.
- E. I. Barakova and L. Spaanenburg Windowed Active Sampling for Reliable Neural Learning [1159 KB pdf]. Journal of Systems Architecture, 44:635-650, 1998.
Abstract
The composition of the example set has a major impact on the quality of neural learning. The popular approach is focused on extensive pre-processing to bridge the representation gap between process measurement and neural presentation. In contrast, windowed active sampling attempts to solve these problems in an on-line interaction between problem selection and learning. This paper provides an unified view on the conflicts that may pop-up within a neural network in the presenceo f ill-ordered data. It is marked that such conflicts become noticeable from the operational learning characteristics. An adaptive operational strategy is proposed that closes the representation gap and its working is illustrated in the diagnosis of power generators.
- D. Vanderelst, R. Ahn, and E. I. Barakova Simulated trust: towards robust social learning [207 KB pdf]. In S. Bullock, J. Noble, R. Watson, and M. A. Bedau, editors, Artificial Life XI: Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems, pages 632-639, 2008. MIT Press, Cambridge, MA.
Abstract
Social learning is a potentially powerful learning mechanism to use in artificial multi-agent systems. However, findings about how animals use social learning show that it is also possibly detrimental. By using social learning agents act based on second-hand information that might not be trustworthy. This can lead to the spread of maladaptive behavior throughout populations. Animals employ a number of strategies to selectively use social learning only when appropriate. This suggests that artificial agents could learn more successfully if they are able to strike the appropriate balance between social and individual learning. In this paper, we propose a simple mechanism that regulates the extent to which agents rely on social learning. Our agents can vary the amount of trust they have in others. The trust is not determined by the performance of others but depends exclusively on the agents’ own rating of the demonstrations. The effectiveness of this mechanism is examined through a series of simulations. We first show that there are various circumstances under which the performance of multi-agents systems is indeed seriously hampered when agents rely on indiscriminate social learning. We then investigate how agents that incorporate the proposed trust mechanism fare under the same circumstances. Our simulations indicate that the mechanism is quite effective in regulating the extent to which agents rely on social learning. It causes considerable improvements in the learning rate, and can, under some circumstances, even improve the eventual performance of the agents. Finally, some possible extensions of the proposed mechanism are being discussed.
- E. I. Barakova, T. Lourens, and Y. Yamaguchi. Life-long learning: consolidation of novel events into dynamic memory representations [327 KB pdf]. In J. Mira and J. R. Alvarez, editors, Proceedings of the International Work-Conference on Artificial and Natural Neural Networks, IWANN 2003, volume 2686 of Lecture Notes in Computer Science, pages 110-117, Menorca, Spain, June 2003. Springer-Verlag.
Abstract
Life-long learning paradigm accentuates on the continuity of the on-line process of integrating novel information into the existing representational structures, and recategorization or update of these structures. This paper brings up the hypothesis, that memory consolidation is a biological mechanism that resembles the features of life-long learning paradigm. A global model for memory consolidation is proposed on a functional level, after reviewing the empirical studies on the hippocampal formation and neocortex. Instead of considering memory as storage, the proposed model reconsiders the memory process as recategorization. Distinct experiences that share a common element can be consolidated in the memory in a way such that they are substrata for a new solution. The model is applied to an autobiographical robot.
- E. I. Barakova Familiarity Gated Learning for Inferential use of Episodic Memories in Novel Situations – A robot simulation [638 KB pdf]. In L. S. Smith, A. Hussain and I. Aleksander, editors, Brain Inspired Cognitive Systems 2004, pages 1-7 (BIS 1-5). Department of Computing Science and Mathematics, University of Stirling, Stirling FK9 4LA, Scotland, UK, 2004, ISBN 1 85769 199 7.
Abstract
This paper presents a method for familiarity gated encoding of episodic memories for the purpose of their inferential use in a spatial navigation task. The method is strongly inspired by the state of the art understanding of the hippocampal functioning and especially its role in novelty detection and episodic memory formation in relation to spatial context. A navigation task is used to provide an experimental setup for behavioral testing with a rat-like agent. The model is build on three presumptions: First that episodic memory formation has behavioral, as well as sensory and perceptual correlates; second, hippocampal involvement in the novelty/familiarity detection and episodic memory formation, experimentally supported by neurobiological experiments; and third, that a straightforward parallel exists between internal hippocampal and abstract spatial representations. Some simulation results are shown to support the reasoning and reveal the methods applicability for practically oriented behavioral simulation.
- E. I. Barakova Emergent behaviors based on episodic encoding and familiarity driven retrieval [431 KB pdf]. In C. Bussler and D. Fensel, editors, Artificial Intelligence: Methodology, Systems, and Applications, 11th International Conference, AIMSA 2004, volume 3192 of Lecture Notes in Artificial Intelligence, pages 188-197. Springer-Verlag, 2004.
Abstract
In analogy to animal research, where behavioral and internal neural dynamics are simultaneously analysed, this paper suggests a method for emergent behaviors arising in interaction with the underlying neural mechanism. This way an attempt to go beyond the indeterministic nature of the emergent behaviors of robots is made. The neural dynamics is represented as an interaction of memories of experienced episodes, the current environmental input and the feedback of previous motor actions. The emergent properties can be observed in a two staged process: exploratory (latent) learning and goal oriented learning. Correspondingly, the learning is dominated to a different extent by two factors: novelty and reward. While the reward learning is used to show the relevance of the method, the novelty/familiarity is a basis for forming the emergent properties. The method is strongly inspired by the state of the art understanding of the hippocampal functioning and especially its role in novelty detection and episodic memory formation in relation to spatial context.
- E. I. Barakova Learning Reliability: a study on indecisiveness in sample selection [3687 KB pdf]. PrintPartners Ipskamp B.V. ISBN 90-367-0987-3, March 1999.
Preface
The design of a product is based on the assumption of how it will be used. Conversely, the product is usually good for only such usage as was assumed during its conception. In a classical sense, the implicit assumption brings an explicit specification from which the design is derived. More often than not, the specification is therefore the starting point of a hopefully structured and well-behaved, but eventually mechanical design effort. Where the customer tends to learn from the design and mandates to change and/or augment the specification during the process, the project planning gets invalidated. Current practice is therefore to fix the specification in advance, for instance by contract.
The interest in Artificial Neural Networks (ANN) is founded on their ability to learn from examples, as derived from the environment in which the product will operate, instead of being designed from an hypothesis about the operation. It is commonly agreed that learning is based on memorization (associating or mapping a set of questions to their answers) and generalization (the ability to answer new questions about the same problem). As such, ANNs promise a perfect fit to their intended usage. But circumstantial evidence still does not equal a witness observation. Despite its historic fame, an Artificial Neural Network will not learn all, let alone under all circumstances. This is probably the most striking difference with a designed product: there will never be a proof by construction!
With the coming of age of neural technology, an impressive number of neural products have found their way to the market place [88]. Some popular applications are indicated in figure 1, which position them in the area spanned by computational complexity and model correctness. The bars indicate the achieved performance: the patterns within the bar indicate the widely achieved results, since the white part stands for the best results in the area. Clearly, none of them achieves a 100% correct functionality. It appears, that for each application a bottom level of functionality can be reached almost without any effort. However, to go beyond requires special attention and has therefore spurred a lot of research to develop new algorithms, to construct alternative architectures, to provide different settings of input parameters or to preprocess input data.
To achieve a product of ultimate performance, two methods can be devised: (a) its function is based on a provably correct algorithm, and (b) an effective redundancy is to be incorporated in the underlying algorithm. As far as ANNs are constructed from analysis of noisy data, they can entirely be considered as systems of the second type. Because statistics is concerned with data analysis as well, there is a considerable overlap between the fields of neural networks and statistics. To analyze learning and generalization of neural networks from noisy/randomized data, statistical inference can also be used.
Performance enhancement can be created by a kind of majority voting. This principle suggests that, instead of providing one neural network solution to a problem, a set of neural networks can be combined to form a neural net system which performs better than any of the networks on its own [116] [138]. The conclusion made in [112] is that mere redundancy does not necessarily increase reliability. Empirically it is common practice to train many different candidate networks to select the winner on basis of predefined criteria. A disadvantage of this method is that training of the losing networks does not help in a further development. Another weak point is that the criterion for choosing the best network is usually the performance on a validation set, which can not guarantee the modeling quality of the underlying data generator. But when the networks are incomplete versions of the same functionality, the combination might raise the functional correctness to a higher level (Figure 2).
The committee arrangement generalizes this idea. It can have significantly better predictions on new data at an acceptable increase of the computational complexity. The performance of the committee can be much better than the performance of each single network in isolation. The committee contains a set of a trained networks diversified in a distinct way. Diversity can appear in the number of hidden neurons, in the kind of network model, in the mixture of networks, in the optimization criteria, in the initial weight configuration, training parameters in the training samples, etc. The extent to which reliability can be improved by combining neural net solutions depends on the type of diversity, present in the set of nets.
All such techniques assume that the basic neural network is optimally trained. However, we have noticed that training algorithms are often slow and sometimes unable to converge, even though the underlying techniques often perform very well on other problems. In other words, even though an ANN can be trained to some functionality, there appears to be an underlying problem that causes unreliability in learning. This thesis will therefore be devoted to unravel such circumstances and to contribute ways in which reliable learning can be achieved. By large, the neural paradigm problem is represented as a stream of examples (data) and that guides the learning algorithm to adapt the network parameters until the network is trained to give the right answers to the posed questions. Thus the success and the reliability of this training depends to a large extent on the content and composition of this data stream.
Overall unreliable learning can be considered to result from the interaction between three factors: network, problem, and algorithm. In an attempt to answer questions like why and when the learning process will become unreliable and when a systematic failure can appear, backpropagation (still the algorithm with highest practical significance) has been used. The restricted class of architectures it is supposed to be used for and the feedforward architecture allow us to elaborate in more detail on the problem with respect to the chosen architecture and algorithm.
As we found that the conventional focus on network, problem and algorithm leaves much to be desired, we propose here to base the discussion rather on symmetry, randomness (as basic network design principles), and knowledge (the problem to be learned) as the basic ingredients of the universe of discourse. A high degree of symmetry in the initially designed network is historically viewed to favor the learning algorithm in providing an equal chance to move in several directions. However, this has also a drawback: the freedom of choice may lead to indecisiveness. Admittedly, randomness may in turn help the network escape from such a dilemma. But then again, randomness may wipe away the knowledge; hence a working balance should be found.
Symmetry can be dominant in the beginning of, but also at specific moments during, learning. Randomness (for instance as stochastic variable in the learning algorithm or as additional noise at the network input, output or internal parameters ) is then required to force the presentation of examples to follow alternative itineraries. When the amount of randomness is not sufficient to counteract symmetry, learning will not be completed: instead of being adapted to ensure the right mapping between input/output data strings, the initial parameters will eventually become zero. If the noise (the randomness) of the system is dominant, learning will also be unsuccessful, because the network will rather learn the noise than the exemplified knowledge. The fundamental issue of learning is therefore the creation of a functional balance between symmetry and randomness directed by the examples (the knowledge).
To bring this idea into tangible borders, the interaction between learning components is represented in the error surface paradigm. The network will be able to extract the necessary information by adapting itself to map the questions posed to the right answers. This adaptation is in fact an optimization procedure and is thus equivalent to finding the minimum energy state on an error landscape. The steps, that the learning algorithm takes on this landscape, are directed by the presented examples and form a learning trajectory on this surface. Directing this itinerary properly can help to escape some difficulties to pass surface areas, at which the learning algorithm normally spends a lot of time on or from which it can never escape. For finding an optimal trajectory on the error surface, the so–called regularisation methods have been used. An alternative effect has the introduction of extra noise during training. Our objection here is that the task complexity or the convergence accuracy may be changed in an unwanted direction. The investigation of the statistical long-run effects of example presentation when traveling on the difficult forms of the global error surface brings us to a constructive algorithm which helps in escaping them.
Therefore, the work in this thesis takes an alternative route to ensure reliable learning by focussing on sample diversity [116]. On basis of the instantaneous characteristics of the current training set we will conclude on learnability, reorder the set if necessary to establish the best sample sequence and train eventually a single network with success.
In conclusion, this thesis aims to give directions on how learning can be guaranteed so that its duration will be short and stable and its success unquestionable from the outset. In this respect, we aim to contribute to move neural technology from the realm of ‘Learning by Examples’ to ‘Design by Examples’.