Multi-modal Feedback for Affordance-driven Interactive Reinforcement Learning

• Autores: Cruz, Francisco; Parisi, Germán; Wermter, Stefan
• Año de publicación: 2018
• Idioma: inglés
• Tipo de recurso: documento de conferencia
• Materia: Ciencias Informáticas; interactive reinforcement learning; affordances; audio-visual feedback; parent-like trainer
• Institución: Universidad Nacional de La Plata

Interactive reinforcement learning (IRL) extends traditional reinforcement learning (RL) by allowing an agent to interact with parentlike trainers during a task. In this paper, we present an IRL approach using dynamic audio-visual input in terms of vocal commands and hand gestures as feedback. Our architecture integrates multi-modal information to provide robust commands from multiple sensory cues along with a confidence value indicating the trustworthiness of the feedback. The integration process also considers the case in which the two modalities convey incongruent information. Additionally, we modulate the influence of sensory-driven feedback in the IRL task using goal-oriented knowledge in terms of contextual affordances.We implement a neural network architecture to predict the effect of performed actions with different objects to avoid failed-states, i.e., states from which it is not possible to accomplish the task. In our experimental setup, we explore the interplay of multi-modal feedback and task-specific affordances in a robot cleaning scenario. We compare the learning performance of the agent under four different conditions: traditional RL, multi-modal IRL, and each of these two setups with the use of contextual affordances. Our experiments show that the best performance is obtained by using audio-visual feedback with affordance-modulated IRL. The obtained results demonstrate the importance of multi-modal sensory processing integrated with goaloriented knowledge in IRL tasks.
In press. Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN), IEEE World Congress on Computational Intelligence (WCCI), Rio de Janeiro, Brazil, July 2018.
Sociedad Argentina de Informática e Investigación Operativa