Policy search for learning robot control using sparse data

B. Bischoff, D. Nguyen-Tuong, H. Van Hoof, A. McHutchon, C. E. Rasmussen, A. Knoll, J. Peters, M. P. Deisenroth

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

18 Scopus citations

Abstract

In many complex robot applications, such as grasping and manipulation, it is difficult to program desired task solutions beforehand, as robots are within an uncertain and dynamic environment. In such cases, learning tasks from experience can be a useful alternative. To obtain a sound learning and generalization performance, machine learning, especially, reinforcement learning, usually requires sufficient data. However, in cases where only little data is available for learning, due to system constraints and practical issues, reinforcement learning can act suboptimally. In this paper, we investigate how model-based reinforcement learning, in particular the probabilistic inference for learning control method (Pilco), can be tailored to cope with the case of sparse data to speed up learning. The basic idea is to include further prior knowledge into the learning process. As Pilco is built on the probabilistic Gaussian processes framework, additional system knowledge can be incorporated by defining appropriate prior distributions, e.g. A linear mean Gaussian prior. The resulting Pilco formulation remains in closed form and analytically tractable. The proposed approach is evaluated in simulation as well as on a physical robot, the Festo Robotino XT. For the robot evaluation, we employ the approach for learning an object pick-up task. The results show that by including prior knowledge, policy learning can be sped up in presence of sparse data.

Original languageEnglish
Title of host publicationProceedings - IEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3882-3887
Number of pages6
ISBN (Electronic)9781479936854, 9781479936854
DOIs
StatePublished - 22 Sep 2014
Event2014 IEEE International Conference on Robotics and Automation, ICRA 2014 - Hong Kong, China
Duration: 31 May 20147 Jun 2014

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729

Conference

Conference2014 IEEE International Conference on Robotics and Automation, ICRA 2014
Country/TerritoryChina
CityHong Kong
Period31/05/147/06/14

Fingerprint

Dive into the research topics of 'Policy search for learning robot control using sparse data'. Together they form a unique fingerprint.

Cite this