Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning

Recent success in deep reinforcement learning for continuous control has been dominated by model-free approaches which, unlike model-based approaches, do not suffer from representational limitations in making assumptions about the world dynamics and model errors inevitable in complex domains. However, they require a lot of experiences compared to model-based approaches that are typically more sample-efficient. We propose to combine the benefits of the two approaches by presenting an integrated approach called Curious MetaController. Our approach alternates adaptively between modelbased and model-free control using a curiosity feedback based on the learning progress of a neural model of the dynamics in a learned latent space. We demonstrate that our approach can significantly improve the sample efficiency and achieve nearoptimal performance on learning robotic reaching and grasping tasks from raw-pixel input in both dense and sparse reward settings.

@InProceedings{HWKW19a, 
 	 author =  {Hafez, Burhan and Weber, Cornelius and Kerzel, Matthias and Wermter, Stefan},  
 	 title = {Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning}, 
 	 booktitle = {Proceedings of the International Joint Conference on Neural Networks (IJCNN)},
 	 editors = {},
 	 number = {},
 	 volume = {},
 	 pages = {},
 	 year = {2019},
 	 month = {Jul},
 	 publisher = {},
 	 doi = {10.1109/IJCNN.2019.8852254}, 
 }