Lifelong Learning of Action Representations with Deep Neural Self-Organization

Lifelong learning is fundamental in autonomous robotics for the incremental acquisition of knowledge through experience. However, most of the current deep neural models for action recognition from videos do not account for lifelong learning, but rather learn a batch of training actions. Consequently, there is the need to design learning systems with the ability to incrementally process available perceptual cues and to adapt their behavioral responses over time. We propose a self-organizing neural network architecture for incrementally learning action sequences from videos. The architecture comprises growing self-organizing networks equipped with recurrent connectivity for dealing with time-varying patterns. We use a set of hierarchically-arranged recurrent networks for the unsupervised learning of action representations with increasingly large spatiotemporal receptive fields. The recurrent dynamics modulating neural growth drive the adaptation of the networks to the non-stationary input distribution during the learning phase. We show how our model accounts for an action classification task with a benchmark dataset also in the case of occasionally missing or incorrect sample labels.

@InProceedings{PW17, 
 	 author =  {Parisi, German I. and Wermter, Stefan},  
 	 title = {Lifelong Learning of Action Representations with Deep Neural Self-Organization}, 
 	 booktitle = {AAAI Spring Symposium Series},
 	 editors = {},
 	 number = {},
 	 volume = {},
 	 pages = {608--612},
 	 year = {2017},
 	 month = {Mar},
 	 publisher = {},
 	 doi = {}, 
 }