Recognition of Transitive Actions with Hierarchical Neural Network Learning

The recognition of actions that involve the use of objects has remained a challenging task. In this paper, we present a hierarchical self-organizing neural architecture for learning to recognize transitive actions from RGB-D videos. We process separately body poses extracted from depth map sequences and object features from RGB images. These cues are subsequently integrated to learn action-object mappings in a self-organized manner in order to overcome the visual ambiguities introduced by the processing of body postures alone. Experimental results on a dataset of daily actions show that the integration of action-object pairs significantly increases classification performance.

@InProceedings{MPW16, 
 	 author =  {Mici, Luiza and Parisi, German I. and Wermter, Stefan},  
 	 title = {Recognition of Transitive Actions with Hierarchical Neural Network Learning}, 
 	 booktitle = {Proceedings of the 25th International Conference on Artificial Neural Networks},
 	 number = {},
 	 volume = {},
 	 pages = {472--479},
 	 year = {2016},
 	 month = {Sep},
 	 publisher = {},
 	 doi = {10.1007/978-3-319-44781-0_56}, 
 }