Recognizing FineGrained and Composite Activities

Authors: Marcus Rohrbach Anna Rohrbach Michaela Regneri Sikandar Amin Mykhaylo Andriluka Manfred Pinkal Bernt Schiele

Publish Date: 2015/08/22

Volume: 119, Issue: 3, Pages: 346-373

PDF Link

Abstract

Activity recognition has shown impressive progress in recent years However the challenges of detecting finegrained activities and understanding how they are combined into composite activities have been largely overlooked In this work we approach both tasks and present a dataset which provides detailed annotations to address them The first challenge is to detect finegrained activities which are defined by low interclass variability and are typically characterized by finegrained body motions We explore how human pose and hands can help to approach this challenge by comparing two posebased and two handcentric features with stateoftheart holistic features To attack the second challenge recognizing composite activities we leverage the fact that these activities are compositional and that the essential components of the activities can be obtained from textual descriptions or scripts We show the benefits of our handcentric approach for finegrained activity classification and detection For composite activity recognition we find that decomposition into attributes allows sharing information across composites and is essential to attack this hard task Using script data we can recognize novel composites without having training data for themThis work was supported by a fellowship within the FITweltweitProgram of the German Academic Exchange Service DAAD by the Cluster of Excellence “Multimodal Computing and Interaction” of the German Excellence Initiative and the Max Planck Center for Visual Computing and Communication

Keywords: