Paper Info
Title
Pattern Theory-Based Interpretation of Activities
Abstract
We present a novel framework, based on Germander's pattern theoretic concepts, for high-level interpretation of video activities. This framework allows us to elegantly integrate ontological constraints and machine learning classifiers in one formalism to construct high-level semantic interpretations that describe video activity. The unit of analysis is a generator that could represent either an ontological label as well as a group of features from a video. These generators are linked using bonds with different constraints. An interpretation of a video is a configuration of these connected generators, which results in a graph structure that is richer than conventional graphs used in computer vision. The quality of the interpretation is quantified by an energy function that is optimized using Markov Chain Monte Carlo based simulated annealing. We demonstrate the superiority of our approach over a purely machine learning based approach (SVM) using more than 650 video shots from the You Cook dataset. This dataset is very challenging in terms of complexity of background, presence of camera motion, object occlusion, clutter, and actor variability. We find significantly improved performance in nearly all cases. Our results show that the pattern theory inference process is able to construct the correct interpretation by leveraging the ontological constraints even when the machine learning classifier is poor and the most confident labels are wrong.
Year
DOI
Venue
2014
10.1109/ICPR.2014.28
Pattern Recognition
Keywords
Field
DocType
Markov processes,Monte Carlo methods,image classification,image motion analysis,image sensors,learning (artificial intelligence),ontologies (artificial intelligence),simulated annealing,support vector machines,Markov Chain Monte Carlo,SVM,actor variability,camera motion,clutter,high-level semantic interpretations,machine learning based approach,machine learning classifier,machine learning classifiers,object occlusion,ontological label,pattern theory inference process,pattern theory-based interpretation,simulated annealing,video activities
Histogram,Pattern theory,Markov chain Monte Carlo,Computer science,Artificial intelligence,Learning classifier system,Simulated annealing,Computer vision,Pattern recognition,Inference,Support vector machine,Feature extraction,Machine learning
Conference
ISSN
Citations 
PageRank 
1051-4651
2
0.37
References 
Authors
8
4
Name
Order
Citations
PageRank
Fillipe Dias Moreira de Souza1555.81
Sudeep Sarkar22839317.68
Anuj Srivastava32853199.47
Jing-yong Su415610.93