Title
MonuMAI: Dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification
Abstract
An important part of art history can be discovered through the visual information in monument facades. However, the analysis of this visual information, i.e, morphology and architectural elements, requires high expert knowledge. An automatic system for identifying the architectural style or detecting the architectural elements of a monument based on one image will certainly help improving our knowledge in art and history. Building such tool is challenging as some styles share architectural elements, the bad conservation state of some monuments and the noise included in the image itself. The aim of this paper is to introduce MonuMAI (Monument with Mathematics and Artificial Intelligence) framework. In particular, (i) we designed MonuMAI dataset rich with expert knowledge considering the proposed architectural styles taxonomy and key elements relationship, which allows addressing several tasks, e.g., monument style classification and architectural elements detection, (ii) we developed MonuMAI deep learning pipeline based on lightweight MonuNet architecture for monument style classification and MonuMAI Key Elements Detection (MonuMAI-KED) model, and (iii) we built citizen science based MonuMAI mobile app that uses the proposed MonuMAI deep learning pipeline trained on MonuMAI dataset for performing in real life conditions. Our experiments show that both MonuNet architecture and the detection model achieve very good results under real life conditions.
Year
DOI
Venue
2021
10.1016/j.neucom.2020.09.041
Neurocomputing
Keywords
DocType
Volume
Convolutional Neural Networks,Architectural information extraction,Architectural style classification,MonuMAI,Monumental heritage
Journal
420
ISSN
Citations 
PageRank 
0925-2312
0
0.34
References 
Authors
0
7
Name
Order
Citations
PageRank
Alberto Lamas1100.99
Siham Tabik216718.92
Policarpo Cruz300.34
Rosana Montes4467.43
Álvaro Martínez-Sevilla500.34
Teresa Cruz600.34
Francisco Herrera7273911168.49