Paper Info
Title
Search by Screenshots for Universal Article Clipping in Mobile Apps
Abstract
To address the difficulty in clipping articles from various mobile applications (apps), we propose a novel framework called UniClip, which allows a user to snap a screen of an article to save the whole article in one place. The key task of the framework is search by screenshots, which has three challenges: (1) how to represent a screenshot; (2) how to formulate queries for effective article retrieval; and (3) how to identify the article from search results. We solve these by (1) segmenting a screenshot into structural units called blocks, (2) formulating effective search queries by considering the role of each block, and (3) aggregating the search result lists of multiple queries. To improve efficiency, we also extend our approach with learning-to-rank techniques so that we can find the desired article with only one query. Experimental results show that our approach achieves high retrieval performance (F1 = 0.868), which outperforms baselines based on keyword extraction and chunking methods. Learning-to-rank models improve our approach without learning by about 6%. A user study conducted to investigate the usability of UniClip reveals that ours is preferred by 21 out of 22 participants for its simplicity and effectiveness.
Year
DOI
Venue
2017
10.1145/3091107
ACM Trans. Inf. Syst.
Keywords
Field
DocType
Universal clipping,search by screenshots,article clipping,mobile apps
Data mining,Market segmentation,Information retrieval,Keyword extraction,Computer science,Usability,Chunking (psychology),Artificial intelligence,Mobile apps,Machine learning,Clipping (audio)
Journal
Volume
Issue
ISSN
35
Issue-in-Progress
1046-8188
Citations 
PageRank 
References 
1
0.36
35
Authors
6
Name
Order
Citations
PageRank
Kazutoshi Umemoto1193.12
Ruihua Song2113859.33
Jian-yun Nie33681238.61
Xing Xie49105527.49
Katsumi Tanaka51349160.89
Yong Rui67052449.08