Paper Info
Title
On building a reusable Twitter corpus
Abstract
The Twitter real-time information network is the subject of research for information retrieval tasks such as real-time search. However, so far, reproducible experimentation on Twitter data has been impeded by restrictions imposed by the Twitter terms of service. In this paper, we detail a new methodology for legally building and distributing Twitter corpora, developed through collaboration between the Text REtrieval Conference (TREC) and Twitter. In particular, we detail how the first publicly available Twitter corpus - referred to as Tweets2011 - was distributed via lists of tweet identifiers and specialist tweet crawling software. Furthermore, we analyse whether this distribution approach remains robust over time, as tweets in the corpus are removed either by users or Twitter itself. Tweets2011 was successfully used by 58 participating groups for the TREC 2011 Microblog track, while our results attest to the robustness of the crawling methodology over time.
Year
DOI
Venue
2012
10.1145/2348283.2348495
SIGIR
Keywords
Field
DocType
new methodology,information retrieval task,twitter term,twitter corpus,real-time search,available twitter corpus,reusable twitter corpus,twitter data,specialist tweet,crawling methodology,twitter real-time information network,reproducibility,information retrieval,real time
Data mining,World Wide Web,Crawling,Social media,Information retrieval,Identifier,Computer science,Microblogging,Robustness (computer science),Software,Terms of service,Text Retrieval Conference
Conference
Citations 
PageRank 
References 
39
1.80
1
Authors
6
Name
Order
Citations
PageRank
Richard Mccreadie140332.43
Ian Soboroff21907218.39
Jimmy Lin34800376.93
Craig Macdonald42588178.50
Iadh Ounis53438234.59
Dean McCullough6592.94