Title
Objective evaluation of Spider Detection Techniques.
Abstract
Spider is a program for harvesting internet resources. Spiders Detection Techniques(SOT) are used to regulate and monitor behaviors of spiders visiting website. In this paper, an Evaluation Method based on Trap technique(EMT) is proposed to calculate the recall rate and precision rate of SOT. Without relying on manual analysis, it is more objective and more adaptive to the development of SOT. T he principles of EMT bases on the statistical hypothesis that the distribution of users captured by trap obeys binomial distribution theory. The experiment of EMT indicates three conclusions: (1)EMT has the consistent conclusion with the manual analysis result. (2)EMT is little impacted by time span of analysis.(3)EMT is little impacted by the traps layout rate when it changes in ±10%. ©2010 IEEE.
Year
DOI
Venue
2010
10.1109/WCINS.2010.5541838
WCNIS
Keywords
Field
DocType
internet,trap,information retrieval,robots,evaluation,data privacy,search engines,binomial distribution,tin,cryptography
Binomial distribution,Recall rate,Spider,Simulation,Computer science,Artificial intelligence,Machine learning,Statistical hypothesis testing,Distributed computing,Internet resources
Conference
Volume
Issue
ISSN
null
null
null
ISBN
Citations 
PageRank 
978-1-4244-5850-9
0
0.34
References 
Authors
7
3
Name
Order
Citations
PageRank
Chunlong Fan132.10
Zhouhua Yu200.34
Lei Xu300.34