Paper Info
Title
A machine learning approach to web page filtering using content and structure analysis
Abstract
As the Web continues to grow, it has become increasingly difficult to search for relevant information using traditional search engines. Topic-specific search engines provide an alternative way to support efficient information retrieval on the Web by providing more precise and customized searching in various domains. However, developers of topic-specific search engines need to address two issues: how to locate relevant documents (URLs) on the Web and how to filter out irrelevant documents from a set of documents collected from the Web. This paper reports our research in addressing the second issue. We propose a machine-learning-based approach that combines Web content analysis and Web structure analysis. We represent each Web page by a set of content-based and link-based features, which can be used as the input for various machine learning algorithms. The proposed approach was implemented using both a feedforward/backpropagation neural network and a support vector machine. Two experiments were designed and conducted to compare the proposed Web-feature approach with two existing Web page filtering methods - a keyword-based approach and a lexicon-based approach. The experimental results showed that the proposed approach in general performed better than the benchmark approaches, especially when the number of training documents was small. The proposed approaches can be applied in topic-specific search engine development and other Web applications such as Web content management.
Year
DOI
Venue
2008
10.1016/j.dss.2007.06.002
Decision Support Systems
Keywords
DocType
Volume
web page classification,web content management,link analysis,topic-specific search engine,keyword-based approach,existing web page,benchmark approach,web application,web page,web mining,web structure analysis,machine learning,web content analysis,support vector machine,search engine,content management,content analysis,web pages,structure analysis,information retrieval
Journal
44
Issue
ISSN
Citations 
2
Decision Support Systems
40
PageRank 
References 
Authors
1.29
40
2
Name
Order
Citations
PageRank
Michael Chau1147197.79
Hsinchun Chen29569813.33