Paper Info
Title
Simulation Based Approach to Evaluate a Distributed Search Engine
Abstract
ABSTRACT We present a simulation model used to evaluate a distributed search engine by modelling its performance. The model is designed to aid performance-driven evolution of the system. We present the results of simulations investigating the architecture of the system and how it will behave when scaled up, and show how the simulation gave insights into the behaviour of the real system. KEYWORDS Simulation, Search Service Management, Search Engine Performance 1. INTRODUCTION The goal of this paper is to present a simulation tool for modelling our Adaptive Distributed Search u0026 Advertising search engine (ADSA) (Khoussainov et al., 2001a). The ADSA project has as its aim the development of a distributed search system that can scale with Web growth. A distributed system of topic-specific search engines has been developed. Such an architecture partitions its information databases into multiple smaller databases in an attempt to relieve the scalability problems of centralised search architectures, and also has the potential to incorporate privately indexable material into a logically unified database. Such a system can scale with Web growth since each engine will process only pages and queries relevant to a selected topic(s). It can also provide comprehensive coverage of selected topics, sophisticated query processing, and lower costs due to the smaller index sizes and query loads at each engine. Distributed topic-specific search will enable both large and small organisations to participate in a global search services market by providing specialist services in selected topic domains. A pilot service has been established and is expected to run until the end of 2003 (http://www.cnds.ucd.ie/adsa). To support this development, we have also built a simulation system which can be used to investigate the properties of ADSA with larger numbers of databases and users than are available in reality. While distributed architectures are not an innovative idea, ADSA is unique in so far as it provides independent ownership of individual databases that dramatically reduces the entry cost for an organisation or even an individual into the global information search market (Khoussainov et al., 2001b). Anybody can install a small, private search engine without the prohibitive costs of maintaining large monolithic databases, and yet be a part of a global search system that can match the growth of the Internet for the foreseeable future. Independent ownership is especially attractive in the context of providing search service for privately indexable information. Providers can retain full control over indexing and information access, yet at the same time integrate themselves into a wide-scale search engine. Due to the proprietary nature of many search systems very little research has been published on their performance evaluation (Floyd and Paxson, 2001; Lorenz, 1997; Unger et al., 1998). With this in mind existing simulation models cannot be applied to measure its performance and hence we developed our own unique model to assist future research in the area of workload modelling of distributed information retrieval.
Year
Venue
Field
2003
ICWI
Service management,Search engine,Information retrieval,Computer science,Workload,Information access,Search engine indexing,Distributed algorithm,Database,The Internet,Scalability
DocType
Citations 
PageRank 
Conference
2
0.39
References 
Authors
3
3
Name
Order
Citations
PageRank
Thomas Phelan120.39
Ahmed Patel216723.33
Seamus O. Ciardhuáin3525.36