Name
Abstract | ||
|---|---|---|
Search engines facilitate efficient discovery of information in large information environments such as the Web. As the amount of information rapidly increases, search engines require greater computational resources. Similarly, as the user base increases search engines need to handle increasing numbers of user requests. Existing solutions to these scalability problems are often designed for large computer clusters. This paper presents a flexible solution that is deployable also on small clusters. The solution is based on the allocation and dynamic re-adjustment of indexing and querying roles to cluster nodes in order to optimize cluster utilisation. By allocating cluster machines to the job that requires the most computational power, indexing and querying may both realize performance gains, while neither overwhelms the limited resources available. A prototype system was built and tested on a small cluster using a dataset of over 100 000 Web pages from the uct.ac.za domain. Initial results confirm an improved system resource utilisation, which warrants further investigation. |
Year | DOI | Venue |
|---|---|---|
2007 | 10.1145/1292491.1292502 | SAICSIT Conf. |
Keywords | Field | DocType |
cluster node,user base increases search,cluster machine,large computer cluster,flexible solution,large information environment,search engine,small search engine cluster,computational power,cluster utilisation,dynamic role allocation,small cluster,indexation,indexing,web pages | Cluster (physics),Resource (disambiguation),Search engine,Web page,Computer science,Search engine indexing,Computer cluster,Database,Scalability | Conference |
Citations | PageRank | References |
0 | 0.34 | 8 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ndapandula Nakashole | 1 | 394 | 19.48 |
| Hussein , Suleman | 2 | 170 | 40.89 |
| Calvin Pedzai | 3 | 0 | 0.34 |