Document Type : Original/Review Paper
Author
panjab university
Abstract
The enormous growth of the World Wide Web in recent years has made it necessary to perform resource discovery efficiently. For a crawler it is not an simple task to download the domain specific web pages. This unfocused approach often shows undesired results. Therefore, several new ideas have been proposed, among them a key technique is focused crawling which is able to crawl particular topical portions of the World Wide Web quickly without having to explore all web pages. Focused crawling is a technique which is able to crawled particular topics quickly and efficiently without exploring all WebPages. The proposed approach does not only use keywords for the crawl, but rely on high-level background knowledge with concepts and relations, which are compared with the texts of the searched page.
In this paper a combined crawling strategy is proposed that integrates the link analysis algorithm with association metric. An approach is followed to find out the relevant pages before the process of crawling and to prioritizing the URL queue for downloading higher relevant pages, to an optimal level based on domain dependent ontology. This strategy make use of ontology to estimate the semantic contents of the URL without exploring which in turn strengthen the ordering metric for URL queue and leads to the retrieval of most relevant pages.
Keywords
)