ABSTRACT

The World Wide Web is a rapidly growing and changing information source. This reality is gradually replacing the traditional way users obtain news or information. Traditionally, individuals get their news or information from print media, such as newspapers and magazines. The advent of the internet has made things a lot easier by making this digitalized news accessible from anywhere in the world, either through news websites or dedicated application. However, its growth and change rates make the task of finding relevant and recent information harder. Users are still faced with the challenge of visiting numerous websites just to get updated or informed on a specific type of news. This creates a problem as users have to always memorize different URLs and visit numerous website just to view a specific type of news. Therefore the need to develop an intelligent web based dynamic news aggregator that will provide a digital platform for individuals to easily find news pertaining to a particular topic in real time becomes imperative. This aim was achieved via developing an algorithm for the news aggregator that will serve as the output for crawled syndicated web pages based on different categories. InfoSpiders and Incremental web crawling methods were also used to develop an algorithm for the web crawler to download syndicated web pages of different categories of news from different Nigerian news websites. The code was realized in PHP scripting language that is suited for web development and can be embedded into HTML. The web crawler frontier was interfaced with seed URLs to identify all the hyperlinks in the page and add them to the list of URLs to visit. This was possible via applying a stochastic selector and incremental web crawling technology to crawl the entire seed URLs. It crawled the web, searched for news agencies and returned specific news of interest to the user. The system was deployed and tested using Apache web server and a personal computer as the testing machine. In order to ascertain the accuracy and performance measure of the developed system, a comparative analysis of the developed system against existing news aggregators was done, and the developed system yielded an accurate result of 93%.