How will the next generation of search engines outperform Google’s all-conquering Pagerank algorithm?
One route might be to hire Vwani Roychowdhury at the University of California, Los Angeles and his buddies who have found a fascinating new way to tackle the problem of website rankings.
Their breakthrough is to have found that the structure of the web is determined by three factors: the number of inbound links to a page, the rate at which pages are created and deleted and the likelihood that somebody visiting a page will link to it.
This last factor is the forehead smacker. Google’s PageRank cannot easily identify new sites with huge potential because their very newness means they they don’t have a large number of inbound links and so feature poorly in the rankings.
But by looking at the ratio of visitors to incoming links, Roychowdhury and co can get a good handle on a site’s potential, even when it is new.
In fact, these sites stick out like sore thumbs. It turns out that in the year’s worth of data the team examined, only 6.5 per cent of the 10 million or so sites they monitored received more than two new incoming links.
It is these up-and-coming sites that go on to displace more established sites in the popularity stakes, leading to the constant slow churn of content we see on conventional Pagerank-based search results. The UCLA team simply see them earlier.
That’s a fascinating and valuable insight.
Roychowdhury and co liken this process to the age-old battle between “experience” (well established sites with many incoming links) and “talent” (up-and-coming sites with potential).
Their algorithm won’t replace Pagerank but it could help to significantly fine tune it, and that could pique the interest of a well known company based in Mountain View, not to mention numerous other pretenders to the search engine crown.
Ref: arxiv.org/abs/0901.0296: Experience Versus Talent Shapes the Structure of the Web