It shouldn’t really happen to a nice company like Google. You try to do someone a favor and it blows up in your face.
A few years back, they realized that it would be difficult to give a speedy response to a search query if they had a single database of all the web pages they were spidering. So they decided to put web pages that might come up more frequently in search queries in their regular database. Other less popular web pages they would put in a supplementary or secondary index. By this means, they could keep cataloguing all the web pages they could find and still deliver fast results to most key word searches by using the regular index. Technically it was the right solution. The mistake was they told people about it. At the time the approach seemed a positive move.
Move on to 2005, and this two-index system begins to upset a lot of people. With the explosive growth of the Internet, it is impossible to put the majority of all web pages in the primary index. Since the primary index is spidered more frequently and its web pages are more likely to appear in keyword searches, you can understand why people got upset. Of course if Google hadn’t mentioned the supplemental index, then people would never have known of this possible problem.
The whole issue has become a can of worms for Google. A great many people were upset. SEO (search engine optimization) experts worked hard to figure out how to keep web pages out of the supplemental index. Google has tried to lower the temperature on this topic by reducing the differences between the two indexes (the regular index and the supplemental index). In mid-December, Yonatan Zunger of the Google Search Quantity Team reported on progress.
We improved the crawl frequency and decoupled it from which index a document was stored in, and once these “supplementalization effects” were gone, the “supplemental result” tag itself-which only served to suggest that otherwise good documents were somehow suspect-was eliminated a few months ago. Now we’re coming to the next major milestone in the elimination of the artificial difference between indices: rather than searching some part of our index in more depth for obscure queries, we’re now searching the whole index for every query.
From a user perspective, this means that you’ll be seeing more relevant documents and a much deeper slice of the web, especially for non-English queries. For webmasters, this means that good-quality pages that were less visible in our index are more likely to come up for queries.
You might have hoped that would satisfy searchers. However Barry Schwartz of SERoundtable felt that the announcement drove only more confusion.
I’ll call Google out on this one, and I rarely do.
Google, we need you to stop hiding this index from us. We really need an explanation of what this index does, why a page would be placed in the supplemental index. When Google actually searches it? In what examples would a page in the supplemental index rank better than a page in the main index?
The confusion over the supplemental index has gone on too long.
Andy Beard was equally concerned that his method of identifying web pages in the supplemental index no longer seemed to be working. Apparently he really would like to know which web pages are still in the supplemental index.
Google may well be upset that people do not seem to be accepting its explanation of the “closeness” of the two indexes. On this one I agree with Google. People seem to be fixated on the notion of the supplemental index, as if this was an important issue in the keyword search algorithms.
It’s interesting to compare this with another Google invention, which used to be a hot topic and is now a yawn topic for most SEO keyword searchers. That’s the Google Toolbar Page Rank indicator. It may well be broken and is possibly only kept around for marketing reasons. It has almost zero connection now with how web pages are ranked in keyword searches.
So people, let’s get over it. Forget about that supplemental index and work on the more important things that make a web page memorable, authoritative, trust-worthy and ultimately search-engine visible. You’ll get much better rewards for your efforts.
Supplemental Result in Google – Hell or Help – March 31st, 2007
Google Supplemental Results Index – A Word To The Wise – July 9th, 2006
Google Supplemental Label Out, PageRank Next? – August 1st, 2007