Note – July 31, 2007
Google has announced that it will no longer show the Supplemental Result label in the SERPs. For details see Google Supplemental Label Out, PageRank Next?.
Most of the content of this post is still relevant although the specific tests to identify which posts are in the Supplemental Index no longer apply. Perhaps Google will heed the calls to identify Supplemental Results via Google Webmaster Central.
Google Hell was an evocative term when Jim Boykin coined it in 2005 to describe the Google Supplemental Results Index. Many webmasters earlier that year had been devastated by the Florida update when their Google generated website traffic in some cases had disappeared. Paranoia was everywhere. Google Hell struck a responsive chord and since then it’s been a term to make webmasters shudder. Even this year, Aaron Wall is still offering good advance on how to escape from that Hell.
It’s a strange term to associate with a company like Google, with its slogan ‘Do No Evil‘. Mark Cutts, one of the best known Google bloggers, has recently suggested that supplemental results are not all that bad and will likely be getting better. Here is what he wrote:
As a reminder, supplemental results aren’t something to be afraid of; I’ve got pages from my site in the supplemental results, for example. A complete software rewrite of the infrastructure for supplemental results launched in Summer o’ 2005, and the supplemental results continue to get fresher. Having URLs in the supplemental results doesn’t mean that you have some sort of penalty at all; the main determinant of whether a URL is in our main web index or in the supplemental index is PageRank. If you used to have pages in our main web index and now they’re in the supplemental results, a good hypothesis is that we might not be counting links to your pages with the same weight as we have in the past. The approach I’d recommend in that case is to use solid white-hat SEO to get high-quality links (e.g. editorially given by other sites on the basis of merit). I think going forward, you’ll continue to see the supplemental results get even fresher, and website owners may see more traffic from their supplemental results pages.
Perhaps it’s the time to set the record straight on Google’s supplemental results. People have leapt too quickly to vilify Google for putting some results in their Supplemental Results Index. There is a silver lining to this Google Hell. Let me explain.
The True Nature Of Google’s Supplemental Results
First we should acknowledge that at first sight it is somewhat surprising that Google has split all web pages into two separate databases. This is not a strategic decision but rather a very practical operational decision. Let us remember that Google’s mission here is to produce relevant results fast when people are doing keyword queries. Google is not applying some moral judgement to each web page as it is assigned to one database or the other.
Web pages are being created every second at a staggering rate. This explosive growth rate is now accelerating as blogging and social media such as MySpace and YouTube become more and more popular. It would be an impossible operational process to give the same attention to every new web page that is created. So web pages that are more likely to come up as most relevant for particular keyword queries get more attention. These are registered in the main database. This means that these web pages are spidered on a regular basis to ensure they still exist and that they still have the same content. In a sense Google separates the sheep from the goats.
Web pages that are less likely to feature in responses to keyword queries are registered in the Supplemental Results Index. There may be two reasons for that:
- For common keyword queries, there are other web pages that are deemed by Google to be more relevant.
- The keyword query for which they are the most relevant web page is very rarely posed by searchers.
Note that the supplemental result web pages do appear if they are the most relevant for a given keyword enquiry. Supplemental results are not permanently hidden or buried.
The Silver Lining Of Supplemental Results
Silver lining in the Google Hell
The division of all web pages into two distinct databases is likely done for operational reasons. Whether we like it or not that is the way Google has arranged matters. This division does help us understand how Google is assessing the individual web page. With both the Yahoo! and the MSN/Live search engines, it is not possible to know how valuable a given web page may be.
With all three engines, it is relatively easy to check whether a given web page is indexed. If a search is done for relatively long patch of text from the web page, then the search engine should produce only that web page as the result of the query.
Is this web page a sheep or a goat?
For Yahoo! and MSN/Live, the result will show whether it is included or not: it’s a simple Yes or No. It gives no indication whether the search engine thinks the page is valuable or worthless. The result for the Google search is much more informative. It may turn out that the web page is shown as a supplemental result. So the web page is less valuable in Google’s view. Either other similar web pages are viewed as more valuable, or it is very rare that a search will be looking for this particular web page.
If the web page is a ‘goat’, improve the quality.
If the web page is rightly assessed as being rarely searched for, then there need be no concern if it turns up in the supplemental index. If the web page should be visible in keyword queries, then being in the supplemental results index is a clear Call to Action. Good advice on what to do is easily found. In short, look to improve the content and improve the inlinks (or in Google-ese the back links) to the web page.
Evaluating A Web Site
Checking a single web page helps to understand the principles involved, but what is a practical way to evaluate a whole website. This can be done by using the Google site: search. This section is written given the way such searches function at the date of writing this post. Changes are always possible in the way Google does such searches and in general there is no announcement on when this may happen.
If the search is done for site:www.mydomain.com, then this will show a listing of all the web pages indexed by Google. At the foot of the results, you will find the following:
In order to show you the most relevant results, we have omitted some entries very similar to the N already displayed.
If you like, you can repeat the search with the omitted results included.
Clicking on the link to include the omitted results shows all results. This listing shows all the web pages in the main index first followed by the results in the supplemental results index. These will appear on the later search result pages.
For example, the SMM Strategic Marketing Montreal website by this analysis has currently 345 web pages in the main index and 8 web pages in the supplemental results index. There is no duplicate copy effect here, it is just that presumably the final 8 have less PageRank.
As another example, one of our clients with an e-store as part of the website shows the following statistics. There are 397 web pages in the main index and 235 web pages in the supplemental index. It is likely that this occurs because there is a great deal of similarity (duplication) between web pages in the supplemental index and those in the main index.
Such situations are not a cause for alarm. If there are a satisfactory number of web pages in the main index, and if these are suitably visible in keyword queries, then Google will deliver a good volume of traffic to the website.
A More Precise Listing Of Supplemental Index Web Pages
Google searches giving numbers such as the site: search will usually give slightly varying numbers. The number of web pages in the Supplemental Results Index as calculated in the previous section will usually be an under-estimate. The following method will give a more precise estimate. Again be aware that this method may cease to work at some time but does work at the time of writing. The results are given by a search for the expression shown on the next line:
site:www.mydomain.com *** -dxdxdx(anynonsensetext)
For the SMM website, this search lists 22 web pages in the Supplemental Results Index (as compared with the 8 shown in the previous analysis). For the e-commerce website, this search lists 323 web pages in the Supplemental Results Index (as compared with the 235 shown in the previous analysis). If this count seems too high or if there are important web pages in the list then again this should be a Call to Action for corrective measures.
Web pages registered in the Supplemental Results Index have not been lost for ever in some Google Hell. Indeed they will appear in some keyword queries. However if they are important pages then this should serve as a Call to Action to improve their quality and their inlinks.
Related: Google Supplemental Results Index – A Word To The Wise
View All Your Google Supplemental Index Results