One of the hottest topics these days in SEO forums, message boards, and blogs is Google’s Supplemental Index. We’ve had some hearty discussions here at Intrapromote about GSI!
If you have a lot of web pages in Google’s Supplemental Index, then you may refer to it as “Google Hell”. A kinder, gentler reference I’ve seen is “The Google Poo”.
Hell or poo, both stink and no one wants to have anything to do with them.
Perhaps most interesting ….. I have a feeling that hundreds of thousands of web sites have stepped in the poo unknowingly.
Heaven or Hell
Google’s Supplemental Index (GSI), as defined by Google:
Supplemental sites are part of Googleâ€™s auxiliary index. Weâ€™re able to place fewer restraints on sites that we crawl for this supplemental index than we do on sites that are crawled for our main index. For example, the number of parameters in a URL might exclude a site from being crawled for inclusion in our main index; however, it could still be crawled and added to our supplemental index. The index in which a site is included is completely automated; thereâ€™s no way for you to select or change the index in which your site appears.
Did you get through that without falling asleep?
If you want to see if you have web pages in the GSI, simply replace “www.abc.com” below with your web address and search at Google for:
site:www.abc.com *** -view
Welcome To Hades. Have A Nice Day.
Here’s what you need to know if you find you have pages in the GSI. Searches conducted at Google, with rare exception, will return pages in Google’s Main Index. So, your pages in the GSI will have little opportunity to ever be found by potential site visitors. Pages in the GSI simply do not rank well.
However, when it comes to the GSI, you’ll be glad to know that the wages of sin are not always death.
Here’s what to do if you want to smell better to Google. Take a close look at your web pages in the GSI:
First, check the quality of your web page content. Especially look to see if your pages contain duplicate content from other web pages in Google’s Main or Supplemental index. The majority of the pages we’ve seen in the GSI have been banished there because they contain duplicate content. So, one way out of Google’s lake of fire is to make sure your web pages have unique content.
Second, check the quantity of your web page content. We’ve seen a lot of pages in the GSI that have little or no content. Again, make your web pages as unique as possible.
Third, check the backlinks to your pages. We’ve seen a lot of GSI pages that do not have any links to them. If your page(s) are linkless, securing quality internal and external links from pages not in the GSI may prove to be your online salvation.
Lastly, once you have addressed the items above, resubmit your HTML or XML sitemap to Google.
Suggested Best Practices:
A good first question to ask is â€œWho exactly are we targeting?ï¿½? If you are targeting a specific country, targeting a specific language-speaking audience, or your web site copy is specifically for a country or language-specific audience, use a ccTLD (country code top level domain) that relates to your target country rather than a general .com domain. For example, a ccTLD would look like www.domain.fr, www.domain.ca, www.domain.jp, or www.domain.co.uk. Always use ccTLDs for each language of your site.
Avoid having multiple language sites on the same domain, e.g., www.domain.com for English language content and www.domain.com/fr/ for French language content.
Make sure that there is not any duplicate content on your .com and any other sites.
Make sure your pages identify what language they are in, e.g., meta http-equiv=”Content-Language” content=”jp”
If you cannot use a ccTLD, use a subdomain, e.g., fr.domain.com. Google views a subdomain as a separate site.
Benefits Of Best Practices:
A ccTLD communicates to search engines the focus of your site.
A ccTLD is the quickest and most accurate way to communicate regionality to the search engines.
A ccTLD assigns more weight for local search. It allows your site to be more easily included in Google Canada, Google Mexico, etc.
Search engines tend to have higher confidence and often give a ranking boost to a ccTLD site for local searches. For example, Google France may give a more favorable ranking to a France-specific (.fr) site.
Q: What about using subdirectories such as www.domain.com/fr/? Can we do a 301 redirect from a subdirectory to a ccTLD, e.g., from www.domain.com/fr/ to www.domain.fr?
A: From a search engine perspective, it is always best to use a ccTLD. If a ccTLD is not possible, then consider using a subdomain. We do not recommend using subdirectories for international sites or language-specific sites.
Q: What does Google say about the use of TLDs, ccTLDs, subdomains, etc.?
A: â€œUse TLDs. To help us serve the most appropriate version of a document, use top level domains whenever possible to handle country-specific content. We’re more likely to know that www.domain.de indicates Germany-focused content, for instance, than www.domain.com/de/.” (Source: Google Blog)
If you’ve had a backstage pass to the SEO Speedwagon for the last few months, you should know by now that we are official groupies for the Sitemaps Protocol. In fact, if we ever take SEO Speedwagon on tour, Sitemaps Protocol would open up for us.
There actually was some interesting news this week re: the Sitemap Protocol.
1. Ask.com has drank the kool-aid so you can now share your sitemap with them.
2. Although MSN isn’t “…ready to consume sitemaps just yet“, all three major engines announced the sitemap protocol will now include Autodiscovery.
Autodiscovery allows site owners to add a link to their sitemap within their robots.txt file. Here is what it should look like:
- All Categories
- Thanks for Joining Intrapromote at #AkronB2B
- The Intrapromote Blogging Team
- About AJ Allen
- About Angela Moore
- About Annalise Kaylor
- About Aric Allen
- About Bil Gaines
- About Bobby Pham
- About Brent Sharp
- About Caleb Dann
- About Crystal Schauf
- About Doug Ausbury
- About Dylan Price
- About Erik Dafforn
- About Erik Dafforn, President of IP
- About James Gunn
- About John Castronova
- About John Karp
- About John Lustina
- About Katie Hehn
- About Kyle Misencik
- About Linda O’Neill
- About Lisa Santora
- About Mary Sutter
- About Matt Ross
- About Sean Bolton
- The IP Newsletter
- 3 Pieces of Facebook Data That Are Often Ignored
- A Quick Guide to Understanding EdgeRank
- Big Changes at Google Mean Business As Usual at Intrapromote
- Community Management and Value Based Listening
- Facebook Timeline Changes – What You Need to Know
- Google Places Becomes Google+ Local
- Google+ and Local: Places, Reviews and Engagement
- Organic Analysis in a (Not Provided) World
- Pinterest for SEO
- Pinterest Separates Personal and Business Pages
- Search: Red Carpet | White Glove
- The New Frontier Of Link Development
We highly recommend that you add this line to your robots.txt, especially since you will not have to resubmit your sitemap file when it is updated (which should be often if your site content is dynamic).
If you are a fan of the statistics, etc. provided by Google Webmaster Tools, then also be sure to submit your sitemap there. Along with statistics, you will also be able to see if there are any errors in your sitemap which can be very important, especially for large web sites (trust me…..been there).
An SEO guy walks into a bar and asks the bartender, “Can you submit a sitemap to MSN?” The bartender looks at him, scratches his head, and asks, “Why?”
One of my favorite classes in college was Consumer Behavior so I guess that explains my fascination with search behavior. I thought Iâ€™d comment on a recent search behavior study conducted by De Vos & Jansen Market Research.
De Vos & Jansen compared the search behavior of two groups of people: Buyers and Information Seekers. From the two groups of 25, their study concludes that the viewing habits of buyers and info seekers are different.
No shock there.
However, one interesting thing about the study is their conclusion that those searchers with the intent to buy viewed more search results and focused more on familiar brand names. More interesting is that while 98% of searchers reviewed natural search results, only 31% of those in the study viewed the sponsored (paid search) listings.
That 31% is probably a generous number, but with a sample size of just 50, I wasnâ€™t surprised that much by it. Other search behavior studies have put this number in the 10-20% range.
The search behavior takeaway here is not rocket science.
If all your Search Engine Marketing eggs (dollars) are in the Paid Search basket (Adwords/YSM), you are missing out on a large percentage of your overall target audience.
Can your business afford to do that?
Recently heard near the Intrapromote water cooler:
â€œItâ€™s not all about search engine placements. Itâ€™s also about attracting the right visitors, site visibility, traffic from search engines, and increased sales/conversions.â€?
Hereâ€™s a recent example from a campaign that I direct:
â€¦ Site Visibility
In the process of working on site visibility, we created and submitted a sitemap to Yahoo through Yahooâ€™s Site Explorer. The number of pages in Yahooâ€™s index prior to this submission was 2.9 million. Three days later, the number of pages in Yahooâ€™s index was 3.9 million. Yahoo inlinks also increased by 1 million.
â€¦ Traffic From Search Engines
Is better site visibility a means to an end for increased traffic from search engines?
Curious to see how this might affect the amount of organic traffic from Yahoo, we were able to pull this data from their analytics program. Before site map submission, the clientâ€™s site was receiving 950 visitors a day from Yahoo. The site is now receiving over 2,000 visitors a day from Yahoo.
â€¦ Increase Sales / Conversions
Do more visitors from search engines mean more sales?
If they are searching for your product or service, these are exactly the visitors you want and should be the bullseye of your SEO campaign. For our client, sales from Yahoo search traffic increased 122%.
Now that Yahoo has agreed to accept the Sitemap protocol , I’ve been going through the process of getting client sites authenticated via Yahoo’s Site Explorer so we can submit sitemap files.
To be knighted as an authenticated site by Yahoo, you have to create an authentication key file then upload it to the root of your site. Once uploaded, you can request authentication.
We’re finding it takes about 24 hours to get the thumbs up or down from Yahoo.
Just a tip for any of you doing the same…..
Once you upload the authentication key file and your site is authenticated, don’t remove the authentication key. Yahoo will check periodically for the presence of the key and if it’s removed, your site will be unauthenticated.