Removing Supplementals in Wordpress or Other Sites
09. Feb 2007 | 21 Comments
I’ve had a few blogs hit supplementals lately even though they’ve got a pretty decent Inbound Link Authority. Generally whenever this happens I’ll query the site in question with Google’s site: operator to see if I can see anything out of the ordinary.
Well chances are that if you’re using wordpress & you’ve not set up your robots.txt correctly then you’ll be getting heaps of pages indexed that you don’t want.
What don’t you want indexed:
- Your Plugins & Content Folder
- Categories
- Galleries & Images
- Feeds & Trackbacks
What you DO want indexed:
- Posts
- Pages
- Sitemap
Shoemoney had a similar problem a few weeks ago (And most people think he’s an SEO?). What happens in that in order to rank a page in the main index Google must attribute a minimum pagerank value to it. If it falls below the threshold it’ll hit supplemental. So you need to make sure that your site has a good distrubution of pagerank or authority. So if most of your links are hitting your homepage & you’re second level of navigation are your categories then you’re losing heaps of link popularity that should be going to your posts instead.
Ony my blogs I’ve implemented the following robots.txt with reasonable success (50% more pages indexed):
User-Agent: Googlebot
Disallow: /index.php
Disallow: /category/
Disallow: /feed/
Disallow: /feedUser-agent: *
Disallow: /wp-
Disallow: /trackback/
There is a good set of reasoning behind doing this on all of your sites. You should almost be telling the engines which pages you want to rank & which pages you don’t. Removing the pages that are worthless to the engines & only serve as a navigation between different parts of the site increase the relevance & internal linking power of other pages.
I have a Joomla content site that has more category pages than static pages. So I exlcluded all the category pages & my traffic has jumped from around 500 uniques/day to 1500 in about 8 days. Gotta love Everflux
Remember if you have unique content on your blog & you’re still suffering from Supplementals you need to get some links to your main page & internals, never stop this cycle of linkbuilding or linkbaiting.
I’ve received about 4 or 5 reviews about Earners Blog that I asked for last week, I’m still looking for a lot more. So keep em coming. There’s $25 & an SEO Audit in it for you.
I also think that the DNS is still settling after the switch to the new hosting, so some of you may still be seeing & commenting on the old version of the site. I’ll try to get the databases synced so we don’t lose some of the comments
In other news check out Eli’s new post about Power Indexing sites with a large number of pages.
Enjoy this post? Get the RSS FeedTags: Search Engine Optimisation | Wordpress or Share This
|
| ![]() |
21 Comments on "Removing Supplementals in Wordpress or Other Sites"
Cool, thanks for the tip. I had noticed recently that I’ve been getting supplemental results so this was great timing. I have implemented this robots.txt with a couple tweaks, so we’ll see how quickly I get out of supplemental.
Ben
http//idleprofit.com
I think I’ve got a fairly similar setup to the one you described (several category pages), but I’m a little confused. Here’s what I got:
Homepage (with links to category pages) > Category pages (with links to individual product pages)
If I disallow my category pages won’t that stop the bot from finding my item pages (since item pages are only linked to from the category pages)? I’ve got too many items to link them from the homepage (thus the need for category pages in the middle).
Thanks.
Thanks for the great tip. I am going to make sure to put it in place on my blog.
My site isn’t in Wordpress, but I think I can still incorporate/test a lot of the suggestions. Thanks.
Just a quick question from a not-so-technically-savy guy…..
So I copy the text given above, save it as robots.txt, then which folder do I upload it to?
Thanks in advance ![]()
Thanks!
If my blog is domain.com/blog, do I upload the robots.txt on domain.com/blog or domain.com?
Thanks again!
I use feedburner, should I restrict feedburner links from begin indexed?
With regards to:
http://www.earnersblog.com/methods/rss-feeds-for-backlinks
Since you say that, so can I say that the robots.txt that you gave us will still allow our feedburner links to be indexed? ![]()
” User-Agent: Googlebot
Disallow: /index.php
Disallow: /category/
Disallow: /feed/
Disallow: /feed
User-agent: *
Disallow: /wp-
Disallow: /trackback/
”
I cannot understand why you disallowed the index.php?
Can you please explain why?
TIA
I made the changes to my robots.txt file, but for some reason my file disappears from my server. I don’t know what’s going on.
How long does it take for Google to read your robots.txt again?
on my website, I have about 19,300 pages index and most of it is from the galleries which have no real content only images.
ps: my galleries are domain.com/gallery/subgallery/picname.html
should I exclude them from the index? Will this result in higher google traffic? what about google image traffic?
@Ant Eksiler
@Stuart
If on your’s 19300 pages you put some simple content,some comments and the image page has got his own title and links point to some pages from your site/blog.. your traffic will grow a lot
Add a Comment
Categories
- Affiliate Marketing
- Affiliate Marketing News
- Affiliate Networks
- Copywriting
- Earners Blog News
- Earnosphere
- Email Marketing
- General Musings
- Guides
- Keyword Research
- Link Building
- Marketing Tools
- Monetization
- Outside the Box
- PPC
- Productivity
- Reviews
- Search Engine Optimisation
- Social Media Optimisation
- Tips & Tricks
- Top Posts
- Traffic Generation
- Wealth Creation
- Web Development
- Wordpress





Hi Garrett, there’s loads of other ways to help the spiders find your other pages without Category Pages.
Check out this post on Internal Linking Structures for Wordpress.