Most Combative Search Engine
Posted by Phil Aaronson at 8:54 AM
Just for fun I've been running an informal "Most Aggressive Search Engine" competition. The Tour has a most aggressive (most combative) competition where the leader gets to wear a red version of their number on the road ... I'll have to create a red badge for the winner, or maybe they'll get their picture taken with the red podium babes. We'll think of something.
For a long time I had set this site up up so that search engines would not to index this site. It was more family blog than anything else. In June I opened up part of the site to the search engines (basically everything under /blogger/), the first part then, who would be first to start indexing under that part of the site?
The results of stage one:
So yesterday I opened up everything under /weblog/ to the search engine indexers. That content is linked off of the main page of this blog, its any content from 2004. The second part of the most combative competition: I wonder which search engine indexes that content first?
[Update 8/12/2005] Here's another interesting little combative factoid. I went through my logs and looked at how often each search engine re-checked my robots.txt and index.html file back when I disallowed everything. This was from March 8, 2004 to June 1, 2005. On average the big three re-checked every:
[Standard disclaimer: I work for Yahoo!, but not in Search]
For a long time I had set this site up up so that search engines would not to index this site. It was more family blog than anything else. In June I opened up part of the site to the search engines (basically everything under /blogger/), the first part then, who would be first to start indexing under that part of the site?
The results of stage one:
msnbot: 26/Jun/2005:02:39:26 -0400MSN is off to a strong three day lead. Interestingly you can find hinkty results at Yahoo! and MSN, but they have not appeared in Google yet (try typing "site:hinkty.com ant crash course" for example into the big three).
Yahoo! Slurp: 29/Jun/2005:18:16:15 -0400
Googlebot: 28/Jul/2005:14:11:16 -0400
So yesterday I opened up everything under /weblog/ to the search engine indexers. That content is linked off of the main page of this blog, its any content from 2004. The second part of the most combative competition: I wonder which search engine indexes that content first?
[Update 8/12/2005] Here's another interesting little combative factoid. I went through my logs and looked at how often each search engine re-checked my robots.txt and index.html file back when I disallowed everything. This was from March 8, 2004 to June 1, 2005. On average the big three re-checked every:
msnbot: 10.5 days
Yahoo Slurp: 13.3 days
Googlebot: 15.1 days
[Standard disclaimer: I work for Yahoo!, but not in Search]
0 Comments:
Post a Comment
<< Home