Boycotting Microsoft's Search Engine

The calls to use technical means to deny content to Microsoft’s new search service will fail on both technical and political grounds.

Chad Fowler recently blogged about a growing movement to boycott Microsoft’s attempts to build a search engine database capable of competing with Google.

I like Chad a lot, and respect what he says, but I feel that perhaps this is movement is missing a couple of important points.

First, there’s the technical side. The suggested approach (for example here) is to add

  User-agent: MSNBOT
  Disallow: /

to your robots.txt file. Crawlers are supposed to take robots.txt as a suggestion of what to include in their trawl, and the given lines ask any crawler called ‘MSNBOT’ to ignore the entire site.

So let’s pretend for a minute that this movement takes off, rather than simply excluding a couple of thousand Geek sites from MSN (a situation that the rest of the world would probably applaud). Let’s instead imagine that millions of sites join in, and that MSN starts to hurt from lack of content. What happens? Perhaps MSN simply ignores robots.txt, or changes their agent name (to googlebot, perhaps). Technically, this approach ignores the reality of the situation. If Microsoft want to index the content, using robots.txt isn’t going to stop them. Denying IP addresses might, but even this could prove troublesome given the worldwide reach of Redmond.

But the more troubling aspect of this for me is the message that such a move sends to the world. The protesters have been complaining about Microsoft tactics for many years (often with justification). "Microsoft doesn’t play fair," they complain. "The playing field isn’t level." And now they set out to deny Microsoft the opportunity to compete with Google.

If as a community we want to foster certain values, then we have to be prepared to live by those values. If we don’t, how can we expect to be taken seriously?