The People's Search Service

Jason Fried asks “Would you pay $5/month for Google if it wasn’t free”?

Dave Winer says “Clone the Google API”.

I’d like to join those together and ask “Would you pay $80/month to own a part of the infrastructure that powered a free implementation of the Google API”? In other words, rent a dedicated (low-end) server from the likes of Layered Technologies, or EV1Servers, and use it to run a distributed part of the crawling, indexing, or querying part of a full-on search engine.

Why would this be a good thing?

• It seems like the “right” thing to do – sort of like joining the EFF.

• No-one really owns the Internet. We all (mostly) pay for little parts of the internet infrastructure, but what we’re really paying for is simply end-to-end transport of IP packets. I’d like search infrastructure to become like that – something (those who need it) pay their bit for, and that we can all use as a basic commodity.

• The next generation of tools to help us do our work (and play); to help us manage information overload; are going to need to be heavily search based (in an automated, behind-the-scenes, kind of way). That infrastructure needs to be open and free to use in the same way that IP transport is now.

What would this take?

• An open specification for how individual nodes in the system talk. Individual nodes need to come and go as they please – if I suddenly can’t afford to use so many of my resources on search, then the redundant index fragment that my machine(s) build need to slip away quietly, without disrupting the behaviour of the the whole system.

• Open-source crawling, indexing, and querying code that can work within the above constraints.

• Focus on providing an API, from the beginning.

• The collective will (and time) to make this work (ouch!).

What could go wrong?

• Getting organized. Starting small and being open to change seems to be the only way to get anything off the ground.

• Freeloaders. I don’t think this would be such a problem – anyone with a massive query volume would probably also feel their own pressure to contribute proportionally.

• Spamming.

• I know things like Grub have tried something like this in the past, but I think the need is here more than ever, now.

So would I pay?

• If it’s $5 for the web-UI for Google, then while others are free, maybe not.

• If it’s for an (almost) unfettered Google API (from Google), then probably.

• If it’s to be part of a universal search infrastructure, then of course :-)

So would you pay, or contribute?