SerendeputyBot

Welcome, and thanks for stopping by.

If you're here, you probably found SerendeputyBot in your server logs and are wondering what it is.

Serendeputy is a personal newsfeed engine. My mission is to build an index of the open web that people can use to build a newsfeed engine they can control, without getting trapped in the walled gardens.

I've been working on the open web since 1997, and I built this application to live up to those ideals. Every link goes directly to the source, without cloaking, framing or any of the other dirty tricks people sometimes play.

The crawler follows the robots.txt protocol. If you think it's misinterpreting your robots.txt file, please let me know so that I can fix the bug.

The feed crawler does conditional gets (using etags) for the feeds. This should work, but if something is being weird, please let me know so I can fix it.

Cheers,

Jason Butler
Founder,
Serendeputy
Concord, Massachusetts, USA
jason@serendeputy.com