Beta tester update, 11/2/18 edition

Posted: 1 year ago

Hello again from Concord, Massachusetts, USA.

I’ll send out these updates periodically when I have something useful to say. (Hopefully every week or two.)

What have I been doing

So, what’s been happening here in the last couple of weeks:

  1. Stability and scaling. Different things break at different scales, and I’ve spent the last couple of weeks solidifying the infrastructure and the core parts of the application. This is (ideally) transparent for you.

  2. Performance Tuning. I don’t want to optimize prematurely, but I’m a fan of start fast and stay fast. You should be seeing <250ms load times throughout the site, and I’m trying to make sure that doesn’t slip as I’m adding new features.

My other key performance metric is “transit time.” This is how long it takes from the time the system sees a new document or new tweet to the time it’s live and fully indexed on the site. My goal is have this be consistently less than 60 seconds, so that you have a realtime-ish index to use. Right now, it’s hovering around 135 seconds. I hope to keep improving this metric.

What am I doing next?

  1. Onboarding, help text and FAQs. Right now, it’s not entirely clear what Serendeputy does or how to use it well. I’m going to take a day and do a pass at this next week.

  2. Search Suggest. Life will be a lot better once you don’t have to type the entire name of the tag into the search box to find it. This should be straightforward, but I haven’t had to integrate any real front-end code yet, so random things may come up.

  3. Noise Cancelling. This is the mathiest part. The profiles learn from the data, which is swamped (at least right now) by US politics. I’ve sketched out a noise-cancelling framework which should help more-tightly isolate what’s unique about a given tag and make it stand out from the background noise. I’ll likely do a quick and dirty implementation next week to see how it works in the wild. In theory, the math says it should work; we’ll see how it does in production.

Once I’m happy with Beta One, I’ll move onto Beta Two, which is building out your personal profiles and providing the tools to manage and tune them. I hope to have something around this by Thanksgiving, but that’s a toss-up right now.


I set up a subreddit for you to talk about what you’re looking for in a newsfeed engine — I hope to see you and hear what you’re thinking.

RSS feeds

A couple of people have mentioned that this performs some of the functions of an RSS Reader. If enough people are interested in that, I can write some tools to make that use case a little easier.

In the meantime, if you have any RSS feeds or OPML files you want me to make sure I index, please send them along and I can load them for you.

Ok, that’s all for now. Lots to do.