Happy first day of fall, everyone.

It’s been a busy summer, but now the children are at school – taking the school buses, even – and I have the house to myself for most of the day. It’s a miracle of productivity!

Here’s what’s new: I rewrote the logic of the homepage for you, and it’s live now.

People use (and maybe even love, a little) Serendeputy for many reasons, but the primary one is something along the lines of “Find me something interesting to read right now.” Some people do more – watch for my PRO release, coming later this fall – but most people are looking for something to read.

So, what does that mean?

I’ve talked to a lot of customers over the past few months, and it broke down into three main things:

• Show me what I expect to see. Your deputy is working off the people you follow on Twitter, and things you’ve explicitly told it. So, if everyone you follow on Twitter is tweeting a specific link out right now, you’d expect to see it.

• Show me what’s popular in general. This is the “trending” piece. Even if my world isn’t on fire for a particular link, it’s good to see what everyone else is linking to.

• Delight me. This is the most esoteric need, but one that pays off the most. People want to see things that are just a little off their radar. Not everything is going to hit, but some will – and those hits can be delightful.

So, here’s what I did: I moved the model of your deputy homepage from “one list” to “a series of clusters.” This is an oversimplification, but it’s essentally accurate: in the old system, your deputy would score everything for you and return the top 40 or so items; in the new system, your deputy will still give you the top 10 or so items, but will use different heuristics for the rest of the page.

Sidebar: the advantage of building Serendeputy as “Explainable AI” is that I can, um, kinda explain it.

You should see clusters that are:

• Top for me. This is similar to what you saw before. Your deputy will score all the documents in the system on your behalf, take out all the ones you’ve already clicked on or skipped, and give you the top 3.

• Newly Tweeted for me. This is a fun bit of randomness. Your second cluster will be the three newest documents coming through your Twitter feed.

• Sample from Tags I follow. Your deputy will grab a random selection of tags you follow and pull the top three documents. This is useful because you may be very interested in a particular site or topic, but it’s not always breaking into your most-highly-scored docs list. This is way to break through that logjam.

• Sample from tags I’ve tuned highly. Your deputy will pull a weighted sample of tags you’ve given a high score. This way, it can surface tags you care a lot about but which don’t have a lot of general pull.

• Sample from deputies I follow. You automatically follow other Serendeputy customers who you also follow on Twitter. This is a way to surface what the world looks like a little through their eyes. This is often one of my most-delightful clusters – especially with some of the people who have vastly different interests.

• Generally Popular. Your deputy will pull in the top ranking documents from around Serendeputy in general, so that you can see what else is trending at the moment.

I’m currently playing with the algorithms and weightings, so please let me know if you have any thoughts or suggestions.

Ok, so what’s next?

Fun with optimization! The old homepage was rendering in 225ms or so. The new one is averaging between 750-1200ms. It makes sense, since I’m essentially running 25 complex queries per page instead of one, but it’s a lot longer than I’d like.

The good thing is that Elixir and Erlang are quite good at concurrency. It’s not magic – I still have just the one database – but I think I can chop about 350ms off by running some of the build in parallel.

I’m also going to finish porting the rest of the application (including the homepage and tag pages) to LiveView. This should help save a lot of round-trips and make the perceived experience a little snappier. Again, no magic, but it should help.

I’d love to hear what you think of the new interactive design. Please drop me a line: jason@.

Cheers,

Jason