tiinker blog

Digg recently announced changes to their algorithm for promoting stories to the front page, citing a need for diversity in content. I used to visit Digg frequently (back when someone else was paying my salary) but over time I found that the type of stories I’d find there narrowed to just a few popular topics, many of which didn’t interest me. Digg has built up an amazing community around their popular-election model of content and has truly changed the world of online news, but any socially-recommended method of content selection finds itself on the horns of a dilemma: you can improve the service to your core community or you can diversify and appeal to more people, but it’s very difficult to do both.

If you’re a regular Digger who’s interests closely match those of the core community members then Digg is just great; an endless stream of stories, images, opinion and occasionally even news. But if you don’t match that profile or have significantly different interests then Digg is far less relevant. Digg’s growth speaks of a very large and valuable community, but growth seems to have peaked around July last year. Perhaps they reached saturation.

They have a problem, though, as evidenced by the backlash of some core Diggers (discussion on VentureBeat and TechCrunch). The algorithmic changes reduce the influence of core contributors, who used to get a hidden bonus to their vote. The changes seem to move the story selection closer to a fair vote and the upset Diggers feel (rightly) that the collective control they had over the community is under threat. This illuminates the problem brilliantly: giving highly contributing members more influence serves that community well, taking advantage of good track records to bump good content to the front page sooner. But this continues to narrow the range content available, and hence broadness of appeal, as more people who like those stories are motivated to contribute and hence gain influence. The changes to diversify content are an attempt to appeal to a wider audience, but they do not serve this core group well. Diversification weakens the community that Digg has built up, but without it appeal is narrow.

There are alternative models of content selection of course, and tiinker captures one of them: treating each user as an individual and learning what they like. Our story selection algorithms pays little attention to how popular an article is but pay close attention to what it’s about and how you have rated previous similar stories. There are middle grounds too, such as that adopted by Reddit, which compares you to other Reddit users and chooses stories based on shared interest. But they too have some narrow popular topics, and the fact that a user’s votes affect content delivered to others opens up the system to gaming.

We think personalisation will be the next stage in our evolving online media, and it seems plenty of big Internet properties think similarly. It’s going to be a lot of fun finding out.