Digg describes itself as a “user driven social content website”. What that means on Digg is that users submit urls to news, blogs, videos, etc with a short description and other users can “digg” them and comment on them. With enough diggs a submission can make it onto the front page of Digg (which increases it’s exposure and therefore it’s diggs).

The problem

A old problem has appeared on Digg. It’s a problem familiar to Slashdot users, a website with a similar premise (user url+story submissions) but tighter editorial control. It’s the problem of duplicates submissions. A user may submit a url, which has already been heavily digged, along with his own description.
The problem is not the duplicate itself but rather that there is a market for the duplicate. To put it a bit clearer, the real problem is that I as a Digg user am not seeing the original interesting submissions and therefore these duplicate entries are useful to me. When I and other digg them they get promoted and get in the way of users who already have seen them => noise up the system.

The solution?

There may not be a simple elegant solution, but I think this one would help significantly.

Since Digg knows what stories I digg and who else diggs those stories, then Digg should be able to recommend to me stories I have not reviewed, but might like. “Stories you may digg”. That should at least cut down on the duplicates since the system would actively push stories I may digg to me for review.*

A more immediate solution is to allow each user to set a preference to allow him to filter out stories containing urls duplicate to stories he has seen (although that may require an impractical amount of computing power).

*excluding Amazon’s recommendation engine, a good example of this is on Standpoint where you get a “Things you may believe” recommendation after feeding the system with data for a while.