I have been into this discussion of Serendipity and Homophily for a while. I consider this an extremely interesting topic that arises with all the discussion about how digital changes information usage and value. But also personal surroundings, user behaviour, group thinking, etc. Some time ago I found an interesting post: O’Reilly Radar „Homophily in Social Software“
In short, you hang out with people who are like you, a phenomenon known as homophily. This happens online, and indeed the Internet can lower the costs of finding people like you. But homophily raises the question for social software designers of how much they should encourage homophily and how much they want to mix it up.
So the internet is – according to this sofar – the main cause of homophily:
It’s often been asked whether this filtering just encourages people to see the news that supports their prejudices and never see news that counters them.
I don’t think so. There are tips of how you can avoid that and provide more serendipity:
Doing this creates serendipity: pleasantly surprising the user. For example, don’t show just the top 10 most similar items in your recommendations list, but show the eight most similar and two from the mid-range. Or call the „less relevant but also likely to be interesting“ results out like you’re advertising them: put a heading like „Take a walk on the wild side“ or „Break out“ on top and act like it’s a feature you’re offering, not a bug you’re fixing.
I think that most platforms will do that quite well. Purely, because people are too different to have too many alike recommendations. There will always be people who add new input to the recommendation system. And secondly, this variable increase, the more likes&dislikes from other parts of life are taken into consideration. If you shop at amazon for books, but the recommendation system takes your preferences for food into account when offering books, you get to see books from people who enjoy the same type of food and read books you might never have heard or thought of…
However, just to make this complete: TechDirt doesn’t believe in technical recommendation systems, though.
And read/write web has an interview with the chief architect StumbleUpon, one of the major „serendipity engines“, if you like.