Musings on the wisdom of crowds and machine intelligence

This is a very provisional post.  It has been swirling around my head since I was talking to Saul Klein before Xmas.  These thoughts are interesting to me as I think more about what it means to build social networks that do stuff, but they are new thoughts, and hence even more than normal I’d welcome your comments. 

It seems to me that you can build an online web service to derive insights from the wisdom of crowds and/or adopt a more mathematical approach.  Most social networks are about the wisdom of crowds – i.e. socially derived insights.  LastFM’s music recommendation service is a great example – finding music you might like by looking at what other people like.  Similarly Crowdstorm tells you what products are hot at the moment by figuring out what people are talking about most.

The other, more unusual, approach is to use machine based intelligence.  Pandora’s Music Genome Project is a great example of this in practice.  They have mapped out the different elements of music, much as the human genome maps out the different elements of our DNA, and are using that to make music recommendations.  Leiki also take this kind of approach – using taxonomies and categorisation at the heart of their recommendation engine.

People feel good about recommendations that come from people based (i.e. socially intelligent) systems, particularly if they come from friends.  The problems come with systems that are below critical mass where you can either get no meaningful intelligence, or sometimes unhelpful echo effects.  These systems are also typically less responsive to new categories/ideas/things than machine systems.  Competitive advantage is likely to come from scale and being the first in a space to hit scale.

Maths based systems are quick to add value once live, they can provide value for the first user of a system and they can be adapted quickly to changing circumstances, but they are more complicated to build in the first place, and I guess easier to get completely wrong.  You have to code the intelligence at the start and there will be no coming back from a big mistake.   Competitive advantage will come from scale AND the depth/uniqueness of your algorithm.

The other obvious thing is that the efficacy of each approach will vary with the problem you are trying to solve.  Some things lend themselves to mathematical approaches – e.g. the flight price forecasting on Farecast.  That said if Pandora can bring maths to music then I guess the same can be true of pretty much anything else.

I guess the so-what of this is that the more value that is crammed into a service the stronger it will be, so a start-up which adds a machine based component to its wisdom of crowds story might well be a more fundable proposition.  Assuming it works.