non-obvious, surprise, reality

From March 6th

There's a guy in Las Vegas who builds computer systems and trains them in Non-Obvious Relationship Awareness. The casinos there are regular targets for fraudsters and tricksters, often colluding with employees, hoping to sneak themselves some undetected winnings under cover of the mass of gaming transactions that happen every minute. So these systems have to act incredibly quickly; matching information from employee databases with known lists of bad guys, arrest records, customer information, credit reports and all sorts of publicly available data. They're looking for non-obvious relationships - like the fact that the guy who's winning lots of cash on table fourteen shares a cellphone number with the dealer on the adjacent table. The kind of thing that should ring an alarm-bell somewhere and might save you thousands of dollars.

And there's a group at Microsoft working on what they call 'Surprise Modeling'. They've devoured huge amounts of data about Seattle's notoriously horrible traffic and about everything that might cause delays; the weather, sports fixtures, public holidays, and they set out to warn drivers about bottlenecks and jams. Which isn't really that exciting, because any experienced driver in the area already knows where the bottlenecks and jams will be, and no-one likes software that tells you what you already know. So this system goes a little further and works out which of the jams it's predicting are likely to be surprising to drivers; and only lets them know about those surprises.

And there's a team at MIT working on the 'reality mining' of personal devices like phones and iPods. They're trying to determine whether the things our phones know about us - our movements, our conversational style, our location, our music preferences - can be used to create aggregate pictures of the society we belong to; it's health, it's relationships, it's behaviour.

Why am I bothering you with this silly new jargon; 'Non-Obvious Relationship Awareness', 'Surprise Modeling', 'Reality Mining'? Because the future of marketing is in data, real data. Not the spurious marketing constructs we're used to - wooly things like spontaneous awareness and day-after-recall - but the live tracking of real world behaviour. The planet is full of objects beaming information about their status to anyone authorised to listen, from Nike trainers to Rolls-Royce aero engines. The objects we use are starting to tell stories about our behaviour. And the people who can find the patterns in that informational fog (who can mine the reality and find the non-obvious relationships) and those who can sort through it and find the stuff we should be worrying about (modeling the surprises) will be the people we'll be desperate to employ. Them and the folk who can connect all those databases together, without losing CDs in the post.

Comments

[Charles Frith]

Yep. There's no point asking people what they think because they'll tell us any old rubbish. But find out non obvious relationships and it seems they are exist.

I've often wondered if there was anyone in the city who could data mine all the fluctuations in the markets and link it with something that nobody had thought of before.

[Chris Solomi]

With any sufficiently large pool of data you will always "see" patterns that that seem to correlate with aparently unrelated phenomena.
As an example,if you charted the position Barnet FC has finished at over the last 20 years you could probably find a stock somewhere in the world who's price exactly correlates with their league postion. (not including stock in Barnet FC!).A fool be he who then buys or sells this stock based on how well Barnet FC's striker is performing!
The temptation is to think this correlation is meaningful in any real way when it's probably not.
The main problem with data minning is the creation of these false correlations. We think we've discovered meaningful insight when in fac tall we are seeing is 2 random patterns that seem to be linked.
Nicholas Taleb makes this point much more elloquently that I ever will be in his excellent book "Fooled by Randomness".