O’Reilly’s Radar has just published a really fascinating interview with Marie Wallace, a social analytics strategist from IBM. She believes that social networks in search of a monetization strategy should turn themselves into data analytics businesses rather than concentrate on serving advertising to their users. It’s certainly an interesting thesis, but it also made me realise that the biggest limiting factor on Big Data achieving its vast potential is a lack of imaginative applications for the technology.

Look through any number of articles, presentations and pitches on big data technologies, and you’ll be hard pushed to find many that aren’t led by marketing or advertising. It is a logical connection to make, especially with businesses such as social networks, whose only tangible assets are data stores full of user information. But I’m convinced that there have to be more imaginative applications for technology capable of crunching billions of records than identifying how and where they should be served adverts for dog food and new brands of breakfast cereal. Indeed as Wallace herself acknowledges in her interview with O’Reilly, the business of placing adverts on social networks is fraught with difficulties that even better analysis of user data might not solve.

“The key is that in most cases ads only work in a search-like context, however with most social media sites people are not going there to search. They are going to converse with friends and family, which makes ads interruptive and frequently invasive. This is further exacerbated by mobile, where limited real estate makes ads even more offensive as they are distracting and clutter the screen.”

The more closely Big Data is associated with marketing (put “’big data’ marketing into Google and it returns over 9 million results”) the greater the risk that a technology which has the power to pervade the whole enterprise will find itself limited to a single business function. And nowhere will this be more limiting than on the Big Data industry itself. What will vendors do, for example, when they realise that their customer insight engines, dashboard and intelligence-gathering algorithms, are all in direct competition over marketing budgets that represent a tiny slice of overall corporate spending power? I would argue that many Big Data businesses’ ability to survive the first phase of industry hype will lie in their success in innovating their way out of the marketing silo.

Instead of pushing ‘me-too’ analytics products that are currently too complex and expensive for all but the largest and most-technology savvy businesses, Big Data vendors should be creating software that enables other business functions to leverage their data stores. And instead of promising the ‘jam tomorrow’ of improved sales from better customer insight, smart developers will instead concentrate on getting the technology to deliver value in areas where Return on Investment case is easier to demonstrate. We can therefore expect to see Big Data analytics products that enable banks or insurers to stay compliant with new legislation requiring better disclosure of customer information (see our blog on Midata), or help finance departments track the profitability of projects by comparing fee income to workers’ timesheets. They waste billions each year on bodged manual processes when improved analytics could easily automate the whole process, thus allowing Big Data products to colonise business functions as diverse as Legal, Finance and HR and giving vendors exposure to a bigger spread of corporate budgets.

As we explored in The Big Data Hype Cycle, before it can make the leap to prime-time, Big Data needs to become boring. Ironically, its success in becoming boring will rely on Big Data vendors having the imagination to apply the technology as a solution to a greater range of business problems. This will mean less reliance on wishy-washy statements about customer insight, and more thinking about how companies can use the data already in their possession to solve the problems that drive them, their employees and their customers nuts.