Technology evolves and for those of us who spend their lives adapting and endeavoring to keep up with the advancements it’s hard not to notice a curious underlying dynamic. Data and our ability to calculate or manipulate it for greater meaning is a little like resolving the chicken and egg paradox. More of one begs more of the other, and yet we continue to ask which came first as if that question were important. For many of us, our interest in closing the uncertainty gap wishes for more data. We expect it will help minimize the error or noise because the present picture of relationships remains a little too ambiguous. The constraint in this case is often our own experience and knowledge.
Professionally, my own work warns against this unconscious bias. I simply ask people to imagine three dots and ask that they line them up. I then remind them that all three dots are coincident data points in time, and ask whether this new piece of information has changed their vision of the dots? I then ask them to place the dots on an axis of time, and tell them that the dots now represent demand, growth or performance like ROI. Does the way you’ve visualized the dots changed again? I explain that the context I’ve added snapped into their own experience to create an image that creates a new puzzle as what they see fights with their expectation and they need more data to explain it.
The Economist in revisiting the Growth Matrix in 2009, put it another way. Bruce Henderson, credited with originating this framework reportedly believed “while most people understand first-order effects, few deal well with second-and third-order effects. Unfortunately, virtually everything interesting in business lies in fourth-order effects and beyond”.
Big Data and the volume variety and velocity of its availability now has several partners, real time processing power and plummeting data storage costs and lest we forget, simple access and manipulation tools placing the data in an ever increasing number of users’ hands. It is the number of people who now want to use the power of analytics that lends Big Data its influence, or at least that’s what several Chicago Booth alums who shared their thoughts last week recognized.
On May 18, 2012 Chicago was busy preparing for the arrival of NATO delegates and support. The result was many businesses strongly encouraged their employees to work from home, leaving the monthly strategy discussion homeless. We took advantage of the opportunity to launch our first virtual discussion combining a real-time interaction platform (Group Systems Thinktank) and conference call (freeconferencecall.com). Interest in the topic proved overwhelming prompting us to open up a second lunch interaction following our usual early morning time. The comments that follow represent a condensed version of the conversation. Note, links to the discussion prep video and articles we encouraged participants to review in advance can be found at the bottom of this post. Also, a full transcripts are available to those who interested in seeing the automated output from ThinkTank, just drop me a note.
What’s the deal
Its a toss up whether mobility or big data has captured the imagination of business media more. The duel isn’t the point. Other driving forces and a growing need for critical thinking skills that were already in short supply. Data reduction may be an emerging competency. As the earlier references to Henderson point out, the questions you are trying to answer don’t get any easier just because you suddenly have access to more data. What to do with this new wealth of rich information are the bigger questions and challenges not merely for business but for consumers as well.
In the process of generating the following list of examples, the interaction on Thinktank let participants also provide some links, raise new questions and add additional comments.
Twitter ,Telephone call records, Smart Grid, Real time Electricity meter data , Nike + ,Scanner data, Comments from Call Centers, Providers’ case and disease management notes, EMR records, Geospatial (GoogleEarth, Navistar, etc), Mobile and GPS, Gov’t DBs (big-data in an unstructured/non-uniform sense) , The quantified self, Output or processed Data from SAS, Salesforce.com, other enterprise databases, Loyalty program, Amazon purchase history, Mint.com, Moneyball (Big data in baseball), the new NSA data warehouse in Utah, QR, The internet of things, Data.gov, The London Datastore, created by the Greater London Authority (GLA–Chicago has similar initiative) offers citizens open access , Netflix movie recommendations, SAP’s HANA usage (profiled in the report on their Sapphire Conference )
Twitter for example has evolved in ways that surprised their founders and also launched a number of new businesses with very unusual purposes. As one article pointed out routing the data can be equally important as tallying it, as illustrated by Procter and Gamble’s practice of funneling social media conversation/data to the appropriate person’s screen for monitoring and response. In other words to be meaningful, sometimes just knowing something happened is enough, it doesn’t necessarily have to be mathematically manipulated to derive value.
Persistent challenges remain in dealing with the enormous variety of formats in which data are presented — some sources are difficult to analyze — their pages long data dictionaries often include details about its collection.Add to that the realization that Data is not just numbers anymore. The automatic semantic annotation required to make sense of this has also entered a new era.
Facebook, LinkedIn, Google+, Pinterest and similar social media sites would fit in here, as well. All offer a richness of information, much of it real-time, that can be monitored, mined and used to drive decisions and actions. The customer center conversations , or customer audio recordings , transcribed to text, and then subjected to text analytics is helping improve performance management practice, allows for campaign conversion performance tracking etc
The impact or convergence of the evolving technologies with all of this new data, is as overwhelming as the scanner data that was available and stored historically but few had resource capability or interest in mining it. That era has passed and with it, new questions and promises arise.
Can we better understand and use consumers sentiment as in do retail customers use more electricity/phone service/etc in a method that correlates to changes in the economy? weather? Or SAPs HANA data, now makes cancer DNA genome type analysis possible in minutes.
FB and Twitter, may be drawing more attention, but those who bring together the different streams and mining more deeply old sources such as scanner data are also causing quite a stir. Target found out that teenage girl was pregnant before her dad did using methods such as these.
Add the RFID and imagine the benefit to stores knowing the quantity of each SKU that they currently have? Can they coordinate sharing of product between their brick-and-mortar stores?
Perhaps the focus on twitter, etc., rather than scanner data comes from hoping that trending sentiment will precede and can be used to predict purchase, rather than log it as with scanner data. At cars.com, a participant shared that vehicle search data on our website is predictive of sales.
It looks like QR codes might achieve the same benefits of RFID as smart phones are becoming ubiquitous.
The Geo-spatial data, according to McKinsey’s recent report on Big Data as the next big innovation, quantified billions in time savings from just helping consumers avert traffic.
Several other behavioral nudging based on real time feedback is now possible. Nike is exploring and furthering this automatic feedback. Check this article for more brands using the quantified self, and more information
SAP’s HANA, Google’s BigQuery , Splunk, Hadoop and NoSQL databases , Tableau, Tibco Spotfire, Omniture ,Pentaho (open source BI), Amazon’s Web Service suite (more of a platform), Cloud computing platforms, Data visualization, most business users, make the data preparation easier and allows them to focus more on analysis and develop insights in combination with Machine learning tools (neural networks, support vector machines, natural language processing, etc.)
Decision making , can and does BIG data make more accuracy possible?
It offers higher granularity — like Target’s ability to create coupon books customized to individual households, Or integrate GPS level stuff — e.g. texting coupons to customers right while they’re standing next to a certain store. Or 2nd/3rd order analysis – correlating Target’s sales with weather data; creating ‘real-time’ personalized coupons; identifying ‘trend-setters’ among the customer base to influence ‘trend-followers’ coupons
The downside? Detecting or separating out spammers from these data sets or paid to express a certain sentiment. Totally! Like those girls paid to say great things about clothes on Facebook — not maybe necessarily analyzing big data, but using the platform.
Greater real-time evidence can reduce risk and insure assumptions in product/service development and marketing are on or off target.
Hard to appreciate the analytic without the qualitative context or understanding; but maybe some strange new ideas can come out of the data, like the “diapers next to beer” epiphany. Data needs some drivers to make it meaningful, as in cause and effect. In part because qualitative data is harder to analyze.
Is quant vs. qual or the social science methods to data collection really that different from the scientific data analysis approach? Both approaches seek to explain cause and effect, or the relationship between a stimulus to produce a predicted response. The problem is too many people will extend a model beyond it’s capacity. Claiming “the data said so” lets people off the hook and avoids responsibility for decision-making.
Suggested tips include avoid extending a model beyond its capacity, or understand and differentiate descriptive and predictive Stats. Likewise, be wary of finding trends that don’t exist (e.g. “data mining” or “straws that look like needles”) and confusing correlation with causality.
Perhaps cross-validation from trained analysts can help avoid these . Danger of expecting tools to automatically extract value from large datasets. Need to ensure good analysis, disciplined hypothesis generation, etc.
The data, even when analyzed, does not represent the decision. This is true with small and big data.
- Big data is here to stay – need to figure out how to use it effectively.
- I liked the point that lots of data is around and that people just don’t know what to do with it. The best BIG DATA process or engine in the world still won’t create the insights that are needed.
- Corporate culture is a huge factor–the problem is not availability of data, but commitment and focus of corporate leaders to shape a culture that moves the organization in that direction.
- Big data is here. It’s a tool and like any other, it’s the latest and greatest on the block, with a bit too much hype. But it has a definite value in providing and stronger qualitative base to identifying trends and activities.
- My realization is that, once again, the technology is interesting, but it is the corporate culture and will that will matter. The culture and vision lead; the strategy and models follow.
- sharing the questions with a wider audience confirms concerns and clearly lots of assumptions that need to be played out. There is a large dark side that we still don’t understand; but the positives and opportunities for real time decision.
- Big Data! The piles get higher and higher and wider and wider…to what purpose? That implies the need to “mine” the data, reduce it and subject it to analysis before it can be made useful.
- Big data will revolutionize business but it is not strategy, potential for a lot of false positives .
- The wise use of big data offers a huge opportunity for developing differentiating strategies and for finding new product/service needs.
- “Big data” is the current term for things that have existed for a long time. All types and sizes of organizations can benefit from big data if they recognize the importance of the human component (not just the data and software) and have specific objectives in mind before starting.
- Much of the expertise about analytics developed over many decades still applies, and there are new dimensions to integrate and understand because of the availability of the technologies and data.
- Everyone on the line has experience with Big data, so I don’t think it’s so scary. Most people have business perspectives, wanting to teach the Analysts that their conclusions need to be driven by business needs. My comment is that as leaders, and those trained with some behavioral awareness through business school, it is _OUR_ responsibility to try and massage the analysts towards an enthusiasm for our world view…;-)
- The human side of utilizing the technology and expertise is just as challenging as ever. (Cognitive biases, communication skills, influencing skills) Garbage in – garbage out is a big risk without proper attention and skill in applying the technology and in communicating. The data, even when analyzed, does not represent the decision. This is true with small and big data.
In closing, let me return to my observations about the limitations of developing strategy rooted in an expectations of the experience curve relationship. The frame with which you approach the problem often has far more bearing than the data, your analysis or the tools. Or at least, in June we plan to look at some of the assumptions around growth as the ultimate strategy.
Please throw your responses, or continue to post links for others as Big Data continues to be quite newsworthy as its impact and influence continues to unfold.
Articles and links:
We suggest an Optional short 5 min. video tutorial , EMC produced to understand what Big Data IS? http://youtu.be/eEpxN0htRK
The following are required advance reading.
1. IBM’s Institute for Business Value, in collaboration with MIT Sloan Management Review 2010 research findings
Analytics: The new path to value: How the smartest organizations are embedding analytics to transform insights into action
2. Strata keynote short 7 min. video by Google’s Digital Marketing Evangelist Avinash Kaushik- a bit irreverent and a little over the top – a bit irreverent, bordering on over the top – but not boring effort to help us understand the problems and the approach undertaken by Google.
Big Data Imperative
3. Tom Davenport’s Culture of Analytics
April 5, 2011
4. Inside P&G’s digital revolution
McKinsey Quarterly November 2011
overseeing the large-scale application of digital technology and advanced analytics across every aspect of P&G’s operations and activities—from the way the consumer goods giant creates molecules in its R&D labs to how it maintains relationships with retailers, manufactures products, builds brands, and interacts with customers. The prize: better innovation, higher productivity, lower costs, and the promise of faster growth…
One more optional overview
The age of big data
NYTimes, Feb 2, 2012