Why Big Data Will Never Beat Human Intuition
Big Data might well be “the new oil,” but it must not be the new religion.
Big Data is big business. Sensors, GPS tracking, math modeling, and artificial intelligence offer companies real-time market insights at massive scale and open the door to unprecedented ways of monitoring, targeting, and measuring employees and customers. Analyst firm Gartner predicts that enterprises adopting Big Data technologies will “outperform competitors by 20 percent in every available financial metric.”
Big Data might well be “the new oil,” but I would caution us not to worship it as the new religion. I’m afraid that amidst all the data frenzy we are not only losing a more holistic view of business but also an important part of our humanity. How do we appreciate quality if we capture it only in quants? How much (or little) space do we leave for creativity and human expression if we equate better living with better algorithms?
I am not a dataphobe, but I am concerned about relying only on data. I am not against quantitative metrics, but I question their authority as the main indicators of business performance, prosperous societies, and meaningful lives.
Big Data comes with many benefits, but let’s complement it with Big Intuition. Here are six reasons why:
- Big Data = Big Brother? The New York Times’ Steve Lohr describes Big Data as a descendant of Taylor’s “scientific management.” Instead of performance in the workplace, which was the focus of Taylorism, we are now measuring happiness and well-being, our consumption preferences, social interactions, physical activities, our attitudes, moods, emotions, behaviors, and bodily functions—in other words, we are measuring our lives.Sure, to some degree, “quantified self” apps may empower the individual to exert more control over their lives. However, by doing so, we are expanding the dictate of business to once-private terrain, commodifying and colonializing it under the mandate of self-improvement. For many of us, the refusal to measure and quantify these few remaining “sacred spaces” of our lives is the last bastion against the pervasiveness of the commercial.
- Big Data is not social. We humans are social animals. Research shows that relationships, especially friendship and marriage, are key factors of happiness and fulfillment. Our brains are wired to care, and our hearts and minds have developed an astounding capacity to empathize and sympathize with fellow humans. We can show compassion, sense mood swings, detect subtle non-verbal cues, tolerate or embrace, accept and reject, love and hurt, experience with all of our senses, act irrationally, and even lose our self-control. These key traits of our humanity are threatened by the “mathematization of subjectivity,” as Leon Wieseltier calls it.Recent social genomics studies suggest that not only our productivity, but also our evolutionary capacity to connect with others is diminished by digital overload.
- Big Data creates small worlds. Morality is gained by way of empathy. Yet, in our age of hyper-connectivity we are increasingly facing the challenge of connecting with the other whose opinions, values, beliefs, faith, and culture may be unlike ours.As digital technology has become more sophisticated in personalizing and customizing our social experiences, online and offline, based on our preferences, our “Likes,” and online paths, we are increasingly stuck in our own world—the “Filter Bubble,” as Eli Pariser called it, designed by smart algorithms to serve us with content, culture, and company that we are already familiar with and that fall squarely within our comfort zones. We don’t “like” the people and things that are unlike us, and thus feed a vicious cycle of social and cultural narrow-mindedness.
- Big Data makes us smarter, not wiser. Our data-driven worlds are not only becoming smaller, they are becoming faster. The real-time flow of information persuades us to react to feedback constantly and instantly. Playing on the title of Alvin Toffler’s 1970 book Future Shock, Douglas Rushkoff calls our current state-of-mind Present Shock, lamenting “a diminishment of everything that isn’t happening right now—and the onslaught of everything that supposedly is.” Real-time feedback loops make it harder for us to step outside the tyranny of the now and see the future—to envision what could be.Vision is what Big Data can’t deliver. Data might give us information fast, but for quick but profound decisions, intuition is much better suited. Prasad Kaipa and Navi Radjou, in a recent book, urge business leaders to move “from smart to wise.” They have a point. Smart organizations and leaders thrive on constant feedback. Smart is fast. Wise, however, is slow. Wise organizations and leaders need time and take it. Time for pause. Time for reflection. Time for not doing anything. Time to find the signal amid all the noise.
- Big Data is (too) obvious. “You can only manage what you measure”—really? The financial crisis has shown that we manage poorly what we measure, and failed mergers, failed product launches, reputational crises, and social media PR disasters, in short, the cultural disconnects within organizations and between brands and their audiences, indicate that we need to get better at managing what we cannot measure.Leaders now need to have “opposable minds,” as design thinker Roger Martin puts it. Data doesn’t parse ambiguity. The business leader of the 21st century will no longer be measured by how much uncertainty he or she can eliminate but how much uncertainty he or she can tolerate.
- Big Data doesn’t give (or forgive). Data might be able to predict new problems or find new solutions to existing problems, but only human intuition and ingenuity can come up with groundbreaking new ideas. That is a uniquely human gifts—one that creates a “meaning excess,” a sense of wonder and significance that goes beyond reciprocity, beyond merely fixing a problem or meeting a functional need.By the same token, if we quantify all of our relationships, we will not leave any wiggle room for human discretion and our unique ability to forget, and even if we can’t forget, to forgive. Because we often have mixed feelings about people and their behavior, our judgment can be more than just binary. This means we can assess and respond to ambivalent behaviors with more nuance—we can appreciate the intention over the outcome, if we want, and we can choose to accept failure as a prerequisite of innovation. It is hard to see how we can collaborate with one another, how we can make progress towards any goal, without the ability to forgive.
So how can we become data-savvy, but not data-obsessed? As innovators, marketers, and business leaders, we must constantly defend and push for spaces for Big Intuition. Let’s resist the rush to data and take the time to lean back so we can be fast when it matters. Let’s grant ourselves a data moratorium from time to time that we can use to reflect on what really is important to us and our organizations. Let’s “hack” Big Data with small acts of friction. Let’s de-stigmatize “gut feel”—it is a better lodestar than we might think. And let’s use data to tell our stories, but let’s not allow data be our only story.
We must be giving and listening instead of filtering and targeting and analyzing. We must not reduce people to profiles and graphs, and aim for short-term victories on what Rushkoff terms the “algorithmic battleground,” whilst losing the hearts and souls of the people we serve in the long-term.
Data can give us the illusion of objective truth, yes, but at the end of the day, our employees and customers are not interested in the truth, they seek experiences that feel true. In other words, they seek authenticity. It is this small, but critical, gap between truth and authenticity that gives our brands, our organizations, and our lives meaning. No data can fill it. Only human beings.