Looking on the bright side of Big Data

What Monty Python teaches us about creativity amid the deluge.

March 13, 2015

What Monty Python teaches us about creativity amid the deluge

John Cleese tells a story about working on a new script with his Monty Python writing partner Graham Chapman – and after weeks of careful development, finding he’d lost it. Once he realised the script was missing he hastily rewrote it from memory, so as to try and capture as much of the original as possible. A while later, the original turned up and he had the opportunity to compare the two: the first draft, painstakingly put together; the second scribbled down based on a vague memory of the first. No prizes for guessing which of the two was better – and better by far.

“The only thing I could think was that my unconscious had been working on the sketch and improving it ever since I wrote it,” said Cleese. “I began to see a lot of my best work seemed to come as a result of my unconscious working on things when I wasn’t really attending to them."

Cleese’s insight might be familiar to anybody who’s studied a course on advertising. We’re used to the concept of creativity being fuelled by ‘unthinking’ time, when we switch off and enable our brain to start making all kinds of free-form connections without our stubborn, slow-moving consciousness getting in the way. We’re far less accustomed to applying this to how we work with data. But as Big Data builds up steam, it’s high time we started.

How to stay creative in the age of Big Data

Psychologists from Harvard spent six years trying to understand creativity and innovation. In what is considered one of the most thorough examinations of the subject, researchers interviewed 3,000 executives and concluded the number one skill that separates creative from non-creative people is the ability to associate seemingly unrelated problems, questions or ideas: to connect the things that other people wouldn’t think of connecting. The big question is: can we keep doing this as the volume of information we are dealing with keeps increasing?

After all, the amount of data headed our way, and the speed with which it piles up, can make it easy for businesses to feel swamped. One of the most obvious coping mechanisms is to classify, categorise and organise data as fast as it arrives, deciding in advance what it means and what it will be useful for. But imposing our own conscious structure on Big Data prevents us connecting ideas and information in the creative way that Cleese describes – it might well mean we get less out of it, not more. We will work with Big Data best when we are able to explore it freely, unhindered by our preconceptions of what’s important and what isn’t.

Remembering the principles of creativity

How can we go about doing so? For starters we need faith in our own capacity for creativity. As all good creatives know, unconscious, unstructured thinking doesn’t just happen out of the blue. It often requires imposing a personal routine that will enable you to think about something intensely for a while and then go off and do something that stops you consciously thinking about it altogether.

Perhaps we need to force ourselves into similar routines when it comes to data: doing the sensible things with it first, but then allowing ourselves time, space and freedom to try something completely different, as Monty Python would have put it. When we start seeking new patterns, and new significance, we start to tap the potential of data for creativity.

It helps that we’re increasingly able to use our full range of senses to grapple with the data around us. When we visualise connections they appear beautiful, stimulating and thought-provoking. We can start to sense patterns as well as consciously seeking them out (one of my personal favourites is this visualisation of the tasting note connections between different scotch whiskies). A colleague of mine also recently wrote about using data visualisation to find the perfect bar for Happy Hour in San Francisco. Appropriately enough for this blog, the scripts he used to do so go by the name of Python.

A vision for Big Data

As someone who works for LinkedIn, I’m part of a vision to describe the entire economic value of the world, through a concept known as “The Economic Graph”. This is quite an objective to have: documenting every economic opportunity on the planet, and everyone with the skills and potential to fill them. It’s a lot of data, and there will be a huge amount of potential for connecting that data in creative and meaningful ways. And that’s the kind of future I think we should be demanding of Big Data: one that sparks the creativity inside us and gives our unconscious imagination something to get to grips with. After all, there’s always been a strong link between the freedom to express yourself creatively and general wellbeing. It would be a terrible shame if Big Data constrained such creativity when it should be unleashing its potential. As a certain comedy troupe might have put it: it’s important to “always look on the bright side of life.”