How organisations can overcome big data’s two biggest obstacles

5 min read

TwitterLinkedInFacebookEmailCopy LinkPrint

Still haven’t found what you’re looking for?

Most organisations across the world are acutely aware of the significant competitive advantage big data and advanced analytics strategies could create for both their own businesses and those of their clients.

When successful, data-analytics strategies should enable organisations to gain unique insights, acquire competitive intelligence, solve problems, optimise operations, identify new markets, target customers more effectively and even create entirely new business models.

Recent research suggests that there are two main obstacles to the successful implementation of big data strategies that are yet to be overcome by a significant number of organisations.

The problem facing many firms can be neatly summed up by Bono: “I have climbed the highest mountains, I have run through the fields, I have run, I have crawled, I have scaled city walls…but I still haven’t found what I’m looking for!”

Obstacle 1: Success for data-driven strategies relies heavily on technically brilliant ‘data scientists’ who may lack key skills and behaviours

Success in big data and analytics endeavours ultimately comes down to the knowledge and expertise of a relatively scarce group of technically brilliant experts (‘data scientists’ or ‘quant jockeys’) who can capture, search and query huge amounts of data then use advanced analytics to turn the information into strategy and recommendations.

For businesses and their clients to reap the benefits of these data-driven strategies, their data scientists must be able to produce insights, conclusions and practical recommendations that are then communicated in a compelling way that can also be understood and acted upon by more commercial, non-technical decision makers, either internally or at their client’s organisation.

Clearly technical skills are vital but success in analytics strategies requires data scientists to also have a commercial understanding of the ‘nuts and bolts’ of the business as well as the ‘softer’ skills and behaviours which will enable them to communicate their findings to the decision makers.

Obstacle 2: There is lack of a clear, consistent and robust process to manage ‘information overload’

Recent research[i] by the American Institute of Certified Public Accountants (AICPA) and Chartered Institute of Management Accountants (CIMA) highlighted the main obstacles hindering the advancement of big data initiatives. In particular, organisations struggle with ‘information overload’ and how to turn ever-expanding volumes of data into strategic insight, conclusions and recommendations.

An overwhelming 80% of respondents to the research admitted that their organisation had used flawed information to make a strategic decision at least once in the last three years. The report also found that firms:

Do not take enough time evaluating the specific question they are trying to answer

Struggle to determine whether the right information exists or where it is located

Find it difficult to take a long-term view and define the right metrics

Have organisational silos and entrenched bureaucracy that create coordination problems

In short, there is overwhelming data and a host of underwhelming solutions.

Overcoming the obstacles

1. Develop and cultivate ‘bilinguals’ (or ’integrative thinkers’)

2. Embrace a firm-wide, cohesive, structured analytical thinking process

For big data and advanced analytics strategies to reach their full potential, firms need to address both of the identified obstacles head on.

David Meer of consultancy firm Strategy& has written of the need for organisations who want to profit from big data and advanced analytics strategies to cultivate ‘bilinguals’ who can “simultaneously speak the language of advanced data analysis and nuts-and-bolts business operations”. Crucially, Meer argues

“Without this translation function, the investments companies are making in data and analytics are unlikely to yield maximum results”.[ii]

Similarly, in the AICPA and CIMA report, the authors argue that the relatively few organisations that have successfully used big data have cultivated ‘integrative thinkers’ – collaborative leaders who can see horizontally across their whole organisation and connect the dots.

The question is how to cultivate these ‘bilinguals’ or ‘integrative thinkers’? This could begin with the development of the crucial ‘softer’ skills which may not come naturally to technical experts. Fortunately, these behaviours and skills can be learned by explanation, role-playing, modelling and ongoing coaching.

We know that most business problems stem from a lack of diagnosis and the more data involved the harder the diagnosis. So how can businesses ensure that their ‘bilinguals’ are able to formulate a ‘nuts and bolts’ business objective, know which data is relevant and where to look, generate insight, collaborate effectively and produce outputs and recommendations that can be both understood and acted upon by those with decision-making authority?

The answer is to establish a cohesive, structured analytical thinking process firm-wide that is understood, embraced and acted upon by ‘technical’ and ‘non-technical’ employees alike. The analytical thinking process should enable everyone in the firm to apply a consistent consultative approach to problem solving and opportunity generation.

The process needs to be robust, reliable and valid and it needs to be efficient (you only get the data you need) and effective (it solves the problem).

A recent survey of 714 global organisations in McKinsey Quarterly (February 2016) supports such a strategy:

“Although companies struggle to roll out big data initiatives across the whole organization, [our] results suggest that efforts to democratize usage – getting analytics tools in the hands of as many different kinds of frontline employees as possible – will yield broad performance improvements”.[iii]

Once everyone involved with big data initiatives has learned and understood each aspect of the analytical thinking process and the associated skills and behaviours, they should be able to:

  • Clearly define the client, the objectives, the stakeholders and the metrics
  • Identify what data they need to find and where to find it in order to generate results
  • Produce actionable, insightful conclusions and recommendations
  • Present and communicate these results in a clear, coordinated manner that can be understood by all stakeholders throughout the organisation (to overcome the ‘silo’ issue).

There will be those who believe that there will always be a minority of technically brilliant data scientists who will never be able to or want to act as ‘bilinguals’ due to a lack of ‘softer’ skills.  While we acknowledge this viewpoint, we argue that these individuals still need to have an understanding of the firm-wide analytical thinking process in order to know how to feed their data, insights and outputs into a process that everyone can understand and act upon.


There is little doubt that data-analytics strategies could provide great opportunities and benefits for organisations and their clients. However, many firms are struggling to reap the full benefits of these initiatives as a result of ‘information overload’ and a lack of understanding and communication between data scientists and non-technical decision makers.

Firms need to develop their data scientists into ‘bilinguals’ (or ‘integrative thinkers) who can understand the intricacies of the data but also the ‘nuts and bolts’ of business and then communicate any findings, conclusions and recommendations throughout the organisation. To achieve this requires the adoption of a robust, reliable and cohesive structured analytical thinking process firm-wide and the embracing of associated skills and behaviours.

There remain key obstacles for businesses to overcome in order to reap the full rewards of advanced analytics and big data. It will not be easy to overcome these challenges but, for those businesses who can, the rewards could become as large as the data sets themselves.

Analytical Thinking Process


If you would like to know more about how we can help your firm adopt a structured analytical thinking process and the associated skills and behaviours, particularly regarding communication and the development of ‘bilinguals’, we would be delighted to have a conversation with you.


[i] Research by AICPA and CIMA entitled: “Joining The Dots Decision Making For A New Era”
[ii] Educating the Next Analytics “Bilinguals” –
[iii] Big data: Getting a better read on performance – By Jacques Bughin, McKinsey Quarterly February 2016

Photo from by Todd Poirier used under ‘Unsplash Licence

Latest Points of View