The revival of data management: trends & challenges

Data management – an umbrella term for data integration, data quality and also data governance – has long taken a back seat within many companies worldwide. But as more and more companies are now rapidly entering the digital era, data management is moving to the forefront. There is a massive amount of – often complex – data to be managed. At the same time, organizations need to take more preventive or real-time action and respond more promptly to market developments in order to get (and stay) ahead. The traditional way of working – whereby business owners receive data reports from IT – is being called into question.

Good data management is a prerequisite for useful data analytics, the importance of which has been underlined by recent studies. The McKinsey Sales Growth 2015 Survey, for example, notes sales improvements of 131% for companies who make extensive use of advanced analytics.

3 striking trends

Rein Mertens, Principal Business Solutions Manager at SAS, has identified three evolutions which are having a significant impact on the data management domain.

  1. The rise of self-service: the need for agility requires businesses to rapidly combine and analyse their own data in a user-friendly way. “We’ve already seen a self-service evolution for reporting, whereby business users adapt their own filters in a report. Today, the self-service capabilities are extending to the data sources themselves. Business users want to extract information from data streams very quickly without needing advanced IT skills. The SAS® Data Loader for HadoopTM, released last year, is a good example of how SAS is responding to this trend. It allows business users to perform search actions on large data volumes quickly and easily, without knowledge of YARN or MapReduce,” says Rein Mertens.
  2. The need for an open platform: to gain insights, organizations need input from different internal and/or external sources. Easy data exchange is necessary for ‘analytics on the edge’ too. “Analytics on the edge refers to automated analytical computation directly performed on data by a sensor or other device, instead of having to wait for the data to be transmitted to a centralized data store. For example, a wind turbine can compare its performance against the other wind turbines at the wind farm. This shortens reaction times in the case of problems. Our newly launched architecture SAS® ViyaTM meets the need for a modern and open architecture, and the SAS Forum will be the ideal opportunity to learn more about it.”
  3. Suggestive data management: modern data management tools make use of analytics to improve data management. The system can provide a roadmap for combining different data sources or can help end users to interpret the data. It’s also no longer inconceivable that data management tools perform comparisons with other external sources to make suggestions about standardisation of columns, for example.

Current hurdles

While today’s organizations are transforming quite fast to keep pace with ever-changing customer demands, they are also facing a lot of challenges – including in the data management domain. In the first place, it is difficult to achieve the right balance between creating the required flexibility and keeping enough management control. If the pendulum swings too far over to the controlling side, a ‘shadow’ IT set-up emerges: business users introduce their own systems to reduce their dependency on the IT department and to receive faster business insights. On the other hand, in a fully self-service system, organizations can lose the ability to capitalize on previously acquired insights. When business users discover relevant insights in a dataset, IT is responsible for automating the process and building the required monitoring and management mechanisms.

“Secondly,commercial and IT departments have to work together closely to achieve the best results for the company overall, but in practice this is an arduous task. Most C-level managers emphasize the importance of data as a corporate asset, but how is this translated within the rest of the company? Many data management initiatives are still driven by the legislative framework. Few companies proactively place data governance at the top of the agenda,” Rein Mertens comments. “We often see Master Data Management (MDM) projects that are driven
by IT alone. Unsurprisingly, business colleagues are unhappy with the end result and the project fails.”

A third challenge is to use appropriate tools for data management – tools that are appropriate for IT and for the business end users alike. According to a study conducted by Bain & Company, 56% of companies do not have the right systems to capture the data they need or are not collecting useful data, and 66% lack the right technology to store and access data. “Moreover, the tools that are needed today to prepare data for analysis and reporting are different from the toolset traditionally used by IT in the operational context of a data warehouse. Companies need a strategic plan for collecting and organizing data, one that aligns with the business strategy for how they will use data to create value.”

Furthermore, many companies are struggling with the Internet of Things and the huge amount of data generated by it. “To me, it’s quite clear that companies first need to get a grip on their ‘small data’ – e.g. internal data, master data – before investing in a Big Data project. You can’t run before you can walk. What’s the point of Big Data insights if you can’t link them to qualitative customer data?” continues Mertens.