Decade Of Investment In Big Data And AI Yield Mixed Results

Cloud computing technology and internet of Things concept,Big data and connection technology getty It was nearly a decade ago that Fortune 1000 companies began to embrace Big Data, and within a few years, embrace AI as well. About this time, beginning in 2012, NewVantage Partner began to track the progress of these efforts, as leading companies sought to undertake data-transformation efforts with the objective of becoming more data-driven in their business decision making.

Source: Decade Of Investment In Big Data And AI Yield Mixed Results

It was nearly a decade ago that Fortune 1000 companies began to embrace Big Data, and within a few years, embrace AI as well.  About this time, beginning in 2012, NewVantage Partner began to track the progress of these efforts, as leading companies sought to undertake data-transformation efforts with the objective of becoming more data-driven in their business decision making. Based upon findings from NewVantage Partners 9^th annual survey of senior corporate c-executives, the picture is decidedly mixed. 

This year’s theme was the progress of leading companies in their data and AI initiatives  The survey, and summary report entitled The Journey to Becoming Data-Driven: A Progress Report on the State of Corporate Data Initiatives, attracted 85 Fortune 1000 or industry leading companies. 

Participating firms included top-tier banks – Bank of America, CitiGroup, JP Morgan Chase, Morgan Stanley, and Wells Fargo; payment processing leaders – American Express, Capital One, Mastercard, and VISA; healthcare services firms – Anthem, Cigna, CVS Health, and United Health; pharmaceutical giants – Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Merck, Pfizer, and Sanofi; leading insurance providers – AIG, Berkshire Hathaway, Liberty Mutual, MetLife; and consumer goods and services giants – McDonalds, Starbucks, and Walmart.

Among the principal findings of this year’s survey, established companies continue to struggle in their efforts to build data-driven organizations, as illustrated by some stark findings from these companies:

• Only 48.5% are driving innovation with data
• Only 41.2% are competing on analytics
• Only 39.3% are managing data as a business asset
• Only 30.0% have a well-articulated data strategy for their company
• Only 29.2% are experiencing transformation business outcomes
• Only 24.4% have forged a data culture
• Only 24.0% have created a data-driven organization.

What is noteworthy in these results is that while investment in Big Data and AI remains high – 99.0% — and investment levels increase – 91.9%, results continue to lag. Only 24.0% of executives report that their companies have realized the goal of becoming a data-driven organization. 

There appears to be a recognition that data-driven business transformation may be harder than once anticipated. The findings illustrate that even as progress remains elusive, most companies have not progressed, and have even regressed in these efforts in recent years – see Table 1. 

These results suggest that most companies continue to remain at an early stage in their transformation efforts, and it will take time for firms to realize the full fruits of their investments of time, energy, and business resources. This is the case even after a decade of investment in these efforts. Transformation comes slowly and transformation efforts represent a long-term journey.

NewVantage Partners LLC

Why is it that as firms continue to invest in data and AI initiatives, and increase their rate of investment, progress remains elusive? One answer may lay in where companies perceive the greatest barriers and impediments to progress with their data and AI initiatives. This is around cultural change and transformation. According to this year’s survey, company executives overwhelmingly report – 92.2% — that the greatest challenges to becoming data-driven are, and continue to be, due to cultural barriers, not technology limitations or a lack of viable technology options. This is an instance of technology being ahead of capacity for adoption.

The cultural impediments and barriers to data-driven transformation efforts arise largely from challenges in the areas of organizational alignment, business processes, change management, communication, people skill sets, and resistance or lack of understanding that enable change. Investments in data governance, data literacy, programs that build awareness of the value and impact of data within an organization, may represent an eventual step in the right direction. As evidenced in Table 2, progress has been slow and cultural impediments continue to loom large, with no immediate improvement in site.

NewVantage Partners LLC

As the survey indicates however, the picture is not all gloom and doom. There are notable bright spots on the horizon. For example, the adoption of Artificial Intelligence (AI) represents one potential bright spot, as companies appear to be progressing steadily in their adoption of AI initiatives. This year, 77.8% of companies report that AI capabilities are now deployed in widespread or limited production, up from 65.8% last year, with only 4.1% reporting no applications of AI in use. This represents steady progress. With overall investment levels in data and AI increasing, adoption of AI capabilities can be expected to continue.

Another potential bright spot can be observed in the long-term optimism of data executives. Despite the many challenges that leading companies are facing as they pursue efforts to become data-driven, an overwhelming majority of executives – 81.0% — are hopeful for the future, expressing optimism about the outlook for data/AI within their firms. Within these companies, 91.9% reported that the pace of investment in data/AI continues to accelerate, another positive indicator. A further positive indicator is the number of executives – 45.4% — that describe their companies as leaders in making progress on data/AI. Even in this most challenging of years, 91.6% of companies report that, while facing the headwinds of the COVID-19 epidemic, their companies would be spending the same or more on data and AI initiatives. Good news indeed.

One final bright spot is reflected is the commitment that leading companies have made to establishing a central data function, in the form of the Chief Data Office, and appointment of Chief Data Officers to fill these leadership positions. During the past half decade, well over half of leading companies have taken steps to establish a CDO function and appoint a Chief Data Officer. Although the role continues to evolve, companies appear to be steadfast in their commitment to this role’s inevitable success, with nearly two-thirds – 65.0% — having a sitting executive in this role today.  As illustrated in Table 3, the Chief Data Officer role is firmly established and there is no turning back. 

NewVantage Partners LLC

A decade into the data-transformation journey, leading companies are making slow but gradual headway in their efforts to become data-driven. While investment levels and signs of optimism continue to increase, results reflect lagging adoption and remaining challenges in organizational transformation efforts. It is decidedly a mixed bag. 

Over the course of the past decade, Big Data and AI have been gradually absorbed into the corporate mainstream. Those firms that continue to make progress and begin to show results from these investments will have transformed themselves to compete more effectively in the 21^st century digital business world.  For legacy companies, this will represent a significant, and hard won, accomplishment.