It’s hunting season! Fortunately, the hunt isn’t for cute forest animals but rather for invaluable business insight. Businesses today are increasingly hunting for fresh insights to help them differentiate their products, services, and customer experiences. They need new and diverse data sources to deliver those insights. As one seasoned Chief Data Officer noted, “With our own data we can only look internally. We need to see industry benchmarks, regional trends, what waves we can ride on.”
“With our own data we can only look internally. We need to see industry benchmarks, regional trends, what waves we can ride on.”
Chief Data Officer, mid-sized U.S. bank
Diverse data drives richer insights
Each new data source adds information that can improve the accuracy of data models, and, as a result, the impact of a new marketing campaign, a new product offer, or an improved customer experience. Take, for example, customer data. A base layer includes core traits that (for the most part) remain constant, such as date of birth or gender. To better understand customers, however, businesses generally need more context, such as an address, marital status, education levels, employment status, and other preferences.
These data points can change over time, and keeping them current is key to effective engagement. But the holy grail is having input from a more dynamic layer of data on recent purchases, current location, or recent life events. Equipped with this data, companies can anticipate the needs and wants of their customers. An insurer, for example, could offer life insurance to new parents or home insurance to a new homeowner. A telecom operator offers an international roaming plan to digital nomads or parental controls to families.
Similarly, location data can be broken down in layers that add depth to analyses. A base layer includes mapping coordinates and unchanging characteristics, such as rivers, mountains, or coastlines. Think about tracking assets or mapping out a supply route. Are the goods close to a border or near a coast? This information has implications for shipping options. But, again, having more context improves the decision-making. Are there roads? How good are they? How far away is the nearest bridge over the river? And, finally, the dynamic layer adds additional richness that drives decision-making in real time or by revealing patterns over time. For example, footfall traffic and crowd density influence retail site selection or the placement of outdoor advertising.
Decision-makers want richer insights
In today’s competitive environment, decision-makers want to get any insights they can—from any source. Where margins are thin, unique insights can create differentiation and competitive advantage. Internal data isn’t enough. Take, for example, the competitive mortgage industry in the United States. Borrowers constantly search for better rates. The threat of churn is growing as are costs of customer acquisition. Having a clear picture of customer segments, what they are looking for, and how they respond to specific drivers is critical to the success of the business.
In a recent conversation with the former Chief Data Officer of a U.S. mortgage lending company, he described how external data was a key ingredient of loan origination and servicing. He oversaw the acquisition of 36 different data sources for use in improving customer experience, ensuring regulatory compliance, and managing risk. In the origination business, data acquisition meant a multimillion-dollar budget to acquire credit scores, employment status, and salary verification. The more data, the more complete the underwriting and the lower the risk of loss. On the servicing side, the risks were lower but the need to keep down churn was a constant challenge and required knowing everything, from a customer’s search terms to requests for credit scores.
In dynamic markets, companies are constantly on the lookout for new data. According to the 2021 State of External Data Acquisition study by Explorium, customer data leads the pack: 62% of firms acquire demographic data and 56% acquire company data. Over one half of firms also acquire anonymized individual data and financial data; 47% acquire geo-spatial data, and 19% acquire weather data. Some companies even hire data hunters to scout for relevant and unique data sets. And, during the pandemic, new data sources appeared. In a recent Snowflake podcast, the CIO of 1-800-Flowers described the importance of assessing the impact of changing Covid-19 infection rates on its business in local regions, to better predict supply and demand.
Increasing demand drives expanding budgets
According to IDC research on the data-as-a-service market, well over 50% of North American companies reported overspending their budget for external data in 2020, with half of them saying they spent significantly more than planned. The pandemic has only served to increase demand as companies have added health-related data sets to their go-to sources, like 1-800-Flowers. The Explorium study found that 78% of respondents plan to increase their budgets for external data acquisition, and only 1% do not have a budget in place at all.
Clearly, companies are taking the hunt for new data very seriously, and putting more wood behind their arrows. The study also found that, in 2020, 81% of companies spent more than $100K each month on external data acquisition, while 31% spent more than $500K. These numbers only paint part of the total investment picture since almost half of respondents said they spend over 50 hours per month on external data acquisition. Yet, the cost of the data is the least challenging aspect of data acquisition: only 34% report that data cost is a significant challenge. The biggest challenge for many is the discovery itself: 46% are not sure what to look for. Other challenges include regulatory constraints (44%), data prep/integration (43%), and lack of skills and tools (39%).
“Clearly, companies are taking the hunt for new data very seriously, and putting more weight behind their arrows.”
Despite these challenges, most companies set their sights on even more external data in 2022. According to IDC’s forecast, on average external data budgets for 2022 will come in just shy of the $1 million mark ($954,895) versus $707,807 in 2020—an increase of about 35%. And, that data will be applied to a greater number of use cases, with significant growth in a few specific processes including HR, legal and compliance, and commerce.
Yet companies need help with acquisition strategy and execution
Although companies overwhelmingly recognize the value of external data, only 28% of respondents in the Explorium study had an acquisition strategy in place. Almost the same percentage (26%) rely on ad-hoc practices or general guidelines for data acquisition.
With a growing number of use cases, efforts and costs of discovering, evaluating, and integrating new data sources is exploding. Organizations must take a more holistic approach, investing in the leadership and skills to drive mature data practices. They must establish processes that scale. And, they must invest in a data platform that can facilitate discovery and access to external data and simplify the path to delivering relevant insights to business stakeholders.
The post When Hunting for Competitive Advantage, Set Your Sights on New Data Sources appeared first on Snowflake.