How Data Teams can evolve into CDO Organizations?

Data orgs are well poised to become CDO organizations. Data orgs should aspire and should be encouraged to evolve into one

Chief Data Officer Orgs is a critical department in modern organizations

In the current era of “Data is the New Oil” [1] and the tech organizations getting fined millions while trying to mine this new oil [2], a Chief Data Officer (CDO) organization is a must-have department in most mid-to-large-sized organizations. While many enterprises establish CDO organizations from the ground up, this blog contends that existing data engineering organizations are poised to evolve into CDO organizations.

What is a CDO organization and why do we need one?

There are multiple definitions of the CDO role; I highlight some of the more prominent ones below:

The primary role of a CDO is to underpin business strategy with data. They are responsible for maximizing the strategic potential of data across the organization — Caroline Carruthers
[
3
]

The CDO plays more of a risk, compliance, policy management, and business role. It serves to drive information and analytics strategy, serving a business purpose. — Gartner [
4
]

While a Chief Data Officer’s day-to-day duties may vary from organization to organization, a CDO’s overarching mission statement remains to own the “data-driven culture” within the company while satisfying the expectations of internal (data assets management/monetization) and external stakeholders (compliance).

CEOs have started to recognize data as a corporate asset that should be managed and monetized [4]

Why are Data Engineering orgs primed to become CDO orgs?

1. Higher ROI and probability of success

There is a considerable overlap of responsibilities between data management and CDO organizations. As a result, these teams are able to make use of their current team, talent, resources, and network while also being aware of the weaknesses of the current systems.

2. Aspirational goal for the team and the natural succession plan for the org

For data engineering, data science, and business intelligence organizations, becoming a CDO organization might be an aspirational objective. We can rally the whole team behind this aspirational goal.

Leaders may find it easier to handle internal politics during such transitions if they have such an aspirational objective combined with a higher ROI for the firm.

How data organizations can evolve into CDO orgs?

Any startup needs two strategies to establish itself in the market:

1. External view — Product, Brand, Marketing, Sales, Stakeholders
2. Internal view — How to achieve the external view — Purpose, People, Technologies, Skills, Team Topology, Operations, Project Management

Let us explore these two views concerning the data orgs' evolution into the CDO org

External View — Branding and Offerings

Most of the current data organizations already own and manage some of the most critical assets that a CDO organization is supposed to offer, such as Enterprise Data Warehouses or lakes, Data Pipelines, Data Science platforms, BI and Reporting, Big Data platforms, and Data Infrastructure.

To evolve into CDOs, I believe that the following additional offerings can help build the case that these orgs are already functioning as a CDO organization:

1. Organization-wide Queue/ Message bus for the asynchronous communication between apps

While most data teams generally own an organization-wide Enterprise Data Warehouse (EDW), Enterprise Queue as a PAAS offering is not as prevalent. While many software teams build such queues for their apps, many less technical or resource-constrained teams turn to less-than-ideal integration patterns instead, like:
– Synchronous communication — Resource intensive, not reusable and creates tight coupling between systems
– EDW as the intermediate layer — When two systems are not able to create direct connections between them, they also resort to using EDW as the buffer storage.
This is sub-optimal because the downstream apps become tightly coupled, event-based integration is not possible, and implementing an archival strategy in a data warehouse gets challenging.

Enterprise Queue should be offered as PAAS to promote decoupled integrations between apps[
5]

Like an Enterprise Data Warehouse, an Enterprise Message Queue may be a PAAS service provided by the CDO organization to create an organization-wide architecture pattern.

2. Data processing systems as a service (PAAS)

The data processing ecosystem is gravitating towards open-source tools (KAFKA, Airflow, Flink, etc.), that are easy to set up and experiment with but are hard to manage in production without a dedicated team to own the infrastructure, patch, upgrade, be compliant, etc., especially in the on-prem environment.

Additionally, these tools must be evaluated, procured, and, if necessary, upgraded to enterprise versions by an independent authority. The CDO organization must also adopt policies to decentralize these tools across the board.

To create a data-driven culture, data teams need to evolve from just services teams to enablement teams. Owning infra and platforms remains the biggest impediment for the less data-oriented teams to adopt a data-driven culture

3. Data Governance

Data governance remains the most important role that a CDO organization needs to play. This includes managing the data lifecycle, from creation, changes, and deprecation.

Data orgs need the right tools for data cataloging, profiling, column-level lineage, data quality, sampling, and discovery. These tools also require the backing of a dedicated team and the executive's support.

4. Promote Data Interfaces besides reporting

While most of the data organizations already own reporting tools such as Tableau and Looker, to influence actions from the analysis, the insights need to reach the apps where the intended user spends the most amount of time. Think of Salesforce for sales and Adobe Analytics for marketing.

Identifying the appropriate interfaces for the user groups and personas where the intended user spends the most time is something that CDO organizations must work to do. We can then leverage Reverse ETL tools to let the insights flow to these interfaces.

5. Own the Policies and Processes to enable a data-driven culture

Platform, tools, and services are important for CDO organizations however to be able to create a true data-driven culture, CDOs need to create and promote the right policies to enable self-service, promote the right architecture and reduce the cost and operational challenges of data access throughout the organization

5.1. Any project is not considered ‘Done’ without providing a default data externalization mechanism to the rest of the org

In 2002, according to tech legend, a mandate was issued by Amazon founder Jeff Bezos [6]

All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.

Anyone who doesn’t do this will be fired. — Jeff Bezos

Siloed data is one of the biggest impediments to a data-driven culture in organizations. Data organizations should work to develop and implement a policy governing apps’ data access. Any new application shouldn’t be considered “done” until it allows data externalization for analytical needs.

The path to data products goes through a data externalization policy, and it should be the first and foremost goal for the CDO organization to implement.

5.2. The contract between the Data producer and the Consumer

A data contract is the extension of the data externalization policy. The applications and the data source owners should design the data products, which can be used by the rest of the organization for analytical purposes. These analytical data products will be governed by the data contract, so that the data source owner will try not to break this contract with its downstream consumers, and any change should go through a change management process, if necessary.

6. Data Monetization Policy

Photo by Mathieu Stern on Unsplash

After covering the above basics, the next inflection point in the journey for a CDO organization is to find opportunities to monetize its current data assets by creating new products or services.

If successful, a CDO organization can aim to transform from a cost center to a profit center for the enterprise.

Internal operational structure to execute the CDO brand

A CDO team can be organized in the 6P structure. The first P stands for Purpose, followed by PMO, Project, Platform, People, and Partnerships.[7]

Although the exact team topology will differ for each organization depending on its size, needs, and maturity, here are some pointers that can be applied while trying to grow into CDO teams.

1. Data Product management team

The mantra for the data product management team is “design thinking” while keeping the customer-first mindset. This team has three main responsibilities:

a. BU’s representative to Dev: This team is responsible for advocating the business requirements in the sprint meeting. This team is proactive about the business’s data needs by observing the business process closely and how digitization or analytics can aid in data-driven decision making without waiting for the BU to raise such requests to IT or CDO

b. CDO Representative to BUs: This team is also vocal about the translation of analytical insights into business actions. E.g., they will like to understand the actions that a field representative takes once the system identifies a customer as churn-risk.

c. CDO and BUs representative to source teams: This team acts as a SPOC for all the analytical products requirements that are to be created by the application teams

2. Platform-As-A-Service (PAAS) Team

CDO org should have a dedicated team to manage platforms-as-a-service to democratize data and enable self-service. The PAAS team can be comprised of software engineers who would be responsible for the automation, high availability, license cost optimization, best practices, and governance of the platform.

3. Data Governance Team

The CDO org should have a strong data governance team that is committed to its mission, has the authority to manage policies, has the support of the appropriate executives, and has influence over the source and target teams.

This team’s policy creation and policy execution sub-groups should have their responsibilities segregated so that the quality of the policy is not sacrificed in favor of easier implementation process

4. Each large project can have a triumvirate of the product manager, the project manager, and the engineering manager. These three roles should have an independent reporting structure to ensure that the delivery team is not the only one dictating the timelines and specifications

Conclusion

A slight expansion of the offering portfolio by data teams can add a ton of value to enterprises, with or without the official CDO title. Just like the best strategy for individuals to get promoted is to start taking on the responsibilities of the next level, even departments need to create their growth strategies to be able to add more value for enterprises, with or without the official title and the official title will follow.

Please reach out to me on Linkedin for further conversation.

Many thanks!

References

1. https://www.wired.com/insights/2014/07/data-new-oil-digital-economy/
2. https://www.tessian.com/blog/biggest-gdpr-fines-2020/
3. https://www.cio.com/article/230880/what-is-a-chief-data-officer.html
4. https://www.gartner.com/smarterwithgartner/understanding-the-chief-data-officer-role
5. https://docs.confluent.io/5.5.1/kafka/introduction.html
6. https://nordicapis.com/the-bezos-api-mandate-amazons-manifesto-for-externalization/
7. https://analyticsindiamag.com/who-should-a-chief-data-officer-report-to/
8. Five Strategies to set up the Chief Data Officer (CDO) for Failure | by Pradeep Menon
9. 10 trends shaping the chief data officer role

---

How Data Teams can evolve into CDO Organizations? was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.