Drain your data lakes with a Data-Products-as-a-Service model

Key takeaways:

What is a Data Product and how to implement a Data-Products-as-a-Service model
Think in trusted data products, execute in data domains
Data governance must live inside — not beside or outside — business

With more and more of our industrial operations data readily in the clouds, Chief Data Officers (CDO) are confronted with the hard reality that moving data to the cloud is not even a third of the journey to value. As Forrester (2021) put it elegantly, “data has no value unless the business trusts it and uses it.”

Think in Trusted Data Products, Execute in Data Domains

With always-on secure data access for all data consumers within and around our industrial enterprises becoming the new norm, CDO focus is turning to the intersection of data engineering/data custodianship (IT) and the business domain expertise for that data (OT/ET). This in turn is fuelling a reversal of the past years’ rush towards the data lake as the single source of all data — replaced by interconnected domain Data-Products-as-a-Service.

Learn more: The data liberation paradox: drowning in data, starving for context →

The driving force for the shift is the realisation that for data to be operationally used at scale, especially for critical operations, it needs to be trusted. For data to be trusted, it need to be productized. The shift from data availability to data products as a service (see figure below) is what will transform our data swamps into operational data lakes of real business value.

Data governance must live inside — not beside or outside — business

Any organization hoping to successfully transition into offering Data-Products-as-a-Service, needs to embrace a shift in data governance ownership. The necessary move is from a centralized data team, such as digital or data CoE, into a collaborative setup where each data domain is co-owned by the respective business function producing the data in their primary business tools.

"It is the business operations teams that understands the data in context best, and are therefore best placed to communicate and provide Data-Products-as-Service to other data consumers"

What is a Data Product and what defines a Data-Product-as-a-Service model?

Data products are a team sport: The data team partners with business operations to tackle specific problems using data.
Data products have an owner, support, SLA, and clear definition.
Data products have an SLA from the entire data domain team, not just the data engineer.
Data-product-as-a-service flow is bi-directional, from the domain data team to the company and back.
Domain expertise is blended directly into the data products themselves.
Data product team members have more business functional experience for their data products, and are responsible for providing insight as opposed to rows and columns.
Data product as a service fuses a service-oriented business partnership focusing on value with a product-oriented SLA focusing on reliability and data quality.

To successfully implement Industrial DataOps, it is essential to move from a conventional centralized data architecture into a domain data architecture (also referred to as a data mesh). This solves many of the challenges associated with centralized, monolithic data lakes and data warehouses. The goal becomes domain-based data as a service, not providing rows and columns of data.

For domain data architecture to work, the data product owner teams need to ensure their data is discoverable, trustworthy, self-describing, interoperable, secure, and governed by global access control. In other words, they need to manage their Data-Products-as-a-Service, not as data.

Learn more: Data management (r)evolution in the age of AI and the citizen data scientist →