Data Governance in the Cloud Era – Accelerating, Not Hindering, Data Democratization

Cloud technology can be empowering for end users within organizations of all sizes. But without effective data governance, what at first appears to be a trend toward greater freedom for end users and higher quality enterprise data will inadvertently become a trend in the exact opposite direction on both counts.

Cloud capabilities remove much of the friction that has historically plagued self-service data provisioning and analytics. With a few mouse clicks, end users can build even large-scale data products to support their own analytic needs. And with just a little extra technical skill, modern tools within the cloud data and analytic ecosystem make it easy to build all the supporting capabilities needed to load, transform, manage, and utilize data.   

Cloud technology can also offer relief to IT departments as end user self-service shrinks the traditionally extensive backlog of requests for data deployment projects. In the past, resourceful analysts would semi-secretly work around IT to avoid waiting in line, sometimes storing data in rogue database servers positioned under desks or in storage closets. Now this work is out in the open, even encouraged, as end users no longer need to acquire, configure, and store anything physical, other than the laptops and mobile devices they already use.

To enable resource sharing, modern data catalogs allow data consumers and producers to publicize, rate, and comment on assets within the data ecosystem, reducing the need for reinvention. For example, once someone learns how to acquire new telemetry data from equipment in the field for proactive maintenance, these processes and data structures can be extended to monitor the uptake of new product features. That’s the idea, anyway.

This “data democratization” is revolutionizing the way enterprises deploy and leverage data for all manner of business value and should be celebrated.

But it isn’t enough.

Without appropriate data governance – positioned as an enabler to self-service rather than a hindrance – this new age of empowerment will see organizations slide even deeper into a morass of inconsistent data, excessive rework, and slow projects. End users will be burdened with ever-growing data management responsibilities, distracting them from the real value-added work of analyzing data for business action.

It would be nice if advanced, AI-enabled tools and a “crowdsourced” approach to data management could lead to both coherent enterprise data and freedom to innovate, without the need for any central coordination. But that just isn’t the case. At least not yet. Extending existing data resources initially built for a single purpose requires more than adding compute and storage capacity. It requires conscious and professional attention to data structures and processes so that extensibility is built in. And the best way to ensure that end users build a coherent set of reusable and extensible data resources is to do it for them.   

Chief data officers (CDOs), and the data governance functions under their direction, must assert a proactive role to provide the end user community with the data and capabilities that will accelerate their work. That is, CDOs should shoulder the common, tedious, under-the-waterline data delivery and management work so that end users (and application development teams leveraging the same data) are free to focus on analyzing data and building business-facing solutions.

To plan and implement data governance as a helpful service, CDOs and other data management leaders must:
 

• Identify top-priority, funded business initiatives (sponsored by other CxOs) and the data needed within those initiatives
• Take direct responsibility, in partnership with IT and a federated network of data domain owners and stewards, to implement highly reusable and cross-functional data to support the initiatives – just-in-time, just-enough, and in just the right condition to meet the needs of the applications and analytic use cases within the initiatives
• Ensure that data resources are flexible and scalable, both within the infrastructure, enabled by cloud services, and the design, enabled by careful attention to architectural principles
• Work with end users and application developers to understand common data management struggles and address them within data delivery projects
• Establish “sandbox” environments with easy-to-use tools that support experimental data provisioning combined with easy access to authorized production data

In other words, data governance in the cloud era in many ways is just an opportunity to institutionalize the kind of discipline that should have been in place all along. Cloud technology has only increased the urgency along with the implications of getting it wrong. And if a healthy data governance program is already in place, it should be modernized and improved, not tossed aside.

As the CDO directly drives delivery of more and more shared data and capabilities, in a virtuous cycle of continuously improving quality, breadth, and trustworthiness, end users will have less and less of a need to do that work on their own. End users across the enterprise don’t acquire and manage the same data repeatedly because they want to. They do it because they believe they don’t have a choice.

Of course, good service from the CDO doesn’t absolve analysts and developers of their responsibilities to contribute to good governance. In addition to openly communicating what would best support their work, they must also agree to abide by regulations and company policy in the proper handling of data, whether experimental or production.  

So, as we rejoice about the freedom and flexibility enabled by modern cloud data and analytics technology, let’s not abandon our responsibilities beyond infrastructure and tools. Let’s establish a data governance partnership that accelerates self-service and consistently enhances foundational data assets at the same time.