Introduction to agile data governance
Recently, it has become increasingly hard to attend a meeting without hearing the expression data governance. However, this subject is nothing new! Even so, with the arrival of Big Data technologies: data and its use become the cornerstone of approaches to innovation. An old subject evolving in a very new context.
Digital transformation is happening at a daunting scale and speed, and no business or industry is immune. With data at the core of digital transformation, scalable and sustainable governance is more important than ever. Your data is always changing, and your data governance policy needs to change, too. If it can’t, it’s not agile, and if it’s not agile, it won’t work in today’s IT environment.
In today’s big-data-fast reality, many companies have hundreds of thousands of data sources, potentially millions of different data sets, and an increasing number of self-service users consuming that information broadly. Traditional top-down, workflow-driven data governance can’t keep up. What’s needed is a new approach to governance that’s agile, scalable, and takes advantage of machine learning to meet the requirements of data-driven businesses, such as with DQLabs.ai.
Agile data governance is oriented to provide support much closer to the point where data is used, focusing on self-service analytics. It is supported by tools that assist deliver knowledge about the data to the data users.
What is Agile Data Governance?
Agile Data Governance is creating and improving data assets by iteratively capturing knowledge as consumers and data producers work together so that everyone can benefit. It takes up the proven best practices of Agile and Open software development to data and analytics.
Agile Data Governance begins by identifying a business problem, then gathering stakeholders who know about the problem and are trying (or have tried) to solve it.
Importance of Agile Data Governance
These are bottlenecks at the data producer threshold. Data consumers can’t keep up while servicing one impromptu request for data after another. Consumers become disappointed with the delay in getting what they need. Analytics projects turn into endless email chains. Using agile principles, data consumers, data producers and domain experts iterate together to build reusable assets that lower the frequency of ad-hoc requests. New impromptu requests will be preserved next to the cataloged data assets and analyzed for the other person to find and use before asking data producers for help.
Data silos and rogue databases
Everyone has encountered that “one person” who gate keeps a particular dataset and is the only one who can create a necessary report. Maybe this person built a one-off system to give some analytics or scripts that only run on their laptop. Agile Data Governance helps data consumers have a direct, clear way to get and iterate on data assets. This reduces the occurrence of “emailed spreadsheets.” Plus, information assets will be well-documented, so more users can get, understand, and use them.
Data obscurity and lack of understanding
In many organizations, those who try to understand the availability and use of data assets encounter partial answers, inefficiencies and confusing systems. This is primarily a documentation issue, and disconnected tools that aren’t built for agile processes make documentation both a chore and an afterthought. Agile Data Governance helps you do the documentation while doing the work. This near-real-time documentation increases global knowledge about what data exists, what it means, and how to use it.
When data work is not transparent, people will not trust it. People come up with various versions of the same analysis after months of work. They argue about data sources, small details, even project goals. With Agile Data Governance, transparency means correction and peer review happen as the analysis unfolds. This creates a shared understanding which can pour into business glossaries and other alignment tools.
You cannot have a data-driven culture if your people don’t understand the simple workings of statistics. They need to appreciate and have easy ways to follow the scientific method and other best practices that make analytics valid and useful. This may be the biggest long-term benefit of practicing Agile Data Governance. Humans learn by copying and doing. An agile process encourages participation and observation of talented people doing great work. This improves data literacy and skill across your entire organization.
Want to learn how DQLabs’ agile data governance initiatives work? Try it free for 7 days.
The same as how software developments have slowly shifted away from traditional methods (V-model, Waterfall, etc.) to agile methods, must think about data governance. Such a perspective is not only iterative but also applied gradually to your data governance strategy allowing greater flexibility, necessary to consider the ever-increasing complexity of your IS.