Data quality is the most important requirement for using AI effectively. As such, data governance is necessary for an organization to use AI, both in day-to-day operations and to assist in analytics. But what about the operations and analytics of data governance itself? Data governance is a business function like finance, supply chain, or marketing and can benefit from AI much in the same way as these other business functions.
AI can help small businesses receive the same benefits that large enterprises do. Larger companies have people dedicated to data governance, whereas in a smaller organization data governance is a shared responsibility across teams with support provided by data and technical specialists who also have other responsibilities.
WHAT IS AI?
Providing practical examples of how AI can help data governance is difficult due to the “AI Effect.” The AI Effect refers to the occasion when an achievement once considered AI is no longer identified as AI. In other words, once that AI capability is widely adapted or integrated, it no longer seems particularly intelligent—artificial or otherwise.
Because of the AI Effect, we won’t worry too much about the ever-changing definition of “AI.” Instead, we will consider AI as a way to augment human intelligence and make basic decisions using a computer program and then move on to how we can use AI to help data governance efforts.
WHO CAN AI HELP?
Though the scope of AI continues to expand, humans are often the ones carrying out data governance and defining policies; AI merely assists. First among those responsible for data governance are data stewards (or data domain stewards), generally responsible for defining data best practices and identifying data sets.
Next are data custodians (or data governance engineers), who are typically responsible for the functional aspects of data governance such as data connections, data transformations, and data access security. From a business perspective, these individuals are responsible for data security and measuring compliance.
Finally, there are departmental leaders who are accountable for the success of the organization’s data governance program. They are often organized into a data governance council, which must be informed of all activity related to the quality and security of data such as security breaches, failed data processes, data processing time, calculation errors, duplicate marketing, do-not-call list violations, and shipping errors.
HOW DOES AI HELP?
Given the quantity and complexity of tasks associated with data governance, it isn’t surprising that successful data governance has historically been possible only for large organizations, but with AI enhancing the capability of human resources, smaller businesses are able to benefit from data governance as well.
AI is democratizing data governance. For example, process mining or automated process discovery uses AI to examine user behavioral data created during data processing. Behavioral data becomes digital records.
AI can determine which processes may be more efficient and which processes may have vulnerabilities. The AI informs the data stewards so that they can better define which practices are best and which need improvement.
IDENTIFYING DATA SETS
There is a cost associated with adding new data to a data warehouse or data lake. The new data requires standards, rules, definitions, and more. Once added, the new data must also be maintained and updated. AI data discovery (or automated data discovery) uses an automated process to trawl internal and external data sets and evaluate if the data sets have some relevance to your organization’s target performance metrics.
For example, an organization may have several social media profiles, dozens of websites with online reviews, an internal customer database, call center logs, manufacturing specifications, and email messages between employees, as well as external data like industry sentiment, unemployment, inflation, regional demographics, etc. These factors could influence its sales rate, cost of goods sold, cost of talent acquisition, employee theft rate, and so on.
Effective data governance calls for an understanding of the cost vs. benefit of adding new data to the environment. But how do we determine the value of a given data set if we haven’t yet added it to our environment? Because AI is capable of exploring any internal and external data sets and evaluating if that data may be related to the business performance metrics, AI can give us a sense of which data sets can better inform our decisions. This allows data stewards to prioritize data set evaluation.
INFORMING RELEVANT PARTIES
Determining how data governance performance and compliance are measured can be just as difficult as implementing data governance in the first place. Although some metrics, such as shipping error rate or percent data elements defined, may be easily identified, business performance metrics may be more difficult.
Performance metrics are focused more on revenue generation and/or cost savings. Data governance may reduce customer service call time or enhance marketing effectiveness, but the direct relationship between increased data governance compliance and the associated impact of data governance compliance on overall business performance can be difficult to isolate in traditional reporting.
Through AI-assisted reporting, the data governance compliance metrics can continuously be compared to business performance metrics, and otherwise complicated relationships can be identified and communicated to the stakeholders accountable for data governance performance.
Even simple metrics may have complex patterns both between multiple metrics and within the same metric over time. AI focusing on pattern recognition and natural language generation can identify such complex patterns and communicate how the pattern is meaningful (or at least that a pattern exists) in simple text format, which is used as summary text within reports to stakeholders.
In today’s data-driven economy, high-quality data provides a competitive advantage. Quality data enables data solutions to lower cost and predictive analytics to better respond to customer needs. Data governance is how businesses keep and maintain high-quality data. Without data governance, small businesses would be at a competitive disadvantage to large organizations with data governance.
Data governance isn’t easy for any size business, but, with AI assistance, data governance activities like best practice determination, data set identification, and operational reporting are all possible without dedicated full-time staff, giving smaller businesses the same data-quality competitive advantage seen by only the largest companies.
September 2019
