The top four influences accelerating change are focused now on small and wide data instead of Big Data resting in silos or data lakes, and data fabric was described as necessary architecture to support data analytics.
Gartner listed the need for smarter, more responsible, scalable AI as the number one trend. And as more is demanded of AI systems, it explained, the greater the need will be to scale the technologies supporting AI. That will mean the ability to operate with “less data via ‘small data’ techniques and adaptive machine learning.” Both of these will require “composable data and analytics” (trend number two). By composable, it means “to use components from multiple data, analytics and AI solutions for a flexible, user-friendly and usable experience that will enable leaders to connect data insights to business actions.”
Trend two leads directly to trend three: data fabric as the foundation. This is the architecture needed to support the new composable data and analytics and their components. Gartner claims that data fabric could reduce the time for integration design by 30%, deployment by 30%, and maintenance by a significant 70%.
And finally, trend number four calls for a shift from Big Data to small and wide data. Instead of relying on warehouses of Big Data, small and wide data solves “increasingly complex questions on AI and challenges with scarce data use cases…. Small data, as the name implies, is able to use data models that require less data but still offer useful insights.”
Gartner predicts these four trends will become significantly disruptive over the span of the next three to five years.
DATA FABRICS
Thankfully, innovators have recently refrained from acronyms and dense technical phrasings in favor of metaphorical names for their efforts. The word “fabric” is a clue and also a way to remember what a data fabric is and does.
Gartner defines data fabric as “a design concept that serves as an integrated layer (fabric) of data and connecting processes. A data fabric utilizes continuous analytics over existing, discoverable and inferenced metadata assets to support the design, deployment and utilization of integrated and reusable data across all environments, including hybrid and multi-cloud platforms.” A weave of many threads then creates an interwoven warp and weft of data and tools.
Two of the advantages of the design are agility and ease of use, even for those who aren’t data scientists. Gartner offers an example very relevant to current supply line problems due to external forces like COVID-19. “A supply chain leader using data fabric can add newly encountered data assets to known relationships between supplier delays and production delays more rapidly, and improve decisions with the new data (or for new suppliers or new customers).” According to Ehtisham Zaidi, Gartner’s senior director analyst, data fabric “allows less technical users and subject matter experts to find and integrate data themselves without relying on expert data engineers and data system experts.”
Using a comparison with self-driving cars, Gartner describes two working scenarios for data fabrics. The first is a parallel to the autonomous car that runs with a fully attentive driver and then autonomously with a minimum or no intervention by the driver. Data fabric works in both modes. “It monitors the data pipelines as a passive observer at first and starts suggesting alternatives that are far more productive. When both the data ‘driver’ and the machine-learning are comfortable with repeated scenarios, they complement each other by automating improvisational tasks (that consume too many manual hours), while leaving the leadership free to focus on innovation.”
InformationWeek adds a different perspective on the value of data fabrics in “Why You Need a Data Fabric, Not Just IT Architecture.” It’s needed because it completes a necessary set of controls. “Data fabrics offer an opportunity to track, monitor and utilize data, while IT architectures track monitor and maintain IT assets. Both are needed for a long-term digitalization strategy.”
The data fabric market in the United States is estimated at $424.9 million this year, and the global data fabric market is expected to reach about $3.7 billion by 2026. If your AI needs are expanding, it’s definitely worth a look.
ADDITIONAL RESOURCES
Gartner’s white paper Data Fabrics Modernize Data Integration is available online. Hitachi Vantara offers the e-book Data Fabric for Dummies for download. It includes chapters on how to effectively build and deploy a data fabric, how to optimize data access and support compliance, and how to modernize a data fabric with DataOps.
October 2021
