Taming organizational data (and big data, in particular) for the coming year will involve firms helping themselves to the numerous opportunities the aforementioned approaches deliver for processing, analyzing, storing, and integrating big data. That much is clear.<\/p>\n\n\n\n
What\u2019s somewhat surprising is the end result of adroitly managing big data with these leading capabilities. The emergence of a data marketplace, facilitating a free flowing exchange of big data within and across organizations, is swiftly becoming a reality aided by composable data management and its technological underpinnings.<\/p>\n\n\n\n
\u201cIn certain cases it is not so much buying and selling; it\u2019s more about binding [data],\u201d Saptarshi Sengupta, Denodo<\/a> Director of Product Marketing, acknowledged about this trend. \u201cBut then, there are cases where it is buying and selling data.\u201d<\/p>\n\n\n\n The Data Marketplace<\/strong><\/p>\n\n\n\n The ascendance of a data marketplace, which typifies the consumerization of big data with parallels to other marketplaces such as Amazon\u2019s or Reuters\u2019 for financial companies, has been a longstanding ideal. It\u2019s finally coming to fruition because of the subsequently described approaches for data fabrics, data meshes<\/a>, data service layers, active metadata, and edge computing. At its finest, a data marketplace is an unexampled opportunity for monetizing data for like-minded consumers. \u201cIt\u2019s the enterprises: the Fortune 100, the Fortune 500,\u201d disclosed Purnima Kuchikulla, Privacera<\/a> Director of Customer Success. \u201cThese guys are already leading the data marketplace. They want to sell data. They\u2019re sellers; they\u2019re buyers; they buy and then want to sell it again.\u201d<\/p>\n\n\n\n Whether the exchange of data is for direct monetization purposes or for inter-departmental use cases between business units, the more data organizations have groomed for this purpose, the more advantageous it becomes. \u201cOn top of all this they\u2019re selling datasets; they\u2019re not selling one table of data,\u201d Kuchikulla specified. \u201cThey\u2019re selling it as a dataset that\u2019s part of a domain.\u201d As Sengupta posited, those dataset exchanges can also be between different domains in the same organization. He described a university system that\u2019s implemented a \u201cdecision support system\u201d in which the school \u201chas a bunch of different campuses and from those campuses there\u2019s faculty members, staff, students, and everybody\u2019s looking at data. That data can be about books, libraries, course offerings, registration, enrollment, etc. It\u2019s more like a data consumption model through a particular website or portal.\u201d<\/p>\n\n\n\n The Data Mesh<\/strong><\/p>\n\n\n\n Conceptually, a data mesh is an architectural approach that is both similar and assistive to an enterprise data fabric, which Gartner termed the top strategic trend for 2022<\/a>. The latter is a holistic means of connecting all data throughout an organization, regardless of its location, so they\u2019re accessible on demand. Despite the sundry of implementation approaches, several competencies have emerged for defining a data fabric. \u201cThere\u2019s a data catalog competence, an active metadata competence, the semantic layer, all the data integration materials, data preparation, etcetera,\u201d Sengupta enumerated.<\/p>\n\n\n\n A data mesh builds on this distributed architectural approach by including domain specific information about data\u2019s creation, storage, and cataloging so it\u2019s applicable to users across domains. \u201cIt gives you some level of persistency and storage of where your data can reside, but it\u2019s not set in stone,\u201d explained Calyptia<\/a> Co-Founder Anurag Gupta. The domain specific attributes of data meshes address semantic differences for cross-departmental use while provisioning governance measures for exposing data. Meshes are frequently overseen by centralized teams. According to Gupta, \u201cA mesh is almost representative of your central nervous system where all your data is sitting in this actionable manner ready to be sent to various end destinations.\u201d<\/p>\n\n\n\n The Data Service Layer<\/strong><\/p>\n\n\n\n Having decentralized data assets uniformly connected and controlled for delivery to multiple locations (and users) is arguably the definition of a data marketplace. Nonetheless, this paradigm, nor that for data fabrics and data meshes, wouldn\u2019t work without what Commit<\/a> Chief Customer Officer Nathan Cayzer called a \u201cservice layer\u201d. With an obvious allusion to the cloud\u2019s Service Oriented Architecture<\/a>, real-time service layers are instrumental for delivering data to end users within and across organizations. \u201cA real time serving layer allows you to materialize responses in real time or close to real time to the end user,\u201d Cayzer mentioned. Such service layers either support, or are in turn supported by, the following data management constructs:<\/p>\n\n\n\n Active Metadata<\/strong><\/p>\n\n\n\n Gartner has embraced the notion of inverting metadata\u2019s value from passive data lineage deployments to low-latent action<\/a> in production settings. In some instances, this functionality entails organizations \u201cusing metadata to do some sort of AI or ML,\u201d Sengupta remarked. \u201cYou basically look into your metadata and your log files and turn it into AI and machine learning so you can recommend what type of activities will come out of that.\u201d <\/p>\n\n\n\n Sometimes doing so involves determining the best way to integrate data. On other occasions, this capability includes \u201cdynamic tagging that represents metadata for how data flows from an edge device to, say, your data mesh,\u201d Gupta denoted. \u201cThis metadata is vital because it can represent important factors like a team and what team owns what slice of data. With privacy concerns growing, you want to make sure that slice of data is under the proper compliance and governance.\u201d<\/p>\n\n\n\n Edge Infrastructure<\/strong><\/p>\n\n\n\n The ability to readily exchange low latent data at the cloud\u2019s edge (like weather data, traffic updates, or manufacturing developments) within a data marketplace broadens its enterprise worth. Doing so hinges on \u201cbringing compute and storage infrastructure to the edge to enable the infrastructure for the post-cloud world,\u201d Cloudian<\/a> CTO Gary Ogasawara specified. Although edge deployments<\/a> typically transmit some data to centralized clouds, growing use cases for this architectural model include:<\/p>\n\n\n\n Composability<\/strong><\/p>\n\n\n\n Developments in a data mesh, data service layer, active metadata, and edge computing enhance big data management with granular controls for disseminating data, on request, in real-time. Sometimes that delivery encompasses selling data within the data marketplace, a concept that\u2019s expansive enough to include exchanging data between departments for timely action, too. As far as their interrelation, however, these developments are derived from the composability tenet at the foundation of adaptable business resilience<\/a>\u2014and capitalization\u2014for the years to come.<\/p>\n\n\n\n Composability is a modular approach to designing the foregoing inputs because \u201corganizations are realizing it is unrealistic to have a single enterprise standard for data and analytics,\u201d reflected Franz<\/a> CEO Jans Aasman. \u201cIn 2022 and beyond, companies will embrace a lego-like approach to analytics and AI solutions where\u2026 [they\u2019re] used in multiple, different applications to connect data insights to business actions across the enterprise.\u201d<\/p>\n\n\n\n About the Author<\/strong><\/p>\n\n\n\n Jelani Harper is an editorial consultant servicing the information technology market. He specializes in data-driven applications focused on semantic technologies, data governance and analytics.<\/em><\/p>\n\n\n\n Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n![\"\"](\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/10\/Jelani-Harper.jpg\")
<\/figure><\/div>\n\n\n\n