{"id":33702,"date":"2023-10-21T03:00:00","date_gmt":"2023-10-21T10:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=33702"},"modified":"2023-10-18T10:37:57","modified_gmt":"2023-10-18T17:37:57","slug":"research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/","title":{"rendered":"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity"},"content":{"rendered":"\n

Dremio<\/a>, the easy and open data lakehouse, has published “The Data Lakehouse: Data Warehousing and More,” a novel research paper now available on\u00a0arXiv<\/a>. The paper explores the data lakehouse model, offering modern insights for businesses looking to optimize their data utilization. The idea through this preprint publication is to gather feedback from the open source research and scientific community and make it available to the wider community of practitioners.<\/p>\n\n\n\n

The paper decomposes commonly used but overloaded terms like data warehouse, data warehousing, and data lakehouse into discrete components (such as query engine, table format, etc.), then offers clear terms and definitions based on these components to bring clarity to communication that uses these common terms.<\/p>\n\n\n\n

Data warehousing has long been a cornerstone of modern data-driven organizations, serving as a strategic asset for informed decision-making. However, the emergence of data lakehouses has challenged the traditional paradigms by providing a new approach to achieving the goals of data warehousing, while overcoming its limitations and adding new dimensions of capability.<\/p>\n\n\n\n

The paper begins by rigorously defining the often-ambiguous terms “data warehousing,” “data warehouse,” and “data lakehouse.”<\/p>\n\n\n\n

“We have seen some folks in the market say that \u2018data lakehouse\u2019 is just another marketing buzzword. We understood the arguments on both sides, but whether the statement was right or not was essentially rooted in how you interpret certain terms. We wanted to provide some clarity and, using an approach similar to a math-based proof, show that with clarity on definitions, \u2018data lakehouse\u2019 is definitely more than a marketing term. Rather, it\u2019s a practical and valuable approach to data warehousing,\u201d said Jason Hughes, director of technical advocacy.<\/em><\/p>\n\n\n\n

The paper breaks down what is commonly referred to as \u201cdata warehousing\u201d into its fundamental requirements, categorizing them into technical components, technical capabilities, and technology-independent practices. It then shows how a data lakehouse addresses all of these core requirements, therefore demonstrating that a data lakehouse can be used to achieve what is traditionally thought to require an RDBMS-OLAP. It also highlights the shortcomings of traditional data warehousing on RDBMS-OLAP, including limitations with semi-structured and unstructured data, lock-in and lock-out, and cost issues, prompting a reevaluation of architectural approaches. Additionally, the paper provides a concrete example of data lakehouse implementation to demonstrate its practical benefits.<\/p>\n\n\n\n

The ultimate goal of a data lakehouse is to combine the strengths of RDBMS-OLAP data warehousing and data lakes, fulfilling data warehousing requirements on open data architecture and expanding to additional analytic capabilities.<\/p>\n\n\n\n

Dremio’s research paper solidifies the concept of data lakehouses and provides a practical roadmap for organizations looking to harness the full potential of their data while optimizing their data architecture.<\/p>\n\n\n\n

To read the full paper please visit:\u00a0https:\/\/arxiv.org\/abs\/2310.08697<\/a><\/p>\n\n\n\n

Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

Join us on Twitter: https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n\n\n\n

Join us on LinkedIn: https:\/\/www.linkedin.com\/company\/insidebigdata\/<\/a><\/em><\/p>\n\n\n\n

Join us on Facebook: https:\/\/www.facebook.com\/insideBIGDATANOW<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Dremio, the easy and open data lakehouse, has published “The Data Lakehouse: Data Warehousing and More,” a novel research paper now available on\u00a0arXiv. The paper explores the data lakehouse model, offering modern insights for businesses looking to optimize their data utilization. The idea through this preprint publication is to gather feedback from the open source research and scientific community and make it available to the wider community of practitioners.<\/p>\n","protected":false},"author":10513,"featured_media":33572,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[115,1054,180,268,56,84,1303,1],"tags":[1108,96],"acf":[],"yoast_head":"\nResearch Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"Dremio, the easy and open data lakehouse, has published "The Data Lakehouse: Data Warehousing and More," a novel research paper now available on\u00a0arXiv. The paper explores the data lakehouse model, offering modern insights for businesses looking to optimize their data utilization. The idea through this preprint publication is to gather feedback from the open source research and scientific community and make it available to the wider community of practitioners.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2023-10-21T10:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-10-18T17:37:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/10\/Data_Pipeline_shutterstock_9623992_special.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1100\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/\",\"url\":\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/\",\"name\":\"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2023-10-21T10:00:00+00:00\",\"dateModified\":\"2023-10-18T17:37:57+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/","og_locale":"en_US","og_type":"article","og_title":"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity - insideBIGDATA","og_description":"Dremio, the easy and open data lakehouse, has published \"The Data Lakehouse: Data Warehousing and More,\" a novel research paper now available on\u00a0arXiv. The paper explores the data lakehouse model, offering modern insights for businesses looking to optimize their data utilization. The idea through this preprint publication is to gather feedback from the open source research and scientific community and make it available to the wider community of practitioners.","og_url":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2023-10-21T10:00:00+00:00","article_modified_time":"2023-10-18T17:37:57+00:00","og_image":[{"width":1100,"height":550,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/10\/Data_Pipeline_shutterstock_9623992_special.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/","url":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/","name":"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2023-10-21T10:00:00+00:00","dateModified":"2023-10-18T17:37:57+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2023\/10\/21\/research-highlights-dremio-demonstrates-data-lakehouse-value-with-math-style-proof-and-technical-clarity\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Research Highlights: Dremio Demonstrates Data Lakehouse Value with Math-Style Proof and Technical Clarity"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/10\/Data_Pipeline_shutterstock_9623992_special.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-8LA","jetpack-related-posts":[{"id":28606,"url":"https:\/\/insidebigdata.com\/2022\/03\/02\/dremio-announces-open-and-forever-free-lakehouse-platform-dremio-cloud-in-partnership-with-aws\/","url_meta":{"origin":33702,"position":0},"title":"Dremio Announces Open and Forever-Free Lakehouse Platform, Dremio Cloud, In Partnership with AWS","date":"March 2, 2022","format":false,"excerpt":"Dremio, the lakehouse company, announced the general availability of Dremio Cloud, the free data lakehouse platform and the addition of two new services: Dremio Sonar, a lakehouse engine built for SQL, and Dremio Arctic, a metadata and data management service for Apache Iceberg that provides a unique Git-like experience for\u2026","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/10\/data-lakes_SHUTTERSTOCK.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":32724,"url":"https:\/\/insidebigdata.com\/2023\/06\/27\/cloudera-expands-open-data-lakehouse-for-trusted-enterprise-ai\/","url_meta":{"origin":33702,"position":1},"title":"Cloudera Expands Open Data Lakehouse for Trusted Enterprise AI\u00a0","date":"June 27, 2023","format":false,"excerpt":"Cloudera, the hybrid data company, announced today an expansion of its Open Data Lakehouse offerings enabling customers to have a foundation for analytic and AI capabilities in their enterprises for all their data - in the cloud and now on-premises. Cloudera was an early proponent of Apache Iceberg,\u00a0introducing support in\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/Industry_Perspectives_shutterstock_1127578655_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":28258,"url":"https:\/\/insidebigdata.com\/2022\/01\/17\/databricks-launches-data-lakehouse-for-retail-and-consumer-goods-customers\/","url_meta":{"origin":33702,"position":2},"title":"Databricks Launches Data Lakehouse for Retail and Consumer Goods Customers","date":"January 17, 2022","format":false,"excerpt":"Databricks, the Data and AI company and pioneer of the data lakehouse architecture, announced the Databricks Lakehouse for Retail, the company\u2019s first industry-specific data lakehouse for retailers and consumer goods (CG) customers. With Databricks\u2019 Lakehouse for Retail, data teams are enabled with a centralized data and AI platform that is\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":30797,"url":"https:\/\/insidebigdata.com\/2022\/11\/03\/video-highlights-modernize-your-ibm-mainframe-netezza-with-databricks-lakehouse\/","url_meta":{"origin":33702,"position":3},"title":"Video Highlights: Modernize your IBM Mainframe & Netezza With Databricks Lakehouse","date":"November 3, 2022","format":false,"excerpt":"In the video presentation below, learn from experts how to architect modern data pipelines to consolidate data from multiple IBM data sources into Databricks Lakehouse, using the state-of-the-art replication technique\u2014Change Data Capture (CDC).","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/10\/data-lakes_SHUTTERSTOCK.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":31820,"url":"https:\/\/insidebigdata.com\/2023\/03\/07\/databricks-launches-simplified-real-time-machine-learning-for-the-lakehouse\/","url_meta":{"origin":33702,"position":4},"title":"Databricks Launches Simplified Real-Time Machine Learning for the Lakehouse","date":"March 7, 2023","format":false,"excerpt":"Databricks, the lakehouse company, announced the launch of Databricks Model Serving to provide simplified production machine learning (ML) natively within the Databricks Lakehouse Platform. Model Serving removes the complexity of building and maintaining complicated infrastructure for intelligent applications. Now, organizations can leverage the Databricks Lakehouse Platform to integrate real-time machine\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":30114,"url":"https:\/\/insidebigdata.com\/2022\/08\/17\/cloudera-launches-all-in-one-data-lakehouse-cloud-service\/","url_meta":{"origin":33702,"position":5},"title":"Cloudera Launches All-in-One Data Lakehouse Cloud Service","date":"August 17, 2022","format":false,"excerpt":"Cloudera, the hybrid data company, announced the launch of Cloudera Data Platform (CDP) One, an all-in-one data lakehouse software as a service (SaaS) offering that enables fast and easy self-service analytics and exploratory data science on any type of data. A simple yet powerful cloud service, only CDP One has\u2026","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/33702"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=33702"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/33702\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/33572"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=33702"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=33702"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=33702"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}