{"id":14853,"date":"2016-04-19T06:00:37","date_gmt":"2016-04-19T13:00:37","guid":{"rendered":"http:\/\/insidebigdata.com\/?p=14853"},"modified":"2016-06-13T15:44:21","modified_gmt":"2016-06-13T22:44:21","slug":"the-lambda-architecture-simplified","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/","title":{"rendered":"The Lambda Architecture Simplified"},"content":{"rendered":"<p><a href=\"http:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch.png\" rel=\"attachment wp-att-14854\"><img decoding=\"async\" loading=\"lazy\" class=\"alignright size-full wp-image-14854\" src=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch.png\" alt=\"Lambda_Arch\" width=\"232\" height=\"299\" \/><\/a>In this special technology white paper, <a href=\"http:\/\/insidebigdata.com\/white-paper\/memsql-lambda-architecture\/\" target=\"_blank\">The Lambda Architecture Simplified<\/a>, you\u2019ll learn about how the Lambda Architecture aims to satisfy the needs for a robust system that is fault-tolerant, both against hardware failures and human mistakes, being able to serve a wide range of workloads and use cases, and in which low-latency reads and updates are required.<\/p>\n<p>In his book, <span style=\"text-decoration: underline;\">Big Data: Principles and Best Practices of Scalable Real-time Data Systems<\/span>, Nathan Marz coined the\u00a0 term <em>Lambda Architecture<\/em> to describe a generic, scalable and fault-tolerant data processing architecture based on his experience in working on distributed systems at Backtype and Twitter.<\/p>\n<p>The paper explores what The Lambda Architecture is and what it isn\u2019t. What it is, is a sensible set of data engineering practices, which you should be applying anyway, plus a clever (but transitional) double-write approach to add a low-latency fast track to existing big data systems. Throughout the white paper, you&#8217;ll find details of the technologies and data processing requirements that will help you implement a simplified Lambda Architecture. The white paper includes the following high level topics:<\/p>\n<ul>\n<li>A Brief History of the Lambda Architecture<\/li>\n<li>Rethinking the Lambda Architecture<\/li>\n<li>Lambda in Production<\/li>\n<\/ul>\n<p><a href=\"http:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch_diagram.png\" rel=\"attachment wp-att-14856\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-14856\" src=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch_diagram.png\" alt=\"Lambda_Arch_diagram\" width=\"813\" height=\"462\" srcset=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch_diagram.png 813w, https:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch_diagram-300x170.png 300w, https:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch_diagram-768x436.png 768w\" sizes=\"(max-width: 813px) 100vw, 813px\" \/><\/a>The pace of data is not slowing. Applications of today are built with infinite data sets in mind. As these real-time applications become the norm, and\u00a0 batch processing becomes a relic of the past, digital enterprises will implement memory-optimized, distributed data systems to simplify Lambda\u00a0 Architectures for real-time data processing and exploration.<\/p>\n<p>The <em>Lambda Architecture Simplified\u00a0<\/em> white paper is\u00a0available\u00a0for\u00a0download in PDF from the<a href=\"http:\/\/insidebigdata.com\/white-paper\/memsql-lambda-architecture\/\" target=\"_blank\">\u00a0insideBIGDATA White Paper Library<\/a>, courtesy of <a href=\"http:\/\/www.memsql.com\/\" target=\"_blank\">MemSQL<\/a>. <a href=\"http:\/\/www.brightcomputing.com\/\" target=\"_blank\"><br \/>\n<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this special technology white paper, The Lambda Architecture Simplified, you\u2019ll learn about how the Lambda Architecture aims to satisfy the needs for a robust system that is fault-tolerant, both against hardware failures and human mistakes, being able to serve a wide range of workloads and use cases, and in which low-latency reads and updates are required.<\/p>\n","protected":false},"author":37,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[115,180,109,183,268,56,84,1,58],"tags":[329,95],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>The Lambda Architecture Simplified - insideBIGDATA<\/title>\n<meta name=\"description\" content=\"The Lambda Architecture Simplified\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Lambda Architecture Simplified - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"The Lambda Architecture Simplified\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2016-04-19T13:00:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-06-13T22:44:21+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch.png\" \/>\n<meta name=\"author\" content=\"Daniel Gutierrez\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AMULETAnalytics\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Gutierrez\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/\",\"url\":\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/\",\"name\":\"The Lambda Architecture Simplified - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2016-04-19T13:00:37+00:00\",\"dateModified\":\"2016-06-13T22:44:21+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed\"},\"description\":\"The Lambda Architecture Simplified\",\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Lambda Architecture Simplified\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning &amp; Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed\",\"name\":\"Daniel Gutierrez\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g\",\"caption\":\"Daniel Gutierrez\"},\"description\":\"Daniel D. Gutierrez is a Data Scientist with Los Angeles-based AMULET Analytics, a service division of AMULET Development Corp. He's been involved with data science and Big Data long before it came in vogue, so imagine his delight when the Harvard Business Review recently deemed \\\"data scientist\\\" as the sexiest profession for the 21st century. Previously, he taught computer science and database classes at UCLA Extension for over 15 years, and authored three computer industry books on database technology. He also served as technical editor, columnist and writer at a major computer industry monthly publication for 7 years. Follow his data science musings at @AMULETAnalytics.\",\"sameAs\":[\"http:\/\/www.insidebigdata.com\",\"https:\/\/twitter.com\/@AMULETAnalytics\"],\"url\":\"https:\/\/insidebigdata.com\/author\/dangutierrez\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Lambda Architecture Simplified - insideBIGDATA","description":"The Lambda Architecture Simplified","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/","og_locale":"en_US","og_type":"article","og_title":"The Lambda Architecture Simplified - insideBIGDATA","og_description":"The Lambda Architecture Simplified","og_url":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2016-04-19T13:00:37+00:00","article_modified_time":"2016-06-13T22:44:21+00:00","og_image":[{"url":"http:\/\/insidebigdata.com\/wp-content\/uploads\/2016\/04\/Lambda_Arch.png"}],"author":"Daniel Gutierrez","twitter_card":"summary_large_image","twitter_creator":"@AMULETAnalytics","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Daniel Gutierrez","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/","url":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/","name":"The Lambda Architecture Simplified - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2016-04-19T13:00:37+00:00","dateModified":"2016-06-13T22:44:21+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed"},"description":"The Lambda Architecture Simplified","breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2016\/04\/19\/the-lambda-architecture-simplified\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"The Lambda Architecture Simplified"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning &amp; Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed","name":"Daniel Gutierrez","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g","caption":"Daniel Gutierrez"},"description":"Daniel D. Gutierrez is a Data Scientist with Los Angeles-based AMULET Analytics, a service division of AMULET Development Corp. He's been involved with data science and Big Data long before it came in vogue, so imagine his delight when the Harvard Business Review recently deemed \"data scientist\" as the sexiest profession for the 21st century. Previously, he taught computer science and database classes at UCLA Extension for over 15 years, and authored three computer industry books on database technology. He also served as technical editor, columnist and writer at a major computer industry monthly publication for 7 years. Follow his data science musings at @AMULETAnalytics.","sameAs":["http:\/\/www.insidebigdata.com","https:\/\/twitter.com\/@AMULETAnalytics"],"url":"https:\/\/insidebigdata.com\/author\/dangutierrez\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-3Rz","jetpack-related-posts":[{"id":16468,"url":"https:\/\/insidebigdata.com\/2016\/11\/15\/high-performance-data-architectures-for-the-internet-of-things\/","url_meta":{"origin":14853,"position":0},"title":"High-Performance Data Architectures for the Internet of Things","date":"November 15, 2016","format":false,"excerpt":"In this special technology white paper, High-Performance Data Architectures for the Internet of Things, you\u2019ll discover how Lambda architectures can provide the capabilities required to create hybrid transactional\/analytical platforms (HTAP) that are capable of providing real-time insights into the massive amount of streaming data to power useful actions at the\u2026","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"IoT Reference Architecture with GridGain","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2016\/11\/Gridgain_IoT.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":33719,"url":"https:\/\/insidebigdata.com\/2023\/10\/23\/lambda-and-vast-data-partner-to-accelerate-ai-training-across-public-and-private-cloud-leveraging-nvidia-technology\/","url_meta":{"origin":14853,"position":1},"title":"Lambda and VAST Data Partner to Accelerate AI Training Across Public and Private Cloud, Leveraging NVIDIA Technology\u00a0","date":"October 23, 2023","format":false,"excerpt":"VAST Data, the AI data platform company and Lambda, a leading Infrastructure-as-a-Service and compute provider for public and private GPU infrastructure, today announced a strategic partnership that will enable the world's first hybrid cloud experience dedicated to AI and deep learning workloads. Together, Lambda and VAST are building an NVIDIA\u2026","rel":"","context":"In &quot;AI Deep Learning&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/09\/AI_data_storage_shutterstock_1107715973_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":13241,"url":"https:\/\/insidebigdata.com\/2015\/06\/15\/the-lambda-dilemma-uncovering-the-layers-of-complexity-to-achieve-your-fast-data-goals\/","url_meta":{"origin":14853,"position":2},"title":"The Lambda Dilemma: Uncovering the Layers of Complexity to Achieve your Fast Data Goals","date":"June 15, 2015","format":false,"excerpt":"This Hadoop 101 article on the Lambda Architecture was contributed by John Hugg. John is the Founding Software Engineer for VoltDB.","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25036,"url":"https:\/\/insidebigdata.com\/2020\/09\/24\/the-future-outlook-of-serverless\/","url_meta":{"origin":14853,"position":3},"title":"The Future Outlook of Serverless","date":"September 24, 2020","format":false,"excerpt":"In this special guest feature, Emrah Samdan, Vice President of Products for Thundra, takes a look at the future outlook of serverless technology. The future of serverless and the production readiness of serverless for many use cases will continue to improve and the potential to cover many others is arriving.\u2026","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2020\/09\/emrahsamdan_thundra.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":20423,"url":"https:\/\/insidebigdata.com\/2018\/05\/17\/accelerate-business-value-serverless-computing\/","url_meta":{"origin":14853,"position":4},"title":"Accelerate Business Value with Serverless Computing","date":"May 17, 2018","format":false,"excerpt":"In order to answer questions about serverless computing, Continuous Data Platform Iguazio offers a whitepaper \"Accelerate Business Value with Serverless Computing\" that weighs the pros and cons of function-as-a-service, or serverless computing in the cloud. The report defines exactly what serverless means (newsflash: it\u2019s a misnomer), examining the offerings and\u2026","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2018\/05\/Iguazio_arch.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":15100,"url":"https:\/\/insidebigdata.com\/2016\/05\/29\/sumo-logic-enables-devops-with-deep-data-insights-into-aws-lambda\/","url_meta":{"origin":14853,"position":5},"title":"Sumo Logic Enables DevOps with Deep Data Insights into AWS Lambda","date":"May 29, 2016","format":false,"excerpt":"Sumo Logic, a leading cloud-native, machine data analytics service, announced the availability of its Sumo Logic App for Amazon Web Services (AWS) Lambda allowing DevOps and developers to monitor the health of all AWS Lambda functions and better manage the associated costs.","rel":"","context":"In &quot;Analytics&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/14853"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=14853"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/14853\/revisions"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=14853"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=14853"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=14853"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}