{"id":31488,"date":"2023-01-28T06:00:00","date_gmt":"2023-01-28T14:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=31488"},"modified":"2023-01-25T17:43:48","modified_gmt":"2023-01-26T01:43:48","slug":"deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/","title":{"rendered":"Deci delivers breakthrough inference performance on Intel’s 4th Gen Sapphire Rapids CPU"},"content":{"rendered":"
\n
\"\"<\/figure><\/div>\n\n\n

Deci<\/a>, the deep learning company building the next generation of AI, announced a breakthrough performance on Intel\u2019s newly released 4th<\/sup> Gen Intel\u00ae Xeon\u00ae Scalable processors, code-named Sapphire Rapids<\/a>. By optimizing the AI models which run on Intel\u2019s new hardware, Deci enables AI developers to achieve GPU-like inference performance on CPUs in production for both Computer Vision and Natural Language Processing (NLP) tasks.<\/p>\n\n\n\n

Deci utilized its proprietary AutoNAC (Automated Neural Architecture Construction) technology to generate custom hardware-aware model architectures that deliver unparalleled accuracy and inference speed on the Intel Sapphire Rapids CPU. For computer vision, Deci delivered a 3.35x throughput increase, as well as a 1% accuracy boost, when compared to an INT8 version of a ResNet50 running on Intel Sapphire Rapids. For NLP, Deci delivered a 3.5x acceleration compared to the INT8 version of the BERT model on Intel Sapphire Rapids, as well as a +0.1 increase in accuray. All models were compiled and quantized to INT8 with Intel\u00ae Advanced Matrix Extensions (AMX) and Intel extension for PyTorch.<\/p>\n\n\n

\n
\"\"
Figure 1: <\/strong>Chart illustrating the improved throughput by running an AutoNAC generated model for Intel Sapphire Rapids<\/em><\/figcaption><\/figure><\/div>\n\n\n
\n

\u201cThis performance breakthrough marks another chapter in the Deci-Intel partnership which empowers AI developers to achieve unparalleled accuracy and inference performance with hardware-aware model architectures powered by NAS,\u201d said Yonatan Geifman, CEO and Co-Founder of Deci. \u201cWe are thrilled to enable our joint customers to achieve scalable, production grade performance, within days\u201d.    <\/p>\n<\/blockquote>\n\n\n

\n
\"\"
Figure 2: <\/strong>Chart illustrating the boost in NLP throughput performance by running an AutoNAC generated model onIntel Lake Sapphire Rapids.<\/em><\/figcaption><\/figure><\/div>\n\n\n

Deci and Intel have maintained broad strategic business and technology collaborations since 2019, most recently announcing<\/a> the acceleration of deep learning models using Intel Chips with Deci\u2019s AutoNAC technology . Deci is a member of the Intel Disruptor program<\/a> and has collaborated with Intel on multiple MLPerf submissions<\/a>. Together, the two are enabling new deep learning based applications to run at scale on Intel CPUs, while reducing development costs and time to market. <\/p>\n\n\n\n

If you are using CPUs for deep learning inference or planning to do so, talk with Deci\u2019s experts to learn how you can quickly obtain better performance and ensure maximum hardware utilization. To learn more about the Deci-Intel collaboration, go to https:\/\/deci.ai\/intel\/<\/a> <\/p>\n\n\n\n

Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

Join us on Twitter: https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n\n\n\n

Join us on LinkedIn: https:\/\/www.linkedin.com\/company\/insidebigdata\/<\/a><\/em><\/p>\n\n\n\n

Join us on Facebook: https:\/\/www.facebook.com\/insideBIGDATANOW<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Deci,\u00a0the deep learning company building the next generation of AI,\u00a0announced\u00a0a\u00a0breakthrough performance on Intel\u2019s newly released 4th\u00a0Gen Intel\u00ae Xeon\u00ae Scalable processors, code-named\u00a0Sapphire Rapids. By optimizing the AI models which run on Intel\u2019s new hardware, Deci enables AI developers to achieve GPU-like inference performance on CPUs in production for both Computer Vision and Natural Language Processing (NLP) tasks.<\/p>\n","protected":false},"author":10513,"featured_media":28944,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,115,180,210,268,56,1],"tags":[437,581,896,264,865,284,635,915,96],"acf":[],"yoast_head":"\nDeci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Deci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"Deci,\u00a0the deep learning company building the next generation of AI,\u00a0announced\u00a0a\u00a0breakthrough performance on Intel\u2019s newly released 4th\u00a0Gen Intel\u00ae Xeon\u00ae Scalable processors, code-named\u00a0Sapphire Rapids. By optimizing the AI models which run on Intel\u2019s new hardware, Deci enables AI developers to achieve GPU-like inference performance on CPUs in production for both Computer Vision and Natural Language Processing (NLP) tasks.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2023-01-28T14:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-01-26T01:43:48+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2022\/04\/Deci_logo.png\" \/>\n\t<meta property=\"og:image:width\" content=\"222\" \/>\n\t<meta property=\"og:image:height\" content=\"77\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/\",\"url\":\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/\",\"name\":\"Deci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2023-01-28T14:00:00+00:00\",\"dateModified\":\"2023-01-26T01:43:48+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Deci delivers breakthrough inference performance on Intel’s 4th Gen Sapphire Rapids CPU\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Deci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/","og_locale":"en_US","og_type":"article","og_title":"Deci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU - insideBIGDATA","og_description":"Deci,\u00a0the deep learning company building the next generation of AI,\u00a0announced\u00a0a\u00a0breakthrough performance on Intel\u2019s newly released 4th\u00a0Gen Intel\u00ae Xeon\u00ae Scalable processors, code-named\u00a0Sapphire Rapids. By optimizing the AI models which run on Intel\u2019s new hardware, Deci enables AI developers to achieve GPU-like inference performance on CPUs in production for both Computer Vision and Natural Language Processing (NLP) tasks.","og_url":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2023-01-28T14:00:00+00:00","article_modified_time":"2023-01-26T01:43:48+00:00","og_image":[{"width":222,"height":77,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2022\/04\/Deci_logo.png","type":"image\/png"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/","url":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/","name":"Deci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2023-01-28T14:00:00+00:00","dateModified":"2023-01-26T01:43:48+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2023\/01\/28\/deci-delivers-breakthrough-inference-performance-on-intels-4th-gen-sapphire-rapids-cpu\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Deci delivers breakthrough inference performance on Intel’s 4th Gen Sapphire Rapids CPU"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2022\/04\/Deci_logo.png","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-8bS","jetpack-related-posts":[{"id":28941,"url":"https:\/\/insidebigdata.com\/2022\/04\/06\/deci-boosts-computer-vision-nlp-models-performance-at-mlperf\/","url_meta":{"origin":31488,"position":0},"title":"Deci Boosts Computer Vision & NLP Models\u2019 Performance at MLPerf\u00a0","date":"April 6, 2022","format":false,"excerpt":"Deci, the deep learning company harnessing Artificial Intelligence (AI) to build AI, announced its results for both Computer Vision (CV) and Natural Language Processing (NLP) inference models that were submitted to the MLPerf v2.0\u00a0Datacenter Open division. These submissions demonstrated the power of Deci\u2019s Automated Neural Architecture Construction (AutoNAC) technology, which\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25781,"url":"https:\/\/insidebigdata.com\/2021\/03\/16\/deci-and-intel-collaborate-to-optimize-deep-learning-inference-on-intels-cpus\/","url_meta":{"origin":31488,"position":1},"title":"Deci and Intel Collaborate to Optimize Deep Learning Inference on Intel\u2019s CPUs","date":"March 16, 2021","format":false,"excerpt":"Deci, the deep learning company building the next generation of AI, announced a broad strategic business and technology collaboration with Intel Corporation to optimize deep learning inference on Intel Architecture (IA) CPUs. As one of the first companies to participate in Intel Ignite startup accelerator, Deci will now work with\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2021\/03\/Deci_Intel.jpeg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":30345,"url":"https:\/\/insidebigdata.com\/2022\/09\/12\/decis-natural-language-processing-nlp-model-achieves-breakthrough-performance-at-mlperf\/","url_meta":{"origin":31488,"position":2},"title":"Deci\u2019s Natural Language Processing (NLP) Model Achieves Breakthrough Performance at MLPerf","date":"September 12, 2022","format":false,"excerpt":"Deci, the deep learning company harnessing Artificial Intelligence (AI) to build better AI, announced results for its Natural Language Processing (NLP) inference model submitted to the MLPerf Inference v2.1 benchmark suite under the open submission track.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2022\/09\/Deci_table1.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":33161,"url":"https:\/\/insidebigdata.com\/2023\/08\/17\/generative-ai-report-deci-releases-powerful-open-source-generative-ai-model-decicoder-to-redefine-code-generation-for-developers\/","url_meta":{"origin":31488,"position":3},"title":"Generative AI Report: Deci Releases Powerful Open-Source Generative AI Model, DeciCoder, to Redefine Code Generation for Developers\u00a0","date":"August 17, 2023","format":false,"excerpt":"Deci, the deep learning company harnessing AI to build AI, released DeciCoder, its inaugural foundation model in generative AI helping users generate programming language code. This groundbreaking Large Language Model (LLM), dedicated to code generation with 1 billion parameters and an expansive 2048-context window, surpasses results released in equivalent models\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/GenerativeAI_shutterstock_2313909647_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":29303,"url":"https:\/\/insidebigdata.com\/2022\/05\/11\/deci-launches-version-2-0-of-its-deep-learning-development-platform-to-democratize-the-power-of-nas-and-eliminate-the-ai-efficiency-gap\/","url_meta":{"origin":31488,"position":4},"title":"Deci Launches Version 2.0 of its Deep Learning Development Platform, to Democratize the Power of NAS and Eliminate the AI Efficiency Gap\u00a0","date":"May 11, 2022","format":false,"excerpt":"Deci, the deep learning company harnessing AI to build AI, today launched Version 2.0 of its deep learning development platform, making it easier than ever before for AI developers to build, optimize, and deploy computer vision models on any hardware and environment including cloud, edge and mobile with outstanding accuracy\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":30272,"url":"https:\/\/insidebigdata.com\/2022\/09\/03\/amax-launches-gpu-servers-powered-by-intels-newest-data-center-gpu-flex-series-for-ai-gaming-media-streaming\/","url_meta":{"origin":31488,"position":5},"title":"AMAX Launches GPU Servers Powered by Intel’s Newest Data Center GPU Flex Series for AI, Gaming, & Media Streaming","date":"September 3, 2022","format":false,"excerpt":"AMAX, a leading provider of turnkey rack-scale High Performance Computing (HPC) solutions, Deep Learning\/AI applications and server appliance manufacturing, announces the new AceleMax X-122-Flex server solution featuring Intel's next-generation Data Center GPU Flex Series, (formerly code-named Arctic Sound-M), providing the capability of a graphics processing (GPU) solution handling high density\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/31488"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=31488"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/31488\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/28944"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=31488"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=31488"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=31488"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}