{"id":29628,"date":"2022-06-19T06:00:00","date_gmt":"2022-06-19T13:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=29628"},"modified":"2023-06-23T12:41:01","modified_gmt":"2023-06-23T19:41:01","slug":"research-highlights-emergent-abilities-of-large-language-models","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/","title":{"rendered":"Research Highlights: Emergent Abilities of Large Language Models"},"content":{"rendered":"\n

Title of paper:<\/strong> Emergent Abilities of Large Language Models<\/a><\/p>\n\n\n\n

Author(s):<\/strong> Jason Wei, et al<\/p>\n\n\n\n

Abstract: <\/strong>Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

Join us on Twitter: @InsideBigData1 \u2013 https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

In this regular column we take a look at highlights for breaking research topics of the day in the areas of big data, data science, machine learning, AI and deep learning. For data scientists, it\u2019s important to keep connected with the research arm of the field in order to understand where the technology is headed. Enjoy!<\/p>\n","protected":false},"author":37,"featured_media":22835,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,87,180,67,56,84,1303,1],"tags":[264,947,948,96],"acf":[],"yoast_head":"\nResearch Highlights: Emergent Abilities of Large Language Models - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Research Highlights: Emergent Abilities of Large Language Models - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"In this regular column we take a look at highlights for breaking research topics of the day in the areas of big data, data science, machine learning, AI and deep learning. For data scientists, it\u2019s important to keep connected with the research arm of the field in order to understand where the technology is headed. Enjoy!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2022-06-19T13:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-23T19:41:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/06\/Data-Scientist-shutterstock_768047488.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"200\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Daniel Gutierrez\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@AMULETAnalytics\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Gutierrez\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/\",\"url\":\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/\",\"name\":\"Research Highlights: Emergent Abilities of Large Language Models - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2022-06-19T13:00:00+00:00\",\"dateModified\":\"2023-06-23T19:41:01+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research Highlights: Emergent Abilities of Large Language Models\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed\",\"name\":\"Daniel Gutierrez\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g\",\"caption\":\"Daniel Gutierrez\"},\"description\":\"Daniel D. Gutierrez is a Data Scientist with Los Angeles-based AMULET Analytics, a service division of AMULET Development Corp. He's been involved with data science and Big Data long before it came in vogue, so imagine his delight when the Harvard Business Review recently deemed \\\"data scientist\\\" as the sexiest profession for the 21st century. Previously, he taught computer science and database classes at UCLA Extension for over 15 years, and authored three computer industry books on database technology. He also served as technical editor, columnist and writer at a major computer industry monthly publication for 7 years. Follow his data science musings at @AMULETAnalytics.\",\"sameAs\":[\"http:\/\/www.insidebigdata.com\",\"https:\/\/twitter.com\/@AMULETAnalytics\"],\"url\":\"https:\/\/insidebigdata.com\/author\/dangutierrez\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Research Highlights: Emergent Abilities of Large Language Models - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/","og_locale":"en_US","og_type":"article","og_title":"Research Highlights: Emergent Abilities of Large Language Models - insideBIGDATA","og_description":"In this regular column we take a look at highlights for breaking research topics of the day in the areas of big data, data science, machine learning, AI and deep learning. For data scientists, it\u2019s important to keep connected with the research arm of the field in order to understand where the technology is headed. Enjoy!","og_url":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2022-06-19T13:00:00+00:00","article_modified_time":"2023-06-23T19:41:01+00:00","og_image":[{"width":300,"height":200,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/06\/Data-Scientist-shutterstock_768047488.jpg","type":"image\/jpeg"}],"author":"Daniel Gutierrez","twitter_card":"summary_large_image","twitter_creator":"@AMULETAnalytics","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Daniel Gutierrez","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/","url":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/","name":"Research Highlights: Emergent Abilities of Large Language Models - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2022-06-19T13:00:00+00:00","dateModified":"2023-06-23T19:41:01+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2022\/06\/19\/research-highlights-emergent-abilities-of-large-language-models\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Research Highlights: Emergent Abilities of Large Language Models"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2540da209c83a68f4f5922848f7376ed","name":"Daniel Gutierrez","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5780282e7e567e2a502233e948464542?s=96&d=mm&r=g","caption":"Daniel Gutierrez"},"description":"Daniel D. Gutierrez is a Data Scientist with Los Angeles-based AMULET Analytics, a service division of AMULET Development Corp. He's been involved with data science and Big Data long before it came in vogue, so imagine his delight when the Harvard Business Review recently deemed \"data scientist\" as the sexiest profession for the 21st century. Previously, he taught computer science and database classes at UCLA Extension for over 15 years, and authored three computer industry books on database technology. He also served as technical editor, columnist and writer at a major computer industry monthly publication for 7 years. Follow his data science musings at @AMULETAnalytics.","sameAs":["http:\/\/www.insidebigdata.com","https:\/\/twitter.com\/@AMULETAnalytics"],"url":"https:\/\/insidebigdata.com\/author\/dangutierrez\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/06\/Data-Scientist-shutterstock_768047488.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-7HS","jetpack-related-posts":[{"id":34133,"url":"https:\/\/insidebigdata.com\/2023\/12\/19\/insidebigdata-ai-news-briefs-bulletin-board\/","url_meta":{"origin":29628,"position":0},"title":"insideBIGDATA AI News Briefs Bulletin Board","date":"December 19, 2023","format":false,"excerpt":"Welcome insideBIGDATA AI News Briefs Bulletin Board, our timely new feature bringing you the latest industry insights and perspectives surrounding the field of AI including deep learning, large language models, generative AI, and transformers. We\u2019re working tirelessly to dig up the most timely and curious tidbits underlying the day\u2019s most\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/07\/AI-News-Briefs-column-banner.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":33975,"url":"https:\/\/insidebigdata.com\/2023\/11\/23\/new-report-the-definitive-guide-to-large-language-models-and-high-performance-marketing-content\/","url_meta":{"origin":29628,"position":1},"title":"New Report: The Definitive Guide to Large Language Models and High-Performance Marketing Content","date":"November 23, 2023","format":false,"excerpt":"Phrasee, a leading innovator in brand language optimization, just released a new white paper \"The Definitive Guide to Large Language Models and High-Performance Marketing Content,\" on how enterprise marketers can build an in-house LLM solution and use it at its full potential.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/GenerativeAI_shutterstock_2284999159_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":31639,"url":"https:\/\/insidebigdata.com\/2023\/02\/15\/video-highlights-attention-is-all-you-need-paper-explained\/","url_meta":{"origin":29628,"position":2},"title":"Video Highlights: Attention Is All You Need – Paper Explained","date":"February 15, 2023","format":false,"excerpt":"In this video presentation, Mohammad Namvarpour presents a comprehensive study on Ashish Vaswani and his coauthors' renowned paper, \u201cAttention Is All You Need.\u201d This paper is a major turning point in deep learning research. The transformer architecture, which was introduced in this paper, is now used in a variety of\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/10\/NLP_shutterstock_299138114.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":31384,"url":"https:\/\/insidebigdata.com\/2023\/01\/11\/ai-generated-texts-could-increase-peoples-exposure-to-threats\/","url_meta":{"origin":29628,"position":3},"title":"AI-generated Texts Could Increase People\u2019s Exposure to Threats","date":"January 11, 2023","format":false,"excerpt":"Research finds that large language models are susceptible to abuse through creative prompt engineering, forcing people to become even more skeptical about what they read.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":27099,"url":"https:\/\/insidebigdata.com\/2021\/09\/14\/best-of-arxiv-org-for-ai-machine-learning-and-deep-learning-august-2021\/","url_meta":{"origin":29628,"position":4},"title":"Best of arXiv.org for AI, Machine Learning, and Deep Learning \u2013 August 2021","date":"September 14, 2021","format":false,"excerpt":"In this recurring monthly feature, we will filter all the recent research papers appearing in the arXiv.org preprint server for subjects relating to AI, machine learning and deep learning \u2013 from disciplines including statistics, mathematics and computer science \u2013 and provide you with a useful \u201cbest of\u201d list for the\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2013\/12\/arxiv.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":30420,"url":"https:\/\/insidebigdata.com\/2022\/09\/21\/cerebras-wafer-scale-cluster-brings-push-button-ease-and-linear-performance-scaling-to-large-language-models\/","url_meta":{"origin":29628,"position":5},"title":"Cerebras Wafer-Scale Cluster Brings Push-Button Ease and Linear Performance Scaling to Large Language Models","date":"September 21, 2022","format":false,"excerpt":"Cerebras Systems, a pioneer in accelerating artificial intelligence (AI) compute, unveiled the Cerebras Wafer-Scale Cluster, delivering near-perfect linear scaling across hundreds of millions of AI-optimized compute cores while avoiding the pain of the distributed compute. With a Wafer-Scale Cluster, users can distribute even the largest language models from a Jupyter\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/29628"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=29628"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/29628\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/22835"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=29628"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=29628"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=29628"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}