{"id":32644,"date":"2023-06-16T03:00:00","date_gmt":"2023-06-16T10:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=32644"},"modified":"2023-06-23T12:36:14","modified_gmt":"2023-06-23T19:36:14","slug":"research-highlights-llms-can-process-a-lot-more-text-than-we-thought","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/","title":{"rendered":"Research Highlights: LLMs Can Process a lot more Text Than We Thought"},"content":{"rendered":"\n

A team of researchers at AI21 Labs<\/a>, the company behind generative text AI platforms Human or Not<\/a>, Wordtune<\/a>, and Jurassic 2<\/a>, has identified a new method to overcome a challenge that most Large Language Models (LLMs) grapple with – a limit as to how much text they can process before it becomes too expensive and impractical. <\/p>\n\n\n\n

The findings emerged from a study<\/a>, in which the researchers showed that two simple changes to the attention mechanism enabled LLMs to tap into its inherent ability to read multiple pieces of text simultaneously, therefore bypassing the problem in the first place. The team illustrated through extensive testing that these models have the built-in ability for \u201cparallel reading,\u201d which makes the processing of many texts more efficient and accurate. <\/p>\n\n\n\n

Imagine you own a hotel and would like to categorize its reviews according to various parameters like cleanliness, check-in, and amenities. Previously, a large LLM would eventually run-into problems scanning all the reviews in their entirety and trying to put them into multiple categories. But by allowing the LLM to scan the texts simultaneously at various intervals, the LLM can improve its ability to categorize existing, and future, reviews. <\/p>\n\n\n\n

Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

Join us on Twitter: https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n\n\n\n

Join us on LinkedIn: https:\/\/www.linkedin.com\/company\/insidebigdata\/<\/a><\/em><\/p>\n\n\n\n

Join us on Facebook: https:\/\/www.facebook.com\/insideBIGDATANOW<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

A team of researchers at\u00a0AI21 Labs, the company behind generative text AI platforms\u00a0Human or Not,\u00a0Wordtune, and\u00a0Jurassic 2, has identified a new method to overcome a challenge that most Large Language Models (LLMs) grapple with – a limit as to how much text they can process before it becomes too expensive and impractical.\u00a0<\/p>\n","protected":false},"author":10513,"featured_media":32645,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,182,180,67,268,56,84,1303,1],"tags":[437,324,264,1245,1248,96],"acf":[],"yoast_head":"\nResearch Highlights: LLMs Can Process a lot more Text Than We Thought - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Research Highlights: LLMs Can Process a lot more Text Than We Thought - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"A team of researchers at\u00a0AI21 Labs, the company behind generative text AI platforms\u00a0Human or Not,\u00a0Wordtune, and\u00a0Jurassic 2, has identified a new method to overcome a challenge that most Large Language Models (LLMs) grapple with - a limit as to how much text they can process before it becomes too expensive and impractical.\u00a0\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-16T10:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-23T19:36:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/AI_shutterstock_2287025875_special-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1100\" \/>\n\t<meta property=\"og:image:height\" content=\"550\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/\",\"url\":\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/\",\"name\":\"Research Highlights: LLMs Can Process a lot more Text Than We Thought - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2023-06-16T10:00:00+00:00\",\"dateModified\":\"2023-06-23T19:36:14+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Research Highlights: LLMs Can Process a lot more Text Than We Thought\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Research Highlights: LLMs Can Process a lot more Text Than We Thought - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/","og_locale":"en_US","og_type":"article","og_title":"Research Highlights: LLMs Can Process a lot more Text Than We Thought - insideBIGDATA","og_description":"A team of researchers at\u00a0AI21 Labs, the company behind generative text AI platforms\u00a0Human or Not,\u00a0Wordtune, and\u00a0Jurassic 2, has identified a new method to overcome a challenge that most Large Language Models (LLMs) grapple with - a limit as to how much text they can process before it becomes too expensive and impractical.\u00a0","og_url":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2023-06-16T10:00:00+00:00","article_modified_time":"2023-06-23T19:36:14+00:00","og_image":[{"width":1100,"height":550,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/AI_shutterstock_2287025875_special-1.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/","url":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/","name":"Research Highlights: LLMs Can Process a lot more Text Than We Thought - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2023-06-16T10:00:00+00:00","dateModified":"2023-06-23T19:36:14+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2023\/06\/16\/research-highlights-llms-can-process-a-lot-more-text-than-we-thought\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Research Highlights: LLMs Can Process a lot more Text Than We Thought"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/AI_shutterstock_2287025875_special-1.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-8uw","jetpack-related-posts":[{"id":32736,"url":"https:\/\/insidebigdata.com\/2023\/06\/26\/databricks-signs-definitive-agreement-to-acquire-mosaicml-a-leading-generative-ai-platform\/","url_meta":{"origin":32644,"position":0},"title":"Databricks Signs Definitive Agreement to Acquire MosaicML, a Leading Generative AI Platform","date":"June 26, 2023","format":false,"excerpt":"Databricks, the Data and AI company, today announced it has entered into a definitive agreement to acquire MosaicML, a leading generative AI platform. Together, Databricks and MosaicML will make generative AI accessible for every organization, enabling them to build, own and secure generative AI models with their own data.\u00a0The transaction\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/GenerativeAI_shutterstock_2313909647_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":33298,"url":"https:\/\/insidebigdata.com\/2023\/09\/06\/insidebigdata-ai-news-briefs-9-8-2023\/","url_meta":{"origin":32644,"position":1},"title":"insideBIGDATA AI News Briefs \u2013 9\/8\/2023","date":"September 6, 2023","format":false,"excerpt":"Welcome insideBIGDATA AI News Briefs, our timely new feature bringing you the latest industry insights and perspectives surrounding the field of AI including deep learning, large language models, generative AI, and transformers. We\u2019re working tirelessly to dig up the most timely and curious tidbits underlying the day\u2019s most popular technologies.\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/07\/AI-News-Briefs-column-banner.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":33551,"url":"https:\/\/insidebigdata.com\/2023\/10\/01\/video-highlights-vicuna-gorilla-chatbot-arena-and-socially-beneficial-llms-with-prof-joey-gonzalez\/","url_meta":{"origin":32644,"position":2},"title":"Video Highlights: Vicu\u00f1a, Gorilla, Chatbot Arena and Socially Beneficial LLMs \u2014 with Prof. Joey Gonzalez","date":"October 1, 2023","format":false,"excerpt":"LLM Vicu\u00f1a, Chatbot Arena, and the race to increase LLM context windows: In this video presentation, guest Joey Gonzalez joins our good friend\u00a0Jon Krohn, Co-Founder and Chief Data Scientist at the machine learning company\u00a0Nebula,\u00a0to talk about developing models and platforms that leverage and improve LLMs, as well as the future\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/08\/Neural_net_shutterstock_1615182352_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":32878,"url":"https:\/\/insidebigdata.com\/2023\/07\/25\/video-highlights-generative-ai-with-large-language-models\/","url_meta":{"origin":32644,"position":3},"title":"Video Highlights: Generative AI with Large Language Models","date":"July 25, 2023","format":false,"excerpt":"At an unprecedented pace, Large Language Models like GPT-4 are transforming the world in general and the field of data science in particular. This two-hour training video presentation by Jon Krohn, Co-Founder and Chief Data Scientist at the machine learning company Nebula, introduces deep learning transformer architectures including LLMs.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/GenerativeAI_shutterstock_2313909647_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":33169,"url":"https:\/\/insidebigdata.com\/2023\/08\/23\/survey-more-than-75-of-enterprises-dont-plan-to-use-commercial-llms-in-production-citing-data-privacy-as-primary-concern\/","url_meta":{"origin":32644,"position":4},"title":"Survey: More than 75% of Enterprises Don\u2019t Plan to Use Commercial LLMs in Production Citing Data Privacy as Primary Concern\u00a0","date":"August 23, 2023","format":false,"excerpt":"Predibase, the commercially available low-code declarative ML platform for developers, today released a new report, \u201cBeyond the Buzz: A Look at Large Language Models in Production.\u201d Based on survey data from organizations experimenting with LLMs, the report offers insight into real-world concerns, opportunities, and priorities for organizations as they embrace\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/GenerativeAI_shutterstock_2313909647_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":34119,"url":"https:\/\/insidebigdata.com\/2023\/12\/08\/hallucination-index-identifies-best-llms-for-most-popular-ai-use-cases\/","url_meta":{"origin":32644,"position":5},"title":"Hallucination Index Identifies Best LLMs for Most Popular AI Use Cases","date":"December 8, 2023","format":false,"excerpt":"Galileo, a leading machine learning (ML) company for unstructured data, released a Hallucination Index developed by its research arm, Galileo Labs, to help users of today\u2019s leading LLMs determine which model is least likely to hallucinate for their intended application.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/AI_shutterstock_2287025875_special-1.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/32644"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=32644"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/32644\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/32645"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=32644"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=32644"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=32644"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}