{"id":29225,"date":"2022-05-03T07:00:00","date_gmt":"2022-05-03T14:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=29225"},"modified":"2022-05-02T19:13:30","modified_gmt":"2022-05-03T02:13:30","slug":"new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/","title":{"rendered":"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease"},"content":{"rendered":"\n

The novel regression algorithm, CALF, outperforms the current gold standard, LASSO, in statistical tests<\/em><\/p>\n\n\n\n

Researchers from the Renaissance Computing Institute<\/a> (RENCI) at UNC-Chapel Hill, Perspectrix, the UNC School of Medicine, and the WVU Rockefeller Neuroscience Institute have collaborated to develop a new method for finding patterns in data which arguably surpasses the performance of a generally accepted “gold standard.\u201d<\/p>\n\n\n\n

Attempting to find patterns in data is central to all research, and it is particularly important in the medical field\u2019s use of biological samples to predict a patient\u2019s risk for disease formation and progression. Today, researchers can utilize advanced medical technology to produce an ocean of data about one person from various biological samples such as blood, DNA, and saliva, with the goal of identifying particular markers that can be informative about a person\u2019s current health and future outlook. However, this advanced data collection and processing has outpaced current statistical methods for identifying these important patterns and relationships, and this is particularly true for the field of psychiatry. For instance, researchers have yet to fully understand and predict the progression of schizophrenia.<\/p>\n\n\n\n

The goal is to create a sum of values from tests of some kind\u2014ideally only a few from many possible tests\u2014that differentiates cases from controls or correlates with some other outcome such as age of onset. The popular and powerful method known as LASSO, or \u201cleast absolute shrinkage and selection operator,\u201d attempts to find real number weights so sums are close to 1 (cases) and 0 (controls). However, the utility to clinicians of any approximation is typically measured by metrics known as Student p-value, AUC, or Pearson correlation. The new method, by contrast, seeks to optimize a metric directly.<\/p>\n\n\n\n

This new method, CALF, is described in the Scientific Reports paper, “A greedy regression algorithm with coarse weights offers novel advantages<\/a>,” published on March 31, 2022. In terms of the standard metrics, application of CALF to five quite different examples from psychiatric and neurological studies consistently outperformed the gold standard, LASSO regression, and other methods.<\/p>\n\n\n\n

“Frisky CALF outruns LASSO in the five examples we outlined in the paper,\u201d said RENCI scientist and lead author Clark Jeffries, PhD. \u201cThe metric values using CALF are superior to those of LASSO when the researcher seeks a small number of collectively informative predictors\u2014five chosen from hundreds, for example. Interrogating the biochemistry or other relationships among the five can then suggest causality.”<\/p><\/blockquote>\n\n\n\n

The key distinction of CALF is its simplicity. For a given level of metric performance, it typically utilizes only a fraction of the predictors required by the LASSO method. CALF progresses in a \u2018greedy\u2019 fashion, meaning it searches through the data and accepts the immediate next best predictor until the algorithm has optimized model performance. According to Jeffries, the research team originally aimed to develop a baseline for the simplest regression model and, in doing so, discovered that this streamlined model extracts statistically significant results from data where LASSO fails to do so.<\/p>\n\n\n\n

\"\"
The geometry underlying a new data analysis method. Shown are three of 26 weight vectors, which are all combinations of +1, -1, 0. In higher dimensions, the choices become astronomical.<\/figcaption><\/figure><\/div>\n\n\n\n

CALF\u2019s improvements on existing predictive models has great implications. It has the potential to unearth statistically significant patterns in data that may otherwise go undetected. Unlike conventional methods, small alterations in input data do not alter its solution at all. It also accepts repetitious or nearly repetitious input data while LASSO solutions may become wild.<\/p>\n\n\n\n

“It\u2019s likely that there are existing data sets out there that failed to show more than a trend with routine analyses and could show classification significance with CALF,\u201d said Diana Perkins, MD, MPH, a psychiatrist at UNC-Chapel Hill. \u201cThis could be groundbreaking for the field of psychiatry in improving prediction of patients\u2019 risk for psychosis and other mental illnesses, allowing earlier intervention and overall improved outcomes.”<\/p><\/blockquote>\n\n\n\n

“While testing examples with all possible parameters in all possible conventional models is impossible, our best practices showed that CALF can find statistically significant patterns in data that otherwise fail interpretation,\u201d added RENCI scientist Jeffrey Tilson, PhD. \u201cWe encourage fellow researchers to test out CALF on their own data sets.”<\/p><\/blockquote>\n\n\n\n

Darius Bost, a PhD student at UNC-Chapel Hill and Graduate Research Assistant at RENCI, noted that, “Using data from the Alzheimer\u2019s Disease Neuroimaging Initiative, CALF was able to determine a small set of DNA markers that are highly correlated with the age of onset of Alzheimer\u2019s, indicating great potential for CALF as a simple and reliable research tool.”<\/p>\n\n\n\n

Current versions of CALF in R and Python were developed by John R. Ford at Perspectrix. These resources are open source and available at the below links: <\/p>\n\n\n\n

R version: https:\/\/cran.r-project.org\/web\/packages\/CALF\/index.html<\/a><\/p>\n\n\n\n

PyPi Python version: https:\/\/pypi.org\/project\/calfpy\/<\/a>\u00a0\u00a0<\/p>\n\n\n\n

Python 3.x version: https:\/\/github.com\/jorufo\/CALF_Python<\/a><\/p>\n\n\n\n

Sample data that may be used for duplication of all the stated results is available via GitHub as an unrestricted supporting resource at https:\/\/github.com\/jorufo\/CALF_SupportingResources<\/a>.<\/p>\n\n\n\n

Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

Join us on Twitter: @InsideBigData1 \u2013 https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Researchers from the Renaissance Computing Institute (RENCI) at UNC-Chapel Hill, Perspectrix, the UNC School of Medicine, and the WVU Rockefeller Neuroscience Institute have collaborated to develop a new method for finding patterns in data which arguably surpasses the performance of a generally accepted “gold standard.\u201d<\/p>\n","protected":false},"author":10513,"featured_media":23655,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[115,182,180,74,122,67,56,1],"tags":[129,334,277,96],"acf":[],"yoast_head":"\nNew Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"Researchers from the Renaissance Computing Institute (RENCI) at UNC-Chapel Hill, Perspectrix, the UNC School of Medicine, and the WVU Rockefeller Neuroscience Institute have collaborated to develop a new method for finding patterns in data which arguably surpasses the performance of a generally accepted "gold standard.\u201d\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2022-05-03T14:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-03T02:13:30+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/12\/Machine_Learning_shutterstock_344688470.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"212\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/\",\"url\":\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/\",\"name\":\"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2022-05-03T14:00:00+00:00\",\"dateModified\":\"2022-05-03T02:13:30+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/","og_locale":"en_US","og_type":"article","og_title":"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease - insideBIGDATA","og_description":"Researchers from the Renaissance Computing Institute (RENCI) at UNC-Chapel Hill, Perspectrix, the UNC School of Medicine, and the WVU Rockefeller Neuroscience Institute have collaborated to develop a new method for finding patterns in data which arguably surpasses the performance of a generally accepted \"gold standard.\u201d","og_url":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2022-05-03T14:00:00+00:00","article_modified_time":"2022-05-03T02:13:30+00:00","og_image":[{"width":300,"height":212,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/12\/Machine_Learning_shutterstock_344688470.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/","url":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/","name":"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2022-05-03T14:00:00+00:00","dateModified":"2022-05-03T02:13:30+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2022\/05\/03\/new-streamlined-statistical-method-provides-improved-pattern-detection-and-risk-prediction-for-disease\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"New Streamlined Statistical Method Provides Improved Pattern Detection and Risk Prediction for Disease"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/12\/Machine_Learning_shutterstock_344688470.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-7Bn","jetpack-related-posts":[{"id":18436,"url":"https:\/\/insidebigdata.com\/2017\/07\/24\/customized-healthcare-near-term-reality\/","url_meta":{"origin":29225,"position":0},"title":"Is Customized Healthcare a Near Term Reality?","date":"July 24, 2017","format":false,"excerpt":"In this special guest feature, Abdul Hamid Halabi, the global business development lead for healthcare and life sciences at NVIDIA, discusses how personalized or precision medicine is becoming a reality with the help of a machine learning method called deep learning.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2017\/07\/Abdul_Hamid_Halabi.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":7463,"url":"https:\/\/insidebigdata.com\/2014\/02\/19\/challenges-solutions-genomics-age-big-data\/","url_meta":{"origin":29225,"position":1},"title":"Challenges and Solutions for Genomics in the Age of Big Data","date":"February 19, 2014","format":false,"excerpt":"Leading researchers in data science and genomics are recommending strategies to help genomic scientists better manage, share, analyze and archive massive research and clinical data sets in an effort to ensure that the big data explosion results in better health outcomes and faster research discoveries.","rel":"","context":"In "Big Data Hardware"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":18981,"url":"https:\/\/insidebigdata.com\/2017\/09\/29\/catching-gorilla-applying-machine-learning-electronic-health-records\/","url_meta":{"origin":29225,"position":2},"title":"Catching the Gorilla: Applying Machine Learning to Electronic Health Records","date":"September 29, 2017","format":false,"excerpt":"In this special guest feature, Ori Geva, Co-Founder and CEO of Medial EarlySign, discusses how the ue of machine learning can help create new opportunities for earlier intervention and delivery of improved, personalized care by allowing physicians and health systems to increase their scope of attention.","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2017\/09\/Ori-Geva-Medial-EarlySign.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":19292,"url":"https:\/\/insidebigdata.com\/2017\/11\/02\/medial-earlysign-machine-learning-algorithm-predicts-risk-prediabetics-becoming-diabetic-within-1-year\/","url_meta":{"origin":29225,"position":3},"title":"Medial EarlySign Machine Learning Algorithm Predicts Risk for Prediabetics Becoming Diabetic Within 1 year","date":"November 2, 2017","format":false,"excerpt":"Medial EarlySign, a developer of machine learning tools for data-driven medicine, announced the results of its clinical data study on identifying and stratifying prediabetic patients at high risk for progressing to diabetes within one year.","rel":"","context":"In "Google News Feed"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25690,"url":"https:\/\/insidebigdata.com\/2021\/02\/24\/how-machine-learning-and-data-science-can-advance-nutrition-research\/","url_meta":{"origin":29225,"position":4},"title":"How Machine Learning and Data Science Can Advance Nutrition Research","date":"February 24, 2021","format":false,"excerpt":"In this special guest feature, Kyle Dardashti, CEO & Founder of Heali, discusses how machine learning and data science bring exciting potential to the world of personalized nutrition. Combining these two technologies together on a cohesive platform that supports continuous tracking would allow for real-time validated nutrition recommendations tailored to\u2026","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2021\/02\/Heali_fig1.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":17814,"url":"https:\/\/insidebigdata.com\/2017\/05\/06\/medable-launches-cloud-based-machine-learning-solution-healthcare\/","url_meta":{"origin":29225,"position":5},"title":"Medable Launches Cloud-Based Machine Learning Solution for Healthcare","date":"May 6, 2017","format":false,"excerpt":"MEDABLE Inc., a leading application and analytics platform for healthcare, announced Cerebrum, the first cloud-based machine learning solution created specifically for healthcare apps. Cerebrum leverages data gathering smartphones with a first-of-its-kind machine learning engine, resulting in health events becoming more easily predicted, such as warning an elderly relative when he\u2026","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/29225"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=29225"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/29225\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/23655"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=29225"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=29225"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=29225"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}