{"id":31552,"date":"2023-02-03T06:00:00","date_gmt":"2023-02-03T14:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=31552"},"modified":"2023-02-02T19:00:03","modified_gmt":"2023-02-03T03:00:03","slug":"anomaly-detection-its-real-life-uses-and-the-latest-advances","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/","title":{"rendered":"Anomaly Detection: Its Real-Life Uses and the Latest Advances"},"content":{"rendered":"\n<p><strong>What is anomaly detection?<\/strong><\/p>\n\n\n\n<p>Anomaly detection in the context of data science is detecting a data sample that is out of the ordinary and does not fit into the general data pattern (or an outlier). This deviation can result from a rare occurrence or an outlying event.<\/p>\n\n\n\n<p>Identifying these samples, called anomaly detection, is an integral part of any monitoring system. Anomaly detection has been traditionally done manually by inspection, which is a tedious  process typically done by experts with significant domain knowledge.<\/p>\n\n\n\n<p><strong>Real-life Use Cases of Anomaly Detection<\/strong><\/p>\n\n\n\n<p>Anomaly detection is used in a wide variety of applications. The following are just a few examples:<\/p>\n\n\n\n<ul>\n<li>Credit card processing companies widely use anomaly detection to identify the fraudulent transactions<\/li>\n\n\n\n<li>The field of cybersecurity is heavily reliant on anomaly detection to discern between normal behavior and malicious attempts to compromise the system<\/li>\n\n\n\n<li>eCommerce vendors use anomaly detection to identify unusual shifts in various performance metrics, such as sale conversion, bounce rate, and unusual counts of positive or negative customer reviews<\/li>\n\n\n\n<li>Detection of spam emails<\/li>\n<\/ul>\n\n\n\n<p><strong>Critical Factors in Anomaly Detection<\/strong><\/p>\n\n\n\n<p>An effective anomaly detection model must address the following factors and requirements:<\/p>\n\n\n\n<ul>\n<li>Minimize the prediction latency<\/li>\n\n\n\n<li>Minimize false positive predictions<\/li>\n\n\n\n<li>Effectively deal with highly imbalanced training datasets<\/li>\n<\/ul>\n\n\n\n<p>For example, an anomaly detection model tasked to discern between legitimate and fraudulent credit card transactions must be able to render prediction in real-time and deny fraudulent transactions before it is too late. In the same context, erroneously misclassifying a valid transaction can lead to poor customer experience and embarrassment. Aside from these challenges, the model has to be trained using a severely imbalanced dataset since fraudulent transactions are extremely rare.<\/p>\n\n\n\n<p><strong>Machine Learning and Anomaly Detection<\/strong><\/p>\n\n\n\n<p>Searching for anomalies using manual inspection of data in the \u201cbig data\u201d age is impossible. Fortunately, machine learning techniques for anomaly detection have proven to be highly effective since they can handle massive datasets and meet the latency requirements of most anomaly detection use cases.<\/p>\n\n\n\n<p><strong>Machine Learning Comes to the Rescue<\/strong><\/p>\n\n\n\n<p>Machine learning algorithms can recognize complex patterns from massive volumes of data and solve detection tasks that might be impossible to accomplish manually. In the context of spam detection, these algorithms can accurately discern between spam and non-spam emails by examining the content of the messages.<\/p>\n\n\n\n<p>Various machine learning models such as Artificial Neural Networks (ANN), Support Vector Machines (SVM), Autoencoders, and K-Nearest Neighbors have been used for anomaly detection. They each have their strengths and weaknesses. One key criterion for model selection is the time and effort needed to prepare the data for that particular model. Given this metric, ANNs are the most effective since they require the least data preprocessing.<\/p>\n\n\n\n<p><strong>Artificial Neural Networks (ANN) for Anomaly Detection<\/strong><\/p>\n\n\n\n<p>ANNs are among the most popular machine learning models for anomaly detection and various other use cases, such as Computer Vision and Natural Language Processing. They are inspired by how neurons work in the brain to enable us to perceive our senses. One of the critical advantages of ANNs is their ability to tackle various degrees of model complexities by merely accepting additional layers of neurons, aiding the model to deal with increasing data pattern complexity and data granularity. This phenomenon is referred to as \u201cDeep Learning.\u201d<\/p>\n\n\n\n<p>Anomaly detection has enjoyed immense improvement following the introduction of Deep Learning models. Over the past few years, several deep algorithms for anomaly detection have been developed capable of solving previously impossible problems.<\/p>\n\n\n\n<p>To better understand deep anomaly detection algorithms, academic researchers have attempted to categorize them into fine-grained categories. A common categorization process is based on how the deep learning model expresses the anomaly score (the anomaly level). Three distinct types can be made:<\/p>\n\n\n\n<p><strong>1. Deep Learning for Feature Extraction<\/strong><\/p>\n\n\n\n<p>These algorithms use a deep learning framework to extract features (independent variables) from the raw data and transform the input into a new representation. The term \u201cfeatures\u201d is simply the characteristic of a given data stream. In this representation, anomalies can often be detected using a traditional anomaly detection algorithm. Essentially, in this category, deep learning and anomaly detection are fully isolated.<\/p>\n\n\n\n<p>This category of methods is instrumental when dealing with high-dimensional input data. Deep models can map the high dimensional data into a lower dimensional one in which anomalies are easier to detect. Furthermore, many of these algorithms can also work with unannotated and unlabeled data.<\/p>\n\n\n\n<p>The feature extraction methods themselves can also be grouped into three categories:<\/p>\n\n\n\n<ul>\n<li><strong>Supervised<\/strong> learning algorithms can only work with labeled data. They train a classifier to predict the label and discard the last layer. The layer before the last one should ideally learn an efficient feature set from the data that guides the model in identifying its class. Since, in most cases, access to labeled information is challenging, these methods are less popular than other learning techniques.<\/li>\n\n\n\n<li><strong>Unsupervised<\/strong> learning methods can extract data representation even when no labels are available. To this end, they often attempt to fit a probability distribution to the data (aka generative models) or build a model with a smaller capacity to reconstruct the input (aka reconstruction-based models). An anomaly detector can efficiently find abnormal samples by looking at the former&#8217;s probability distribution or the model&#8217;s bottleneck in the letter.<\/li>\n\n\n\n<li>Self-Supervised: Self-Supervised learning is a new methodology in deep learning and is considered the state-of-the-art technique for learning good representations from unlabeled data. Just like unsupervised methods, they can learn from unannotated data and eliminate the expensive and time-consuming labeling process. However, to learn a good representation, these models define proxy tasks, generate pseudo-labels, and try to predict these labels in the output. By learning to solve this proxy task, which is often unrelated to the final task (e.g., anomaly detection or object classification), the model can learn meaningful patterns present in the data that indirectly benefits the downstream task. For instance, if we have an unlabeled dataset of animal images, the network can learn a good representation of them by rotating them in different directions and trying to predict the rotation angle. Since the network itself applies the rotation, it has the angle label and can build a supervised classifier to predict it. Empirically, recent research has shown that such a trained model can learn a representation in which different animals can be reality identified with the help of a simple classifier.<\/li>\n<\/ul>\n\n\n\n<p><strong>2. Learning Feature representations of Normality<\/strong><\/p>\n\n\n\n<p>This category of algorithms uses deep learning models to measure the data sample abnormality level indirectly. They can often learn an efficient score from unlabeled data by assuming the relationship of a network&#8217;s output with the degree of abnormality of the samples. An example of such algorithms is \u201cautoencoders\u201d; they, too, are deep models that attempt to reconstruct their input. Suppose we assume that the autoencoder is only trained on normal data; therefore, it will not be able to reconstruct abnormal samples well. In that case, we can use the reconstruction error to measure the degree of sample abnormality.<\/p>\n\n\n\n<p>It is worth noting that autoencoders can be used as an independent anomaly-scoring models that operate very efficiently and can precede other anomaly detection algorithms to augment their functionality.<\/p>\n\n\n\n<p><strong>3. End-to-End Anomaly Score Learning<\/strong><\/p>\n\n\n\n<p>Finally, the last category of deep learning algorithms in anomaly detection are those that can learn a direct anomaly score from the input data. Unlike the second category, this score directly reflects the degree of input abnormality and is often defined by an expert. Therefore, all models in this category can be considered supervised, as they require costly date-set labeling making them less popular and only used when data sets are already labeled.<\/p>\n\n\n\n<p>The recent surge in the popularity of deep learning has significantly improved anomaly detection algorithms. They opened doors for addressing problems that were previously perceived to be impossible to solve. It is fully expected that further improvements in deep learning models will also benefit the field of anomaly detection.<\/p>\n\n\n\n<p><strong>About the Author<\/strong><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"alignleft size-full\"><img decoding=\"async\" loading=\"lazy\" width=\"125\" height=\"125\" src=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/02\/Al-Gharakhanian.png\" alt=\"\" class=\"wp-image-31553\" srcset=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/02\/Al-Gharakhanian.png 125w, https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/02\/Al-Gharakhanian-110x110.png 110w, https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/02\/Al-Gharakhanian-50x50.png 50w\" sizes=\"(max-width: 125px) 100vw, 125px\" \/><\/figure><\/div>\n\n\n<p>Al Gharakhanian, Machine Learning Development Director, <a href=\"https:\/\/cognityze.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Cognityze<\/a><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><em>Sign up for the free insideBIGDATA\u00a0<a href=\"http:\/\/inside-bigdata.com\/newsletter\/\" target=\"_blank\" rel=\"noreferrer noopener\">newsletter<\/a>.<\/em><\/p>\n\n\n\n<p><em>Join us on Twitter:&nbsp;<a href=\"https:\/\/twitter.com\/InsideBigData1\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n\n\n\n<p><em>Join us on LinkedIn:&nbsp;<a href=\"https:\/\/www.linkedin.com\/company\/insidebigdata\/\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.linkedin.com\/company\/insidebigdata\/<\/a><\/em><\/p>\n\n\n\n<p><em>Join us on Facebook:&nbsp;<a href=\"https:\/\/www.facebook.com\/insideBIGDATANOW\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.facebook.com\/insideBIGDATANOW<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this contributed article, Al Gharakhanian, Machine Learning Development Director, Cognityze, takes a look at anomaly detection in terms of real-life use cases, addressing critical factors, along with the relationship with machine learning and artificial neural networks. <\/p>\n","protected":false},"author":10513,"featured_media":22375,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,115,182,180,67,268,56,97,1],"tags":[718,133,277,652,96],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Anomaly Detection: Its Real-Life Uses and the Latest Advances - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Anomaly Detection: Its Real-Life Uses and the Latest Advances - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"In this contributed article, Al Gharakhanian, Machine Learning Development Director, Cognityze, takes a look at anomaly detection in terms of real-life use cases, addressing critical factors, along with the relationship with machine learning and artificial neural networks.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2023-02-03T14:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-03T03:00:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/03\/Anomaly_detection_SHUTTERSTOCK.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"193\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/\",\"url\":\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/\",\"name\":\"Anomaly Detection: Its Real-Life Uses and the Latest Advances - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2023-02-03T14:00:00+00:00\",\"dateModified\":\"2023-02-03T03:00:03+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Anomaly Detection: Its Real-Life Uses and the Latest Advances\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning &amp; Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Anomaly Detection: Its Real-Life Uses and the Latest Advances - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/","og_locale":"en_US","og_type":"article","og_title":"Anomaly Detection: Its Real-Life Uses and the Latest Advances - insideBIGDATA","og_description":"In this contributed article, Al Gharakhanian, Machine Learning Development Director, Cognityze, takes a look at anomaly detection in terms of real-life use cases, addressing critical factors, along with the relationship with machine learning and artificial neural networks.","og_url":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2023-02-03T14:00:00+00:00","article_modified_time":"2023-02-03T03:00:03+00:00","og_image":[{"width":300,"height":193,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/03\/Anomaly_detection_SHUTTERSTOCK.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/","url":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/","name":"Anomaly Detection: Its Real-Life Uses and the Latest Advances - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2023-02-03T14:00:00+00:00","dateModified":"2023-02-03T03:00:03+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2023\/02\/03\/anomaly-detection-its-real-life-uses-and-the-latest-advances\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Anomaly Detection: Its Real-Life Uses and the Latest Advances"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning &amp; Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/03\/Anomaly_detection_SHUTTERSTOCK.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-8cU","jetpack-related-posts":[{"id":14037,"url":"https:\/\/insidebigdata.com\/2015\/11\/13\/numenta-anomaly-benchmark-evaluates-anomaly-detection-techniques-for-real-time-streaming-data\/","url_meta":{"origin":31552,"position":0},"title":"Numenta Anomaly Benchmark Evaluates Anomaly Detection Techniques for Real-time, Streaming Data","date":"November 13, 2015","format":false,"excerpt":"Numenta, Inc., a leader in machine intelligence, launched the Numenta Anomaly Benchmark (NAB), an open-source benchmark and tool designed to help data researchers evaluate the effectiveness of algorithms for anomaly detection in streaming, real-time applications.","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":12802,"url":"https:\/\/insidebigdata.com\/2015\/02\/27\/blue-coat-systems-prelert-partner-provide-anomaly-detection-security-solutions\/","url_meta":{"origin":31552,"position":1},"title":"Blue Coat Systems and Prelert Partner to Provide Anomaly Detection in Security Solutions","date":"February 27, 2015","format":false,"excerpt":"Prelert, a leading provider of machine learning anomaly detection, and Blue Coat Systems, Inc., a market leader in enterprise security, announced a partnership to integrate Prelert\u2019s machine learning analytics into Blue Coat\u2019s portfolio of security solutions.","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":24465,"url":"https:\/\/insidebigdata.com\/2020\/05\/30\/research-highlights-midas-real-time-anomaly-fake-news-intrusion-detection\/","url_meta":{"origin":31552,"position":2},"title":"Research Highlights: MIDAS &#8211; Real-time Anomaly\/Fake News\/Intrusion Detection","date":"May 30, 2020","format":false,"excerpt":"In the insideBIGDATA Research Highlights column we take a look at new and upcoming results from the research community for data science, machine learning, AI and deep learning. Our readers need to get a glimpse for technology coming down the pipeline that will make their efforts more strategic and competitive.\u2026","rel":"","context":"In &quot;AI Deep Learning&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/img.youtube.com\/vi\/Bd4PyLCHrto\/0.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":11113,"url":"https:\/\/insidebigdata.com\/2014\/08\/26\/prelert-extends-anomaly-detection-elasticsearch\/","url_meta":{"origin":31552,"position":3},"title":"Prelert Extends Anomaly Detection to Elasticsearch","date":"August 26, 2014","format":false,"excerpt":"Prelert, the anomaly detection company, today announced the release of an Elasticsearch Connector to help developers quickly and easily deploy its machine learning-based Anomaly Detective\u00ae engine on their Elasticsearch ELK (Elasticsearch, Logstash, Kibana) stack.","rel":"","context":"In &quot;Big Data&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":9346,"url":"https:\/\/insidebigdata.com\/2014\/05\/20\/prelert-announces-open-api-anomaly-detective-analytics-engine\/","url_meta":{"origin":31552,"position":4},"title":"Prelert Announces Open API to Anomaly Detective\u00ae Analytics Engine","date":"May 20, 2014","format":false,"excerpt":"Prelert, the anomaly detection company, today announced that it is enabling developers and power users to leverage its Anomaly Detective\u00ae advanced analytics engine through an open API.","rel":"","context":"In &quot;Analytics&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":31665,"url":"https:\/\/insidebigdata.com\/2023\/02\/20\/research-highlights-mit-develops-first-generative-model-for-anomaly-detection-that-combines-both-reconstruction-based-and-prediction-based-models\/","url_meta":{"origin":31552,"position":5},"title":"Research Highlights: MIT Develops First Generative Model for Anomaly Detection that Combines both Reconstruction-based and Prediction-based Models","date":"February 20, 2023","format":false,"excerpt":"Kalyan Veeramachaneni and his team at the MIT Data-to-AI (DAI) Lab have developed the first generative model, the AutoEncoder with Regression (AER) for time series anomaly detection, that combines both reconstruction-based and prediction-based models. They\u2019ve been building it for three years\u2014AER has been learning and extracting intelligence for signals and\u2026","rel":"","context":"In &quot;AI Deep Learning&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/31552"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=31552"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/31552\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/22375"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=31552"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=31552"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=31552"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}