{"id":22669,"date":"2019-06-27T06:30:50","date_gmt":"2019-06-27T13:30:50","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=22669"},"modified":"2019-06-28T09:04:45","modified_gmt":"2019-06-28T16:04:45","slug":"machine-learning-predefined-recipes","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/","title":{"rendered":"Machine Learning Beyond Predefined Recipes"},"content":{"rendered":"\n

In this sponsored post, Carlos Pazos, Product Marketing Manager at SparkCognition, explores the expanding requirements of machine leaerning tools today, and highlights SparkCognition’s “Darwin,” a machine learning tool designed to automate the building and deployment of machine learning models at scale. <\/em><\/p>\n\n\n\n

\"\"
Carlos Pazos, Product Marketing Manager, SparkCognition<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

While machine learning has enabled massive advancements across industries, it requires significant development and maintenance efforts from data science teams. The next evolution in human intelligence is automating the creation of machine learning models to not follow predefined formulas, but rather adapt and evolve according to the problem\u2019s data. The same way a tailored suit feels and looks different from generic options, tailored models perform differently than pre-established boxed algorithms because they are custom-fitted to your data. <\/p>\n\n\n\n

To answer this need, SparkCognition has developed Darwin, a machine learning product that automates the building and deployment of models at scale. Darwin uses a patented approach based on neuroevolution that custom builds model architectures to ensure the best fit for the problem at hand. Rather than simply choosing the best performer from a predefined list of algorithms, Darwin uses a blend of evolutionary and deep learning methods to iteratively find the most optimal model tailored to your data. This automated model building process effectively creates unique solutions that correctly and accurately generate predictions for your unique data problems.
<\/strong>
How Does Darwin Work?<\/strong><\/p>\n\n\n\n

Darwin automates three major steps in the data science process: cleaning, feature generation, and the construction of either a supervised or unsupervised model. Each step is performed as a single generation of Darwin\u2019s evolutionary process, which contains dozens of model architecture candidates. At the end of each generation Darwin keeps the best performers, analyzes their architectural characteristics, and spawns a new generation of models based on these features. This way, Darwin automatically generates thousands of models that evolve and improve with each generation to more accurately reflect the relationships in your data.<\/p>\n\n\n\n

\"machine<\/figure><\/div>\n\n\n\n

Cleaning<\/strong><\/p>\n\n\n\n

First, Darwin needs to convert data sets into a usable form for algorithmic development. This includes representing categorical data as numeric and extracting features that preserve temporal relationships in date\/time information. Data is also scaled to normalize data sets so features can be compared to one another.<\/p>\n\n\n\n

Feature Generation<\/strong><\/p>\n\n\n\n

Once data has been cleaned, data scientists often manipulate that data to generate more appropriate features to solve a particular problem. One of the biggest challenges in handling dynamic time series data is determining how to window the time steps for feature generation. Darwin automates this windowing process using one-dimensional convolutional neural networks (CNN). CNNs are a class of deep neural networks that use a type of multilayer perceptions designed to need only minimal preprocessing. The network instead automatically learns the filters that traditionally would need to be engineered by hand.<\/p>\n\n\n\n

Darwin begins by analyzing the characteristics of the input dataset and the specified problem, and then applying past knowledge to construct an initial population of machine learning models which are likely to produce accurate predictions on the problem. <\/p><\/blockquote>\n\n\n\n

Feature Selection and Model Building<\/strong><\/p>\n\n\n\n

Once automated cleaning and feature generation have taken place, the data set is ready to be used to build a model. Through neuroevolution, Darwin is capable of building both supervised learning and normal behavior models. These methods differ in how they work and the problems they solve.<\/p>\n\n\n\n

For supervised learning problems, the goal of Darwin is to ingest the cleaned input data and automatically produce a highly optimized machine learning model which can accurately predict a target of interest specified by the user. Darwin accomplishes this using a patented evolutionary algorithm which simultaneously optimizes and compares various machine learning methodologies, most heavily favoring deep neural networks.<\/p>\n\n\n\n

Darwin begins by analyzing the characteristics of the input dataset and the specified problem, and then applying past knowledge to construct an initial population of machine learning models which are likely to produce accurate predictions on the problem. Then, traits from the best-performing models are combined to yield even better models<\/a> over many generations. This ensures a final model that is highly optimized to the specified problem.<\/p>\n\n\n\n

In the same way that Darwin uses an evolutionary algorithm to solve supervised problems, it is also capable of identifying relationships in data that drift over time using a technique called normal behavioral modeling. Darwin does normal behavior modeling through an autoencoder, which is a neural network-based approach that performs dimensionality reduction. Autoencoders compress data to reduce the feature set to the smallest size possible, and then decompress it with as little as loss possible. <\/p>\n\n\n\n

Like any other neural network, autoencoders have numerous hidden layers, a defined latent space, and different activation functions in their encoding\/decoding process. Darwin automates the creation of this network topology, and then performs backpropagation with dropout to reduce the output loss via weight optimization. When deployed in production, the model\u2019s ability to reconstruct data over time helps to identify shifting relationships in data.<\/p>\n\n\n\n

Darwin uses this approach to build models that go beyond a traditional \u201crisk index\u201d and can identify anomalous operations and systems failures. <\/p>\n\n\n\n

How good is this process? Read Darwin\u2019s Efficacy Report<\/a>to learn more.<\/p>\n\n\n\n

Carlos Pazos is a Product Marketing Manager at SparkCognition<\/a> responsible for automated model building and natural language processing solutions.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

The next evolution in human intelligence is automating the creation of machine learning models to not follow predefined formulas, but rather adapt and evolve according to the problem\u2019s data. While machine learning has enabled massive advancements across industries, it requires significant development and maintenance efforts from data science teams. Enter Darwin, a machine learning tool that automates the building and deployment of models at scale. <\/p>\n","protected":false},"author":10513,"featured_media":22671,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,87,180,67,56,311,57],"tags":[437,748,277,652,746,95],"acf":[],"yoast_head":"\nMachine Learning Beyond Predefined Recipes<\/title>\n<meta name=\"description\" content=\"This sponsored post from SparkCognition explores tools designed to automate the building and deployment of machine learning models at scale.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning Beyond Predefined Recipes\" \/>\n<meta property=\"og:description\" content=\"This sponsored post from SparkCognition explores tools designed to automate the building and deployment of machine learning models at scale.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2019-06-27T13:30:50+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-06-28T16:04:45+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/05\/Carlos-Pazos-Headshot.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"322\" \/>\n\t<meta property=\"og:image:height\" content=\"450\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/\",\"url\":\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/\",\"name\":\"Machine Learning Beyond Predefined Recipes\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2019-06-27T13:30:50+00:00\",\"dateModified\":\"2019-06-28T16:04:45+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"description\":\"This sponsored post from SparkCognition explores tools designed to automate the building and deployment of machine learning models at scale.\",\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Machine Learning Beyond Predefined Recipes\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning Beyond Predefined Recipes","description":"This sponsored post from SparkCognition explores tools designed to automate the building and deployment of machine learning models at scale.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/","og_locale":"en_US","og_type":"article","og_title":"Machine Learning Beyond Predefined Recipes","og_description":"This sponsored post from SparkCognition explores tools designed to automate the building and deployment of machine learning models at scale.","og_url":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2019-06-27T13:30:50+00:00","article_modified_time":"2019-06-28T16:04:45+00:00","og_image":[{"width":322,"height":450,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/05\/Carlos-Pazos-Headshot.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/","url":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/","name":"Machine Learning Beyond Predefined Recipes","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2019-06-27T13:30:50+00:00","dateModified":"2019-06-28T16:04:45+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"description":"This sponsored post from SparkCognition explores tools designed to automate the building and deployment of machine learning models at scale.","breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2019\/06\/27\/machine-learning-predefined-recipes\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Machine Learning Beyond Predefined Recipes"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2019\/05\/Carlos-Pazos-Headshot.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-5TD","jetpack-related-posts":[{"id":22644,"url":"https:\/\/insidebigdata.com\/2019\/05\/30\/darwin-machine-learning-platform\/","url_meta":{"origin":22669,"position":0},"title":"SparkCognition’s Darwin Machine Learning Platform Designed to Accelerate Data Science at Scale","date":"May 30, 2019","format":false,"excerpt":"As machine learning technology becomes more widely available on an enterprise scale, differentiating and studying which platform can be best for your business can be difficult. A new white paper from SparkCognition explores one of the solutions on the market that works to accelerate data science at scale. Its Darwin\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"data catalogs","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/05\/shutterstock_675107101.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":22638,"url":"https:\/\/insidebigdata.com\/2019\/05\/23\/alternative-data-investment-management\/","url_meta":{"origin":22669,"position":1},"title":"How Alternative Data is Paving the Way for the Future of Investment Management","date":"May 23, 2019","format":false,"excerpt":"Many hedge fund managers to mutual funds \u2014 and even private equity managers \u2014 are turning to alternative data to pave the way for the future of investment management. SparkCognition contends that alternative data has the power to improve valuation of securities and ramp up clarity of the investment process.\u2026","rel":"","context":"In "Data Science"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/05\/shutterstock_606840716.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":23720,"url":"https:\/\/insidebigdata.com\/2019\/12\/19\/machine-learning-for-all-the-democratizing-of-a-technology\/","url_meta":{"origin":22669,"position":2},"title":"Machine Learning for All: the Democratizing of a Technology","date":"December 19, 2019","format":false,"excerpt":"Our friends over at H2O.ai have produced a short new eBook \"Machine learning for all: the democratizing of a technology\" which covers machine learning features and automatic AI solutions, and how organizations can benefit from using them.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/12\/H20.ai_WPCover1_2019-12-10_10-46-11.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":23925,"url":"https:\/\/insidebigdata.com\/2020\/02\/03\/do-you-trust-and-understand-your-predictive-models\/","url_meta":{"origin":22669,"position":3},"title":"Do You Trust and Understand Your Predictive Models?","date":"February 3, 2020","format":false,"excerpt":"To help practitioners make the most of recent and disruptive breakthroughs in debugging, explainability, fairness, and interpretability techniques for machine learning, our friends over at H2O.ai have written and exciting eBook \u201cAn Introduction to Machine Learning Intrepretability Second Edition.\u201d This report defines key terms, introduces the human and commercial motivations\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":17908,"url":"https:\/\/insidebigdata.com\/2017\/05\/18\/bringing-artificial-intelligence-life-intel\/","url_meta":{"origin":22669,"position":4},"title":"Bringing Artificial Intelligence to Life","date":"May 18, 2017","format":false,"excerpt":"Artificial Intelligence (AI) may seem like a vision for a distant future, but in truth, AI is all around us as machines are increasingly learning to sense, learn, reason, act and adapt in the real world. This is transforming industries and changing our lives in amazing new ways, by amplifying\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":20183,"url":"https:\/\/insidebigdata.com\/2018\/04\/03\/crowdflower-unveils-new-machine-learning-solutions-changes-name-figure-eight\/","url_meta":{"origin":22669,"position":5},"title":"CrowdFlower Unveils New Machine Learning Solutions; Changes Name to Figure Eight","date":"April 3, 2018","format":false,"excerpt":"CrowdFlower, the essential Human-in-the-Loop artificial intelligence platform for data science and machine learning teams, today unveiled new machine learning solutions that will help companies reduce the time to apply AI to their business and generate business impact from real world AI applications.","rel":"","context":"In "Big Data"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2014\/11\/crowdflower_logo.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/22669"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=22669"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/22669\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/22671"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=22669"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=22669"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=22669"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}