{"id":32673,"date":"2023-06-17T04:00:00","date_gmt":"2023-06-17T11:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=32673"},"modified":"2023-06-16T17:33:15","modified_gmt":"2023-06-17T00:33:15","slug":"synthetic-data-the-cure-to-data-drift","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/","title":{"rendered":"Synthetic Data: The Cure to Data Drift?"},"content":{"rendered":"\n

Recent advancements in AI and computer vision capabilities have massively increased the scale and demand for training data. While real world data continues to dominate AI training, it is often becoming out of date in as short as six months. This is an area of concern as constantly evolving trends and the need for businesses to stay agile, leave little to no room for error in decision making.<\/p>\n\n\n\n

It’s critical that organisations have available reliable, accurate training data more than ever before. Yet we recently found that almost two-thirds of organisations suffer from data drift in their training data.<\/p>\n\n\n\n

Data drift is a discrepancy between the actual data processed by the deployed system and the training data used to train, validate and test the AI model that processes that real world input. This can arise as a result of various factors, including seasonal variations, climate change and even changes in fashion. Regularly monitoring the performance of a computer vision model is essential to successful deployment. If data drift is not identified in time, it can have serious implications on model performance leading to incorrect business decisions being made.<\/p>\n\n\n\n

This phenomenon can be manageable if dealt with appropriately, usually requiring retraining of the model on new data but the effort needed will vary depending on the extent of the issue. This can be disruptive, causing ongoing problems for organisations and be a costly problem to solve. Therefore, detecting data drift should be a key part of the machine learning lifecycle. Ideally this should be an automated process supported by careful action. <\/p>\n\n\n\n

What actions can be taken?<\/strong><\/p>\n\n\n\n

Methods of dealing with data drift are often not mutually exclusive, meaning multiple strategies can and may need to be employed. An effective solution to minimising potential data drift has emerged in the form of synthetic training data. It is artificially generated from computer systems and provides the opportunity to produce greater volumes of accurate training data quickly and more cost-effectively than acquiring real world training data. But, beyond this, it can enhance the robustness of AI models by delivering training data for edge-cases that may be difficult or dangerous to repeat in the real world.<\/p>\n\n\n\n

Systems that create synthetic training data allow users to generate training data on demand as opposed to waiting for real-world occurrences, enabling greater control\u00a0over the training process and providing an opportunity to act before data becomes obsolete.\u00a085% of organisations<\/a>\u00a0are already making use of synthetic data to train computer vision systems and of those who don\u2019t, almost a third (29%) anticipate their organization will start using it in 2023.<\/p>\n\n\n\n

How can synthetic data ensure data drift is a thing of the past?<\/strong><\/p>\n\n\n\n

Synthetic data offers a plethora of advantages. It\u2019s fast to create, easy to update and cost effective when compared to acquiring real world training data. In particular annotation of real-world training data is labour intensive, time consuming, expensive and less accurate than annotation of synthetic data which is an automated and pixel accurate process. Synthetic training data can also be intelligently created in greater volumes, which is particularly beneficial in building more robust AI models. By filling in gaps and supplementing real-world data, the use of synthetic training data can alleviate the fundamental issues leading to data drift.<\/p>\n\n\n\n

Another key advantage of synthetic data is the opportunity to optimise training efficiency. Large volumes of synthetic data can be generated much more rapidly than the alternative of collating real-world data. Users are therefore able to quickly gather training data for cases where new data is needed immediately.<\/p>\n\n\n\n

For example, at the height of the pandemic, the mandate of face masks and social distancing meant that some AI systems were outdated, and needed to be retrained to recognise someone wearing a face covering. Another example is the deployment of electric scooters, which has also harnessed machine vision for harm detection and aids in preventing accidents. In addition to updating datasets to prevent data drift, data that is no longer relevant should be removed too. This can be done efficiently with the help of synthetic data training.<\/p>\n\n\n\n

Training datasets containing private data present a risk of violating privacy regulations when used to train models. Synthetic data avoids this risk as it does not contain information traceable to individuals. Ensuring privacy compliance is essential to protecting individuals and businesses from legal and financial consequences, as well as aiding in building trust in AI. <\/p>\n\n\n\n

Overall, synthetic data provides robust and versatile datasets for AI training purposes. It does not rely on manual efforts and so, is quicker, comprehensive and more cost-effective to gather. With technological advancement and innovation, synthetic data is becoming richer, more diverse, and closely aligned to real world data. It can help to maintain user privacy and keep enterprises compliant, all of which furthers its ability to overcome the potential of data drift.<\/p>\n\n\n\n

About the Author<\/strong><\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

Steve Harris, CEO of\u00a0Mindtech<\/a>, has over 30 years of experience in the technology market sector and holds a masters in Microprocessor Engineering from Manchester University. He has previously been instrumental in creating several European start-up organisations, with a proven track record of success in building strategic relationships and strong revenue streams with tier one companies worldwide. Prior to his current role, he has worked in a number of senior sales and business development positions at leading technology companies, such as:\u00a0Imagination Technologies,\u00a0Gemstar,\u00a0Liberate, and\u00a0Sun Microsystems,\u00a0allowing him to bring a wealth of insight and expertise to Mindtech.<\/em><\/p>\n\n\n\n

Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

Join us on Twitter: https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n\n\n\n

Join us on LinkedIn: https:\/\/www.linkedin.com\/company\/insidebigdata\/<\/a><\/em><\/p>\n\n\n\n

Join us on Facebook: https:\/\/www.facebook.com\/insideBIGDATANOW<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

In this contributed article, Steve Harris, CEO of\u00a0Mindtech, discusses how data drift is a growing problem affecting almost two-thirds of organizations. Synthetic training data is becoming richer, more diverse, and closely aligned to real world data and has emerged as a solution to overcome the potential of data drift.<\/p>\n","protected":false},"author":10513,"featured_media":32610,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,115,182,180,61,67,56,97,1],"tags":[437,1319,277,1069,96],"acf":[],"yoast_head":"\nSynthetic Data: The Cure to Data Drift? - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Synthetic Data: The Cure to Data Drift? - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"In this contributed article, Steve Harris, CEO of\u00a0Mindtech, discusses how data drift is a growing problem affecting almost two-thirds of organizations. Synthetic training data is becoming richer, more diverse, and closely aligned to real world data and has emerged as a solution to overcome the potential of data drift.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-17T11:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-17T00:33:15+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/Synthetic_data_shutterstock_2306922019_special.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1280\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/\",\"url\":\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/\",\"name\":\"Synthetic Data: The Cure to Data Drift? - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2023-06-17T11:00:00+00:00\",\"dateModified\":\"2023-06-17T00:33:15+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Synthetic Data: The Cure to Data Drift?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Synthetic Data: The Cure to Data Drift? - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/","og_locale":"en_US","og_type":"article","og_title":"Synthetic Data: The Cure to Data Drift? - insideBIGDATA","og_description":"In this contributed article, Steve Harris, CEO of\u00a0Mindtech, discusses how data drift is a growing problem affecting almost two-thirds of organizations. Synthetic training data is becoming richer, more diverse, and closely aligned to real world data and has emerged as a solution to overcome the potential of data drift.","og_url":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2023-06-17T11:00:00+00:00","article_modified_time":"2023-06-17T00:33:15+00:00","og_image":[{"width":2560,"height":1280,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/Synthetic_data_shutterstock_2306922019_special.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/","url":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/","name":"Synthetic Data: The Cure to Data Drift? - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2023-06-17T11:00:00+00:00","dateModified":"2023-06-17T00:33:15+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2023\/06\/17\/synthetic-data-the-cure-to-data-drift\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Synthetic Data: The Cure to Data Drift?"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2023\/06\/Synthetic_data_shutterstock_2306922019_special.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-8uZ","jetpack-related-posts":[{"id":28673,"url":"https:\/\/insidebigdata.com\/2022\/03\/08\/appen-invests-in-synthetic-data-business-mindtech\/","url_meta":{"origin":32673,"position":0},"title":"Appen Invests in Synthetic Data Business Mindtech","date":"March 8, 2022","format":false,"excerpt":"Appen Limited, a leader in data for the AI Lifecycle, announced the investment in Mindtech, a synthetic data company specializing in the creation of high-quality training data for AI computer vision models. As part of the investment, Appen has formed a commercial partnership agreement with Mindtech.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":29270,"url":"https:\/\/insidebigdata.com\/2022\/05\/09\/introducing-innodatas-new-ai-data-marketplace\/","url_meta":{"origin":32673,"position":1},"title":"Introducing Innodata\u2019s New AI Data Marketplace","date":"May 9, 2022","format":false,"excerpt":"Innodata is excited to announce its new Innodata AI Data Marketplace \u2013\u00a0an e-commerce portal where users can purchase on-demand\u00a0datasets to accelerate AI\/ML model building and training. With easy access to curated, industry-leading datasets, data science teams can now overcome persistent data challenges that often hamper AI initiatives, such as volume,\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":23280,"url":"https:\/\/insidebigdata.com\/2019\/09\/18\/best-of-arxiv-org-for-ai-machine-learning-and-deep-learning-august-2019\/","url_meta":{"origin":32673,"position":2},"title":"Best of arXiv.org for AI, Machine Learning, and Deep Learning \u2013 August 2019","date":"September 18, 2019","format":false,"excerpt":"In this recurring monthly feature, we will filter all the recent research papers appearing in the arXiv.org preprint server for subjects relating to AI, machine learning and deep learning \u2013 from disciplines including statistics, mathematics and computer science \u2013 and provide you with a useful \u201cbest of\u201d list for the\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2013\/12\/arxiv.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":27623,"url":"https:\/\/insidebigdata.com\/2021\/11\/17\/book-review-synthetic-data-for-deep-learning\/","url_meta":{"origin":32673,"position":3},"title":"Book Review: Synthetic Data for Deep Learning","date":"November 17, 2021","format":false,"excerpt":"\"Synthetic Data for Deep Learning,\" by Sergey I. Nikolenko (published by Springer), represents a very good academic treatment of the subject. But what gives the book more street cred is the fact that the author is also Chief Research Officer for Synthesis AI, a start-up company pioneering this accelerating field.\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2021\/11\/SyntheticData_book_fig1.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":30471,"url":"https:\/\/insidebigdata.com\/2022\/09\/28\/synthesized-solidifies-its-partnership-with-deutsche-bank-providing-high-quality-synthetic-data-for-ai-and-ml-testing-purposes\/","url_meta":{"origin":32673,"position":4},"title":"Synthesized Solidifies its Partnership with Deutsche Bank, Providing High-quality Synthetic Data for AI and ML Testing Purposes","date":"September 28, 2022","format":false,"excerpt":"Synthesized Ltd, a leading synthetic data generation platform, which provides engineering and data science teams a quick way to create and share trusted data through advanced machine learning and automation, announced that Deutsche Bank is investing in its next phase of growth and technology innovation development.\u00a0","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":28431,"url":"https:\/\/insidebigdata.com\/2022\/02\/09\/7-reasons-for-bias-in-ai-and-what-to-do-about-it\/","url_meta":{"origin":32673,"position":5},"title":"7 Reasons For Bias In AI and What To Do About It","date":"February 9, 2022","format":false,"excerpt":"In this contributed article, Alexandra Ebert, Chief Trust Officer at MOSTLY AI, discusses 7 important ways that machine learning models become biased along with techniques for prevention. The power of AI is that it can scale processes so effortlessly that they can amplify both the good and the bad way\u2026","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2019\/05\/Artificial_intelligence_SHUTTERSTOCK.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/32673"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=32673"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/32673\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/32610"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=32673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=32673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=32673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}