{"id":25619,"date":"2021-02-12T06:00:00","date_gmt":"2021-02-12T14:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=25619"},"modified":"2021-02-16T10:47:07","modified_gmt":"2021-02-16T18:47:07","slug":"circular-statistics-in-python-an-intuitive-intro","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/","title":{"rendered":"Circular Statistics in Python: An Intuitive Intro"},"content":{"rendered":"\n

In a wide range of scientific disciplines, the observations are that directions have periodic nature measured in degrees or radians. Such data should be analyzed on an angular scale with respect to a chosen \u201czero-direction\u201d and an essence of \u201crotation\u201d. For instance, due to the fact that 0\u00b0 and 360\u00b0 are identical angles, the sum of 20\u00b0 and 350\u00b0 angles is equal to 10\u00b0, not 370\u00b0.<\/p>\n\n\n\n

In this article we will review some basic principles and tools of circular statistics, as well as the reasons why conventional linear methods would not work well on circular data. Furthermore, we will reveal on how you can construct a simple noise filter from these basic tools.<\/p>\n\n\n\n

Circular Mean<\/strong><\/p>\n\n\n\n

Conventional methods suitable for the analysis of linear data don\u2019t fit angular data. First, let\u2019s consider a zero-mean \u2013 normally distributed angular signal with standard deviation (std) of 0.3 radian.  At first glance, when examining this distribution over the real plane (Fig 1.a), it appears to come from two populations being wrapped between 0 to 2\u03c0 rad. Calculating the familiar linear arithmetic mean will return \u03c0 as a result, but, after the signal is represented as points on the circumference of a unit circle (Fig 1.b), it is clear that the mean is actually 0 rad.<\/p>\n\n\n\n

\"\"
Fig 1. Linear vs circular mean: a. Angular data, linear mean marked in red. b. The angular signal over the unit circle after polar transformation.<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

By assuming linearity, we ignore the data’s real topography: any two points 2\u03c0 apart in the real plane will be transformed into the same location over the unit circle. Disregarding the periodic nature of the signal makes many linear techniques often misleading.<\/p>\n\n\n\n

So, what can we do?<\/strong><\/strong><\/p>\n\n\n\n

By using polar transformation, we can map the distribution to Cartesian coordinates. Any point over the circle can be described as (X,Y) in terms of Cartesian coordinates, using the trigonometric functions cosine and sine for the translation:<\/p>\n\n\n\n

\"\"
Fig 2. Polar transformation<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

Wher e is an angle in [0, 2\u03c0) and r set to a const value of unit length
(r = 1) for convenience.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Algorithm:<\/strong><\/p>\n\n\n\n

1. Convert each sample to 2d representation from polar coordinates to Cartesian coordinates.
2. Calculate the arithmetic mean over each component separately.<\/p>\n\n\n\n

3. Convert back using atan2.<\/p>\n\n\n\n

Code:<\/strong><\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Circular Median<\/strong><\/strong><\/p>\n\n\n\n

In order to compute the median of circular data, we must first address the question of \u2018circular distance\u2019 \u2013 the difference in the measured values of two points when the data that they represent is angular.<\/p>\n\n\n\n

How Do We Estimate The Circular Distance?<\/strong><\/p>\n\n\n\n

The most popular measure for distance is the Euclidean distance, which works perfectly on linear data in the Euclidean space. In the \u201ccircular world\u201d, there are a couple of strong candidates when the most intuitive among them is based on arc lengths, taking the smaller of the two arc-lengths between the points along the circumference:<\/p>\n\n\n\n

\"\"
Fig 3. Circular distance: The red arc is shorter than the purple, hence, we define the circular distance to be the arclength ACB<\/em><\/figcaption><\/figure><\/div>\n\n\n\n

Where  and  represent the angles corresponding to the points A, B over the unit circle. The red arc is shorter than the purple, hence, we define the circular distance to be the arclength ACB. The circular distance always lies in [0,\u03c0) since there are no two points on the circumference of a circle that can be farther than \u03c0.<\/p>\n\n\n\n

Code:<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Let\u2019s use the definition of \u201ccircular distance\u201d for calculating the median of a distribution.<\/p>\n\n\n\n

In statistics, a median is a value separating the higher half from the lower half of a data sample. It can be handy when treating non-Normal distributions because it is less affected by a small proportion of extremely large or small values. The median of a data sample can be defined as the observation with the minimum distance to all the other observation in the sample. What\u2019s left is to combine the definition of the median with the definition of circular distance:<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Where d is the circular distance between two angular observations.<\/p>\n\n\n\n

Code:<\/strong><\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Removing undesirable aspects of a signal is always very important, as it will reduce noise and improve the data quality for better modeling and data inference. Analyzing circular data is tricky, however, analyzing noisy circular data might prove to be quite challenging.<\/p>\n\n\n\n

We discussed the polar transformation and reviewed powerful tools for analysis: mean, median and \u201ccircular distance\u201d. Now if we combine these to construct our filter through the following steps:<\/p>\n\n\n\n

  1. Calculate the distance between two adjacent samples.<\/li>
  2. Set low and high thresholds for this distance.<\/li>
  3. Filter the observation above or below these thresholds.<\/li>
  4. Convert each sample to Cartesian coordinates.<\/li>
  5. Use a \u2018cubic\u2019 interpolation on each component separately.<\/li>
  6. Recombine using atan2.<\/li><\/ol>\n\n\n\n

    We will have this code:<\/p>\n\n\n\n

    \"\"<\/figure><\/div>\n\n\n\n

    Ultimately, there are big differences between linear and circular data, which is why it is so important to use the appropriate techniques and models for such data. By also using algorithms for calculating mean and distance and by filter outliers, these are the first steps one can take in exploring the world of directional data.<\/p>\n\n\n\n

    About the Author<\/strong><\/p>\n\n\n\n

    \"\"<\/figure><\/div>\n\n\n\n

    Amit Babayoff is a data scientist at Deeyook<\/a>, a precise location technology company that develops indoor and outdoor navigation solutions. Amit holds a Master\u2019s Degree in Computation Neuroscience and a Bachelor\u2019s Degree in CS and Neuroscience .She has a strong interest in ML, (biological and computer) vision and robotics, and enjoys solving complicated problems in multidisciplinary fields.<\/em><\/p>\n\n\n\n

    Sign up for the free insideBIGDATA newsletter<\/a>.<\/em><\/p>\n\n\n\n

    Join us on Twitter: @InsideBigData1 \u2013 https:\/\/twitter.com\/InsideBigData1<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

    In this contributed article, Amit Babayoff, a data scientist at Deeyook, discusses the principles of circular statistics, by looking at some its basic principles and tools and why conventional linear methods don\u2019t work well on circular data. She also explores how a simple filtering for handling noise can be constructed from these basic tools.<\/p>\n","protected":false},"author":10513,"featured_media":24971,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[526,68,87,180,67,56,97,1],"tags":[133,277,337,134,96],"acf":[],"yoast_head":"\nCircular Statistics in Python: An Intuitive Intro - insideBIGDATA<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Circular Statistics in Python: An Intuitive Intro - insideBIGDATA\" \/>\n<meta property=\"og:description\" content=\"In this contributed article, Amit Babayoff, a data scientist at Deeyook, discusses the principles of circular statistics, by looking at some its basic principles and tools and why conventional linear methods don\u2019t work well on circular data. She also explores how a simple filtering for handling noise can be constructed from these basic tools.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/\" \/>\n<meta property=\"og:site_name\" content=\"insideBIGDATA\" \/>\n<meta property=\"article:publisher\" content=\"http:\/\/www.facebook.com\/insidebigdata\" \/>\n<meta property=\"article:published_time\" content=\"2021-02-12T14:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-02-16T18:47:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/insidebigdata.com\/wp-content\/uploads\/2020\/09\/data_engineering_shutterstock_669226057.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"168\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Editorial Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:site\" content=\"@insideBigData\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Editorial Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/\",\"url\":\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/\",\"name\":\"Circular Statistics in Python: An Intuitive Intro - insideBIGDATA\",\"isPartOf\":{\"@id\":\"https:\/\/insidebigdata.com\/#website\"},\"datePublished\":\"2021-02-12T14:00:00+00:00\",\"dateModified\":\"2021-02-16T18:47:07+00:00\",\"author\":{\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\"},\"breadcrumb\":{\"@id\":\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/insidebigdata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Circular Statistics in Python: An Intuitive Intro\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/insidebigdata.com\/#website\",\"url\":\"https:\/\/insidebigdata.com\/\",\"name\":\"insideBIGDATA\",\"description\":\"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/insidebigdata.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9\",\"name\":\"Editorial Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g\",\"caption\":\"Editorial Team\"},\"sameAs\":[\"http:\/\/www.insidebigdata.com\"],\"url\":\"https:\/\/insidebigdata.com\/author\/editorial\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Circular Statistics in Python: An Intuitive Intro - insideBIGDATA","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/","og_locale":"en_US","og_type":"article","og_title":"Circular Statistics in Python: An Intuitive Intro - insideBIGDATA","og_description":"In this contributed article, Amit Babayoff, a data scientist at Deeyook, discusses the principles of circular statistics, by looking at some its basic principles and tools and why conventional linear methods don\u2019t work well on circular data. She also explores how a simple filtering for handling noise can be constructed from these basic tools.","og_url":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/","og_site_name":"insideBIGDATA","article_publisher":"http:\/\/www.facebook.com\/insidebigdata","article_published_time":"2021-02-12T14:00:00+00:00","article_modified_time":"2021-02-16T18:47:07+00:00","og_image":[{"width":300,"height":168,"url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2020\/09\/data_engineering_shutterstock_669226057.jpg","type":"image\/jpeg"}],"author":"Editorial Team","twitter_card":"summary_large_image","twitter_creator":"@insideBigData","twitter_site":"@insideBigData","twitter_misc":{"Written by":"Editorial Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/","url":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/","name":"Circular Statistics in Python: An Intuitive Intro - insideBIGDATA","isPartOf":{"@id":"https:\/\/insidebigdata.com\/#website"},"datePublished":"2021-02-12T14:00:00+00:00","dateModified":"2021-02-16T18:47:07+00:00","author":{"@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9"},"breadcrumb":{"@id":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/insidebigdata.com\/2021\/02\/12\/circular-statistics-in-python-an-intuitive-intro\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/insidebigdata.com\/"},{"@type":"ListItem","position":2,"name":"Circular Statistics in Python: An Intuitive Intro"}]},{"@type":"WebSite","@id":"https:\/\/insidebigdata.com\/#website","url":"https:\/\/insidebigdata.com\/","name":"insideBIGDATA","description":"Your Source for AI, Data Science, Deep Learning & Machine Learning Strategies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/insidebigdata.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/2949e412c144601cdbcc803bd234e1b9","name":"Editorial Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/insidebigdata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e137ce7ea40e38bd4d25bb7860cfe3e4?s=96&d=mm&r=g","caption":"Editorial Team"},"sameAs":["http:\/\/www.insidebigdata.com"],"url":"https:\/\/insidebigdata.com\/author\/editorial\/"}]}},"jetpack_featured_media_url":"https:\/\/insidebigdata.com\/wp-content\/uploads\/2020\/09\/data_engineering_shutterstock_669226057.jpg","jetpack_shortlink":"https:\/\/wp.me\/p9eA3j-6Fd","jetpack-related-posts":[{"id":13621,"url":"https:\/\/insidebigdata.com\/2015\/08\/31\/data-science-101-an-introduction-to-scikit-learn-machine-learning-in-python\/","url_meta":{"origin":25619,"position":0},"title":"Data Science 101: An Introduction to scikit-learn – Machine Learning in Python","date":"August 31, 2015","format":false,"excerpt":"The tutorial presentation below offers an introduction to the scikit-learn package and to the central concepts of Machine Learning.","rel":"","context":"In "Data Science"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":14084,"url":"https:\/\/insidebigdata.com\/2015\/11\/23\/book-review-doing-math-with-python\/","url_meta":{"origin":25619,"position":1},"title":"Book Review: Doing Math with Python","date":"November 23, 2015","format":false,"excerpt":"When one of my favorite independent tech book publishers, No Starch Press, notified me about their new title \"Doing Math with Python,\" I was energized to review what potentially could be a good new resource for budding data scientists.","rel":"","context":"In "Book Review"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":19054,"url":"https:\/\/insidebigdata.com\/2017\/10\/17\/book-review-python-data-science-handbook\/","url_meta":{"origin":25619,"position":2},"title":"Book Review: Python Data Science Handbook","date":"October 17, 2017","format":false,"excerpt":"I recently had a need for a Python language resource to supplement a series of courses on Deep Learning I was evaluating that depended on this widely used language. As a long-time data science practitioner, my language of choice has been R, so I relished the opportunity to dig into\u2026","rel":"","context":"In "Book Review"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2017\/10\/Python-Data-Science-Handbook.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":24794,"url":"https:\/\/insidebigdata.com\/2020\/07\/24\/how-to-launch-your-data-science-career\/","url_meta":{"origin":25619,"position":3},"title":"How to Launch Your Data Science Career","date":"July 24, 2020","format":false,"excerpt":"Are you interested in getting into the field of data science? We don\u2019t blame you. Data science is an exciting field that\u2019s constantly changing and developing, which gives data scientists\u2019 work endless potential. Here are six tips for ways that you can launch your data science career.","rel":"","context":"In "AI Deep Learning"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":33215,"url":"https:\/\/insidebigdata.com\/2023\/08\/24\/anaconda-distribution-for-python-brings-data-science-to-hundreds-of-millions-of-microsoft-excel-users\/","url_meta":{"origin":25619,"position":4},"title":"Anaconda Distribution for Python Brings Data Science to Hundreds of Millions of Microsoft Excel Users","date":"August 24, 2023","format":false,"excerpt":"Anaconda Inc., the provider of one of the world\u2019s most widely used and trusted data science and AI platforms, announced the beta availability of Anaconda Distribution for Python in Excel, a new integration with\u00a0Microsoft\u00a0Excel.\u00a0 Anaconda\u2019s Python distribution is fully embedded and integrated into the Excel grid toolboxes for manipulating, analyzing,\u2026","rel":"","context":"In "Cloud"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/insidebigdata.com\/wp-content\/uploads\/2023\/08\/Data_Science_shutterstock_1247255884_special.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":6104,"url":"https:\/\/insidebigdata.com\/2013\/12\/09\/data-science-wars-python-vs-r\/","url_meta":{"origin":25619,"position":5},"title":"Data Science Wars: Python vs. R","date":"December 9, 2013","format":false,"excerpt":"As I frequently travel in data science circles, I'm hearing more and more about a new kind of tech war: Python vs. R. I've lived through many tech wars in the past, e.g. Windows vs. Linux, iPhone vs. Android, etc., but this tech war seems to have a different flavor\u2026","rel":"","context":"In "Analytics"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/25619"}],"collection":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/users\/10513"}],"replies":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/comments?post=25619"}],"version-history":[{"count":0,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/posts\/25619\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media\/24971"}],"wp:attachment":[{"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/media?parent=25619"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/categories?post=25619"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/insidebigdata.com\/wp-json\/wp\/v2\/tags?post=25619"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}