{"id":12345,"date":"2014-11-12T06:00:39","date_gmt":"2014-11-12T14:00:39","guid":{"rendered":"http:\/\/insidebigdata.com\/?p=12345"},"modified":"2014-11-12T08:38:20","modified_gmt":"2014-11-12T16:38:20","slug":"ask-data-scientist-data-science-process","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2014\/11\/12\/ask-data-scientist-data-science-process\/","title":{"rendered":"Ask a Data Scientist: The Data Science Process"},"content":{"rendered":"

\"datascientist2_featured\"<\/a>Welcome back to our series of articles sponsored by Intel \u2013 \u201cAsk a Data Scientist.\u201d Once a week you\u2019ll see reader submitted questions of varying levels of technical detail answered by a practicing data scientist \u2013 sometimes by me and other times by an Intel data scientist. Think of this new insideBIGDATA feature as a valuable resource for you to get up to speed in this flourishing area of technology. If you have a big data question you\u2019d like answered, please just enter a comment below, or send an e-mail to me at: daniel@insidehpc.com<\/a>. This week\u2019s question is from a reader who wonders if there is a general process for conducting data science projects.<\/p>\n

Q:<\/strong> Is there a typical “data science process?”<\/strong>
\n<\/b><\/p>\n

A:<\/strong> Great question! There most definitely is a general formula followed by data scientists in striving to achieve best practices with a data science project. The figure below encapsulates the high-level steps in the so-called \u201cdata sci\u00adence pipeline\u201d that contribute to the success of a project: under\u00adstanding the goal of the project, data access, data munging, exploratory data analysis, feature engi\u00adneering, model selection, model validation, deploy, visualization and communicate the results.<\/p>\n

\"DataScienceProcess\"<\/a><\/p>\n

Data science involves understanding and preparing the data, defining the statistical learning model, and following the data science process. Statistical learning models can assume many shapes and sizes, depending on their complexity and the application for which they are designed. The first step is to un\u00adderstand what questions you are trying to answer for your organization. The level of detail and com\u00adplexity of your questions will increase as you be\u00adcome more comfortable with the analytic process. The most important steps in the data science process are as follows:<\/p>\n