The Data Science Lifecycle

A typical data science project starts with asking the right business question and collecting and preparing data. Success in the later phases is dependent on what occurs in these earlier phases. It is not surprising, therefore, that poor quality data will lead to poor model performance. That is why data science teams must ensure the quality of the data they use as input for predictive modeling.

Data preparation is one of the most important and often time-consuming phases of a data science project. In fact, it is estimated that data preparation usually takes 50-70% of a project's time and effort, leaving data science teams very little time to uncover new insights and deliver them to decision makers. With the amount of stored data now too vast for manual verification, today's data scientists require automated processes to prepare data quickly and accurately.

Often referred to as "data wrangling," data preparation involves cleaning the data and reshaping it into a usable form for performing data science. Examples of common data preparation activities include dealing with non-standard, unstructured or inconsistent data and combining data from different sources and formats. Tools that make it easier to access and manipulate data sources on their own, without technical proficiency in programming languages or coding expertise, can encourage more users to take advantage of enterprise or external data sources.

Once your data is in the right format to work with, you can conduct the next step in the data analysis process: data exploration. This initial exploration of the dataset is critical because it helps data scientists illuminate previously unknown patterns, relationships, or other actionable findings. Some helpful questions to ask at this point include:

• Which attributes seem promising for further analysis?
• Has the exploration revealed new characteristics about the data?
• How have these explorations changed any initial hypotheses?
• Can a specific subset of the data be used later?
• Has the data exploration altered the project goals?

Data scientists commonly use data visualizations to quickly view relevant features of their datasets and identify variables that are likely to result in interesting observations. By displaying data graphically-for example, through scatter plots or bar charts-users can see if two or more variables correlate and determine if they are good candidates for more in-depth analysis.

This is the point at which your hard work begins to pay off. The data you spent time preparing is brought into the data science toolset, and the results begin to shed some light on the business problem posed during the early stages of the project.

Model development is usually conducted in multiple iterations. Typically, data scientists run several models using default parameters and then fine-tune the parameters or revert to the data preparation phase for manipulations required by their model of choice. It's rare for an organization's question to be answered satisfactorily with a single algorithm and a single execution. This is what makes data science so interesting. There are many ways to look at a given problem, and today there are a wide variety of tools to help you do that.

Although you may already have some idea about which types of models are most appropriate for your organization's needs, now is the time to make some decisions about which ones to use. Determining the most appropriate model will typically be based on the data types available, your project goals, and specific requirements for data sizes or types.

Your models have been built and you've selected the best one. Before actual deployment, however, you need to evaluate your model to understand its quality and ensure that it fully addresses the business problem. Model implementation entails computing various diagnostic measures and other outputs such as tables and graphs, enabling the data scientist to interpret the model's quality and its efficacy in solving the problem.

For a predictive model, data scientists use a testing set that is independent of the training set but follows the same probability distribution and has a known outcome. The testing set is used to evaluate the model so it can be refined as needed. Sometimes the final model is applied also to a validation set for a final assessment.

Data scientists may also assign statistical significance tests to the model as further proof of its quality. This additional proof may be instrumental in justifying model implementation or taking actions when the stakes are high-such as an expensive medical protocol or a critical airplane flight system.

Once a satisfactory model has been developed and is approved by the business sponsors, you are ready to deploy it into the production environment or a comparable test environment.

Deploying a model into an operational business process is usually an IT function, although it can involve additional groups. However, the process is not as simple as simply handing the model off to your IT team. Testing the model first is necessary to identify any dependencies in the production environment. In addition, data science teams need to ensure models receive the correct production data and send the scores to the right place, and that the system must be set up for monitoring and scalability.

Questions to ask at this stage of the process include:

• What data does the model expect to ingest and produce?
• What infrastructure is required to run this model at scale?
• Does the model have everything it needs to execute?
• How easy is it to identify points of failure if the model is not running properly?

Models are normally deployed in a limited way until their performance has been fully evaluated. Deployment may be as simple as generating a report with recommendations, or as involved as embedding the model in a complex workflow and scoring process managed by a custom application.

During deployment and integration of modeling results, model management must be a continuous process to ensure optimal performance over time. This means knowing exactly where each model is in the lifecycle, how old the model is, who developed it, and who is using it for which application. Model version control, which includes event logging and change tracking to understand how the model evolves over time, is another critical requirement.

Data science must also be concerned with model decay, and continuously gather metrics to determine when a model should be refreshed or replaced. For example, a model deployed to increase customer retention among high-value customers will need to be revised once a level of retention is reached.