ML development phases

ML projects progress in phases with specific goals, tasks, and outcomes. A clear understanding of the ML development phases helps to establish engineering responsibilities, manage stakeholder expectations, and efficiently allocate resources.

Successfully moving through the phases (often iteratively) is foundational for designing, assembling, and building ML models that solve business problems over the long term.

At a high level, implementing an ML solution consists of the following phases:

1. Ideation and planning
2. Experimentation
3. Pipeline building
4. Productionization

Ideation and planning

During the ideation and planning phase, you frame your problem in terms of an ML solution and estimate the project's feasibility.

• Goal: To determine if ML is the best solution to your problem.
• Tasks: Analyze the business problem to understand your project's constraints.
• Outcome: A design doc outlining how to solve a problem with a ML solution.

Important: During the ideation and planning stage, it can take a long time to understand the data and metrics required for a production-ready system.

Experimentation

Experimentation is the core of machine learning. During this phase, you verify that an ML solution is viable. Finding a solution is an iterative process. It's not uncommon to try hundreds of experiments before finding the right combination of features, hyperparameters, and model architecture that solves the problem.

• Goal: To build a model that solves the business problem.
• Tasks: Experiment with features, hyperparameters, and model architectures.
• Outcome: A model with good enough quality to put into production.

Important: During the experimentation phase, new-to-ML practitioners often underestimate the challenges of designing and implementing the appropriate experimentation tooling and processes.

Pipeline building and productionization

During the pipeline building and productionization phase, you build pipelines for processing data, training a model, and serving predictions. You then deploy the model and pipelines into production with the necessary monitoring and logging infrastructure.

• Goal: To build and implement the infrastructure for scaling, monitoring, and maintaining models in production.
• Tasks: Build pipelines to automate many of the tasks for keeping up-to-date models in production.
• Outcomes: Validated ML pipelines.

Important: During the productionization phase, it's easy to underestimate the complexity of productionizing data pipelines and evaluating models, especially as features evolve. For instance, not only do you have to implement all the monitoring infrastructure required for a non-ML project, but also all the ML-specific monitoring.

End-to-end ML workflow

The following diagram illustrates the entire end-to-end ML workflow, listing each phase and its tasks and outcomes:

Figure 1. The four main phases of an ML workflow.

Keep in mind

Multiple challenges exist at each phase. Not realizing—and planning for—them may lead to missed deadlines, frustrated engineers, and failed projects.

Check Your Understanding

You've just read about some ML technology that might benefit your product. What should you do next?

Before spending time drafting a design doc or writing code, you should first verify that ML is the right solution to your problem.

Correct. Before spending time drafting a design doc or writing code, you should first verify that ML is the right solution to your problem.

Draft a design doc outlining the ML use case and the required infrastructure to implement it.

Before drafting a design doc, you should first verify that ML is the right solution to your problem.

Find code examples and begin experimenting to determine if the model can make good predictions.

Before writing a line of code, you should first verify that ML is the right solution to your problem.

Key Terms:

feature	hyperparameter
model	pipeline
prediction