This section discusses some of the benefits and limitations of AutoML and can help you decide if AutoML is right for your project.
Benefits
Some of the most common reasons you might choose to use AutoML include:
- To save time: Many people choose AutoML to save time by avoiding extensive manual experimentation to find the best model.
- To improve quality of ML model: AutoML tools can comprehensively search for the highest quality model.
- To build an ML model without needing specialized skills: A side effect of automating ML tasks is that it democratizes ML. Users don't need to have a deep understanding of machine learning algorithms or programming to develop an ML model.
- To smoke test a dataset: Even if you're an expert model builder, AutoML can quickly give you a baseline estimate for whether a dataset has enough signal in all of its noise. If the AutoML tool can't build even a mediocre model, it might not be worth spending your time trying to build a good model by hand.
- To evaluate a dataset: Many AutoML systems help you evaluate not just the models they produce but also help evaluate the features that go into training those models. Even if you don't use the resulting model, AutoML tools may help you determine which of your features might not be worth gathering and including in your final dataset.
- To enforce best practices: Automation includes built-in support to apply ML best practices to each model search
Limitations
However, before choosing AutoML there are some limitations you should consider:
- Model quality may not be as good as manual training. Using a generalized optimization algorithm to find the best combination of architecture, hyperparameters, and feature engineering for any random dataset often means that a motivated expert with enough time can create a model with better prediction quality.
- Model search and complexity can be opaque. With AutoML, it is difficult to have insight into how the tool arrived at the best model. In fact, the output models themselves may be made from different architectures, hyperparameters, and feature engineering strategies. Models generated with AutoML are difficult to reproduce manually.
- Multiple AutoML runs may show more variance: Because of how the optimization algorithm iteratively finds what it thinks are the best values for building the model, different AutoML runs may search different portions of the space and wind up in moderately (or even possibly significantly) different places. Retuning with AutoML to create a new model architecture may show more variance than simply retraining the previous output model architecture.
- Models can't be customized during training. If your use case requires customization or tweaking during the training process, AutoML may not be the right choice for you.
Data requirements
Whether you're using custom training or an AutoML system, one thing you can count on when you are building a model from scratch is that you need large amounts of data. The advantage with AutoML is that you can mostly ignore the architecture and hyperparameters search and focus primarily on the quality of your data.
There are also specialized AutoML systems that can train models with significantly less data because they use transfer learning. For example, instead of requiring hundreds of thousands of examples to build an image classification model, these specialized AutoML systems can use only a few hundred labeled images along with transfer learning from an existing image classification framework model.
Is AutoML right for your project?
AutoML can help anyone—from novices to experts—use ML to build products and solve problems. If you are trying to decide if AutoML is right for your project, consider these trade offs:
- AutoML is a great choice for a team with limited experience building ML models or experienced teams that are looking for productivity gains and don't have customization requirements.
- Custom (manual) training is more appropriate when model quality is important and the team needs to be able to customize their model. In these cases, manual training may require more time for experimentation and building a solution, but the team can often achieve a higher quality model than with an AutoML solution.