You may recall from the Feature cross exercises in the Categorical data module, that the following classification problem is nonlinear:
![Figure 1. Cartesian coordinate plane, divided into four
quadrants, each filled with random dots in a shape resembling a
square. The dots in the top-right and bottom-leftquadrants are blue,
and the dots in the top-left and bottom-right quadrants are orange.](https://developers.google.cn/static/machine-learning/crash-course/neural-networks/images/nonlinear_simple.png?authuser=1)
"Nonlinear" means that you can't accurately predict a label with a model of the form \(b + w_1x_1 + w_2x_2\). In other words, the "decision surface" is not a line.
However, if we perform a feature cross on our features $x_1$ and $x_2$, we can then represent the nonlinear relationship between the two features using a linear model: $b + w_1x_1 + w_2x_2 + w_3x_3$ where $x_3$ is the feature cross between $x_1$ and $x_2$:
![Figure 2. The same Cartesian coordinate plane of blue and orange
dots as in Figure 1. However, this time a white hyperbolic curve is
plotted atop the grid, which separates the blue dots in the top-right
and bottom-left quadrants (now shaded with a blue background) from
the orange dots in the top-left and bottom right quadrants (now
shaded with an orange background).](https://developers.google.cn/static/machine-learning/crash-course/neural-networks/images/nonlinear_simple_feature_cross.png?authuser=1)
Now consider the following dataset:
![Figure 3. Cartesian coordinate plane, divided into four quadrants.
A circular cluster of blue dots is centered at the origin of the
graph, and is surrounded by a ring of orange dots.](https://developers.google.cn/static/machine-learning/crash-course/neural-networks/images/nonlinear_complex.png?authuser=1)
You may also recall from the Feature cross exercises that determining the correct feature crosses to fit a linear model to this data took a bit more effort and experimentation.
But what if you didn't have to do all that experimentation yourself? Neural networks are a family of model architectures designed to find nonlinear patterns in data. During training of a neural network, the model automatically learns the optimal feature crosses to perform on the input data to minimize loss.
In the following sections, we'll take a closer look at how neural networks work.