In the interactive exercises below, you'll use what you've learned to train a neural network to fit nonlinear data.
Exercise
In the Feature cross exercises in the Categorical data module, you manually constructed feature crosses to fit nonlinear data. Now, you'll see if you can build a neural network that can automatically learn how to fit nonlinear data during training.
Your task: configure a neural network that can separate the orange dots from the blue dots in the diagram below, achieving a loss of less than 0.2 on both the training and test data.
Instructions:
In the interactive widget below:
- Modify the neural network hyperparameters by experimenting with some
of the following config settings:
- Add or remove hidden layers by clicking the + and - buttons to the left of the HIDDEN LAYERS heading in the network diagram.
- Add or remove neurons from a hidden layer by clicking the + and - buttons above a hidden-layer column.
- Change the learning rate by choosing a new value from the Learning rate drop-down above the diagram.
- Change the activation function by choosing a new value from the Activation drop-down above the diagram.
- Click the Play (▶️) button above the diagram to train the neural network model using the specified parameters.
- Observe the visualization of the model fitting the data as training progresses, as well as the Test loss and Training loss values in the Output section.
- If the model does not achieve loss below 0.2 on the test and training data, click reset, and repeat steps 1–3 with a different set of configuration settings. Repeat this process until you achieve the preferred results.
Click here for our solution
We were able to achieve both test and training loss below 0.2 by:
- Adding 1 hidden layer containing 3 neurons.
- Choosing a learning rate of 0.01.
- Choosing an activation function of ReLU.