This page challenges you to answer a series of multiple choice exercises
about the material discussed in the "Training Decision Trees" unit.
What are the effects of replacing the numerical features with their
negative values (for example, changing the value +8 to -8) with
the exact numerical splitter?
The same conditions will be learned; only the
positive/negative children will be switched.
Different conditions will be learned, but the overall structure
of the decision tree will remain the same.
If the features change, then the conditions will change.
The structure of the decision tree will be completely
The structure of the decision tree will actually be
pretty much the same. The conditions will change, though.
What two answers best describe the effect of testing only half
(randomly selected) of the candidate threshold values in X?
The information gain would be higher or equal.
The information gain would be lower or equal.
The final decision tree would have worse testing accuracy.
The final decision tree would have no better training accuracy.
What would happen if the "information gain" versus "threshold" curve
had multiple local maxima?
It is impossible to have multiple local maxima.
Multiple local maxima are possible.
The algorithm would select the local maxima with the smallest
The algorithm would select the global maximum.
Compute the information gain of the following split:
|Node ||# of positive examples ||# of negative
|parent node ||10 ||6 |
|first child ||8 ||2 |
|second child ||2 ||4 |
Click the icon to see the answer.
# Positive label distribution
p_parent = 10 / (10+6) # = 0.625
p_child_1 = 8 / (8+2) # = 0.8
p_child_2 = 2 / (2+4) # = 0.3333333
h_parent = -p_parent * log(p_parent) - (1-p_parent) * log(1-p_parent) # = 0.6615632
h_child_1 = ... # = 0.5004024
h_child_2 = ... # = 0.6365142
# Ratio of example in the child 1
s = (8+2)/(10+6)
f_final = s * h_child_1 + (1-s) * h_child_2 # = 0.5514443
information_gain = h_parent - f_final # = 0.1101189