A feature cross is a synthetic feature formed by multiplying (crossing)
two or more features. Crossing combinations of features can provide predictive
abilities beyond what those features can provide individually.
Feature Crosses
Feature Crosses
Feature crosses is the name of this approach
Define templates of the form [A x B]
Can be complex: [A x B x C x D x E]
When A and B represent boolean features, such as bins, the resulting crosses can be extremely sparse
Feature Crosses: Some Examples
Housing market price predictor:
[latitude X num_bedrooms]
Feature Crosses: Some Examples
Housing market price predictor:
[latitude X num_bedrooms]
Tic-Tac-Toe predictor:
[pos1 x pos2 x ... x pos9]
Feature Crosses: Why would we do this?
Linear learners use linear models
Such learners scale well to massive data e.g., Vowpal Wabbit, sofia-ml
But without feature crosses, the expressivity of these models would be limited
Using feature crosses + massive data is one efficient strategy for learning highly complex models