Data Mining - Stacking


Stacking is a ensemble of models combined sequentially.

Stacking uses a “meta learner” (not voting) to combine the predictions of “base learners.”

The base learners (the expert) are not combined by voting but by using a meta-learner, another learner scheme that combines the output of the base learners.

For example create a linear model and then as a second step use random forests (residual errors as a response).

  • Base learners: level-0 model
  • Meta Learner: level-1 model

The predictions of the base learners are input for the meta-learner.

Typically, different machines learning schemes are used as based-learners to get different perspective.

It can’t use predictions on training data to generate data for level‐1 model (Instead use cross‐validation‐like scheme)



Stacking C is the more efficient version (allow multiple level‐0 models by specifying a metaclassifier)

Documentation / Reference

Powered by ComboStrap