This article talks about the first step of feature selection in R that is the models generation.
Once the models are generated, you can select the best model with one of this approach:
Best subset regression looks through all possible regression models of all different subset sizes and looks for the best of each size. And so produces a sequence of models which is the best subset for each particular size.
best subset is quite aggressive looking at all possible subsets.
A function in leaps called regsubsets do best subset modeling.
# Load the library library(leaps) myPathOfModel=regsubsets(Response~.,data=myDataFrame, nvmax=8) # By default it only goes up to subsets of size 8, you can change it with the parameter nvmax
Forward stepwise is a greedy algorithm. It produces a nested sequence of models as each time you just add the variable that improves the set the most.
The models selected are nested because each new model includes all the variables that were before plus one new one.
The regsubsets with the method=“forward” option
Summary will summarize the best subset models:
- for the best subset of size 1, it puts 1 star next to the variable that's in the best subset of size 1.
- for the best subset of size 2, it puts 2 stars next to the variables that are in the best subset of size 2
- for the best subset of size n, it puts n stars next to the variables that are in the best subset of size n
Subset selection object Call: regsubsets.formula(Response ~ ., data = myDataFrame) 3 Variables (and intercept) Forced in Forced out Var1 FALSE FALSE Var2 FALSE FALSE Var3 FALSE FALSE 1 subsets of each size up to 8 Selection Algorithm: exhaustive Var1 Var2 Var3 1 ( 1 ) " " " " "*" 2 ( 1 ) "*" "*" " "
The subsets don't have to be nested in a best subset method whereas in forward stepwise, they are.
 "which" "rsq" "rss" "adjr2" "cp" "bic" "outmat" "obj"
Like for each models, the best subset models has the following variables:
You use this data to R - Feature Selection - Indirect Model Selection them
Get the regression coefficient vector for a model
(Intercept) Var1 Var2 .... Varn 162.5354420 -2.1686501 6.9180175 5.7732246 -0.1300798