Statistics - (Multiclass Logistic|multinomial) Regression

Thomas Bayes


Multiclass logistic regression is also referred to as multinomial regression.

Multinomial Naive Bayes is designed for text classification. It's a lot faster than plain Naive Bayes.

also known as maximum entropy classifiers ?


The symmetric form:

<MATH> \begin{array}{rrrl} Pr(Y = k|X) & = & \frac{\displaystyle e^{\displaystyle B_{0k} + B_{1k} . X_1 + \dots + B_{ik} . X_i }}{\displaystyle \sum^K_{l=1} e^{\displaystyle B_{0l} + B_{1l} . X_1 + \dots + B_{il} . X_i }} \\ \end{array} </MATH>

  • k is the index of a outcome class
  • K is the number of outcome classes (ie bigger than 2)
  • in the numerator we've got an exponential to the linear model. This is for the probability that Y is k given X, a small k.
  • In the denominator, we've just got the sum of those exponentials for all the classes. In this case, each class gets its own linear model.
  • And then we just weigh them against each other with this exponential function, sometimes called the softmax function.
  • some cancellation is possible,
  • only K - 1 linear functions are needed as in 2-class logistic regression.


package glmnet

Discover More
Thomas Bayes
Machine Learning - Logistic regression (Classification Algorithm)

The prediction from a logistic regression model can be interpreted as the probability that the label is 1. linear regression can also be used to perform classification problem. Just by transforming the...

Share this page:
Follow us:
Task Runner