Data Mining - Maximum Entropy Algorithm


Maximum Entropy (MaxEnt) models are feature-based classifier models.

In a two-class scenario, it is the same as using logistic regression to find a distribution over the classes.

Feature overlap

MaxEnt makes no independence assumptions for its features, unlike Naive Bayes. This means that features like bigrams and phrases can be added to MaxEnt without worrying about features overlapping.

??? Theoretically, MaxEnt performs better than Naive Bayes because it handles feature overlap better. However, in practice, Naive Bayes can still perform well on a variety of problems.

Documentation / Reference

  • Stanford Classifier is a MaxEnt classifier. The Stanford Classifier shines is in working with mainly textual data. For small data sets and numeric predictors, you'd generally be better off using another tool such as R or weka.

Powered by ComboStrap