Data Mining - Rare Event

A rare event is always rare in function of the population being studied.

In high dimensions, all cases are edge cases

Sam Ross

The rate of an event is related to the probability of an event occurring in some small subinterval (of time, space or otherwise).

Example

• click-through rate prediction

Documentation / Reference

Discover More
Data Mining - (Anomaly|outlier) Detection

The goal of anomaly detection is to identify unusual or suspicious cases based on deviation from the norm within data that is seemingly homogeneous. Anomaly detection is an important tool: in data...
Data Mining - High Dimension (Curse of Dimensionality)

High dimension In high dimension, it's really difficult to stay local. edge cases See this interactive app in R Shiny on the Curse of Dimensionality. Circle...
Loss functions (Incorrect predictions penalty)

Loss functions define how to penalize incorrect predictions. The optimization problems associated with various linear classifiers are defined as minimizing the loss on training points (sometime along with...
Statistics - (Base rate fallacy|Bonferroni's principle)

Every accurate (model|test) can be useless as detection tools if the studied case is sufficiently rare among the general population. The data model will produce too many false positives or false negatives....
Statistics - Power of a test

The power of a test sometimes, less formally, refers to the probability of rejecting the null when it is not correct, the chance that your experiment is right. A test's power is influenced by the choice...
Text Mining - term frequency – inverse document frequency (tf-idf)

tf–idf, short for term frequency–inverse document frequency, is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus. It is often used...
What is Event Detection or Event mining?

Event Mining or Event detection can be defined as a process of finding: the frequent events, the rare events, unknown event (it occurrence can be deduced from observation of the system), the anomaly,...