KL divergence To measure the difference between two probability distributions over the same variable x, a measure, called the Kullback-Leibler divergence, or simply, the KL divergence has been popularly used in the data mining literature. The concept was originated in probability theory and information theory.
The KL divergence, which is closely related to…