What is Jensen's inequality?

Named after the Danish mathematician (Johan Jensen), Jensen's inequality is used to define convex functions. It can be stated as follows:

Jensen's inequality is just a mathematical way of describing that the convex function always lies below the secant line joining them for any two points.

Extensions

This inequality can be extended to many other useful inequalities as well.

Generalization

For example, if $\theta_1, \theta_2, \theta_3,....$ form an affine combinationNon-negative co-efficients having sum of 1., the inequality can be generalized to the following:

Expected values

Since probabilities are non-negative and sum to 1 and hence make an affine combination, we can also extend Jensen's inequality to probabilities.

Note: We can apply Jensen's inequality to any function, $f$ as long as it is convex and the coefficients combination is affine.

Arithmetic and geometric mean

Since $-\mathcal{log} \space x$ is a convex function, we can apply Jensen's inequality to yield:

Taking anti-log returns:

This is a famous inequality, explaining that the geometric mean of two numbers will always be less than their arithmetic mean. Similarly, there can be a number of other inequalities which can be deduced from Jensen's inequality as well.