About MaxEnt

The idea behind the Principle of Maximum Entropy is quite simple: model all that is known and assume nothing about that which is unknown.

The mentions of this concept can be dated as far as biblical times as it is, indeed, a relatively intuitive notion that one should not make assumptions about data without sufficient information. A more recent pioneer of maximum entropy, E. T. Jaynes, tells us:
...the fact that a certain probability distribution maximizes entropy subject to certain constraints representing our incomplete information, is the fundamental property which justifies use of that distribution for inference; it agrees with everything that is known, but carefully avoids assuming anything that is not known. It is a transcription into mathematics of an ancient principle of wisdom...

It may seem like a reasonable approach to dealing with data and, in fact, it does often produce excellent results provided that the features selected to represent the information suit the purpose. This requisite, however, is not unique to MaxEnt and on the whole I personally am a fan of the method and the concept behind it.

posted under |
Thoughts about work and life.

Anonymous Questions Answered