Quantum mechanics

Natural language

Markov model

Statistics

Entropy (arrow of time)

Maximum entropy spectral estimation

Markov chain

Computer science

Maximum-entropy Markov model

Natural language processing

Principle of maximum entropy

Physics

Mathematics

Artificial intelligence

Markov property

Maximum likelihood

Machine learning

Computational Linguistics

The concept of maximum entropy can be traced back along multiple threads to Biblical times. Only recently, however, have computers become powerful enough to permit the widescale application of this concept to real world problems in statistical estimation and pattern recognition. In this paper, we describe a method for statistical modeling based on maximum entropy. We present a maximum-likelihood approach for automatically constructing maximum entropy models and describe how to implement this approach efficiently, using as examples several problems in natural language processing.

A maximum entropy approach to natural language processing