Entropy optimization has become a powerful tool for data analysis and problem solving that has an infinite array of real-world applications. This senior-level textbook provides a unified conceptual framework for the study of probabilistic systems with its elucidation of three key concepts: Shannon's Information Theory, Jayne's Maximum Entropy Principle, and Kullback's Minimum Cross-Entropy Principle. A wide array of real-world problems and applications are included that will establish the usefulness of these methods for any discipline looking at probabilistic systems and information (such as engineering, statistics, economics, and operations research). This textbook, complete with exercises, will leave students with the ability to apply these principles to new problems. The first true textbook that provides an interdisciplinary approach to entropy optimization principles with numerous applications and exercises Applies principles to a diverse assortment of applications in statistics, thermodynamics, pattern recognition, spectral analysis, queuing theory, and parameter estimation problems Will be of use to all engineering students looking at probabilistic systems, as well as to students of statistics, operations research and economics.