Shannon Entropy

+**Shannon Entropy**

+Shannon entropy, a cornerstone of [Information Theory](/wiki/information_theory), quantifies the average uncertainty or "surprise" inherent in the possible outcomes of a random variable. It measures the minimum number of bits required to encode a message from a source, reflecting its inherent unpredictability and enabling efficient [Data Compression](/wiki/data_compression).

+## See also

+- [Information Theory](/wiki/information_theory)

+- [Data Compression](/wiki/data_compression)