Search This Blog

Shannon Entropy - Explained Fast

Shannon Entropy - Explained Fast

Suppose you have a system that has n discrete states, where n is an integer.

And suppose the probability of finding the system in a given state is the same for all states.

Then is you solve n=2^x for the variable x you've just found the Shannon Entropy of the system.

Shannon Entropy is also known as Information Entropy.

For example: Consider a 6 sided dice. It has 6 states (meaning that when thrown it can land in 6 different ways), so solving 6=2^x gives x=2.5849.. and that's the Shannon Entropy of your dice.

Content written and posted by Ken Abbott