Shannon Entropy - Explained Fast
Suppose you have a system that has n discrete states, where n is an integer.
And suppose the probability of finding the system in a given state is the same for all states.
Then is you solve n=2^x for the variable x you've just found the Shannon Entropy of the system.
Shannon Entropy is also known as Information Entropy.
For example:
Consider a 6 sided dice. It has 6 states (meaning that when thrown it can land in 6 different ways), so solving 6=2^x gives x=2.5849.. and that's the Shannon Entropy of your dice.
Content written and posted by Ken Abbott abbottsystems@gmail.com