Question: Prove that the entropy of extremely likely and extremely unlikely message is zero.

0

0

In case of the “extremely likely” message, there is only one single possible message make to be transmitted. Therefore its probability pk =1. The entropy of a most likely message make is :

$H = Pk \log_2 \big(\frac{1}{Pk}\big)=1 \log_2(1) \\ = \frac{\log_{10}1}{\log_{10}2} =0$

For an extremely unlikely message make k, its probability Pk →0

$∴H=Pk \log_2(1/Pk)$

=0

Thus the average information or entropy of the most likely and most unlikely message is zero.

Please log in to add an answer.