Prove that the entropy of extremely likely and extremely unlikely message is zero.

In case of the “extremely likely” message, there is only one single possible message make to be transmitted. Therefore its probability pk =1. The entropy of a most likely message make is :

$H = Pk \log_2⁡ \big(\frac{1}{Pk}\big)=1 \log_2⁡(1) \\ = \frac{\log_{10}⁡1}{\log_{10}⁡2} =0$

For an extremely unlikely message make k, its probability Pk →0

$∴H=Pk \log_2⁡(1/Pk)$


Thus the average information or entropy of the most likely and most unlikely message is zero.

Please log in to add an answer.

Continue reading

Find answer to specific questions by searching them here. It's the best way to discover useful content.

Find more