For a source which generates letters from an alphabet $A= { a1 , a2 , a3 , a4 , a5 }$ with $P (a_1 ) = P (a_3) = 0.2 , P (a_2 ) = 0.4, P (a_4 ) = P (a_5 ) = 0.1$ also calculate entropy of the source.

$P ( a_4‘)= P (a_4 ) + P (a_5 ) = 0.1 + 0.1 = 0.2$

$C (a_4 ) = α_1 * 0$

$C (a_5 ) = α_1 * 1$

$P (a_3’ ) = P (a_3 ) + P (a_4’ ) = 0.2 + 0.2 =0.4$

$C ( a_3 ) =α_2 * 0$

$C ( a_4 ‘ ) = α_2 * 1 = α_1$

$C ( a_4 ) = α_1 * 0 = α_2 * 1 * 0 = α_2 * 10$

$C (a_5) = α_1 * 1 = α_2 * 1 * 1 = α_2 * 1$

$P (a_3” ) = P (a_3’ ) + P (a_1 ) = 0.4 + 0.2 = 0.6$

$C (a_3’ ) = α_2 = α_3 * 0$

$C ( a_1 ) = α_3 * 1 = 0 * 1 = 01$

$C ( a_3 ) = α_3 * 00 = 0 * 00 =000$

$C (a_4 ) = α_3 * 010 = 0 * 010 = 0010$

$C (a_5 ) = α_3 *011 = 0 * 011 =0011$

Verification

$Entropy =_I log_2 (P_i)$

$= - (0.2 log_2 0.2 + 0.4 log_2 0.4 + 0.2 log_2 0.2 + 0.1 log_2 0.1 + 0.1 log_2 0.1 )$

Entropy = 2.122 bits/ symbol