0
26kviews
Calculate the entropy of the source. Also find the huffman code with minimum variance.

A source emits letters A {$a_1$, $a_2$, $a_3$, $a_4$} with probabilities p ($a_1$) = 0.5, p ($a_2$) 0. I5, p ($a_3$) = 0.25, p ($a_4$) = 0.10 calculate the entropy of the source. Also find the huffman code with minimum variance.

1 Answer
0
5.1kviews

p ($a_1$) = 0.5

p ($a_2$) = 0. I5

p ($a_3$) = 0.25

p ($a_4$) = 0.10

Huffman code using minimum variance

p(a1) 0.5 0.75 1
p(a2) 0.25 0.75 1
p(a3) 0.15 0.25 1
p(a4) 0.10 0.25 1
Letter Probability Code No. of bits
a1 0.5 00 2
a2 0.15 10 2
a3 0.25 01 2
a4 0.10 11 2

Average length:

L = ∑ P(i) x No. of bits

= 0.5 x 2 + 0.15 x 2 + 0.25 x 2 + 0.10 x 2

= 1 + 0.3 + 0.5 + 0.2

L = 2 bits/symbol

Entropy:

H = ${-∑_{i=1}^4}$ $P_i$ $log_2$ ($P_i$)

= - 1/log⁡2 [0.5 log (0.5) + 0.15 log (0.15) + 0.25 log (0.25) + 0.10 log (0.10)]

= -3.322 (-0.151 – 0.124 – 0.151 – 0.1)

H = 1.747 bits/symbol

Please log in to add an answer.