An analog baseband signal, band-limited to 100 Hz, is sampled at the Nyquist rate. The samples are quantized into four message symbols that occur independently with probabilities p₁ = p₄ = 0.125 and p₂ = p₃. The information rate (bits/sec) of the message source is __________

In the range: 360 - 363

Concept:

The Entropy of a source, which is also the average information content is given by:

$H=\sum P\left(x_i\right){\log }_2\left(\frac{1}{P\left(x_i\right)}\right)$

Where P(xi) = Probability of each codeword.

Application:

We have P₁ = P₄ = 0.125

Now, P₁ + P₂ + P₃ + P₄ = 1

Since P2 = P3, the above equation becomes:

$0.12+2{\mathrm{P}}_2+0.125=1$

2P₂ = 0.75

P₂ = P₃ = 0.375

Average information will be:

H = - P₁log₂P₁ - P₂log₂P₂ - P₃log₂P₃ - P₄log₂P₄

H = - 0.125.log₂ 0.125 - 0.375.log₂ 0.375 - 0.375.log₂0.375 - 0.125.log₂0.125

H = 0.375 + 0.531 + 0.531 + 0.375

H = 1.812 bits/symbol.

Sampling rate is the Nyquist rate, i.e.

r = 2 × 100 = 200 samples/sec

Thus, the information rate will be: r × H

= 200 × 1.812

= 362.4 bits/sec.