Information Theory_Information Rate

Information Rate

The information rate is represented by R and it is given as,
Information Rate : R = rH
Here R is the information rate.
H is the Entropy or average information
And r is the rate at which messages are generated.
Information rate R is represented in average number of bits of information per second. It is calculated as follows:

R = ( r in ) * ( H in )
= bits / second




Example1: A PCM source transmits four samples (messages) with a rate 2B samples / second. The probabilities of occurrence of these 4 samples (messages) are p1 = p4 = 1/8 and p2 = p3 = 3/8. Find out the information rate of the source.
Solution: (i) to calculate the Entropy (H):
We have four messages with probabilities p1 = p4 = 1/8 and p2 = p3 = 3/8. Average information H (or entropy) is given by equation as;

H = 1.8 bits / message

Message rate r = 2B samples / second
= 2B messages / second

Hence,
R = rH
Putting values of r and H in the above equation ,
R = 2B messages / second * 1.8 bits / message
= 3.6 B bits/second