How to design an optimal code

1. Kraft's inequality theorem tells us that if the Kraft's condition is satisfied, there exists a prefix code C. Also, if the code C is the prefix code, the Kraft's condition is satisfied.

2. The average length of the code is always greater than the entropy of the source S. Therefore, the better code is when L(C) is closer to H(S).

H(S) <= L(C)

where the source S has the probability distribution as P(S)={p1, p2, p3,..,pn}
L(C)=Sum of {l1 x p1 + l2 x p2 + l3 x p3 + ...+ ln x pn}