信息论第三版课后答案【篇一:西电邓家先版信息论与编码第3章课后习题解答】6x11/6y13/41/4x2图3.1 二元信道y2?x??x1x2???=?0.60.4?通过一干扰信道,接收符号y=?y1y2?,信道传递概率如p(x)????图3.33所示。
求:(1)信源x中事件x1,和x2分别含有的自信息。
(2)收到消息yj(j=1,2)后,获得的关于xi(i=1,2)的信息量。
(3)信源x和信源y的信息熵。
(4)信道疑义度h(x|y)和噪声熵h(y|x)。
(5)接收到消息y后获得的平均互信息。
解:(1)由定义得:i(x1)= -log0.6=0.74biti(x2)= -log0.4=1.32biti(xi;xj)= i(xi)-i(xi|yj)=log[p(xi|yj)/p(xi)]= log[p(yj|xi)/p(yj)]则 i(x1;y1)= log[p(y1|x1)/p(y1)]=log5/6/0.8=0.059bit i (x1;y2)= log[p(y2|x2)/p(y2)]=log1/6/0.2=-0.263biti(x2;y1)= log[p(y1|x2)/p(y1)]=log3/4/0.8=-0.093bit i(x2;y2)= log[p(y2|x2)/p(y2)]=log1/4/0.2=0.322bit(3)由定义显然 h(x)=0.97095bit/符号h(y)=0.72193bit/符号(4)h(y|x)=?22p(xy)log[1/p(y|x)]=??i?1j?1p(xi)p(yj|xi)log[1/p(yj|xi)]h(x|y)= h(x)+h(y|x)-h(y)=0.9635bit/符号(5) i(x;y)= h(x)-h(x|y)=0.00745 bit/符号3.2设8个等概率分布的消息通过传递概率为p的bsc进行传送。
八个消息相应编成下述码字:m1=0000, m2=0101, m3=0110, m4=0011, m5=1001, m6=1010, m7=1100, m8=1111, 试问 (1) 接受到第一个数字0与m之间的互信息。
(2) 接受到第二个数字也是0时,得到多少关与m的附加互信息。
(3) 接受到第三个数字仍是0时,又增加多少关与m的互信息。
(4) 接受到第四个数字还是0时,再增加了多少关与m的互信息。
解: (1 ) i(0;m1)= log[ p(0|m1)/p(0)]=1 bit(2 ) i(00;m1)= log[ 1/p(00)]=2 bit2-1=1 bit(3 ) i(000;m1)=3 bit 3-2=1 bit(4 ) i(0000;m1)=4 bit4-3=1 bit3.3 设二元对称信道的传递矩阵为?2?3??1??31?3?? 2??3?(1)若p(0)?3/4,p(1)?1/4,求h(x),h(x|y),h(y|x)和i(x;y);(2)求该信道的信道容量及其达到信道容量时的输入概率分布。
解:(1)已知二元对称信道的传递矩阵,又已知输入的概率分布p(0)?3/4,p(1)?1/4,可以求得输出y的概率分别和后验概率。
p(y?0)??p(x)p(y?0|x)x?p(x?0)p(y?0|x?0)?p(x?1)p(y?0|x?1)32117?????434312p(y?1)??p(x)p(y?1|x)x?p(x?0)p(y?1|x?0)?p(x?1)p(y?1|x?1) 31125?????434312所以p(x?0|y?0)?p(x?0)p(y?0|x?0)6?p(x)p(y?0|x)7xp(x?1|y?0)?p(x?1)p(y?0|x?1)1?p(x)p(y?0|x)7xp(x?0|y?1)?p(x?0)p(y?1|x?0)3?p(x)p(y?1|x)5xp(x?1|y?1)?于是,h(x)??p(x?1)p(y?1|x?1)2?p(x)p(y?1|x)5x?p(x)logp(x)?0.811比特/符号xxyh(x|y)????p(x)p(y|x)logp(x|y)326111313122?[??log??log??log??log]437437435435?0.111?0.234?0.184?0.220?0.749比特/符号 h(y|x)????p(x)p(y|x)logp(y|x)xy322111311122?[??log??log??log??log]433433433433?0.918比特/符号i(x;y)?h(x)?h(x|y)?0.062比特/符号(2)此信道为二元对称信道,所以信道容量2c?1?h(p)?1?h()?0.082比特/符号3根据二元对称信道的性质可知,输入符号为等概率分布(即p(0)?p(1)?道的信息传输率才能达到这个信道容量值。
1)时信2解:x2??x??x1?y??y1y2??p(x)???0.640.36?, ?p(y)???0.70.3? ????????且p(x1∣y1)=0.8 p(x2∣y1)=0.2 由p(x1)=p(y1)p(x1∣y1)+p(y2)p(x1∣y2) p(x2)=p(y1)p(x2∣y1)+p(y2)p(x2∣y2)得p(x2∣y2)=2.2/3 p(x1∣y2)=0.8/3所以h(x∣y)=p(y1)〔-p(x1∣y1)logp(x1∣y1)-p(x2∣y1)logp(x2∣y1)〕+ p(y2)〔-p(x1∣y2)logp(x1∣y2)-p(x2∣y2)logp(x2∣y2)〕=0.7〔-0.8log0.8-0.2log0.2〕+0.3〔-0.8/3log(0.8/3)-2.2/3log(2.2/3)〕=0.7 [0.258+0.464]+0.3[0.509+0.328]=0.505+0.251=0.756h(x) =-0.64log0.64-0.36log0.36=0.412+0.531=0.944 i(x;y)=h(x)-h(x∣y)=0.944-0.756=0.1883.5若x,y,z是三个随机变量,试证明:(1)i(x;yz)=i(x;y)+i(x;z|y)=i(x;z)+i(x;y|z); (2)i(x;y|z)=i(y;x|z)=h(x|z)-h(x|yz); (3)i(x;y|z)?0;证明:(1) i(x)+i(x;z/y)=??p(xy)logxyp(y/x)+?p(y)x??p(xyz)logp(x/y)yzp(x/yz)=??xy?p(xyz)logzp(x/yz)p(y/x)p(y)p(x/y)p(x/yz)p(y/x)p(xy)p(x/yz)p(y/x)p(x)p(y/x)p(x/yz)p(y/x)p(x)=??xyxy?p(xyz)logzz=???p(xyz)log??xy=?p(xyz)logz=i(x;yz) i(x;z)+i(x;y/z) = ?xx?p(xz)logzp(z/x)+?p(z)x?y?p(xyz)logzp(x/yz)p(x/z)=??yxy?p(xyz)logzzp(x/yz)p(z/x)p(z)p(x/z)p(x/yz)p(z/x) p(x)p(z/x)=???p(xyz)log=??xy?p(xyz)logzp(x/yz)p(x)=i(x;yz) (2) i(x;y/z) = ??xyxy?p(xyz)logzp(x/yz)p(x/z)=????xy?p(xyz)logzp(xyz)p(z)p(xz)p(yz)p(y/xz)p(xz)p(z) p(y/z)p(z)p(xz)p(y/xz)p(y/z)=?p(xyz)logz=??xy?p(xyz)logz=i(y;x/z)h(x/z)-h(x/yz) =?p(xz)logxzxyz1?p(x/z)p(x/yz)?p(xyz)logxyz1p(x/yz)=?p(xyz)log=i(x;y/z)i(x;y/z)=i(y;x/z)=h(x/z)-h(x/yz) (3) i(x;y/z)≥0i(x;y/z)=?p(xyz)logp(yz)p(x/z)xyzp(xyz)-i(x;y/z)=?p(xyz)logxyzp(yz)p(x/z)≤log???p(yz)p(x/z)=0p(xyz)xyz得i(x;y/z)≥03.6若三个离散随机变量,有如下关系:x+y=z,其中x和y相互统计独立。
试证明:(1) h(x)=h(z); (2) h(y)=h(z);(3) h(z)= h(xy)= h(x)+ h(y); (4) i(x;z)=h(z)-h(y); (5) i(xy;z)= h(z); (6) i(x;yz)= h(x); (7) i(y;z/x)= h(y);(8) i(x;y/z)= i(x/z)= i(y/z);【篇二:信息论答案第三章】3.1 设信源????通过一干扰信道,接收符号为y = { y1, y2 },信道转移矩??p(x)??0.60.4??51???阵为?66?,求:13???44?(1) 信源x中事件x1和事件x2分别包含的自信息量;(2) 收到消息yj (j=1,2)后,获得的关于xi (i=1,2)的信息量; (3) 信源x和信宿y的信息熵;(4) 信道疑义度h(x/y)和噪声熵h(y/x); (5) 接收到信息y后获得的平均互信息量。
i(x1)??log2p(x1)??log20.6?0.737biti(x2)??log2p(x2)??log20.4?1.322bit2)51p(y1)?p(x1)p(y1/x1)?p(x2)p(y1/x2)?0.6??0.4??0.66413p(y2)?p(x1)p(y2/x1)?p(x2)p(y2/x2)?0.6??0.4??0.464p(y1/x1)5/6i(x1;y1)?log2?log2?0.474bitp(y1)0.6p(y2/x1)1/6i(x1;y2)?log2?log2??1.263bitp(y2)0.4i(x2;y1)?log2i(x2;y2)?log23)p(y1/x2)1/4?log2??1.263bitp(y1)0.6p(y2/x2)3/4?log2?0.907bitp(y2)0.4h(x)???p(xi)logp(xi)??(0.6log0.6?0.4log0.4)log210?0.971bit/sy mbolih(y)???p(yj)logp(yj)??(0.6log0.6?0.4log0.4)log210?0.971bit/sy mbolj4)h(y/x)????p(xi)p(yj/xi)logp(yj/xi)ij55111133??(0.6?log?0.6?log?0.4?log?0.4?log)?log21066664444?0.715bit/symbol?h(x)?h(y/x)?h(y)?h(x/y)?h(x/y)?h(x)?h(y/x)?h(y) ?0.971?0.715 ?0.971?0.715bit/symbol5)i(x;y)?h(x)?h(x/y)?0.971?0.715?0.256bit/symbol?21???3.2 设二元对称信道的传递矩阵为?33?12???33?(1) 若p(0) = 3/4, p(1) = 1/4,求h(x), h(x/y), h(y/x)和i(x;y); (2) 求该信道的信道容量及其达到信道容量时的输入概率分布;解: 1)3311h(x)???p(xi)??(?log2??log2)?0.811bit/symbol4444ih(y/x)????p(xi)p(yj/xi)logp(yj/xi)ij322311111122??(?lg??lg??lg??lg)?log210433433433433?0.918bit/symbol3211p(y1)?p(x1y1)?p(x2y1)?p(x1)p(y1/x1)?p(x2)p(y1/x2)?????0.583 343433112p(y2)?p(x1y2)?p(x2y2)?p(x1)p(y2/x1)?p(x2)p(y2/x2)?????0.416 74343h(y)???p(yj)??(0.5833?log20.5833?0.4167?log20.4167)?0.980bi t/symbolji(x;y)?h(x)?h(x/y)?h(y)?h(y/x)h(x/y)?h(x)?h(y)?h(y/x)?0.811?0.980?0.918?0.749bit/symboli(x; y)?h(x)?h(x/y)??0.811?0.749?0.062bit/symbol2)1122c?maxi(x;y)?log2m?hmi?log22?(lg?lg)?log210?0.082bit/symbo l33331p(xi)?2解:对本题建立数学模型如下:?x阻值??x1?2??x2?5????y瓦数??y1?1/8? ??????0.7??0.3??p(x)???p(y)??0.64p(y1/x1)?0.8,p(y2/x1)?0.2求:i(x;y)以下是求解过程:y2?1/4??0.36?p(x1y1)?p(x1)p(y1/x1)?0.7?0.8?0.56p(x1y2)?p(x1)p(y2/x1)?0.7?0.2?0.14?p(y1)?p(x1y1)?p(x2y1)?p(x2y1)?p(y1)?p(x1y1)?0.64?0.56?0.08?p(y2)?p(x1y2)?p(x2y2) ?p(x2y2)?p(y2)?p(x1y2)?0.36?0.14?0.22h(x)???p(xi)???0.7?log20.7?0.3?log20.3??0.881bit/symbolih(y)???p(yj)???0.64?log20.64?0.36?log20.36??0.943bit/symbol jh(xy)????p(xiyj)logp(xiyj)ij???0.56?log20.56?0.14?log20.14?0.08?log20.08?0.22?log20.2 2??1.638bit/symboli(x;y)?h(x)?h(y)?h(xy)?0.881?0.943?1.638?0.186bit/symbol(1) i(x;yz) = i(x;y) + i(x;z/y) = i(x;z) + i(x;y/z);证明:i(x;yz)????p(xiyjzp(xi/yjzk)k)logijkp(xi) ????p(xp(xi/yjzk)p(xi/yj)iyjzk)logijkp(xi)p(xi/yj)????p(xp(xi/yj)p(xi/yjzk)iyjzk)logp(x?p(xiyjzk)logijki)???ijkp(xi/yj)?i(x;y)?i(x;z/y)i(x;yz)????p(xp(xi/yjzk)iyjzk)logijkp(xi) ????p(xiyjzk)logp(xi/yjzk)p(xi/zk)ijkp(xi)p(xi/zk)????p(xp(xi/zk)iyjzk)log?ijkp(x???p(xlogp(xi/yjzk)iyjzk)i)ijkp(xi/zk) ?i(x;z)?i(x;y/z)(2) i(x;y/z) = i(y;x/z) = h(x/z) – h(x/yz);证明:i(x;y/z)????p(xxi/yjzk)iyjzk)logp(ijkp(xi/zk)????p(xi/yjzk)p(yjzk)iyjzk)logp(xijkp(xi/zk)p(yjzk)????p(xiyjzk)logijkp(xiyjzk)p(xi/zk)p(zk)p(yj/zk)p(xiyjzk)p(xizk)p(yj/zk)p(xiyjzk)p(xi zk)p(yj/zk)p(yj/xizk)p(yj/zk)????p(xiyjzk)logijk????p(xiyjzk)logijk????p(xiyjzk)logijk?i(y;x/z)p(xi/yjzk)p(xi/zk)ijki(x;y/z)????p(xiyjzk)logijkijk?????p(xiyjzk)logp(xi/zk)????p(xiyjzk)logp(xi/yjzk)?? ?????? p(xiyjzk)?logp(xi/zk)?h(x/yz)ik?j?????p(xizk)logp(xi/zk)?h(x/yz)ik?h(x/z)?h(x/yz)(3) i(x;y/z) ≥0,当且仅当(x, y, z)是马氏链时等式成立。