信息论基础（于秀兰 陈前斌 王永）课后作业答案注：X 为随机变量，概率P(X =x)是x 的函数，所以P(X)仍为关于X 的随机变量，文中如无特别说明，则以此类推。
第一章1.6[P (xy )]=[P(b 1a 1)P(b 2a 1)P(b 1a 2)P(b 2a 2)]=[0.360.040.120.48] [P (y )]=[P(b 1)P(b 2)]=[0.480.52] [P (x|y )]=[P(a 1|b 1)P(a 2|b 1)P(a 1|b 2)P(a 2|b 2)]=[0.750.250.0770.923]第二章2.1(1)I (B )=−log P (B )=−log 18=3(bit) 注：此处P (B )表示事件B 的概率。
(2)设信源为X ，H (X )=E [−logP (X )]=−14log 14−2∙18log 18−12log 12=1.75(bit/symbol) (3)ξ=1−η=1−1.75log4=12.5%2.2(1)P(3和5同时出现)=1/18I =−log118≈4.17(bit) (2)P(两个2同时出现)=1/36I =−log 136≈5.17(bit) (3)向上点数和为5时（14，23，41，32）有4种，概率为1/9，I =−log 19≈3.17(bit) (4)(5)P(两个点数至少有一个1)=1−5∙5=11 I =−log 1136≈1.71(bit) (6)相同点数有6种，概率分别为1/36；不同点数出现有15种，概率分别为1/18；H =6∙136∙log36+15∙118∙log18≈4.34(bit/symbol)2.9(1)H (X,Y )=E [−logP (X,Y )]=−∑∑P(x i ,y j )logP(x i ,y j )3j=13i=1≈2.3(bit/sequence)(2)H (Y )=E [−logP (Y )]≈1.59(bit/symbol)(3)H (X |Y )=H (X,Y )−H (Y )=0.71(bit/symbol)2.12(1)H (X )=E [−logP (X )]=−2log 2−1log 1≈0.92(bit/symbol) Y 的分布律为：1/2,1/3,1/6；H (Y )=E [−logP (Y )]≈1.46(bit/symbol)(2)H (Y |a 1)=E [−logP (Y|X )|X =a 1]=−∑P (b i |a 1)logP (b i |a 1)i=−34log 34−14log 14≈0.81(bit/symbol) H (Y |a 2)=E [−logP (Y|X )|X =a 2]=−∑P (b i |a 2)logP (b i |a 2)i=−12log 12−12log 12=1(bit/symbol) (3)H (Y |X )=∑P (a i )H (Y |a i )i =23∙0.81+13∙1≈0.87(bit/symbol)2.13(1)H (X )=H (0.3,0.7)≈0.88(bit/symbol)二次扩展信源的数学模型为随机矢量X 2=(X 1X 2)，其中X 1、X 2和X 同分布，且相互独立，则H (X 2)=2H (X )=1.76(bit/sequence)平均符号熵H 2(X 2)=H (X )≈0.88(bit/symbol)(2)二次扩展信源的数学模型为随机矢量X 2=(X 1X 2)，其中X 1、X 2和X 同分布，且X 1、X 2相关，H (X 2|X 1)=E [−logP (X 2|X 1)]=−∑∑P (x 1,x 2)logP (x 2|x 1)x 2x 1=−110log 13−210log 23−2140log 34−740log 14≈0.84(bit/symbol) H (X 2)= H (X 1,X 2)=H (X 2|X 1)+H (X 1)=0.84+0.88=1.72(bit/sequence)H 2(X 2)=H (X 2)/2=0.86(bit/symbol)2.14(1)令无记忆信源为X ，H (X )=H (14,34)=14×2+34×0.415≈0.81(bit/symbol ) (2)I (X 100)=−logP (X 100=x 1x 2…x 100)=−log [(14)m (34)100−m]=2m +(2−log3)(100−m )=200−(100−m )log3 (bit)(3)H (X 100)=100H (X )=81(bit/sequence)2.15(1)因为信源序列符号间相互独立，且同分布，所以信源为一维离散平稳信源。
(2)H(X)=H(0.2,0.8)≈0.72(bit/symbol)H(X2)=2H(X)=1.44(bit/sequence)H(X3|X1X2)=H(X3)=H(X)=0.72(bit/symbol)H∞=H(X)=0.72(bit/symbol)2.16(1)H(X2|X1)=E[−logP(X2|X1)]=−∑∑P(x1,x2)logP(x2|x1)x2x1=−610log910−230log110−230log210−830log810≈0.55(bit/symbol) H(X3|X2X1)= H(X3|X2)=H(X2|X1)≈0.55(bit/symbol)H(X4|X3X2X1)= H(X4|X3)=H(X2|X1)≈0.55(bit/symbol)(2)H∞=H(X2|X1)≈0.55(bit/symbol)ξ=1−η=1−0.55log2=45%(3)H(X)=H(23,13)≈0.92(bit/symbol)H∞≤H(X)，二维离散平稳信源的极限熵不大于其单符号信源的熵，说明离散单符号信源扩展后的单符号平均熵是非增的。
2.18(1)a iϵA，A是状态集；P(x i+1|s i=E a)表示i时刻状态为E a，i+1时刻输出x i+1。
该马尔科夫链的状态转移矩阵为P=[P(E aj |E ai)]=[P(x i+1|x i)]=[P(x i+1|s i=E a)]=[1/21/41/42/301/32/31/30]，P2=[7/125/245/245/95/181/65/91/65/18]，所以该链为齐次遍历马尔科夫链。
(2)令P(x i=k)=p i(k)，则[p1(1)p1(2)p1(3)]=[1/21/41/4]，[p2(1)p2(2)p2(3)]=[p1(1)p1(2)p1(3)]P=[111][1214142123130]=[712524524]因为[p1(1)p1(2)p1(3)]≠[p2(1)p2(2)p2(3)]，所以该信源不是离散平稳信源。
(3)当信源的输出序列足够长，马尔科夫链达到平稳分布时，该信源可以看作离散平稳信源。
(4)H(X i+1|s i=E1)=−∑P(x i+1=a|s i=E1)logP(x i+1=a|s i=E1)a=−12log12−2∙14log14=1.5(bit/symbol)同理得：H(X i+1|s i=E2)≈0.92(bit/symbol)H(X i+1|s i=E3)≈0.92(bit/symbol)设极限分布为[P(E1)P(E2)P(E3)]，则P(E1)=12P(E1)+23P(E2)+23P(E3)P(E2)=14P(E1)+13P(E3)P(E3)=1P(E1)+1P(E2)P(E1)+P(E2)+P(E3)=1解得P(E1)=4/7，P(E2)=3/14，P(E3)=3/14H∞=H(X i+1|s i)=47×1.5+2×314×0.92≈1.25(bit/symbol)(5)H0=log3≈1.59(bit/symbol)H1=H(12,14,14)=1.5(bit/symbol)ξ1=1−1.51.59≈5.66%H2=H(712524524)≈1.40(bit/symbol)ξ2=1−1.4≈11.95%[p3(1)p3(2)p3(3)]=[p2(1)p2(2)p2(3)]P=[712524524][1214142123130]=[413131]H 2=H(41723114431144)≈1.42(bit/symbol) ξ3=1−1.421.59≈10.69%2.20(1)状态转移矩阵P =[P(E j |E i )]=[0.80.200000.50.50.50.500000.20.8] (2)由P 知此马尔科夫链存在极限分布，设极限分布为[P(E 1)P(E 2)P(E 3)P(E 4)]，则P (E 1)=0.8P(E 1)+0.5P(E 3)P (E 2)=0.2P(E 1)+0.5P(E 3)P (E 3)=0.5P(E 2)+0.2P(E 4)P (E 4)=0.5P(E 2)+0.8P(E 4)P (E 1)+P (E 2)+P (E 3)+P(E 4)=1解得P (E 1)=5/14，P (E 2)=1/7，P (E 3)=1/7，P (E 4)=5/14(3)H (X i+1|s i =E 4)=H (X i+1|s i =E 1)=−0.8log0.8−0.2log0.2≈0.72(bit/symbol)H (X i+1|s i =E 3)=H (X i+1|s i =E 2)=−2∗0.5log0.5=1(bit/symbol)H ∞=H (X i+1|s i )=2×5×0.72+2×1×1=0.8(bit/symbol) (4)P (0)=∑P (0|E i )P(E i )i =0.8×514+0.5×17+0.5×17+0.2×514=0.5 P (1)=0.5(5)初始时刻的P (0),P (1)和(4)中不一样，所以初始时刻的信源不是平稳信源。