Download Wednesday

Agenda
 On Quiz 2
 Network Information Theory
 Typical Networked Communication Scenarios
 Gaussian Networked Communication Channels
 Joint Typical Sequences
1
Network Communication Theory
 Point-to-point Information Theory
 Lossless data compression
 Channel capacity
 Lossy data communication, rate-distortion
 Information Theory for Communication
Network
 Multiple point communication
 Data flow in the network
2
Typical Communication Topology
 Multiple Access Communication System
 Broadcast Communication System
Transmitter
 Relay Communication System
Relay
Transmitter 1 Transmitter 2
Receiver
Transmitter
Receiver 1
Receiver
Receiver 2
3
Typical Communication Topology
 Communication Network
C1
C4
C3
C2
C5
 Max-Flow Min-Cut Theorem
C = min{C1+C2, C1-C3+C5, C2+C3+C4, C4+C5}
4
Gaussian Multi-user Channels
 Gaussian Single User Channel
Y = X + Z, Z ~ Gaussian(0, N)
C = (1/2)log(1 + P/N)
 Gaussian Multi-access Channel with M Users
Y = Σ1≤m≤MXm + Z, Z ~ Gaussian(0, N)
Capacity Region: Σm \in SRm ≤ (1/2)log(1 + |S|P/N)
Two users?
Three users?
5
Gaussian Multi-user Channels
 Gaussian Broadcast Channels
Y1 = X + Z1, Z1 ~ Gaussian(0, N1)
Y2 = X + Z2, Z2 ~ Gaussian(0, N2)
 User Rate Region: (R1, R2), for 0 ≤ α ≤ 1
R1 ≤ C(αP/N1), R2 ≤ C((1-α)P/(αP+N2))
6
Gaussian Interference Channel
 Gaussian Interference Channel
X1
X2
a
a
Y1
Y2
Y1 = X1 + aX2 + Z1, Z1 ~ Gaussian(0, N)
Y2 = X2 + aX1 + Z2, Z2 ~ Gaussian(0, N)
Strong Interference = No Interference;
What is a >> 1?
7
Joint Typical Sequence
 Communication Network Signal Vector
(X1, X2, …, XN)
 Let S Denote any Order Sets of (X1, X2, …, XN)
S = (X1, X2), S = (X1, XN), S = (X1, X3, XN-1),
S1, S2, …, SN be realizations of S
-(1/n)log p(S1, S2, …, SN)  H(S)
 Typical: |-(1/n)log p(S1, S2, …, SN) - H(S)| < ε
All 2N-1 choices of S
8
Joint Typical Sequence
 A(n)ε(S): the ε-typical sequence w.r.t. S
 For sufficiently large n, we have
 Pr(A(n)ε(S)) > 1 – ε
 For s in A(n)ε(S), we have p(s) ≈ 2-nH(s)
 |A(n)ε(S)| ≈ 2nH(s)
 For s1, s2 in A(n)ε(S), we have p(s1|s2) ≈ 2-nH(s1|s2)
9
Joint Typical Sequence
 A(n)ε(S1|s2): set of S1 sequence jointly typical with
s2 sequence
 For sufficiently large n, we have
|A(n)ε(S1|s2)| ≤ 2n(H(S1|S2) + 2ε)
(1-ε)2n(H(S1|S2)-2ε) ≤ Σs2p(s2)|A(n)ε(S1|s2)|
10
Homework
15.2, 15.4, 15.6, 15.23, 15.25, 15.33
11