Download Extrimes of Information Combining

Extremal Problems of
Information Combining
Alexei Ashikhmin
 Information Combining: formulation of the problem
 Mutual Information Function for
the Single Parity Check Codes
 More Extremal Problems of Information Combining
 Solutions (with the help of Tchebysheff Systems)
for the Single Parity Check Codes
Joint work with Yibo Jiang, Ralf Koetter, Andrew Singer
Information Transmission
Channel
Encoder
Channel
APP
Decoder
Channel
Density function of the channel is not known
We only know
Optimization Problem
We assume that
and that the channel is symmetric
Problem 1
Among all probability distributions
such that
determine the probability distribution that maximizes (minimizes)
the mutual information at the output of the optimal decoder
From variable nodes
Interleaver
Check nodes processing
processing
channel
nodes
Variable
Input from
To variable nodes
Decoder of
single parity
check code
Problem is Solved Already
1. I.Land, P. Hoeher, S.Huettinger, J. Huber, 2003
2. I.Sutskover, S. Shamai, J. Ziv, 2003
Repetition code:
The Binary Erasure Channel (BEC) is the best
The Binary Symmetric Channel (BSC) is the worst
erasure
Single Parity Check Code:
is Dual of Repetition Code
The Binary Erasure Channel (BEC) is the worst
The Binary Symmetric Channel (BSC) is the best
Our Goals
 We would like to solve the optimization problem for the
Single Parity Check Codes directly (without using
duality)
 Get some improvements
Soft Bits
Channel
We call
soft bit, it has support on
erasure
Binary symmetric channel,
Gaussian Channel:
Channel
Encoder
Single Parity
Check Code
Channel
Decoder
Single Parity
Check Code
Channel
E.Sharon, A. Ashikhmin, S. Litsyn
Results:
Properties of the moments
Lemma
1.
is nonnegative and nonincreasing
2. The ratio sequence
is nonincreasing
Lemma
In the Binary Erasure Channel all moments are the same
Problem 2
Among all T-consistent probability distributions
such that
determine the probability distribution that maximizes
(minimizes) the second moment
on [0,1]
Solution of Problem 2
Theorem
Among all binary-input symmetric-output channel
distributions with a fixed mutual information
Binary Symmetric Channel maximizes
and
Binary Erasure Channel minimizes
the second moment
Proof: We use the theory of Tchebysheff Systems
Lemma
Binary Symmetric, Binary Erasure and an arbitrary channel
with the same mutual information have the following layout of
Lemma
Let
and
1)
2) if
then
for
and
for
This is exactly our case
satisfy conditions of the previous lemma
Problem 1 on extremum of mutual information
and
Problem 2 on extremum of the second moment
are equivalent
Extrema of MMSE
Channel
It is known that the channel soft bit
is the MMSE estimator fo
the channel input
Theorem Among all binary-input symmetric-output channels with
fixed
the Binary Symmetric Channel has
the minimum MMSE:
and the Binary
Erasure Channel has the maximum MMSE:
How good the bounds are
Problem 3
Channel
Encoder
Single Parity
Check Code
Channel
Decoder
Single Parity
Check Code
Channel
1)
2)
Among all T-consistent channels find
(minimizes)
that maximizes
Problem 4
Among all T-consistent probability distributions
such that
1)
2)
determine the probability distribution that maximizes
(minimizes) the fourth moment
on [0,1]
Theorem
The distribution with mass
and mass
at 0 maximizes
The distribution with mass
and mass
at 1 minimizes
at
, mass
at
at
, mass
at
Extremum densities
Maximizing
Minimizing:
Lemma
Channel with minimum and maximum
and an arbitrary
channel with the same mutual information have the followin
layout of
Problem 3 on extremum of mutual information
and
Problem 4 on extremum of the fourth moment
are equivalent
Assume that
and
is the same as in AWGN channel with this
Tchebysheff Systems
Definition
A set of real continues functions
is called
Tchebysheff system (T-system) if for any real
the linear
combination
has at most distinct roots at
Definition
A distribution is a nondecreasing, right-continues function
The moment space, defined by
(
For
is the set of valid distributions), is a closed convex cone.
define
Problem
For a given
find
that maximizes (minimizes)
Theorem
If
and
and
are T-systems,
then the extrema are attained uniquely with
distrtibutions
and
Lower principal
representation
with finitely many mass points
Upper principal
representation
Soft Bits
Channel
We call
soft bit, it has support on
Lemma (Sharon, Ashikhmin, Litsyn)
If
then
Random variables with this property are called T-consistent
Find extrema of
Under constrains
Theorem
Systems
and
are T-systems on [0,1].
--------------------------------------------------------------------------------the distribution
that maximizes
has only one mass point at
:
has probability mass
and
This is exactly the Binary Symmetric Channel
at
at