Download On the Identifiability of Hierarchic, Sparse and

Nature-inspired Smart Info Systems
Ronald L. Westra, Department of Mathematics
Lars Eijssen, Joyce Corvers, Department of Genetics
Maastricht University
On the identifiability of
piecewise linear gene-protein networks
relative to noise and chaos
G
1
G
3
G
1
G
2
P
3
P
2
P
4
P
3
G
3
P
5
P
1
Σ1
Σ2
G
4
G
6
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
1
Nature-inspired Smart Info Systems
Items in this Presentation
1. Background and problem formulation
2. Modeling and identification of gene/proteins interactions
3. The implications of stochastic fluctuations and deterministic chaos
5. Example 1: Application on artificial reaction model
5. Example 2: Application on Tyson-Novak model for fission yeast
5. Example 3: Application on fission yeast expression data
6. Conclusions
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
2
Nature-inspired Smart Info Systems
1. Problem formulation
Question: Can gene regulatory networks be reconstructed from
time series of observations of (partial) genome wide and protein
concentrations?
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
3
Nature-inspired Smart Info Systems
Problems in modeling and identification
Relation between mathematical model and phys-chem-biol reality
Macroscopic complexity from simple microscopic interactions
Approximate modeling as partitioned in subsystems with local
dynamics
Modeling of subsystems as piecewise linear systems (PWL)
PWL-Identification algorithms: network reconstruction from
(partial) expression and RNA/protein data
Experimental conditions of poor data: lots of gene but little data
The role of stochasticity and chaos on the identifiability
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
4
Nature-inspired Smart Info Systems
2. Modeling the Interactions between
Genes and Proteins
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
5
Nature-inspired Smart Info Systems
2.1 Modeling the molecular dynamics and
reaction kinetics as Stochastic Differential Equations
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
6
Nature-inspired Smart Info Systems
2.2 Gene-Protein Interaction Networks as
Piecewise Linear Models
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
7
Nature-inspired Smart Info Systems
2.3 Problems concerning the identifiability
of PieceWise Linear models
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
8
Nature-inspired Smart Info Systems
3. The Implications of Stochastic
fluctuations and Deterministic Chaos
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
9
Nature-inspired Smart Info Systems
3.1 Stochastic fluctuations
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
10
Nature-inspired Smart Info Systems
3.2 Noise-induced control in single-cell gene expression
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
11
Nature-inspired Smart Info Systems
Influence of stochastic fluctuations on the evolution of
the expression of two coupled genes.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
12
Nature-inspired Smart Info Systems
3.3 Deterministic Chaos
Prerequisite for the
successful
reconstruction of geneprotein networks is the
way in which the
dynamics of their
interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
13
Nature-inspired Smart Info Systems
4. Identification of Interactions
between Genes and Proteins
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
14
Nature-inspired Smart Info Systems
4.2 The identification of PIECEWISE linear networks
by L1-minimization
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
15
Nature-inspired Smart Info Systems
Gene-Protein Interaction Networks as
Piecewise Linear Models
The general case is complex and approximate
Strongly dependent on unknown microscopic details
Relevant parameters are unidentified and thus unknown
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
16
Nature-inspired Smart Info Systems
2. Modeling of PWL Systems as
subspace models
Global dynamics:
Σ5
Local attractors
(uniform, cycles, strange)
Σ4
Basins of Attraction
Σ1
Σ3
Σ2
Σ6
Each BoA is a
subsystem Σi
“checkpoints”
State space
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
17
Nature-inspired Smart Info Systems
Modeling of PWL Systems as subspace models
State vector moves
through state space
driven by local
dynamics (attractor,
repeller) and inputs
in each subsystem Σ1
the dynamics is
governed by the local
equilibria.
approximation of
subsystem as linear
statespace model:
State space
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
18
Nature-inspired Smart Info Systems
Problems concerning the identifiability
of Piecewise Linear models
1. Due to the huge costs and efforts involved in the experiments, only
a limited number of time points are available in the data. Together with
the high dimensionality of the system, this makes the problem
severely under-determined.
2. In the time series many genes exhibit strong correlation in their
time-evolution, which is not per se indicative for a strong coupling
between these genes but rather induced by the over-all dynamics of
the ensemble of genes. This can be avoided by persistently exciting
inputs.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
19
Nature-inspired Smart Info Systems
Problems concerning the identifiability
of Piecewise Linear models
3. Not all genes are observed in the experiment, and certainly most of
the RNAs and proteins are not considered. therefore, there are many
hidden states.
4. Effects of stochastic fluctuations on genes with low transcription
factors are severe and will obscure their true dependencies.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
20
Nature-inspired Smart Info Systems
Problems with stochastic modeling
Such are the problems relating to the identifiability of
piecewise linear systems:
Are conditions for modeling rate equations met?
High stochasticity and chaos
Are piecewise linear approximations a valid metaphor?
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
21
Nature-inspired Smart Info Systems
The identification of PIECEWISE linear networks
by L1-minimization
K linear time-invariant subsystems {Σ1, Σ2, .., ΣK}
Continuous/Discrete time
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
22
Nature-inspired Smart Info Systems
4.2 The identification of PIECEWISE linear networks
by L1-minimization
Weights wkj indicate membership of observation #k
to subsystem
Σj :
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
23
Nature-inspired Smart Info Systems
Rich and Poor data
poor data: not sufficient empirical data is
available to reliably estimate all system parameters,
i.e. the resulting identification problem is underdetermined.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
24
Nature-inspired Smart Info Systems
(un)known switching times,
regular sampling intervals,
rich / poor data,
Identification of PWL models with known switching times and
regular sampling intervals from rich data
Identification of PWL models with known switching times and
regular sampling intervals from poor data
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
25
Nature-inspired Smart Info Systems
1. unknown switching times,
regular sampling intervals,
poor data, known state derivatives
This is similar to simple linear case
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
26
Nature-inspired Smart Info Systems
This can thus be written as:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
27
Nature-inspired Smart Info Systems
with:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
28
Nature-inspired Smart Info Systems
with:
The approach is as follows:
(i) initialize A, B, and W,
(ii) perform the iteration:
1. Compute H1 and H2, using the simple linear system approach
2. Using fixed W, compute A and B,
3. Using fixed A and B, compute W
until: (iii) criterion E has converged sufficiently –
or a maximum number of iterations.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
29
Nature-inspired Smart Info Systems
Linear L1-criterion:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
30
Nature-inspired Smart Info Systems
With linear L1-criterion E1 the problem can be
formulated as LP-problem:
LP1: compute H1,H2 from simple linear case
LP2: A and B, using E1-criterion and extra constraints
for W, H1,H2,
LP3: compute optimal weights W, using E1-criterion
with constraints for W, H1,H2, A and B
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
31
Nature-inspired Smart Info Systems
2. unknown switching times,
regular sampling intervals,
poor data, unknown state derivatives
Use same philosophy as mentioned before
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
32
Nature-inspired Smart Info Systems
Subspace dynamics and linear L1-criterion :
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
33
Nature-inspired Smart Info Systems
System parameters and empirical data :
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
34
Nature-inspired Smart Info Systems
Quadratic Programming problem QP :
Problem: not well-posed: i.e.: Jacobian becomes
zero and ill-conditioned near optimum
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
35
Nature-inspired Smart Info Systems
Therefore split in TWO Linear Programming problems:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
36
Nature-inspired Smart Info Systems
In case of sparse interactions replace LP1 with:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
37
Nature-inspired Smart Info Systems
Performance of robust Identification approach
Artificially produced data reconstructed with this approach
Compare reconstructed and original data
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
38
Nature-inspired Smart Info Systems
The influence of increasing intrinsic noise on the identifiability.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
39
Nature-inspired Smart Info Systems
a: CPU-time Tc as a function of the problem size N,
b: Number of errors as a function of the number of nonzero entries k,
M = 150, m = 5, N = 50000.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
40
Nature-inspired Smart Info Systems
a: Number of errors versus M,
b: Computation time versus M
N = 50000, k = 10, m = 0.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
41
Nature-inspired Smart Info Systems
a: Minimal number of measurements Mmin required to compute A
free of error versus the problem size N,
b: Number of errors as a function of the intrinsic noise level σA
N = 10000, k = 10, m = 5, M = 150, measuring noise B = 0.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
42
Nature-inspired Smart Info Systems
Example 1: how to apply this method on current data sets
Spellman et al. data for cell-cycle of fission yeast :
Components: 6179 genes measured for 18-24 irregular
time instants
Processing: fuzzy C-means, gene annotation with Go
term finder and Fatigo, net recontruction with identification
algorithm
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
43
Nature-inspired Smart Info Systems
Spellman et al. data for cell-cycle of fission yeast :
Processing:
Selection of most up/down-regulated genes: 3107 from 6179
Clustering: fuzzy C-means: best outcome 23 clusters
Gene annotation with Go term finder (4th level) and Fatigo,
both for biological process and cellular component
Net recontruction with identification algorithm on 23 clusters
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
44
Nature-inspired Smart Info Systems
Centroids after clustering 23 clusters
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
45
Nature-inspired Smart Info Systems
Gene ontology
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
46
Nature-inspired Smart Info Systems
Gene ontology
Cluster 1
GO Term Finder: The genes are involved in spindle pole during the cell cycle, with relations to microtubuli and
chromosomal structure.
FatiGO: The main cellular component is the chromosome.
Cluster 2
GO Term Finder: The genes are involved in proliferation and replications, especially bud neck and polarized
growth.
FatiGO: The results found by the GO Term Finder are confirmed.
…………….
Cluster 22
GO Term Finder: Only a few annotations are found and there are many unknown genes. The genes are
involved in respiration and reproduction. The main cellular components are the actin/cortical skeleton and the
mitochondrial inner membrane.
FatiGO: No further clear annotations are found.
Cluster 23
GO Term Finder: The genes are involved in RNA processing. The main cellular components are the nucleus,
the RNA polymerase complex and the ribonucleoprotein complex.
FatiGO: The main cellular component is the ribonucleoprotein complex.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
47
Nature-inspired Smart Info Systems
Reonstructed network
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
48
Nature-inspired Smart Info Systems
Example 2: artificial data of hierarchic/sparse network
Artificial reaction network with:
Components:
2 master genes with high transcription rates
3 slave genes with low transcription rates
4 agents (= RNA or proteins).
Processes: stimulation, inhibition, transcription, and
reactions between ‘agents’
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
49
Nature-inspired Smart Info Systems
Dynamics:
– large hierarchic and sparse network
– implicit relation between genes with expression x
through agents (= proteins, RNA) with concentration a
– system near equilibrium and small perturbations
– inputs: persistent excitation u
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
50
Nature-inspired Smart Info Systems
Dynamics:
– implicit system dynamics:
– linear statespace model makes gene interaction explicit:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
51
Nature-inspired Smart Info Systems
Dynamics:
– estimate gene-gene interaction matrix A from
empirical data:
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
52
Nature-inspired Smart Info Systems
reactions
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
53
Nature-inspired Smart Info Systems
reactions
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
54
Nature-inspired Smart Info Systems
reactions
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
55
Nature-inspired Smart Info Systems
rate equations
Matlab-simulation
y(1)
y(2)
y(3)
y(4)
y(5)
y(6)
y(7)
y(8)
y(9)
=
=
=
=
=
=
=
=
=
-
0.03*x(1)
0.05*x(2)
0.02*x(3)
0.01*x(4)
0.02*x(5)
0.02*a(1)
0.01*a(2)
0.01*a(3)
0.05*a(4)
+
+
+
+
+
+
+
0.2*(1-x(1))*a(2)^2 - 0.2*x(1)*a(3) ;
0.3*(1-x(2))*a(1)
- 0.1*x(2)*a(4) ;
0.1*(1-x(3))*a(2)
- 0.1*x(3)*a(1) ;
0.2*(1-x(4))*a(1)*a(2) - 0.2*x(4)*a(3)^2;
0.3*(1-x(5))*a(3)
- 0.1*x(5)*a(1);
0.4*x(1) - 0.2*a(1)*a(2) - 0.1*a(1)*a(3)^3;
0.15*x(2) - 0.2*a(1)*a(2);
+ 0.2*a(1)*a(2) - 0.1*a(1)*a(3)^3;
+ 0.9*a(1)*a(3);
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
56
Nature-inspired Smart Info Systems
Real network structure: implicit
a
1
2
b
3
c
d
g
gene
p
agent
4
5
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
57
Nature-inspired Smart Info Systems
Real network structure: explicit
master
master
1
2
3
4
5
slave
slave
slave
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
58
Nature-inspired Smart Info Systems
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
59
Nature-inspired Smart Info Systems
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
60
Nature-inspired Smart Info Systems
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
61
Nature-inspired Smart Info Systems
Reconstructed network structure:
low noise
master
master
1
2
3
4
slave
slave
5
slave
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
62
Nature-inspired Smart Info Systems
Reconstructed network structure:
moderate noise
master
master
1
2
3
4
5
slave
slave
slave
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
63
Nature-inspired Smart Info Systems
Reconstructed network structure:
high noise (an example)
slave
master
1
3
slave
2
4
5
slave
master
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
64
Nature-inspired Smart Info Systems
Example 3: data of Tyson-Novak math. model for cell cycle
Tyson-Novak model for cell-cycle of fission yeast :
Components:
9 agents (= RNA or proteins).
Processes: stimulation, inhibition, transcription, and
reactions between ‘agents’
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
65
Nature-inspired Smart Info Systems
The deterministic Tyson-Novak model.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
66
Nature-inspired Smart Info Systems
The stochastic Tyson-Novak model.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
67
Nature-inspired Smart Info Systems
Example: stochastic Tyson-Novak model
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
68
Nature-inspired Smart Info Systems
Example: stochastic Tyson-Novak model
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
69
Nature-inspired Smart Info Systems
4.2 The identification of PIECEWISE linear networks
by L1-minimization
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
70
Nature-inspired Smart Info Systems
5. Epilogue: Lessons from Nature
Prerequisite for the successful reconstruction of
gene-protein networks is the way in which the
dynamics of their interactions is modeled.
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
71
Nature-inspired Smart Info Systems
Discussion …

G
1
G
3
G
1
G
2
P
3
P
2
P
4
P
3
G
3
P
5
P
1
Σ1
Σ2
G
4
G
6
Westra: Piecewise Linear Dynamic Modeling and Identification of Gene-Protein Interaction Networks
72