Download Bayesian Metanetworks for Context

Contextual level
Predictive level
Bayesian Metanetworks
for Context-Sensitive Feature Relevance
Vagan Terziyan
[email protected]
Industrial Ontologies Group, University of Jyväskylä,
Finland
SETN-2006, Heraclion, Crete, Greece
24 May 2006
Contents
 Bayesian Metanetworks


Metanetworks for
managing conditional
dependencies
Metanetworks for
managing feature
relevance
 Example
 Conclusions
Vagan Terziyan
Industrial Ontologies Group
Department of Mathematical
Information Technologies
University of Jyvaskyla (Finland)
http://www.cs.jyu.fi/ai/vagan
This presentation: http://www.cs.jyu.fi/ai/SETN-2006.ppt
2
Bayesian Metanetworks
3
Conditional dependence between
variables X and Y
P(X)
X
P(Y|X)
Y
P(Y)
P(Y) = X (P(X) · P(Y|X))
4
Bayesian Metanetwork
 Definition. The Bayesian Metanetwork is a
set of Bayesian networks, which are put on
each other in such a way that the elements
(nodes or conditional dependencies) of every
previous probabilistic network depend on the
local probability distributions associated with
the nodes of the next level network.
5
Two-level Bayesian C-Metanetwork
for Managing Conditional Dependencies
Contextual level
Predictive level
6
Contextual Effect on Conditional
Probability (1)
X
x1
x2
x3
x4
xk
x6
x7
contextual attributes
predictive attributes
Assume conditional
dependence between
predictive attributes
(causal relation between
physical quantities)…
x5
xt
xr
… some contextual
attribute may effect
directly the conditional
dependence between
predictive attributes but
not the attributes itself
8
Contextual Effect on Conditional
Probability (3)
Xt1 : I am in Paris
xt
Xt2 : I am in Moscow
P1(Xr |Xk )
Xk1
Xk2
Xk1 : order flowers
Xr1
0.3 0.9
Xr1 : visit football match
Xk2 : order wine
Xr2
0.4 0.5
Xr2 : visit girlfriend
xr
xk
Xk : Order present
P2(Xr |Xk )
Xk1
Xk2
Xr1
0.1 0.2
Xr2
0.8 0.7
Xr : Make a visit
10
Contextual Effect on Conditional
Probability (4)
Xt1 : I am in Paris
Xt2 : I am in Moscow
P( P (Xr |Xk ) | Xt )
X t1
X t2
P1(Xr |Xk )
0.7
0.2
P2(Xr |Xk )
0.3
0.8
xt
xr
xk
P1(Xr |Xk )
Xk1
Xk2
P2(Xr |Xk )
Xk1
Xk2
Xr1
0.3 0.9
Xr1
0.1 0.2
Xr2
0.4 0.5
Xr2
0.8 0.7
11
Contextual Effect on Unconditional
Probability (1)
X
x1
x2
x3
x4
X
xk
x7
xt
P(X)
x1 x2 x3 x4
x6
contextual attributes
predictive attributes
Assume some predictive
attribute is a random
variable with appropriate
probability distribution
for its values…
x5
… some contextual
attribute may effect
directly the probability
distribution of the
predictive attribute
12
Contextual Effect on Unconditional
Probability (3)
P( P (Xk ) | Xt )
X t1
X t2
P1(Xk )
0.4
0.9
P2(Xk )
0.6
0.1
xt
P1(Xk)
Xt2 : I am in Moscow
P2(Xk)
0.7
0.5
0.3
0.2
Xk
Xk
Xk1 Xk2
Xk1 Xk2
Xk1 : order flowers
Xk2 : order wine
Xt1 : I am in Paris
xk
Xk : Order present
14
Two-level Bayesian C-Metanetwork
for managing conditional dependencies
Contextual level
P(B|A)
P(Y|X)
A
B
X
Predictive level
Y
16
Two-level Bayesian R-Metanetwork
for Modelling Relevant Features’ Selection
Contextual level
Predictive level
18
Feature relevance modelling (1)
We consider relevance as a
probability of importance of
the variable to the inference
of target attribute in the
given context. In such
definition relevance inherits
all properties of a probability.
P(X)
X
Probability to have this model
is:
Probability to have this
model is:
P((X)=”no”)= 1-X
P((X)=”yes”)=  X
P0(Y)
P(Y|X)
Y
Y
P1(Y)
19
Feature relevance modelling (2)
X: {x1, x2, …, xnx }
1
P(Y ) 
  P(Y | X )  [nx  X  P( X )  (1   X )].
nx X
20
Example (1)
 Let attribute X will be
“state of weather”
and attribute Y, which is influenced by X, will
be “state of mood”.
 X (“state of weather”) ={“sunny”,
“overcast”, “rain”};



P(X=”sunny”) = 0.4;
P(X=”overcast”) = 0.5;
P(X=”rain”) = 0.1;
P(X)
 Y (“state of mood”) ={“good”, “bad”};






P(Y=”good”|X=”sunny”)=0.7;
P(Y=”good”|X=”overcast”)=0.5;
P(Y=”good”|X=”rain”)=0.2;
P(Y=”bad”|X=”sunny”)=0.3;
P(Y=”bad”|X=”overcast”)=0.5;
P(Y=”bad”|X=”rain”)=0.8;
Let:
X=0.6
P(Y|X)
21
Example (2)
P(Y ) 

1
  P(Y | X )  [nx  X  P( X )  (1   X )].
nx X
Now we have:
1
 {P(Y " good " | X " sunny" )  [1.8  P( X " sunny" )  0.4] 
3
 P(Y " good " | X " overcast" )  [1.8  P( X " overcast" )  0.4] 
P(Y " good " ) 
 P(Y " good " | X " rain" )  [1.8  P( X " rain" )  0.4]}  0.517;
P(Y " bad " )  0.483.
!

One can also notice that these values belong to the intervals created by the two
extreme cases, when parameter X is not relevant at all or it is fully relevant:
0.467  P0 (Y " good " ) | X 0  P(Y " good ") | X 0.6  P1 (Y " good " ) | X 1  0.55
0.45  P1 (Y "bad ") | X 1  P(Y "bad " ) | X 0.6  P0 (Y "bad " ) | X 0  0.533
22
General Case of Managing Relevance (1)
Predictive attributes:
X1 with values {x11,x12,…,x1nx1};
X2 with values {x21,x22,…,x2nx2};
…
XN with values {xn1,xn2,…,xnnxn};
Target attribute:
Y with values {y1,y2,…,yny}.
Probabilities:
P(X1), P(X2),…, P(XN);
P(Y|X1,X2,…,XN).
Relevancies:
X1 = P((X1) = “yes”);
X2 = P((X2) = “yes”);
…
XN = P((XN) = “yes”);
Goal: to estimate P(Y).
23
General Case of Managing Relevance (2)
Probability
P(XN)
P(Y ) 
1
N
 nxs
s 1
  ... [ P(Y | X 1, X 2,... XN ) 
X1 X 2
XN
nxr 


r ( ( Xr )" yes ")
Xr
 P( Xr ) 
(1  


Xq
)]
q ( ( Xq )"no")
24
Example of Relevance Bayesian
Metanetwork (1)
Conditional
relevance !!!
1
P(Y ) 
  {P(Y | X )  [nx  P( X ) 
nx X
  P( X |  A )  P( A )  (1   X )]}.
A
25
Example of Relevance Bayesian
Metanetwork (2)
26
Example of Relevance Bayesian
Metanetwork (3)
Ψ(A)
Ψ(X)
Ψ(B)
Contextual level
Ψ(Y)
Predictive level
A
B
X
Y
27
When Bayesian Metanetworks ?
1.
Bayesian Metanetwork can be considered as
very powerful tool in cases where structure
(or strengths) of causal relationships
between observed parameters of an object
essentially depends on context (e.g. external
environment parameters);
2.
Also it can be considered as a useful model
for such an object, which diagnosis depends
on different set of observed parameters
depending on the context.
28
Conclusion
 We are considering a context as a set of contextual
attributes, which are not directly effect probability
distribution of the target attributes, but they effect on a
“relevance” of the predictive attributes towards target
attributes.
 In this paper we use the Bayesian Metanetwork vision to
model such context-sensitive feature relevance. Such
model assumes that the relevance of predictive attributes
in a Bayesian network might be a random attribute itself
and it provides a tool to reason based not only on
probabilities of predictive attributes but also on their
relevancies.
29
Read more about Bayesian
Metanetworks in:
Terziyan V., A Bayesian Metanetwork, In: International
Journal on Artificial Intelligence Tools, Vol. 14, No. 3, 2005,
World Scientific, pp. 371-384.
http://www.cs.jyu.fi/ai/papers/IJAIT-2005.pdf
Terziyan V., Vitko O., Bayesian Metanetwork for
Modelling User Preferences in Mobile Environment, In:
German Conference on Artificial Intelligence (KI-2003),
LNAI, Vol. 2821, 2003, pp.370-384.
http://www.cs.jyu.fi/ai/papers/KI-2003.pdf
Terziyan V., Vitko O., Learning Bayesian Metanetworks from
Data with Multilevel Uncertainty, In: M. Bramer and V. Devedzic
(eds.), Proceedings of the First International Conference on
Artificial Intelligence and Innovations, Toulouse, France, August 2227, 2004, Kluwer Academic Publishers, pp. 187-196 .
http://www.cs.jyu.fi/ai/papers/AIAI-2004.ps
30