Download Granular Computing

Software Engineering of
Granular Systems
Granular Computing and Hypercomputation
Andrzej Bargiela
NAFIPS 2004, Summer School
Overview and motivation



Granular Computing came about through the
realisation of incompatibility between human and
machine information processing
Fuzzy and rough set formalisms of information
granulation provided an excellent framework for
knowledge representation (Zadeh, Pawlak, Kacprzyk,
Yager, Kandel, Bezdek, Pedrycz, Kreinovich, Hirota,
Skowron, Slowinski)
The key remaining question is whether the processing
of information granules (knowledge) can be
represented adequately as a computational process.
Workshop Plan






Fundamentals of computation
Information vs. knowledge processing
Hypercomputational processing of knowledge
as an aspect of human intelligence
Information granulation fundamentals
Granular Computing
Granular Computing and Hypercomputation
Fundamentals of computation
Definitions

Conceptual “computation”


Formal behaviour equivalent to UTM (so
“computable” means “computable by UTM”)
Physical “computation”


Action of a physical system that implements
conceptual computation (man or machine)
Implementation that implements computable
function is a “computational system”
Universal Turing Machine

UTM





Finite but unbounded tape for storage of symbols
A head for reading and writing symbols from/to
the tape
Finite set of symbols used for input/output.
Finite set of internal states
Transitions associated with internal states and
detailing how the machine should function
(symbol1, symbol2, direction, new-state)
Universal Turing Machine

UTM to calculate binary expansion of 1/3


Initial tape
Final tape
0 0 0 0 0 0 0 0 0 0 …
0 1 0 1 0 1 0 1 0 1…
0/0/R/1
1
0
0/1/R/0
Universal Turing Machine

Turing’s halting function


Uncomputable set


Function that takes as its input a natural number
representing a Turing machine and returns 1 if
the Turing machine halts or 0 if it does not.
{n | n represents a Turing machine / input pair
that halts}
Uncomputable real

x such that its n-th digit of its binary expansion is
1 if n represents a Turing machine / input pair
that halts and 0 otherwise
Universal Turing Machine

Proof of uncomputability of a halting function




Assume there exists a Turing machine T for the
halting function H
Construct a modified machine T* which takes an
encoding of T and determines whether T halts
when given its own encoding as input
Construct another machine T** that is like T* but
it loops if T halts on its own encoding and halts if
T loops on its encoding
If T** is given an encoding of itself as input, it
loops if and only if it would halt. A contradiction  T does not exist
Universal Turing Machine


So, what can be evaluated by UTM?
Church-Turing [CT] thesis

“… the numerical functions that can be effectively
evaluated by human clerical labour, working to
fixed rule, and without understanding are
precisely functions that can be evaluated by the
UTM …”
Interpretation of CT thesis

Misunderstanding


!
Any function that can be computed by any means
whatsoever can be computed by UTM
Correct interpretation



Humans can in-principle, simulate computers
through clerk-like behaviour
The computer behaviours are precisely the effective
human behaviours
but … human behaviours go beyond that!!!
especially when it comes to knowledge processing
Illustration of CT thesis
Intelligent
human behaviours
effective
computer
=
(clerk-like)
behaviours
behaviours
Effective = clerical labour, working to fixed rule, and without understanding
processing of symbols (information)
Characteristics of behaviours

Effective




Processing of finite set
of symbols (information)
Operation to “fixed rule”
Operation without
insight
Operation without
understanding

Intelligent

Processing of
uncountable abstractions
(knowledge)



Keeping “open mind”
about operation
Essential insight
Essential understanding
Beyond UTM

Hypercomputational system


IS the Nature computational?


A system that “computes” non-computable
behaviours (such that cannot be simulated by UTM)
All implementations are computational
IS the Nature hypercomputational?

There exist at least one implementable behaviour
that is not computational
Quo Vadis AI ?

Wrong premise

!
“.. If a scientific (empirically verifiable) study of
intelligence (AI) is possible at all it must be capable
of being expressed in computational terms..”
(scientific method does not exclude insight/intelligence but
definition of computability does)

“.. A standard digital computer, given only the right
program, a large enough memory and sufficient
time can compute any rule-governed input-output
function..” (unwarranted extension of the definition of UTM
from manipulation of symbols to any I/O mapping)
The meaning of “A” in AI

AI dilemma
If “artificial” in AI is to be interpreted as “fake”
then it can continue with developing computational
models of (ersatz) intelligence,
but …
 If “artificial” in AI is to be interpreted as “produced
by means that are not of human origin but
equivalent in substance to the original thing”, AI
must go beyond “computation” i.e. focus on
“hypercomputation”.

Illustration of CT thesis
Intelligent
human behaviours
(that include)
Hypercomputational behaviours
effective
computer
=
(clerk-like)
behaviours
behaviours
Effective = clerical labour, working to fixed rule, and without understanding
processing of symbols (information)
Intelligent / hypercomputational
behaviour
Mathematical reasoning

Goedel’s Incompleteness Theorem (1931)


Principia Mathematica is either incomplete or
inconsistent theory of natural numbers.
Proof:



Mapping of arithmetical formulae onto numbers
(so that statements can refer to each other).
Mapping of a statement “is provable in Principia
Mathematica” onto a statement in arithmetic.
Formulation of a statement that says of itself that
it is not provable within Principia Mathematica.
“Finite means” reasoning

Incompleteness Theorem extension


Any formal proof system that operates by final
means is either incomplete or inconsistent
Proof



Mapping of axioms onto numbers (so that axioms
can refer to each other)
Mapping of inference rules onto statements in
arithmetic
Formulation of a statement that says of itself that
it is not provable within a given formal proof
system.
“Computational” reasoning

Interpretation of “finite means” in the context
“mechanical computation”



Turing: Finite means interpreted as a computation
accomplished by a human agent operating without
insight
Kleene: Finite means interpreted as a recursive
evaluation of functions that are rooted in a set of
basic functions (zero, successor, projection,
composition, primitive recursion).
Same set of computable functions
Entscheidungsproblem

Given a set of axioms A decide whether the
theorem T is or is not provable from A.



Failure to find an algorithm (Turing Machine) for
solving such a problem
General argument: The set of functions from N to
N is uncountable while a set of Turing machines is
countable.
Specific examples of functions that cannot be
computed (Turing’s halting function)
HC rationale

We need hypercomputation (HC) to
process knowledge as opposed to
processing just an information
… more than “computing” …

Computing





Finite set of symbols
Operation to “fixed
rule”
Operation without
insight
Operation without
understanding
Single correct result

… more …




Infinite, enumerable
set of symbols
Infinite state
Infinite time
A-priori results
Theoretical forms of HC





Infinite
Infinite
Infinite
Infinite
Infinite
decay)
!
memory (Oracle, initial inscriptions)
state
speed
time
mass/energy (harnessing radioactive
Infinite memory

Turing’s Oracle machine (OTM)



The machine is equipped with three additional
special states: 0-state, 1-state and “call state”
If the Oracle stores the halting set (potentially
infinite) then the presentation of the input on
which the machine halts can stipulate the
reference to the Oracle and successful
“computation” of the halting function
(non harnessable hypercomputation)
!
Infinite memory

UTM with initial inscriptions



The odd squares of the machine tape are filled
with the binary expansion of the real number
representing the halting set and the even squares
are used for normal operation of UTM
The initial inscription is effectively another form of
Oracle (potentially infinite)
(non harnessable hypercomputation)
!
Infinite state

UTM with infinite set of states



The implication is that there is an infinite number
of transitions from which a finite number leads to
a given state
The states (infinite set) of the UTM* can be used
to represent the halting set of the corresponding
UTM
(non harnessable hypercomputation)
!
Infinite speed

UTM with geometrically increasing clock



The first step is assumed to take 1 unit of time
and the subsequent steps are taking ½ of the time
required by the previous step. So the infinite
number of steps can be accomplished in 2 units of
time (infinite execution speed)
The problem with the halting set is bypassed by
taking the output from the first square after 2
time steps
(non harnessable hypercomputation)
!
Infinite time

UTM allowed to operate infinite time



The configuration of the machine is defined from
the preceding configurations. The machine goes
into a special limit-state and each tape square is
defined as: 0 if the square settles to 0; 1 if the
square settles to 1; and 1 if the square alternates
infinitely often between 0 and 1
After deciding the state of a square the machine is
restarted from the first square
(non harnessable hypercomputation)
!
Infinite mass/energy

Probabilistic UTM



The machine has two applicable transitions from a
given state. When in such a state, the machine
chooses the transition with equal probability. The
machine computes a function if the probability of
a correct answer is greater than ½
Machine that outputs randomly 0 or 1 with equal
probability (using eg. Radioactive decay as a
random process) generates a non-recursive real
with probability 1 but there is probability 0 of
generating any specific number
(non harnessable hypercomputation)
!
Practical form of HC

Use ordinary computers

but …

use them in an “intelligent” way to
do more than just “computing”
… more than “computing” …

Computing





Finite set of symbols
Operation to “fixed
rule”
Operation without
insight
Operation without
understanding
Single correct result

… more …




Infinite, enumerable
set of symbols
Infinite state
!
Infinite time
A-priori results
… more than “computing” …

Computing





Finite set of symbols
Operation to “fixed
rule”
Operation without
insight
Operation without
understanding
Single correct result

… more …






Infinite, enumerable
set of symbols
Infinite state
!
Infinite time
A-priori results
Physical simulation
Keep open mind
about the results
Hypercomputational processing
of knowledge as an aspect of
human intelligence
… more than “computing” …
2
1
1
… more than “computing” …
… more than “computing” …
… more than “computing” …
Practical form of HC

Use ordinary computers

but …

allow them to be “open minded” (based
on evidence derived at different levels of
abstraction  granulation)
Information granulation
fundamentals
Granulation as a basis for
being “open minded”



Granulation - a process of instilling additional
semantic content into data (not just a simple
aggregation).
Fundamental distinction from “flat” numerical
interpretation of data.
An insight rooted in the axiomatic set
theory.
Axiomatic set theory
Georg Cantor
foundations of Set Theory 1874-1884
Two primitive notions of: sets and membership
Paradoxes in initial theory:
- The cardinality of “set of all sets” vs. cardinality of the set of all
subsets drawn from this set.
- Construction of sets paradox (Russel)
Axiomatic set theory
Modern Set Theory (ies) -
(eg. Goedel, 1940)
Abandons a uniform view of sets and introduces some
form of “hierarchy”
Emphasises the semantical transformation (qualitative
change) that occurs when grouping individual elements
into sets or conversely when identifying sub-sets within
a given set
Three primitive notions of: class, set and membership
Axiomatic set theory
Modern Set Theory (ies)
Class - an entity corresponding to some but not
necessarily all properties
(-> various classes represent conceptually different
entities)
A class that is a member of some other class is
considered a set, otherwise it is considered a proper
class
(-> a “set of all sets” is a proper class)
Granular Computing
New Insights

Granular Computing:


A paradigm for computational processing of
information granules
A paradigm for harnessable hypercomputation
through concurrent computations along the
dimension of Information Abstraction
New Insights
Infinite Resource Hypercomputing
Turing Machines
Abstractions
Physical
Logical
States
States
Numbers
Sets
Concepts
New Insights
Infinite Resource Hypercomputing
Turing Machines
Granular
Computing
Granular
Computing
Granular
Computing
Granular
Computing
Abstractions
Physical
Logical
States
States
Numbers
Sets
Concepts
Granular Computing
G = <X, G, A, C>
Granular world
data
space
granulation
framework
communication
mechanism
family of
granules
A/D
Digital
processing
D/A
Granular Computing
G = <X, G, A, C>
Granular world
data
space
granulation
framework
communication
mechanism
family of
granules
Digital
A/D
D/A
Digital
D/D processing
D/D
Digital
D/D processing
D/D
Digital
D/D processing
D/D
Digital
D/D processing
D/D
processing
Granular Computing
Concepts:
Encoded as paintings/sculptures/music etc.
Original 
idea

Decoded
idea
Granular Computing
Concepts:
Encoded as paintings/sculptures/music etc.
Processing of pixels
Original 
idea
Image segmentation
Texture/colour analysis
Analysis of metaphores

Decoded
idea
Granular Computing
Concepts:
Encoded as paintings/sculptures/music etc.
Original 
idea

Decoded
idea
Granular Computing
Concepts:
Encoded as paintings/sculptures/music etc.
Processing of pixels
Original 
idea
Image segmentation
Analysis of rendering
Analysis of metaphores

Decoded
idea
Granular Computing and
Hypercomputation
Entscheidungsproblem

Given a set of axioms A decide whether the
theorem T is not provable from A



At the “coarse” level of granularity produce an
instantaneous negative answer
At the “fine” level of granularity perform a
computational process of permutating axioms
If the processing at the “fine” level of granularity
results in proving T then change the answer to
affirmative
Conclusions
• Granular Computing arose through insights from
pattern classification
• Axiomatic set theory underpinning the above insights
• The prominence of fuzzy/rough sets for the
description of information granules (knowledge)
• Knowledge processing as an inherently
hypercomputational process
• Theoretical and practical hypercomputer
implementations
• Software Engineering of Granular Systems concurrent multi-resolution processing as a
harnessable hypercomputation
Granular Computing and
Collaborative Computation
Collaborative processing
Distributed sources of data (databases)
No direct interaction between databases
Goal: reveal structure common to all databases
through a process of collaboration
Approach: explore possibilities of collaboration
through granular communication
Horizontal collaboration
Databases about same N clients at various institutions
COLLABORATION
xxxx --- -----
1
.
.
N
---- yyy -----
1
.
.
N
---- --- zzzzz
Confidentiality : no direct sharing of data
Sharing of information granules
1
.
.
N
Vertical collaboration
Databases about various clients at several institutions
xxxx
1
.
.
N1
yyyy
1
.
.
N2
zzzz
1
.
.
Np
DB-1
DB-2
DB-P
Horizontal collaboration:
Fuzzy C-Means
- Identify patterns by optimising
local objective functions in each data set
ii
N

k 1
jj
c
2
2
u
[
ii
]
d
 ik ik [ii ]
i 1
N
c
 u
k 1
i 1
2
ik
[ jj]d ik2 [ jj]
Horizontal collaboration:
Fuzzy C-Means
- Coordinate partition
matrices in data sets
ii
jj
U[jj]
[ii,jj]
U[ii]
Communication and
collaboration
via partition matrices
U[ii], U[jj]
jj
ii
[ii,kk]
kk
U[kk]
Horizontal collaboration:
Computational details
Modified objective function
N

Q[ii ]  
k 1
c
P
N
jj1
jj ii
k 1
 u [ii ]d [ii ]   [ii , jj]
i 1
2
ik
2
ik
Optimization problem
c
2 2
{
u
[
ii
]

u
[
jj
]}
d ik [ii ]
 ik
ik
i 1
collaboration
Min Q[ii]
subject to
U[ii]  U
c
N
i 1
k 1
U= =. {u ik [ii ]  [0,1] |  u ik [ii ]  1 for all k and 0   u ik [ii ]  N for i}.
Horizontal collaboration:
Computational details
Unconstrained optimisation of V with respect
of partition matrix and Lagrange multipliers
V[ii] 
c
u
i 1
2
ik
[ii]d [ii] 
2
ik
P
c
 α[ii, jj] {u
jj1
jj ii
ik
i 1
The necessary conditions
V[ii]
 0,
u st [ii]
V[ii]
0
λ
2
ik
c
u
[ii]  u ik [jj]} d [ii]  λ(
2
i 1
ik
[ii]  1)
Horizontal collaboration:
Computational details
p
V[ii]
2
 2u st [ii]d st [ii]  2  α[ii, jj](u st [ii]  u st [jj])d st2 [ii]  λ  0
u st [ii]
jj1
jj ii
 p



2
λ  2d st [ii]   α[ii, jj]u st [jj] 
 jjjj1ii



u st [ii] 


p


2
2d st [ii] 1   α[ii, jj]u st [jj] 
 jjjj1ii



Horizontal collaboration:
Computational details
P
 st [ii]   α[ii, jj]u st [jj]
jj1
jj ii
ψ[ii] 
P
 α[ii, jj]
jj1
jj ii
c
u
s 1
st
[ii]  1
c
1 
 st [ii]
1  ψ[ii]
λ c
1
 2
s 1 2d st [ii](1  ψ[ii])
s 1
Horizontal collaboration:
Partition matrix
st [ii]
ust [ii] 

1  [ii]
c  [ii ]
1
jt
[1  
]
2
c
d st
j 1 1  [ii ]
 d2
j 1 jt
Horizontal collaboration:
Prototypes
N
2
A st [ii]   u sk
[ii]x kt [ii]
A [ii ]  C [ii ]
v [ii ] 
B [ii ]  D [ii ]
st
st
s
s
k 1
st
N
2
B s [ii]   u sk
[ii]
k 1
C st [ii] 
P
N
jj1
jj ii
k 1
2
α[ii,
jj]
(u
[ii]

u
[jj])
x kt [ii ]

 sk
sk
D s [ii] 
P
N
jj1
jj ii
k 1
2
α[ii,
jj]
(u
[ii]

u
[jj])

 sk
sk
Quantification of the effect
of collaboration
1 N c
δ
  u ik [1]  u ik [2] |
N * c k 1 i 1
Average distance between the partition matrices
Quantification of the effect
of collaboration
2 -ref
1 -ref
1
1

2

2
1 N c
Δ1 
  u ik [1]  u ik [1  ref] |
N * c k 1 i 1
1 N c
Δ2 
  u ik [2]  u ik [2  ref] |
N * c k 1 i 1
Average distance of the partition matrices obtained
with and without collaboration