Download NNsML chap2

Neural Networks
and Machine Learning Applications
CSC 563
Prof. Mohamed Batouche
Computer Science Department
CCIS – King Saud University
Riyadh, Saudi Arabia
[email protected]
Complex Systems
Emergent Behaviors and Patterns
from Local Interactions
What is a complex system?

A complex system displays some or all of the
following characteristics:

Agent-based


Heterogeneous


Changes are often the result of feedback from the environment
Organization


Characteristics change over time, usually in a nonlinear way; adaptation
Feedback


The agents differ in important characteristics
Dynamic


Basic building blocks are the characteristics and activities of individual agents
Agents are organized into groups or hierarchies
Emergence

Macro-level behaviors that emerge from agent actions and interactions
3
Complex systems
The fundamental characteristic of a complex system is that it
exhibits emergent properties: Local interaction rules between
simple agents give rise to complex pattern and global behavior
4
Complex vs. Simple Systems
Have many parts
• Parts are interdependent in behaviour
• Difficult to understand because:
– behaviour of whole understood from behaviour of
parts
– behaviour of parts depends on behaviour of whole
5
Examples of Complex Systems









Brain
Government
Family
Wold Ecosystem
Local Ecosystem
(desert, ocean,
rainforest)
Weather
University
Ant colony
…
6
Anything to be Learnt from
Ant Colonies?


Fairly simple units generate complicated
global behaviour.
An ant colony expresses a complex
collective behavior providing intelligent
solutions to problems such as:




carrying large items
forming bridges
finding the shortest routes from the
nest to a food source, prioritizing food
sources based on their distance and ease
of access.
“If we knew how an ant colony works,
we might understand more about how all
such systems work, from brains to
ecosystems.”
(Gordon, 1999)
7
Shortest path discovery
8
Adaptation to Environmental
Changes
9
Interactions among Social
Insects
• Direct Interactions
• Food or liquid exchange
• Visual or tactile contact
• Indirect Interactions:
Stigmergy
• Pheromones
• Individual behaviour modifies
the environment (e.g., by
putting up signs = stigma),
which in turn modifies the
behaviour of other
individuals.
10
Demo NetLogo

Massively Parallel MicroWolds
11
Universal properties shared by
complex systems


Emergence: The appearance of
macroscopic patterns, properties,
or behaviors that are not simply
the “sum” of the microscopic
properties or behaviors of the
components
Self-organization: In biological
systems, the emergent order
often has some adaptive purpose
– e.g., efficient operation of ant colony
12
Emergence: Attempt of a
Definition
• From the book:
Steven Johnson, Emergence—The Connected Lives of
Ants, Brains, Cities, and Software
– Emergence is what happens when an interconnected system of
relatively simple elements self-organizes to form more
intelligent, more adaptive higher-level behaviour.
– It’s a bottom-up model; rather than being engineered by a
general or a master planner, emergence begins at the ground
level.
– Systems that at first glance seem vastly different […] all turn out
to follow the rules of emergence.
13
Emergence in complex
systems
• How do neurons respond to each other in a way that
produces thoughts (minds)?
• How do cells respond to each other in a way that
produces the distinct tissues of a growing embryo?
• How do species interact to produce predictable
changes, over time, in ecological communities?
• ...
14
Emergence in complex
systems

Boids of Craig Reynolds
15
Emergence in complex
systems

Boids of Craig Reynolds
16
Emergence in complex
systems

Boids of Craig Reynolds
17
Emergence in complex
systems

Boids of Craig Reynolds
18
Why Are Complex Systems
Important for CS?
• Fundamental to theory &
implementation of massively
parallel, distributed computation
systems
• How can millions of independent
computational (or robotic) agents
cooperate to process information
& achieve goals, in a way that is:
–
–
–
–
efficient
self-optimizing
adaptive
robust in the face of damage or
attack
19
Some of Natural Systems








adaptive path minimization by ants
fish schooling and bird flocking
evolution by natural selection
information processing in the brain
wasp and termite nest building
pattern formation in animal coats
game theory and the evolution of cooperation
computation at the edge of chaos
20
Some of Artificial Systems








artificial neural networks
simulated annealing
cellular automata
ant colony optimization
artificial immune systems
particle swarm optimization
genetic algorithms
other evolutionary computation systems
21
Conclusions

We can learn from nature and take advantage of the problems
that she has already solved.

Many simple individuals interacting with each other can make a
global behavior emerge.

Techniques based on natural collective behavior (Swarm
Intelligence) are interesting as they are cheap, robust, and
simple.

They have lots of different applications.

Swarm intelligence is an active field in Artificial Intelligence,
many studies are going on.
22