Download Research Issues and Challenges on Brain Informatics

Research Issues and Challenges on Brain Informatics
Towards Computing & Intelligence in the Big Data Era
Ning Zhong
“I believe that at the end of the century
the use of words and general educated
opinion will have altered so much that
one will be able to speak of machines
thinking without expecting to be
contradicted.”
Alan M. Turing
ACM Turing Centenary Celebration
June 15 - 16, 2012
Palace Hotel, San Francisco
turing100.acm.org
2
What is the basic model of data representation in the big data era? 3
Web Intelligence (WI’12)
Intelligent Agent Technology (IAT’12)
Active Media Technology (AMT’12)
Brain Informatics (BI’12)
Methodologies for Intelligent Systems
(ISMIS’12)
Human Intelligence
-- An Important Scientific Issue in the Future
Three Important Scientific
Issues in the Future：
 Universe
 Molecular Biology
 Human Intelligence
Edward Feigenbaum
Stanford University, USA
1994 Turing Award Winner
WIC Advisor Turing Keynote Talk
at the 2012 World
Intelligence Congress
(WIC 2012)
December 4-7, 2012
Macau, China
Memoriam of John McCarthy
1927-09-4 – 2011-10-24
(1971 Turing Award Winner)
Roads to Human Level AI
September 2004 at BJUT
WIC Advisor, WI‐IAT’04 Keynote Speaker
6
Human‐Level Capabilities
We need to better understand
- How human being does complex adaptive,
distributed problem solving and reasoning.
- How intelligence evolves for individuals
and societies, over time and place.
Ignoring what goes on in human brain and focusing instead on behavior has been a large impediment to understand how human being does complex adaptive, distributed problem solving and reasoning. Brain: Big Science, Big Data, Big Innovation, Big …
Brain Big Data Computing
Gene
Neuron
Brain Structure
Educational
Psychology
Artificial Intelligence
Cognitive psychology
Neuroinformatics
Forensics
Psychiatry
Brain
Clinical Medicine
Cognitive
Informatics
Neuroscience
NeuroImage
Neuroeconomics
Geriatrics
Web
Intelligence
Intelligent
Information
Technology
Brain Imaging
 Human Brain Project (EU) (2013 – 2023)
 Obama's Brain Project (USA) (2013 -)
 ……
Behavior
EEG
Developing brain inspired intelligent technologies
to provide (human-level) wisdom services
Intelligent Diagnosis and
Treatment Technologies of
Brain and Mental Disorders
New Brain-Computer
Interfaces
Wisdom Marketing
Smart City/
Smart Health
……
Gene
Neuron
Brain Informatics
Brain Structure
Cognitive Psychology
Intelligent Information
Technologies and Services
EEG/ERP
Neuroimaging
New Industry Chain
Computer Science
(AI + ICT)
Brain Informatics
(BI)
Cognitive Science
Neuroscience
AI + ICT
Brain Informatics (BI)
Cognitive Science
Neuroscience
Brain (Big) Data Computing
AI + ICT
Brain Informatics (BI)
Cognitive Science
Neuroscience
fMRI, EEG, MEG, PET, Eye-Tracking, …
Brain Data Computing
AI + ICT
brain research from informatics perspective
brain research supported by information technology
BI = Brain Big Data Science
Cognitive Science
Neuroscience
Brain Data Computing - 1
AI + ICT
Human brain = Information Processing System (HIPS) with big data brain research from informatics perspective
brain research supported by information technology
BI = Brain Big Data Science
Cognitive Science
Neuroscience
Brain Data Computing - 2
AI + ICT
brain research from informatics perspective
brain research supported by information technology
Curation, mining, use of brain big data
BI = Brain Big Data Science
Cognitive Science
Neuroscience
Brain Informatics Initiatives
IEEE-CIS Task Force on Brain Informatics
(http://braininformatics.org)
N. Zhong et al. Brain Informatics, Springer, 2014
N. Zhong, J.M. Bradshaw, J.M. Liu, J.G. Taylor (eds.)
Special Issue on Brain Informatics, IEEE Intelligent Systems,
26(5) (2011)
N. Zhong, J. Liu, Y.Y. Yao (eds.) Special Issue on Brain
Informatics, Cognitive Systems Research, Elsevier, 11(1) (2010)
B. Hu, J. Liu, L. Chen, N. Zhong (eds.) Brain Informatics,
LNAI 6889, Springer (2011)
Y.Y. Yao, R. Sun, T. Poggio, J. Liu, N. Zhong, J. Huang (eds.)
Brain Informatics, LNAI 6334, Springer (2010)
N. Zhong, K. Li, S. Lu, L. Chen (eds.) Brain Informatics,
LNAI 5819, Springer (2009)
N. Zhong et al (eds.) Web Intelligence Meets Brain Informatics,
LNCS 4845, Sate-of-the-Art Survey, Springer (2007)
2006 International Workshop on
Web Intelligence meets Brain Informatics
“WI meets BI”
December 2006, Beijing, China
Beijing Key Lab of MRI & Brain Informatics
International Journal, Conference and Collaboration
Beijing International Collaboration
Base on Brain Informatics and
Wisdom Services
WIC 2014: Web Intelligence Congress
11‐14 August 2014, Warsaw Poland Web Intelligence (WI’14)
Intelligent Agent Technology (IAT’14)
Brain Informatics and Health (BIH’14)
Active Media Technology (AMT’14)
University of Warsaw
Warsaw, Poland
wic2014.mimuw.edu.pl
IEEE Intelligent Systems
Special Issue on
Brain Informatics
Vol 26, No 5, 2011
20
Three Aspects of BI Research 


Systematic investigations for complex brain science problems
New information technologies for systematic brain science studies BI studies based on WI (W2T) research needs
Systematic investigations for
complex brain science problems
Human thinking centric cognitive functions (e.g. reasoning, problem‐
solving, decision‐
making, learning, attention, and emotion)
Clinical diagnosis and pathology of human brain, mind and mental related diseases (e.g. MCI: mild cognitive impairment, epilepsy,
AD: alzheimer disease, depression)
Core Issues in BI Methodology





Human brain is regarded as an information
processing system (HIPS) with big data –
systematic studies
Systematic investigation of complex brain
science problems
Systematic design of cognitive experiments
Systematic data management – the Data Brain
Systematic data analysis/simulation
– multi-aspect approach
Systematic investigations for
complex brain science problems
Human thinking centric cognitive functions (e.g. reasoning, problem‐
solving, decision‐
making, learning, attention, and emotion)
Clinical diagnosis and pathology of human brain, mind and mental related diseases (e.g. MCI: mild cognitive impairment, epilepsy, AD: alzheimer disease, depression)
Core Issues in BI Methodology





Human brain is regarded as an information
processing system (HIPS) with big data –
systematic studies
Systematic investigation of human thinking
centric high cognitive functions
Systematic design of cognitive experiments
Systematic data management – the Data Brain
Systematic data analysis/simulation
– multi-aspect approach
Human Intelligence = Thinking + Perception
Thinking ----------------- Perception
computation
learning
discovery
creativity
Reasoning – Language – Memory – Attention – Vision
decision-making
planning
problem-solving
……
audition
tactile
……
Reasoning Centric, Thinking Oriented Functions
and Their Inter-relationships – A Conceptual View
Problem-Solving
Decision-Making
emotion
memory
deduction
Planning
granularity
Reasoning
uncertainty
search
(commonsense)
Learning
Computation
induction abduction
autonomy
stability
Discovery
attention
Creativity
Language
Reasoning Centric, Thinking Oriented Functions
and Their Inter-relationships – A Conceptual View
Problem-Solving
Decision-Making memory
emotion
deduction
Reasoning
Planning
granularity
search
uncertainty
(commonsense)
Learning
induction abduction Computation
autonomy
stability
Discovery
attention
Creativity
Language
Reasoning Centric, Thinking Oriented Functions
and Their Inter-relationships – A Conceptual View
Problem-Solving
Decision-Making memory
emotion
deduction
Reasoning
Planning
granularity
search
uncertainty
(commonsense)
Learning
induction abduction Computation
autonomy
stability
Discovery
attention
Creativity
Language
Investigation Paradigm in MRI
Functional MRI: Subsequent States (time)
Pre-taskresting
state
Task-on state
(active, passive)
Post-taskresting
state
Representation
Represent
Represent
Represent
Disturbance
Graph model
Graph model
Graph model
Systematic investigations for complex brain science problems
X.Q. Jia, P.P. Liang, J. Lu, Y.H. Yang, N. Zhong, K.C. Li. Common and
Functional MRI:
Subsequent
States (time)
Dissociable Neural Correlates Associated
with
Component
Processes of
Inductive Reasoning. NeuroImage, Elsevier, 56(4):2292-2299, 2011.
Z. Wang, J. Liu, N. Zhong, Y. Qin, H. Zhou, K.C. Li. Changes in the Brain
Intrinsic Organization in Both On-Task State and Post-Task Resting
State. NeuroImage, Elsevier, 62:394-407, 2012.
G. Jin, K.C. Li, Y. Hu, Y. Qin, X. Wang, J. Xiang, Y. Yang, J. Lu, N. Zhong.
Amnestic Mild Cognitive Impairment: Functional MR Imaging Study of
Response in Posterior Cingulate Cortex and Adjacent Precuneus during
Problem-solving Tasks. Radiology, RSNA, 261(2): 525-533, 2011.
M. Li, N. Zhong, K. Li, S. Lu. Functional Activation of the
Parahippocampal Cortex and Amygdala During Social Statistical
Information Processing. Cognitive Systems Research, Elsevier,
17-18:25-33, 2012.
Systematic investigations for
complex brain science problems
Human thinking centric cognitive functions (e.g. reasoning, problem‐
solving, decision‐
making, learning, attention, and emotion)
Clinical diagnosis and pathology of human brain, mind and mental
related diseases (e.g. MCI: mild cognitive impairment, epilepsy, AD: alzheimer disease, depression)
Systematic Study on Depression
Macro
Symptoms
&
Behaviors
BI
Methodology
Multi-modal
Meso
Systematic
Macro
To understand
depression
Micro
Structure &
Function
of Brain
Neuron
Synapse &
gene
Micro
Systematic Study on Depression
Diagnostic
Criteria
Depressive Patients
High Risk People
Target Groups
Healthy People
Unifying Studies of Cognition, Emotion and Depression
Study
Human Information Processing System (HIPS)
Study
Influence
Depression
(MDD)
Emotion
Impaired
Multi-modal Experiments
Three Aspects of BI Research 


Systematic investigations for complex brain science problems
New information technologies for systematic brain science studies BI studies based on WI (W2T) research needs
New information technologies for systematic brain science studies
N. Zhong, J. Chen. Constructing a New-style Conceptual Model of Brain
Data for Systematic Brain Informatics. IEEE Transactions on Knowledge
and Data Engineering, 24(12): 2127-2142, 2012.
J. Chen, N. Zhong. Towards the Data-Brain Driven Systematic Brain Data
Analysis. IEEE Transactions on Systems, Man and Cybernetics (Part A).
43(1): 222-228, 2013.
J. Chen, N. Zhong. P. Liang. Data-Brain Driven Systematic Human Brain
Data Analysis: A Case Study in Numerical Inductive Reasoning Centric
Investigation. Cognitive Systems Research, Elsevier, 15-16:17-32, 2012.
N. Zhong, S. Motomura. Agent-Enriched Data Mining: A Case Study
in Brain Informatics. IEEE Intelligent Systems, 24(3):38-45, 2009.
N. Zhong, J.M. Bradshaw, J.M. Liu, J.G. Taylor. Brain Informatics.
IEEE Intelligent Systems, 26 (5):16-21, 2011.
The shortcomings of existing fMRI and EEG databases
FSPEEG




fMRIDC
They cannot describe cognitive functions from multiple aspects because of only
storing a single type of data, such as EEG or fMRI data.
They cannot effectively identify the data relationships among different
experiments because of only providing descriptions for each dataset.
They cannot help users, who aren’t familiar with recent database structures and
terms of special domains because of only providing the keyword-based query or
term dictionary-based query expansion
They cannot effectively support cloud-based big data mining because of not
describing data analysis and not storing analytical results.
It is an urgent work to develop a multi-functional brain data center for
various requirements from education, research, clinical diagnosis, health care, etc.
Curation, Mining & Use of Brain Big Data
on the Wisdom Web of Things (W2T)


A brain data centre needs to be constructed on
the W2T for effectively utilizing the “data
wealth” as services (i.e. WaaS – Wisdom as
a Service)
Curate BI big data, which can be characterized
by five parameters: volume, velocity, variety,
veracity and value, in order to support data
sharing and reuse among different BI experimental
and computational studies for generating and
testing hypotheses about human and
computational intelligence.
Brain Big Data 5V
Big Data
Value
Big Amount
of Data
Value
Volume
Velocity
Big Speed of
Data In and Out
Veracity
Variety
Big Range of Data
Types and Sources
Big Data Accuracy
and Truthfulness
How to support the whole process of BI methodology
and integrate brain big data for systematic studies?
41
WaaS Standard and Service Platform
WaaS
Content Schedule Standards
WaaS Application Portal
Intelligent Application Standards
Data/Information/Knowledge Buses
WaaS Platform
WaaS Standard System
Data Accessing
Standards
Information
Accessing Standards
Knowledge Accessing
Standards
Data Content and
Format Standards
Metadata Standards
Knowledge
Representation and
Organization Standards
Data Transmission
Protocols
Data-Mining-Related
Standards
Data Collection
Interface Standards
InformationRetrieval-Related
Standards
DaaS Standard System
InaaS Standard System
Knowledge Retrieval
Standards
KaaS Standard System
Knowledge Query
Data Query
Data Management
Data Cleaning
Data Collection
Data Curation
Data Mining
Information Retrieval
Knowledge Development
DaaS Platform
InaaS Platform
KaaS Platform
InaaS
KaaS
DaaS
Private
Cloud
Brain and Intelligence
Big Data Center
Knowledge Management
Knowledge Retrieval
Large Knowledge
Collider
Information Assurance Framework
LarKC - Large Knowledge Collider
• The aim of the EU FP 7 Large-Scale
Integrating Project LarKC is to develop
a platform for massive distributed
incomplete reasoning that will
remove the scalability barriers of
currently existing reasoning systems
for the Semantic Web.
• Fourteen member units, a total
investment of 10 millions Euros.
10M€
• Three use cases: urban computing
and intelligent traffic, semanticsbased medical literature retrieval, and Large
the relationships between cancer and Knowledge
gene.
Collider
43
Data-Brain, BI Provenances and the LarKC
How to model the whole
process of BI methodology?
Conceptual View  Structural View
Experiment Dimension
Function Dimension
Analysis Dimension
Data Dimension
Data-Brain
How to extract and represent
brain big data related
information and knowledge?
Data Provenances
How to integrate brain big
data related information and
knowledge into a global
framework?
Analysis Provenances
Large Knowledge
Collider
Data-Brain
The Data-Brain is a
conceptual model of brain
data which models brain
data from four aspects,
i.e., cognitive functions,
experiments, data
themselves and
analytical processes.
N. Zhong and J. H. Chen. Constructing a new-style conceptual model of brain data for
systematic Brain Informatics. IEEE Transactions on Knowledge and Data Engineering,
24(12), pp. 2127-2142, 2012.
J. H. Chen and N. Zhong. Toward the Data-Brain driven systematic brain data analysis.
IEEE Transactions on Systems, Man, and Cybernetics: Systems, 43(1), pp. 222-228, 2013.
BI Provenances
BI Provenances are the
metadata describing the origin
and subsequent processing of
various human brain data in
systematic BI studies, including
data provenances and
analysis provenances.
Data Provenances
J. H. Chen, N. Zhong, and P. P. Liang.
Data-Brain driven systematic human brain
data analysis: A case study in numerical
inductive reasoning centric investigation.
Cognitive Systems Research, Elsevier,
vol. 15-16, pp. 17-32, 2012.
Analysis Provenances
A Data-Brain Based Brain Data Center
Concept
owl:Thing
Instance
Brain Data
ed:Experimental
-Groupta
fd:CognitiveFunction
dd:BI-Da
dd:…
ad:AnalyticProcess
Function Dimension
ad:…
Experiment Dimension
Relations between dimensions
Data Dimension
Analysis Dimension
Data-Brain
Relations in dimension
BI Provenances
Brain Data
A Data-Brain Based Brain Data Center
Concept
owl:Thing
Instance
Brain Data
ed:Experimental
-Groupta
fd:CognitiveFunction
dd:BI-Da
dd:…
ad:AnalyticProcess
Function Dimension
ad:…
Experiment Dimension
Relations between dimensions
Data Dimension
Analysis Dimension
Data-Brain
Relations in dimension
BI Provenances
Open Data
Brain Data
A Data‐Brain Based Brain Data Center Concept
owl:Thing
Instance
Brain Data
fd:CognitiveFunction
ed:Experimental
-Group
dd:BI-Data
ed:…
ad:AnalyticProcess
ad:…
A brain data and knowledge base
Function Dimension Experiment Dimension
Relations between dimensions
Data Dimension
Analysis Dimension
Data-Brain Relations in dimension
BI Provenances
Brain Data
Wisdom Web of Things (W2T)
Social World
Cyber World
Hyper World
Companies/Societies
ServiceService
Service
ServiceServiceService
Individuals
Developing Transparent Services
Internet
/WWW
SEA-net
Hypw-TSBus
Hypw-DKServer
Wisdom
Intelligent KNOWLEDGE/
INFORMATION/DATA Utilization
Knowledge
Information
(from Web, SEA-net)
Intelligent INFORMATION Analysis
Web-Information
Sensor-Information
Data
Intelligent DATA Pre-processing
Physical World
TSBus – Transparent Service Bus
Data
Service
50
Wisdom Web of Things (W2T)
Social World
Hyper World
 The Wisdom Web of Things (W2T) is
an extension of the Wisdom Web in
the hyper-world with big data.
 The “Wisdom” means that each of things
in the IoT/WoT can be aware of both
itself and others to provide the right
service for the right object at a right time
and context.
Physical World
Cyber World
Research challenges and perspectives on
wisdom web of things (W2T).
Journal of Supercomputing, Springer,
64(3):862-882, 2013.
From the Wisdom Web to W2T

N. Zhong, J. Ma, R. Huang, J. Liu, Y.Y. Yao, Y. Zhang, J. Chen.
Research Challenges and Perspectives on Wisdom Web of
Things (W2T), Journal of Supercomputing, Springer. 2013,
64(3):862-882 (2010, DOI 10.1007/s11227-010-0518-8)






N. Zhong, J. Liu, Y.Y. Yao. Web Intelligence (WI), Encyclopedia of
Computer Science and Engineering, Vol. 5, Wiley (2009) 3062-3072.
N. Zhong, J. Liu, Y.Y. Yao. Envisioning Intelligent Information
Technologies Through the Prism of Web Intelligence.
Communications of the ACM, 50(3), 89-94, 2007.
N. Zhong. Impending Brain Informatics Research from Web Intelligence
Perspective, International Journal of Information Technology and Decision
Making, Vol. 5, No. 4, World Scientific (2006) 713-727.
N. Zhong, J. Liu, Y.Y. Yao. Web intelligence, Springer, 2003.
N. Zhong, J. Liu, Y.Y. Yao. In Search
IEEE Computer, 35 (11), 27-31, 2002.
of the Wisdom Web.
N. Zhong, J. Liu, Y.Y. Yao, S. Ohsuga. Web Intelligence (WI),
Proc. 24th IEEE COMPSAC, IEEE-CS Press (2000) 469-470.
52
W2T Initiatives






N. Zhong et al. Wisdom Web of Things, Springer, 2014
J.H. Chen, J.H. Ma, N. Zhong, Y.Y. Yao, J.M. Liu, R.H. Huang,
W.B. Li, Z.S. Huang, Y. Gao, and J.P. Cao. WaaS – Wisdom as
a Service. IEEE Intelligent Systems (in press)
WIC 2012 Turing-Centenary Panel on Top 10 Questions in
Intelligent Informatics/Computing. wi‐consortium.org/blog/top10qi/index.html
Special Issue on Wisdom Web of Things (W2T)
World Wide Web Journal, Springer, 16(4) 2013
WI-IAT 2011 Panel on Wisdom Web of Things (W2T) :
Fundamental Issues, Challenges and Potential Applications,
wi-iat-2011.org
N. Zhong, J. Ma, R. Huang, J. Liu, Y.Y. Yao, Y. Zhang, J. Chen.
Research Challenges and Perspectives on Wisdom Web of
Things (W2T), Journal of Supercomputing, Springer. 2013,
64(3):862-882 (2010, DOI 10.1007/s11227-010-0518-8)
53
W2T Big Data Cycle
54
BI Methodology vs BI Data Cycle


Guided by such a BI methodology, the whole
process of BI research can be regarded as a
BI data cycle
Implemented by measuring, collecting, modeling,
transforming, managing, mining, interpreting,
and explaining multiple forms of brain big data
obtained from various cognitive experiments by
using powerful equipments, such as fMRI and
EEG.
WaaS - Wisdom as a Service
 A content architecture of the W2T cycle
 A perspective of W2T in services
IEEE Intelligent Systems
(in press)
Providing services based on both
already-created and will-created raw
data, information, knowledge and
wisdom.
DIKW)Hierarchy
Wisdom
WaaS Architecture
Wisdom-as-a-Service
(WaaS)
Knowledge
Information
Data
Knowledge-as-a-Service
(KaaS)
Information-as-a-Service
(InaaS)
Data-as-a-Service
(DaaS)
 Knowledge Query Services
 Knowledge Retrieval Services
 Development and Management Services of
Knowledge Bases
…
 Information Retrieval Services
 Data Mining Services
 Data Curation Services
…
 Data Collection Service
 Data Production Service
 Data Sharing Services
…
WaaS – Wisdom as a Service
Providing the right service, including infrastructure, platform,
software, as well as data, information, and knowledge,
for the right object at a right time and context.
Wisdom-as-a-Service
(WaaS)
Software-as-a-Service (SaaS)
(Applications/Workflows)
Platform-as-a-Service (PaaS)
(Middleware-AS/WS)
Infrastructure-as-a-Service (IaaS)
(Servers/Storage Devices/Internet/IoT/MI)
Interfaces
Knowledge-as-a-Service (KaaS)
(Ontology/Models/Cases)
Information-as-a-Service (InaaS)
(Meta Data/Data Features)
(Big
Data-as-a-Service (DaaS)
Data-Web Pages/Video/Audio/Text/Images)
5V: Volume-Velocity-Variety-Veracity-Value
Cyber World
WaaS – Wisdom as a Service
Agents
Companies/Societies
individuals
Socio-culture &
organizational components
Wisdom-as-a-Service
(WaaS)
interaction, personalization,
Knowledge-as-a-Service (KaaS)
(Ontology/Models/Cases)
context-aware, active-service,
affective/emotion, auto-perception
Software-as-a-Service (SaaS)
(Applications/Workflows)
Platform-as-a-Service (PaaS)
(Middleware-AS/WS)
Infrastructure-as-a-Service (IaaS)
(Servers/Storage Devices/Internet/IoT/MI)
Information-as-a-Service (InaaS)
(Meta Data/Data Features)
Data-as-a-Service (DaaS)
(Big Data-Web Pages/Video/Audio/Text/Images)
5V: Volume-Velocity-Variety-Veracity-Value
Cyber World
Brain Data Center and BI Research Platform
BI Research Community
BI Portal
Brain Data Center
GLS-BI
W2T
Internet
/WWW
Domain
Ontologies
Hypw-TSBus
Hypw-DKServer
Data-Brain
BI
Provenances
Analysis Agents
Data Agents
SEA-net
Data-Brain Based Data/Analysis/Knowledge
Description, Integration and Publishing
BI Experimental
Studies
GLS-BI – Global Learning Scheme for BI
Data
59
Service
Depression Data Center and Service Platform Health-care
Pervasive
Service
Community
Physician
Depression victims
Ambulance
Hospital
SEA-net
Nurse
Internet
/WWW
Mental Health
Research
Community
Hypw-TSBus
Hypw-DKServer
Depression Transparent
Service Platform
W2T
Depression
Unified
Data Center
60
Portable Brain/Mental Health Monitoring System
Data Brain Based Multi-Aspect
Human Brain Data Analysis


Various analysis agents (association, classification,
clustering, manifold, peculiarity-oriented, SPM etc)
are deployed on the data brain for multi-aspect
analysis in multiple data sources.
Studying functional relationships and neural structures
of the activated areas, and trying to understand
- how a peculiar part of the brain operates
- how they are linked functionally and intrinsically
- how they work cooperatively to implement
a whole information processing.
Changing the perspective of cognitive scientists
from a single type of experimental data towards
a long-term, holistic vision.
An Investigation Flow Based on BI Methodology
Experimental
Design Part
Experiments
(Multiple Difficulty Levels)
Data Mining Part
Data Collection
Data Extraction
Model
Transformation
Determination of tasks
Perception
: Vision
: Audition
: Tactile
etc.
Memory,
Attention, etc.
Thinking
: Computation
: Reasoning
: Learning
etc.
Visualization &
Integration of
Results
(Multi-aspect
Data Analysis)
Data
Mining
EEG/
ERP data
Data
Mining
Spectrum
data
Data
Mining
fMRI
Image
Explanation/Synthesis
Knowledge Discovery
ACT-R
Simulation
Three Aspects of BI Research 


Systematic investigations for complex brain science problems
New information technologies for systematic brain science studies
BI studies based on WI (W2T) research needs
Ways for Studying Intelligence



AI ‐‐ Computer Science: 4 types of definition (cognitive, Turing‐test, logic, agents), computational models of intelligence, computer based technologies;
Cognitive Science ‐‐ Psychology: Mind & Behavior based cognitive models of intelligence; Neuroscience ‐‐ Medicine: Brain & Biological models of intelligence.
Memoriam of Herbert A. Simon
1916-06-15 – 2001-02-09
Notable awards
Known for
• Logic Theory
Machine
• General
Problem
Solver
• Bounded
Rationality
Fields:
• Turing Award 1975
• Nobel Prize in
Economics 1978
• National Medal of
Science 1986
• von Neumann
Theory Prize 1988
• Artificial
Intelligence
• Cognitive
psychology
• Computer
science
• Economics
• Political
science
66
Related to Simon’s Contributions These fundamental issues of the W2T are related Simon's contributions, including bounded rationality & decision making etc.
 Human problem solving and GPS as a cognition inspired computer program;
 Human scientific discovery process and Bacon series as a cognition‐inspired computer program. 67
Extension of Simon’s Works


Exploring the neural basis of human problem solving and scientific discovery by using Brain Informatics means;
Developing Web based problem‐solving and knowledge discovery systems with human level capabilities for meeting WI (W2T) real‐world needs.
68
What Are Problems?


The traditional AI research has not produced major breakthrough recently due to a lack of understanding of human brains and natural intelligence.
Most of the AI models and systems will not work well when dealing with dynamically changing, open and distributed big data at Web scale and in a ubiquitous environment.
A New Perspective of WI:
WI Meets Brain Informatics (BI)
New instrumentation (fMRI etc)
and advanced IT are causing an
impending revolution in WI and
brain sciences (BS).


BI for WI: New understanding and discovery
of human intelligence models in BS will yield
a new generation of WI research/development.
WI for BI: WI based technologies will provide
a new powerful platform for BS.
WI = AI + IT
brain = information processing system
Web = information processing system
Unifying Studies
Cognitive Science
Neuroscience
Unifying Study of Human & the Web



Brain ＝ information
processing system (IPS)
Human-Human
question-answering
Human reasoning/
problem solving/
decision making/
learning



The Web ＝ information
processing system (IPS)
Human-Web
question-answering
Web reasoning/
problem solving/
decision making/
e-learning
Human＋the Web ＝ two aspects of IPS
Unifying Study on Human and the Web
Multi‐granule/Multi‐source/?? networks
Rule‐based/Case‐based/?? reasoning
Variable Precision
Common-sense
Common-sense
reasoning
Granule
reasoning
Common-sense
reasoning
……
Granule-based
cognitive model
……
Granule
reasoning
……
Human
commonsense model
Common-sense
reasoning
Distributed & personalized
problem-solving
Distributed network
based reasoning
Personalization
Brain distributed
cooperation
mechanism
PSML: Problem
Solver Markup
Language
Human Level Web Intelligence (WI)
Combining the three
intelligence related areas
Understanding
intelligence in depth
Brain
Sciences
AI
Human Intelligence
Machine Intelligence
Social Intelligence
Habitu
ation
WI
AI
Brain Sciences
Hebbian
Spatial
Constraint representations learning
Motivation
satisfaction
Learning Population
Probabilistic
codes
Reasoning
Habituation
reasoning
Vision
Multi-perception
Multi-agent
Language
Emotion
Planning
Problem-solving
Attention
Knowledge Decision-making Memory and
based methods
forgetting
WI
AI
Brain Sciences
Hebbian
learning
Spatial
Constraint
Motivation
satisfaction representations
Social networks
Population
Probabilistic
codes
Learning
Community
Small
world
Habituation
reasoning
discovery
theory
Multi-agent
Reasoning
Multi-perception
Social brain
Planning
Vision Social Emotion
Social
Knowledge
agents
dynamics
Language Memory
and Attention
based methods
Groupwareforgetting
Social media
Social Intelligence
Emotion Machine vs Logic Machine




Human (Brain): Emotion Machine
Computer (Web): Logic Machine
How to develop Intelligent Web
systems?
How to develop Web based
Emotion Machine?
WI meets BI
Basic Philosophy



Considering both scalability and personalization
in the same importance.
Organizing and using granular networks of
data-information-knowledge as a way
to achieve such a goal – both scalability and
personalization.
Developing a human-like natural reasoning
and problem-solving system that can be
carried out on the granular networks.
Natural Reasoning/Problem‐Solving （Human vs the Web）
Retrieval and Reasoning for Problem-Solving/Decision-Making
rationality
variable precision
context-aware
common-sense
affective/emotion
personalization
Organization and Use of Multiple Information Granule Networks
Both the Web and Brain are huge complex networks with big data [From Google Image]
80
Linked Data Cloud
31 billion RDF triples
81
Complex system, brain and complex network
?
?
WImBI
Research
Object
?
Dynamic information processing system with fast
emergence computing
Dynamic information processing system with
fast emergent computing
Complex networks with the organization of small-world and
scale-free or truncated power-law degree distribution
Case
New
Mechanisms
82
Tool/Model
Sequential States (Time)
Representation
Pre-taskresting state
Task-on state
Semantic-matching task stimuli
Post-taskresting state
Whole brain (large-scale), over states
Represent
Represent
Represent
Disturbance
Network model
Network model
Network model
General Processing of fMRI Data Analysis
- Filtering
- Preprocess
(0.01~0.08Hz)
- Parcellation
- Regression
- Correlation
- The AAL-based DMN
15 Subjects across the 3 states
- Sparsity threshold
3-Layer Perspectives
L1: Globally small-world topology
L2: The global/full DMN topography
L3: The DMN nodal properties
L1:
Globally small-world
topology
Post-task resting state vs.
Pre-task resting state
On-task state vs.
Pre-task resting state
On-task state vs.
Post-task resting state
L2:
The global/full
DMN topography
L3:
The DMN nodal
properties
DMN
DMN
DMN
DMN
Eight Examples of WIC Brain Informatics Research
1. Common and Dissociable Neural Correlates Associated with Component Processes of Inductive Reasoning
2. Different Brain Networks Revealed by Solving Sudoku Puzzles
3. BI Based Study on Depression Mechanisms and Diagnosis 4. Curation, Mining and Use of BI Big Data on the Wisdom Web of Things
5. Agent‐Enriched Data Mining: A Case Study in Brain Informatics
6. Intrinsic Neural Connectivity of ACT‐R ROIs
7. Changes in the Brain Intrinsic Organization in Both On‐Task State and Post‐Task Resting State
8. Effects of Visual Information Forms on Human Information Processing
88
The Brain Informatics studies will yield profound
advances in our analyzing and understanding of the
mechanism of data, information, knowledge and
wisdom,
as
well
as
their
inter-relationships,
organization and creation process.
Informatics-enabled brain studies are transforming
various brain sciences, as new methodologies enhance
human interpretive powers when dealing with big data
increasingly derived from advanced neuro-imaging
technologies, as well as from other sources like eyetracking and from wearable, portable devices.
It will fundamentally change the nature of
IT in general and AI in particular
Towards Computing & Intelligence in the big data era
Thank You !