Download 2007-01-04 DSTA-ESD

DSTA-MIT/ESD: Armament Safety Project
22 January 2007
• MIT Team
– Prof. Stuart Madnick ([email protected])
– Dr. Michael Siegel
– Lynn Wu
– Nathan Minami (MIT & US Army)
– Allen Moulton
– Mihai Lupu (SMA)
– Assistance from:
• Prof. Aykut Firat (MIT & Northeastern University)
• Prof. Hongwei Zhu (MIT & Old Dominion University)
© MIT, 2007
1
DSTA-MIT/ESD: Armament Safety
Raw Knowledge High-Level Project Overview
Sources
Knowledge
Repository
SEM 2007-01-13
High Level
Policy choices
and Interventions
Safety Indicators
External sources
Safety
Forecasting
Model (2)
Knowledge
Re-Use
Internal sources:
Assessment Reports,
live firing trials data,
incident investigation
reports, acceptance
test data, qualification
trials reports, hazard
database for various
items, WDA, defect
rates, range & training
schedule, weapon &
ammo procurement
and ages, etc.
Data Extraction &
Semantic Reconciliation
Technologies (1)
Detailed Indicators
Funding
=
Training
Training Schedule
Pressure
Ammunition Age
2
Knowledge Management Activities
[Phase 1 and 2]
Diverse Raw
Knowledge Sources
Knowledge
Repository
(a) Raw Knowledge Retrieval
(e.g., specific piece of
knowledge) [Phase 1]
External sources
(b) Processed Knowledge
Retrieval (e.g., comparisons
and/or aggregations of
knowledge) [Phase 1]
Knowledge Storage
Internal sources
Knowledge Capture
[Phase 1]
(c) Projections based on
Knowledge (e.g., Forecast
and/or Consequences)
[Phase 2]
Knowledge Re-use
3
Example of Basic Knowledge Capture and Re-Use
• Goal: Demonstrate use of MIT extraction and
analysis technologies
• Constraint: Difficult to get access to diverse data
related to Singapore Armament Safety (so far)
• Approach: Demonstrate using public sources
and interesting strategy questions
• Example issue: What is Singapore military
expenditure per capita?
• How do others compare (+ or - %) with Singapore:
–
–
–
–
Israel =
USA
=
UK
=
Malaysia =
4
Good News – Data Publicly Available
(https://www.cia.gov/cia/publications/factbook/geos/sn.html)
Population
Economy
Military
5
But it is scattered … must be located …
• … and not in exactly easy to use form …
•
Must do calculations:
– Military expenditures = 4.9% (of GDP) x $126.5 billion = $6.2
billion
– per capita = $6.2 billion / 4,492,150 people = $1379.85 per capita
• Need to be careful about scale factors
– (one of our team members was off by a thousand)
• Amount is in US$ … what if you want in SG$?
6
Easier way: Cameleon Web Extraction Tool
( http://coin.mit.edu/DSTA/Mihai/cameleon/cameleon.html
Results as tsv
(data record)
coin\smadnick 6671coin
)
Results as XML file
<?xml version="1.0" encoding="ISO-8859-1" ?>
- <DOCUMENT>
- <ELEMENT>
<country>Singapore</country>
<population>4,492,150</population>
<gdp>126.5</gdp>
<milexpendpercent>4.9%</milexpendpercent>
</ELEMENT>
7
</DOCUMENT>
But also want: (1) multiple countries and (2)
military per capita calculation
• First: Multiple countries.
– Combine Cameleon web extraction capability with IBM
DB2 Federated Database Server
• Technical details explained later
SQL Query:
Select country, population, GDP, MilExpendPercent
from cia where country in ("Singapore" , “Israel”,
“United States” ,“United Kingdom” ,“Malaysia”)
COUNTRY
Singapore
Israel
United States
United Kingdom
Malaysia
POPULATION
4492150
6352117
298444215
60609153
24385858
GDP
126.5
156.9
12.31
1.81
287
MILEXPENDPERCENT
4.90%
7.70%
4.06%
2.40%
2.03%
8
Now need military per capita calculation
• DB2 can provide the computation based on the
data being extracted by Cameleon
Select country, MilExpendPercent * GDP * 1000000000 /
population from cia
where country in (Singapore" , “Israel” ,“United States”,
“United Kingdom” ,“Malaysia”)
COUNTRY
Singapore
Israel
United States
United Kingdom
Malaysia
MilExpendPercent * GDP * 1000000000 / population
1379.85
1901.93
1.67
0.72
238.91
Note: Is USA really only $1.67 per capita and Singapore $1379 per capita?
9
Need to adjust for GDP “billion” and “trillion”
• Use an auxiliary table
– scalefactor
text
scale
Billion
1000000000
Trillion 1000000000000
• Now can use database “Join”
Select country, MilExpendPercent * GDP * scalefactor.scale / population
from cia, scalefactor where country in ('Singapore', 'Israel','
United States','United Kingdom', 'Malaysia') and
scalefactor.text=cia.gdp_unit
COUNTRY
Singapore
Israel
United States
United Kingdom
Malaysia
GDP_UNIT
billion
billion
trillion
trillion
billion
MilExpendPercent * GDP * unit/ population
1379.85
1901.93
1674.64
716.72
238.91
10
That is great: but want to graph comparisons
• Combine Cameleon web extraction capability
with Excel spreadsheet graphing capabilities
• Can put SQL queries to Cameleon in Excel
Query automatically assembled .. And sent to Cameleon
11
Excel provides nice graphics
• Excel can display results in both table form and
graphics
12
Need to adjust scale factors
• Some GDP’s are stated in billion and some on
trillion
– by the way: using USA definition of “billion”
• Can use conditional definition for
military_expenditure per capita that handles
scale factor adjustment.
13
Final results of “military expenditure per capita”
– with scale factor adjusted
• Using Excel formula with conditional definition
(Conditional definition)
14
Much easier to “visualize” graphically
• All kinds of additional calculations and graphics possible
country
Singapore
Israel
United
Kingdom
United
States
Malaysia
population
gdp
gdp_unit
4,492,150
126.5 billion
6,352,117
156.9 billion
milexpendpercent miilitary per capita
4.90%
1379.85
7.70%
1901.93
60,609,153
1.818 trillion
2.40%
719.89
298,444,215
24,385,858
12.31 trillion
287 billion
4.06%
2.03%
1674.64
238.91
Military expense per capita
Israel
2000.00
1800.00
United States
1600.00
US dollars
1400.00
Singapore
Singapore
1200.00
Israel
1000.00
United Kingdom
800.00
United States
United Kingdom
Malaysia
600.00
400.00
Malaysia
200.00
0.00
miilitary per capita
country
15
Military expenditure in multiple currencies
•
The data in CIA Fact book is all in US dollars.
– Might be more helpful to view in other currencies.
•
Currency conversion web sites also exist: www.oanda.com
– We “wrapped” it
– called it “hzoanda”
Select any
currency
Query automatically assembled .. And sent to Cameleon
16
Military expenditure in choice of currencies
date
rate
1/6/2007
country
Singapore
Israel
United States
United Kingdom
Malaysia
1.54
Military in
gdp_ milexpend Military
Singapore
population gdp unit percent
per capita Dollars
4,492,150
6,352,117
298,444,215
60,609,153
24,385,858
126.5
156.9
12.31
1.818
287
billion
billion
trillion
trillion
billion
4.90%
7.70%
4.06%
2.40%
2.03%
1379.85
1901.93
1674.64
719.89
238.91
2124.97
2928.98
2578.94
1108.63
367.93
17
These solutions can be even easier through
COIN context mediation
• MIT has developed a Context INterchange (COIN) technology that
can automatically:
– Adjust scale factors and
– Provide answers in SG$ (or any currency), etc.
• This query:
Select country, MilExpendPercent * GDP * scalefactor.scale /
population
from cia, scalefactor where country in ('Singapore', 'Israel','
United States','United Kingdom', 'Malaysia') and
scalefactor.text=cia.gdp_unit [currency conversion not shown]
• Could be reduced to:
Select country, MilExpendPercent * GDP / population
from cia where country in ('Singapore', 'Israel','
United States','United Kingdom', 'Malaysia')
• This part of project cannot be demonstrated today due to recent
hardware problems that disrupted system and corrupted some
software.
18
Information may be scattered across
multiple sources
• What if want military
expenditure per soldier?
• Can get size of armed
forces from
http://en.wikipedia.org/wiki/List_of_co
untries_by_size_of_armed_forces
• Need to combine this data
with the information from
CIA Fact Book to get
military expenditure per
soldier
19
Can do “Joins” across multiple sources
select cia.country, GDP, GDP_unit, armed_forces,
MilExpendPercent * GDP* scalefactor.unit /
armed_forces/1000 as milpersoldier
from cia, armforces, scalefactor
where cia.country in ('Singapore', 'Israel', 'United States',
'United Kingdom') and armforces.country = cia.country and
scalefactor.text=cia.gdp_unit
COUNTRY
GDP
Israel
156.9
Singapore
126.5
United Kingdom
1.81
United States
12.31
GDP_ ARMED_ MILPERSO
UNIT FORCES LDIER
billion
168
71912.50
billion
60 103308.33
trillion
190 228631.58
trillion
1426 350481.07
20
What about Armament Knowledge?
• More good news …
• There is (we believe) lots of armament
information on SAF intranets
– Though we have not been able to access much yet
• There is lots on public Internet – for fee
– Jane’s Information Group (http://www.janes.com )
• There is lots on public Internet – for free
– Small arms, Tanks, Artillery, etc
• We would need assistance of Subject Matter
Experts to determine needed information
• In meantime, here is a brief demonstration …
21
Some Example Questions?
• Firearms & RPG: How do the weights of
an AK-47, AK-74, M-15A4, and M16 A2
compare ?
• Tanks: Where is the Centurion Mk13, M1,
M1A2, and T-54 made and what is the
range and speed of each?
22
How do the weights of an
AK-47, AK-74, M-15A4, and M16 A2 compare ?
• Information scattered across multiple sites:
http://www.warfare.ru/?lang=&catid=246&linkid=1814
http://www.armalite.com/sales/catalog/rifles.htm
http://www.fnmfg.com/products/
http://www.armalite.com/sales/catalog/rifles/m15a2.htm
http://www.fnmfg.com/products/m16fam/m16a2.htm
23
Can be accomplished by a single query
select weapon, weight, weight_unit from armlite where weapon
in('AK-47', 'AK-74', 'M-15A2', 'M16 A2')
union select weapon, weight, weight_unit from warfareru where
weapon in('AK-47', 'AK-74', 'M-15A2', 'M16 A2')
union select weapon, weight, weight_unit from fnmanufacturing
where weapon in('AK-47', 'AK-74', 'M-15A2', 'M16 A2') *
WEAPON
AK-74
AK-47
M16 A2
M-15A2
WEIGHT
3340
4300
7.5
8.2
WEIGHT_UNIT
gr
gr
lbs.
POUNDS
* This query could be further simplified by using a database View of the 3 db’s.
24
Where is Centurion Mk13, M1, M1A2, and T-54
made and what is the range and speed of each?
• Even when all the information is on a single site,
it is hard to pull together for comparisons
http://www.fas.org/man/dod-101/sys/land/row/tanks.htm
25
But can be done with a simple query
select weapon, range, speed from tanks where
weapon in ('CENTURION Mk5', 'T-55', 'M1A2',
'M1')
WEAPON
CENTURION Mk5
M1
M1A2
T-55
RANGE
102
498
465
500
SPEED
34.6
72
66
50
26
How does it work? Web-Wrapper Technology
User or
Program (via SQL Query)
Select Edgar.Net_income
From Edgar
Where Edgar.Ticker=intc
and Edgar.Form=10-Q
Web page
spec file *
SQL
Side
Ticker
INTC
Web
Wrapper
Generator
HTML
Side
Net Income
1,983
Data record returned
* Spec file contains:
Schema, Navigation rules,
and Extraction rules.27
Recent Technology Extensions to Support
DSTA Effort
• Cameleon Studio
– Makes it much easier/faster to create spec files
• Incorporate Cameleon into IBM’s DB2 Federated
Database System *
– To handle much more complex SQL queries
• Calculations
• OR and IN features
– Required creation of a general-purpose “capability
handling” engine
• Web sites are more limited than databases (e.g., may only
take one “key” at a time, only certain fields can be used as a
“key”, etc.
* For more information see: Lynn Wu, Aykut Firat, Stuart Madnick, Tarik Alatovic, “Querying
Web-Sources within a Data Federation”, Proceedings of the International Conference on
28
Information Systems (ICIS), Milwaukee, Minnesota, December 2006.
Cameleon Studio tool makes spec file creation (relatively) easy
29
Previous Cameleon + POE Architecture
• Our Planner, Optimizer, Executioner (POE) facility
extended Cameleon web wrapper
SQL
Data
Application
Retrieval
Extraction
HTTPClient
Relational
Front End
-Divides query into simpler
single source queries
Planner, Optimizer,
-Dispatches the queries to core
Executioner (POE)
Cameleon or SQL db
Simple SQL Query - Assembles results together
Output in
Desired Format
Core Cameleon
Regular Expression
Engine
& Output Format
Query Handling
Spec File Parsing
Authentication, etc
Web or Database
POE handles complex
multi-source query
(including “join”s):
Spec Files
Registry
Problem - POE does not
handle full SQL needed:
- No calculations ( e.g.,
gdp/capita)
- Allows JOINS and UNIONS
but not IN or OR
30
IBM’s DB2 Federated Database System
• Uses wrappers to access non-relational data sources.
• DB2 first decomposes the original query into query
fragments and then sends them to wrappers.
– Some generic wrappers provided (e.g., for XML sources)
– For arbitrary sources (such as web sites), custom wrappers
must be created
• Wrapper sends the result back to DB2 which then
assembles the final results.
DB2 XML Wrapper (Adapted from IBM).
31
IBM’s DB2 Federated database system architecture
Query: Select ..from s1,s2,s3
…
Federation Engine
IBM DB2
Wrapper: Request-Reply Protocol
Wrapper
For each web site (S1,
S2, S3), a wrapper
must be custom
crafted.
Wrapper for S1
Wrapper for S2
Wrapper for S3
Capability
Handler
Capability
Handler
Capability
Handler
Data
Extraction
Data
Extraction
Data
Extraction
Web
Sources
S1-website
S2-website
S3-website
32
Our Solution: (1) Cameleon as general-purpose
wrapper for web sources (2) with new capability engine
Two-Layered Architecture
Three-Layered Architecture - current IBM solution
with general-purpose engines
Query: Select ..from s1,s2,s3…
Query: Select ..from s1,s2,s3…
Federation Engine
Federation Engine
IBM DB2
IBM DB2
Wrapper: Request-Reply Protocol
Wrapper
Wrapper for S1
Capability
Handler
Wrapper: Request-Reply Protocol
Wrapper,
Capability
Engine
Wrapper for S2
Capability
Handler
Wrapper for S3
Data
Extraction
Data
Extraction
Data
Extraction
Engine
Data
Extraction
Engine
Capability
Record
Declaration
CR for S2
CR for S3
DE for S1
Data
Extraction
Spec Files
DE for S2
DE for S3
Web
Sources
Web
Sources
S1-website
CR for S1
Capability
Handler
Cameleon
Data
Extraction
Query
planning
with
capability
declaration
S2-website
S3website
S1-website
S2-website
S3-website 33
Phase 2 – Evaluation of Policy/Intervention Options for
Improving Armament Safety
• Approach: Use of System Dynamics (SD)
Modeling
– Used by US Government/Military for many purposes
– Recent example – evaluating interventions on “State
Stability” (see http://web.mit.edu/smadnick/www/wp/2005-13.pdf )
• Goal: Develop a system dynamics model for
Singapore Armament Safety
– Represent causalities and linkages
– Uncover root causes
• Outcome: Model can be used to predict the
likelihood of future accidents and help evaluate
policy options and preventive measures.
34
Philosophy of System Dynamics
• Every action has consequences
• Often through complex feedback loops
Do you feel crowded in – and frustrated?
35
See if you can get a bit more space by pushing on that wall …
36
Oops …
37
Quick Primer:
The What (and Why) of System Dynamics
Consider the domain of Software Development
• Classic “knee jerk” reaction to a project behind schedule is to add
people.
• “Brooks Law” noted that “Adding people to a late project, just makes it
later” (because the new people must be trained, this takes productive
people off the project).
• Both of these points are fairly well-known by most software developers –
but both are much too naïve. There are many other factors: length of
project, type of project, expertise of staff available, approach to and time
needed to do training, stage of project, etc. Over the years, all of these
individual factors have been well-studied individually – but how do they
interact ?
• System dynamics makes it possible to model & study the dynamics of
these interdependencies. Many non-obvious outcomes have been found
(e.g., sometimes Brooks is wrong! Important to know when and Why?)
38
Source: Software Project Dynamics: An Integrated Approach, by T.K. Abdel-Hamid and S. Madnick, Prentice-Hall, 1991.
History of System Dynamics
SDM used as modeling & simulation method over 30 years
• Designed to eliminate limitations of linear logics and oversimplicity
• Typical human assumptions and behaviours
• Based on understanding system structure, behavior patterns,
interconnections of positive & negative feedback loops, and
intended & unintended consequences of action
SDM has been applied to numerous domains, e.g.,
•
•
•
•
•
Software development projects
Process Improvement projects
Crisis and threat in the world oil market
Stability and instability of countries
… many many others …
SDM helps to uncover ‘hidden’ dynamics in system
•
•
•
Helps understand ‘unfolding’ of situations,
Helps anticipate & predict new modes
Explore range of unintended consequences
39
Armament Safety Example:
consequences and proceeding actions
Held up due to traffic accident
Was busy dealing with
an earlier accident
Safety officer arrived late
Hypothetical
Example
Training started late
Training was behind schedule
Did not stop to cool off
Weapon had overheated
Weapon had jammed, which
damaged ammunition
Damaged ammunition
Chamber explosion
Soldier injured
40
Preliminary Example of Armament Safety SD Causal Model
Quality of Information
Management
Size of Warstock
Ammunition
+
Ammunition War-Stock
- to Training Cycle Time
+
Age of Training
Ammunition
B3
+
Ammo Quality
Ammunition
Purchase Rate
+
+
B1
+
-
+
Follow regulations for
ammo/ WPNs use
+
Weapon Age
-
Safety Efforts
Serviceability of
WPNs
Availability of
Replacement Parts
Ammunition
+
+
Risk Assessment
Quality
B2
WPN Cleanliness/
Maintenance
-
Level of Training
-
Weapon
Malfunctions
-
B4
+
Immediate Action
Occurences
+
+
Follow
Regulations for IA
Preparation
Quality of
Pre-Marksmanship
Instruction
Weapons
Other Competing
Budget Requirements
-
Money Available for
Training, Weapons &
Ammo
+
Pressure to
Complete Training
Attention to
Warning Signs
+
B5
Overconfidence &
Complacency
-
R1
Training Level
+
Chamber
Explosions
Incident Reporting
+
+
+
Ceasfires/
Delays
B9
-
Damaged Ammo
Resolution of
Conflicting Goals
-
+
Range Availability
+
Time
-
Injuries
B6
-
Direct Safety
Improvement
B7
Protective
Equipment
Training/Operation
Tempo
+
Safety Equipment
+
+
+
Emphasis on Safety &
Process Improvement
B8
Developed based upon Singapore reports provided
and Subject Matter Expertise (SME) provided by US
Army Major Nathan Minami, Company Commander 25th Infantry Division and West Point instructor.
Indirect Safety
Improvement
41
Plans for Phase 2
• Focus: Armament Safety Projection & Mitigation
(e.g., Understanding Consequences)
- Maybe include Vehicle Safety
• Needs:
– Access to Subject Matter Experts
(Plan to also explore US Army sources)
• To develop causal model
• To determine “tipping points” and other “non-obvious”
behaviour
– Access to Data
• To parameterize model for simulation
42