Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Data Mining Combat Simulations: an Emerging Opportunity Barry A. Bodt [email protected] (410) 278-6659 Computational and Information Sciences Directorate Army Research Laboratory (ARL) The U.S. Army’s Corporate Laboratory Motivation • Simulation and statistical analysis are underutilized in helping the commander’s staff to analyze courses of action. • Battle results are infinite in scope, yet the outcome of any one battle is defined by a unique set of battlefield interactions. • Key is to recognizing those interactions through development of more informative performance measures unique to the scenario at hand. Approach Use statistical methods and combat models to create a methodology that identifies nontraditional metrics for plan evaluation. Background Military Decision Making Process •Focus on wargame •Disciplined rules •Synchronization matrix COA Joint Tactical Operation Center, Qatar Network Centric Warfare Communicate… Smart Logistics On-board Diagnostics Soldier Health Sensor information … Information Requirements in NCW The key to any analysis is the set of measures used to represent the performance and effectiveness of the alternatives being considered. We are relatively good at measuring the performance of sensors and actors, but less adept at measuring command and control. Command and control, to be fully understood, cannot be analyzed in isolation, but only in the context of the entire chain of events that close the sensor-to-actor loop. To make this even more challenging, we cannot isolate on one target, or even a set of targets but need to consider the entire target set. Furthermore, network centric warfare is not limited to attrition warfare … It is not sufficient to know how many targets are killed, but exactly which ones and when… Ref: Network Centric Warfare, 2002 Simulation Data • • • • Scenario development OneSAF lay down of forces OneSAF modified output Data supporting modeling Scenario BMP-2 BMP-2 BMP-2 T-72M T-80 T-72M T-72M Town T-72M T-80 T-72M T-80 T-80 Company Objective OneSAF Screen Dump Automated Data Collection • OneSAF Modifications OBJECT_ID: 100A31 X = 24396.82 Y = 25828.75 Z = 755.72 Vehicle Authorized Undamaged Catastrophic Firepower Damage Damage M2 1 0 1 0 Equip/Supplies: Current Lvl Resupply Lvl Avg Per 25mm HE (M792) 625.00 625.00 625.00 25mm APFSDS-T (M919) 325.00 325.00 325.00 TOW (TOW) 0.00 5.00 0.00 7.62mm MG (M240) 2340.00 2340.00 2340.00 Fuel (Fuel) (gallons) 171.00 174.00 171.00 Mobility Damage 0 Veh OneSAF Modification Killer/Victim Scoreboard Time Stamp 1010070890 Vehicle ID 1076 Firer ID 1087 Projectile 1143670848 Firer Position: • • • • Firer and Target Identity and Location Type of Ammo Range Outcome X = 220217.00 Target Position: X = 222454.38 Y = 146765.00 Y = 149117.80 Z = 12.37 Z = 9.99 Vehicle 1076: Hit with 1 "munition_USSR_Spandrel" (0x442b0840) Comp DFDAM_EXPOSURE_HULL, angle 19.53 deg Disp 0.889701 ft Kill Thermometer is: Pk:1.00, Pmf:1.00, Pf:0.90, Pm:0.80 Pn:0.80 RANGE 3246.773576 r = 0.990835 kill_type = MF 1076 100A41 vehicle_US_M1 1087 100A23 vehicle_USSR_BMP2 Data Supporting Classification Models Response – mission • 228 OneSAF runs accomplished (success) • 3 situational snapshots per if an undamaged platoon run occupies objective at – 10% blue ammo expended battle end (MA) – 25% blue ammo expended – 40% blue ammo expended • 429 data points per run (143 -other responses include MBT and “Eric” strength per stopping time) – Number of K, M/F, F, and M kills and forces on objective – – – – – – Ammunition levels Number of hits delivered Range of hits Number of side hits delivered Distance to objective Number of Blue on objective Data Matrix 228 x 434 Model Performance Pred Pred Obs Obs Slice 1 ~ 2000m Slice 2 ~ 4000m Or ~ 5 ½ minutes Or ~ 10 minutes Slice 3 ~ 5800m 00 11 34 00 85 98 21 Pred 0 741 Obs 11 35 25 84 0 105 14 1 20 89 Or ~ 20 minutes Company Company Objective Objective Correctly Classified Correctly Classified Loss: 71% Loss: 82% Win: 68% Win: 77% Correctly Overall:Classified 70% Overall: 80% Loss: 88% Win: 82% Overall: 85% Method Comparison Percent Correct Classification by Stopping Time and Method Stopping Discriminant CART Logistic Time (min) Analysis Regression 5½ 70% 70% 69% 10 80% 75% 74% 20 85% 82% 85% Advantages – Support prediction for COA performance evaluation – Provide models identifying key battle parameters for a given engagement, influencing both COA development and commander’s critical information requirements – Input to CCIRs – Input to contingency plans – Input to tolerances for synchronization Implementation Models Reach back Advantages -computational power (ARL 9th) -more complex analyses Distributed Disadvantages -latency -can’t smell gunpowder Advantages -cheaper boxes (250 OneSAF boxes used at Ft. Leavenworth) -closer to action Disadvantages -depth of a field analysis -automation required Why Aren’t We Already Doing This? A few reasons … • Computer simulation focus has been mainly strategic or oriented toward acquisition. Tactical application has been limited. • Simulations did not have high enough fidelity for tactical application. • Simulations were unstable. • Computing resources were inadequate. • Necessary communication of inputs had not been imagined. • Simulation creators do not always talk to statisticians. Improvements Here and On the Way • Stability • Power Point force laydown of forces • MS Word OPORD •Terrain, weather wizzards • Composable simulations • After Action Report data • Man-in-loop allowed • Sensor advances • Communication advances • Computation speed and cost Catching On? After Action Review PURPOSE: The OneSAF After Action Review component provides the capability to correlate, rollup, and analyze simulation outputs and visualize the results of the simulation exercise. The toolset allows the analyst to preplan the AAR prior to exercise execution. • Situation awareness during the execution of the exercise and afterwards during exercise playback: – PVD & 3D Stealth display – Statistical charts, tables – OPORD paragraphs – Task Organizations Summaries – Radio/audio playback (Future) • Mining of collected data to construct MOPs/MOEs • Automatically build AAR presentations & Take Home Package using COTS Office Automation 20 Next Up Wei-Yin Loh, Regression Tree Analysis of Battle Simulation Data David Kim, Robust Modeling Based on L2E Applied to Combat Simulation Data Warren Liao, Discovery of Battle States Knowledge from MultiDimensional Time Series Data