* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download What is Data Mining?
Document related concepts
Phase of Flight & Rule of Flight Calculator By: Cliff Johnson, FAA Adrian Rusu, Anthony Breitzman, Nicholas Laposta, John Bucknam, Rowan University Presented to: ICRAT 2016 Jun. 21, 2016 Drexel University, Philadelphia, PA Federal Aviation Administration Overview • Introduction • Motivation • Direction • Phase/Rule of Flight Background • Phase of Flight/Rule of Flight Calculator • Case Study • Future Work • Conclusion • Questions and Answers Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 2 Introduction • Federal Aviation Administration – Cliff Johnson, who was the project lead and collaborator between Rowan University and the Federal Aviation Administration (FAA). • Rowan Collaboration – 5 undergraduate students (Nick Laposta) under Dr. Adrian Rusu using the Federal Aviation Administration’s equipment and data. – Dr. Anthony Breitzman who helped us create the algorithms used through data mining. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 3 Motivation • Increase safety by improving analytical tools for the safety analyst. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 4 U.S. Helicopter Fatal Accident & Fatality Rates *CY 14, Jan-Jul Only Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 5 5 Direction • Q: But how do we achieve a lower or nearzero accident rate? • A: Proactively identify risk via projects that research methods and tools such as data mining of flight data monitoring data and FAA surveillance (i.e. aircraft tracking) data. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 6 Concept of Rotorcraft ASIAS Rotorcraft ASIAS All Missions Flight Training Charter EMS Oil/Gas Logging Police News Cargo Lift … Participating Operators Own-Access User FAA, Sponsor Secure System Database Analysis Toolkit Interface & Display Aggregate Results De-identified Data Access User Analysis Tools System Developer & Administrator Visualizations, Algorithms, Event Definitions PEGASAS Research Team for Rotorcraft ASIAS Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 7 Rotorcraft Research for ASIAS Industry Involvement Potential Mitigation Partner FDM Equipment Working Group Program Management Outreach Flight Tests HFDM Device Integration & Calibration Data Analysis Modeling & Simulation ASIAS Rotorcraft Analysis Capabilities Outreach Data Transcription HFDM Research Repository HFDM Architecture Design HFDM Working Group Safety Metrics HFDM Analysis Techniques HFDM Modeling Techniques HFDM Cockpit Audio/Video Analysis Data Mining for Safety Events Identification of High-Risk Safety Events Supports the USHST Goal of 20% Reduction in the Fatal Accident Rate for Helicopters by 2020 Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 8 Rotorcraft ASIAS Research Team Members Ed DiCampli, Chief Operating Officer– PI Harold Summers, Director of Flight Operations Jay Clark, Manager, Information Systems Kipp Lau, RASIAS FDM Consultant Robert Liguori, RASIAS IT Consultant Keith Cianfrani, FDM Specialist/Outreach Prof. Dimitri Mavris – PI Hernando Jimenez, Ph.D. – Co-PI Kyle Collins Simon Briceno Alek Gavrilovski Alexia Payan Aviation Research Division, ANG-E Cliff Johnson – Rotorcraft ASIAS Research Task Lead FAA WJHTC Atlantic City, NJ Alex Ahlstein – ASIAS Group Leader Mike Yablonski – Principal Aviation Systems Engineer MITRE-CASSD *Note: MITRE role involves collaboration with research activities for integration with overall ASIAS. Prof. Karen Marais – PI Arjun Rao – Co-PI Inseok Hwang Arjun Rao Sanghuyn Shin Tyler Travis Lana Manovych Jean-Christophe Geffard Steve Cullen Scott Sunder FOR OFFICIAL USE ONLY Prof. Stephen Cusick – PI Keith Cianfrani Nicholas Currie Federal Aviation Administration 9 Why Rotorcraft ASIAS? Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY 10 Federal Aviation Administration 10 Background • How do we determine Phase of Flight Detection and Rule of Flight Detection and implement them into the prototype for Rotorcraft HFDM research for ASIAS? • Two approaches: – 1. Examine fixed-wing aircraft tracking data (focus of the initial effort) – 2. Examine rotorcraft flight data monitoring data (focus of current and future efforts) Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 11 Background • Currently, the only way for an analyst to determine the Phase of Flight (PoS) of an aircraft is to either look at the flight plan or depend on a signal received from the aircraft Rule of Flight is important when analyzing traffic and can change multiple times throughout the flight. A pilot can switch between Visual Flight Rules (VFR) and Instrumental Flight Rules(IFR) depending on weather conditions and flight status. Our system aims to determine general phases to help analysts determine possible faults in procedure, manufacture, or operation of aircraft in post-accident analysis Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 12 Background (Conflicts) • There are many different phases that an aircraft can be in at any given time and this quantity can be even greater when also taking rotorcraft into account. • We looked to create a simple classification tool that can be expanded upon in future research. • Although an aircraft broadcasts its Rule of Flight constantly the signal will not always be picked up by primary radar. Therefore, we are attempting to determine the Phase and Rule of FLight when only given primary radar. • It is possible that different airports may have different protocols that require aircrafts to be in different position, however this model should be feasible for any airport with minimum changes, possibly using parameters. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 13 Phases & Rules of Flight • • • • • • • • • • • • • Phases of Flight Standing Pushback/Towing Taxi Takeoff Initial Climb En Route Maneuvering Approach Landing Emergency Descent Uncontrolled Descent Post-Impact Unknown • • • Aviation Research Division, ANG-E Rules of Flight Visual Flight Rules – Thousands of feet plus 500 feet. Odd number thousands of feet (2500, 3500, 4500, etc.) within our Altitude data. We also found that aircraft points with very low altitude were more likely to be VFR as well based on predefined points. Instrument Flight Rules - Even number of thousands of feet (2000, 3000, 4000, etc.) within our Altitude data. The exception to this is near the airport environment where 1000 feet is typically the altitude of the VFR traffic pattern. Unknown - In the case that our data was flawed or couldn’t be defined as VFR or IFR. FOR OFFICIAL USE ONLY Federal Aviation Administration 14 Initial Case Study In our research, we used the National Offload Program (NOP) data taken from different locations, such as the Atlantic City International Airport (ARTS). Although we were given other locations as well, such as Philadelphia International Airport, we found it easier to focus on one location, especially the amount of data given which as ~7 million data points. With these points, we chose to focus on the most commonly defined columns within our database, such as the altitude and speed of a given aircraft at a given point, as well as our own custom column called Aircraft ID (ACID). Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 15 Sample Data F3_Date F4_Time F5_ Callsign F6_ AcType 10/01/2015 00:00:00.0 02 4004 280 274 193 34.95895 -85.46546 592 10/01/2015 00:00:00.0 33 3602 342 469 326 32.16565 -82.83178 499 10/01/2015 00:00:00.0 61 1461 350 464 332 34.61182 -85.05031 478 10/01/2015 00:00:00.0 61 7466 410 566 62 34.90963 -85.28257 216 10/01/2015 00:00:00.1 33 RBJ67 MD88 640 103 262 94 33.53683 -84.48752 205 10/01/2015 00:00:00.1 33 K4QC C150 5227 42 84 60 32.54313 -86.00119 220 Aviation Research Division, ANG-E F8_BeaconC ode F9_ Altitude F10_ Speed FOR OFFICIAL USE ONLY F11_ Heading F12_ Latitude F13_ Longitude F14_ TrackNum Federal Aviation Administration 16 Aircraft ID (ACID) • The creation of the Aircraft ID was one of the major hurdles for our project as it was necessary for any progress to be made. • We needed to create a unique identifier in order to connect different data points to one aircraft, and to do this we used a timestamp and a beacon code. – Timestamp: Time at which data point was recorded – Beacon Code: An identifier used by radar to identify an aircraft at a given time. • Using these 2 columns, we looked at time intervals using the same beacon code. By doing this, we created lists of data points with the same ACID, called tracks that were used for later analysis by our Phase of Flight/Rule of Flight Calculator. • Some issues with this method include the major use of Beacon Code 0000, which was consistently changing different aircrafts and was against our original model. However, this should be easily resolved by using more of the given columns. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 17 Phase of Flight/Rule of Flight Calculator • Simplified Phases – We considered the following as phases of flight in our software: • • • • • Ascending Descending Cruising Taxiing Unknown – These phases can also be used in order to figure out larger phases as well, such as landing, approach, or other phases due to possible emergencies. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 18 GUI of Calculator Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 19 Basic Model Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 20 Future Work – Transition to Vertical Flight Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 21 Helicopter Research at the FAA • Focus: Certification of New Technologies, Standards for Continued Airworthiness, Occupant Protection, and Hazard Avoidance/Risk Mitigation, Training, etc. • Two Projects I oversee that require phase of flight determination: – Helicopter Flight Data Monitoring Research for Aviation Safety Information Analysis & Sharing (ASIAS) – Helicopter Advanced Vision Systems Research Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 22 Data Mining in Aviation Safety • FDM/FOQA paradigm is underpinned by the a-priori definition of known safety events • Safety event definition is overtly simplistic • Assertion: safety benefits of FDM can be significantly enhanced by incorporating formal data mining methods. – avoids, or at least greatly mitigates, the dependence on previously defined safety events – Allows for more sophisticated event characterizations commensurate with the complexities of flight safety • Flight Data Mining – Formal techniques in aviation are not new. – Systems health monitoring and prognostics for commercial aviation – Operational safety data mining in FDM is sparsely documented, limited to commercial aviation. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 23 What is Data Mining? • “Data mining […] consists of applying data analysis and discovery algorithms that, under acceptable computational efficiency limitations, produce a particular enumeration of patterns (or models) over the data.” – U. Fayyad, G. Piatetsky-Shapiro and P. Smyth, "From data mining to knowledge discovery in databases," AI magazine, pp. 37-54, 1996. Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 24 Unsupervised learning - Clustering Distance based Density Based Hierarchical - Variable Density Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 25 Data Mining for Aviation Safety Supervised Unsupervised Intrinsic similarity - dissimilarity Training data set Data in parameter space Label Data within each of these groups looks alik Analysis data set NEW Data in parameter space Estimate Label These groups look different These points look very different from all others! Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 26 Opportunity and Objectives Opportunity • Data mining can provide operational safety benefits in the aviation segments where it is most needed • Leverage on growing adoption of FDM in GA and Rotorcraft Objectives • Establish the best techniques and practices for rotorcraft safety data mining, addressing unique challenges and features – What data processing techniques are best suited? – What data mining techniques are most effective for intended purpose? – How do we best make sense of, and utilize, results? • Implement, test, and refine current/new data mining algorithms – Phase of flight identification – Anomaly detection Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 27 Phase of Flight -Experimental Strategy Prescriptive Approach • Identify phase of flight definitions • Smooth data as needed • Apply definitions as data filters • Revise as needed Data-driven Approach Compare – Is this the right phase? SME inspection – Exhaustive – all points are assigned – Exclusive – points are assigned to one phase • Unsupervised (no labels) data mining to entire flights • Use only parameters in phase of flight definitions • Use all parameters available How well do existing phase of flight definitions match actual operations? Does the answer change based on mission type? Do we need revised phase of flight definitions for rotorcraft? What techniques work best? Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 28 Phase of flight – Prescriptive Definition Rotorcraft phase of flight definition based on authoritative sources 1. ICAO common taxonomy team – ICAO 2. European coordination center for accident and incident reporting systems – ECCAIRS 3. FAA helicopter flying handbook – FAA 4. Learning to fly helicopters from Randall Padfield – LOF 5. Principles of helicopter flight from W. J. Wagtendonk POH Implementation of phase of flight definition as mathematical inequalities for parameters Data processing / smoothing 1. 2. 3. 4. 5. 6. 7. Low pass filter (FFT with Gaussian filter) Moving average (backward, center) Local regression (weights, 1st or 2nd order) Robust local regression (neglect outliers) Savitzky-golay (1st or 2nd order, successive fitting) Piece-wise linear regression Digitization … and others Example: Low Pass Filter 29 Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 29 FAA HFDM Test Flights Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 30 FAA R&D Test Flights • Test Platform • HFDM / HFDR Devices • Research Goals • Research Outputs – FAA’s Sikorsky S-76A Helicopter, Equipped with ADS-B Out (1090ES) – Current devices: Appareo Vision 1000, L3 Light Data Recorder, Honeywell Skyconnect Tracker 3, Ballard, Stratus, MEMSIC AHRS, iLevil AHRS, GoPro Cameras (6), General Aviation Airborne Recording Device (GAARD) – Additional devices: Skytrac, North FDS/Outerlink IRIS, Latitude iONode, others – Identify, examine, and install several different HFDM/HFDR units – Collect/process data from each HFDM/HFDR system for different scenarios/conditions – Helicopter FDM Data to be used to define and validate future analysis of events, parameters, exceedances, recording rates, etc. from anomalous data – Installation guidelines and optimal locations for each system Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 31 FAA R&D Test Flights • GAARD & Foreflight Mobile Applications during data collection Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 32 Data Mining with Rowan University’s CAVE® Virtual Reality Environment Aviation Research Division, ANG-E • The South Jersey Technology Park at Rowan University • Integrating the CAVE® as extension of Tech Center Labs over the NextGen Prototyping Network (NPN) • Analyzing data generated from FAA Flight Tests using data mining techniques (CAVE ® simulations add realism and offer ability to reanimate flights and simulate anomalies inside a virtual helicopter) FOR OFFICIAL USE ONLY Federal Aviation Administration 3333 Aviation Research Division, ANG-E FOR OFFICIAL USE ONLY Federal Aviation Administration 34