Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Systems of Systems: Cybersecurity Vulnerabilities and Opportunities Donald Wunsch, ACIL Director Ann Miller, TSL Director Applied Mathematics for Deregulated Electric Power Systems: Optimization, Control, and Computational Intelligence Crystal City, November 2003 Applied Computational Intelligence Lab & Trustworthy Systems Lab University of Missouri - Rolla Acknowledgements Personnel – Funding – NSF – Sandia – Boeing – MK Finley Professorship – Cindy Tang Professorship Senior Personnel – – – – – – Ganesh Kumar Venayagamoorthy Ron Harley Daryl Beetner Danil Prokhorov Raonak Uz-Zaman Frank Harary – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – Narayan Vishwanathan Amit Agarwahl Sam Mulder Wenxin Liu Nian Zhang Alexander Novokhodko Xindi Cai Rohit Dua Hu Xiao Rui Xu Brian Blaha Paul Pigg Arvind Rapka Nath Qiang Yao Kevin Bollum Anjaya Shrestra Karthik Balasubramanian Pinar Demircan Daniel Treat Ian Downard Eyad Salah Tagiedin Ganesh Sridharan Jason White Krishnaprasad Balasubramanian Dayle Majors Nartaj Lakshminarasimhan Siddarth Panchal Robert Wayne Denier Tongquan Wei Jimish Doshi Ravikiran Sharda Systems of Systems: Interdependencies “system of systems” – – Grown/evolved by adding components not initially designed to be part of the system Interdependencies not easily identified Potential for cascading failures Potential for hidden robustness Issues in Systems of Systems Trustworthiness Testing Market Demands Complexity Safety Life-Cycle Model Integration Recommended disk space, MB Complexity: Software Size Growth 100 10 Math package 1 Math package 2 Math package 3 1 1984 86 88 90 92 94 96 1998 Source: IEEE Spectrum, January 1998 Complexity: Software Size Growth Complexity: Interdependencies A graph representing almost 6 million lines of computer code. The graph contains approximately 33 thousand nodes and 34 thousand relations. Source: NATO Report on Visualization, 1999. Failure Rates – System Calls Memory management File directory Access I/O Primitives Process Primitives Process environment Memory management File directory Access I/O Primitives Process Primitives Process environment Memory management File directory Access I/O Primitives Process Primitives Process environment Memory management File directory Access I/O Primitives Process Primitives Process environment LINUX NT Win-2000 Win-CE 0 (Source: 5 20 15 10 Carnegie Mellon, CS Dept.) Normalised Failure Rate, % 25 30 Effects of Complexity and Growth Cascading failures Opportunities for errors Control, Communication, IT – – Pres. Commission on Critical Infrastructure Protection Particularly EMS & SCADA Voltage Collapse High-Consequence Even brief – expensive – Circuit fab: 20 min = ($30 M) Recent large disruption caused deaths Backups no guarantee – Well-known in software safety circles Therac 25 classic example Complexity: Ripple Effect Example At 0903 CST on 18 December 1997, at the Olathe (Kansas City) Air Route Traffic Control Center, a technician routed power through half of the redundant uninterruptible power system, preparatory to performing the annual preventive maintenance on the other half. Apparently the wrong board was pulled. Complexity: Ripple Effect Example Results: – – – Power only out for 4 minutes Radar and communications working within 17 minutes However, at least 300 planes were in the Olathecontrolled airspace; domino effect: hundreds of flights canceled, diverted, or delayed with problems well into the evening. Back-up Systems Are Not a Guarantee Not only did the Air Route Traffic Control Center have redundant systems, there were also standby generators and emergency batteries. Yet, that December morning, these backup systems were bypassed. Why? Complex Interactions: States and Inputs The back-up systems were bypassed because the system was in a maintenance state. This particular combination of inputs was not anticipated to occur when the system was in maintenance mode. Tempting Target Dramatic growth in number of knowledgeable experts Potential to insert incorrect data or Denial of Service attacks High leverage / low risk Computational Intelligence Tools Can Help Neural Net Intrusion Detection ADP Robust Controls Combinatorial Optimization for reconfigurability Intrusion Detection with Neural Nets RBFNs can be used for misuse and anomaly detection using sequences of system calls Data are obtained from 1998 DARPA Intrusion Detection Evaluation program Also collaboration with Sandia Red Team 1 RBFNN Generalization on unknown test data 0.9 0.8 True positives 0.7 0.6 0.5 0.4 0.3 0.2 accuracy=0.74 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 False positives 0.7 0.8 0.9 1 PNN + ADABOOST Multi-Machine Power System Control Multi-Machine Power System with Conventional Controllers Dw1 Governor 1 4 900 Km Dw2 5 2 G1 Governor G2 S S Turbine Pref1 Turbine Exciter Ve1 Vt1 AVR Exciter 900 Km 900 Km Vt2 3 Vref1 G3 Pref2 Ve2 Vref2 AVR Multi-Machine Power System with DHP Neurocontrollers D P1 Governor 4 1 900 Km 5 Governor 2 G1 DP2 G2 S S Turbine Turbine Exciter Pref1 Vf1 Vref1 DVref1 S Vt1 TDL Exciter 900 Km 900 Km D w2 D w1 3 TDL Neurocontoller TDL Vt2 TDL Pref2 Vf2 S Vref2 DVref2 Neurocontoller G3 DHP Critic Network Adaptation Yref TDL ACTION Neural Network PLANT Y(t) U (t ) A(t ) A(t) (t+1) MODEL Neural TDL Network TDL TDL + S+ ^ D Y (t 1) ^ D Y (t ) ^ D Y (t 1) (t 1) J (t 1) ^ D Y (t 1) + - S U (t ) DY (t ) MODEL Neural Network CRITIC Neural Network ^ D Y (t ) ^ TDL D Y (t 1) TDL D Y (t 2) ^ EC2(t) CRITIC Neural Network (t 1) J (t 1) ^ D Y (t 1) Terminal Voltage of Generator G2 for a 5% Step Change in its Desired Terminal Voltage & Operating Point Changed 1.08 Terminal voltage in pu 1.07 1.06 1.05 1.04 1.03 1.02 1.01 AVR 1 DHP 0.99 0.98 1 2 3 4 5 Time in seconds 6 7 8 Speed Deviation of Generator G2 Operating Point Changed -3 x 10 1.5 Speed deviation of G1 in pu 1 0.5 0 -0.5 Conventional -1 Neurocontroller -1.5 0 1 2 3 4 Time in seconds 5 6 7 8 Traveling Salesman Problem Great benchmark NP – complete – Maps to other NP – complete problems Public databases Big need – get learning capability of NN without brittleness of other techniques. Previous contributions -- disappointing Paper Method Largest Quality Instance (percent Test bed excess over optimal ) [11] 1st 100 14.6% NS [13] 1st 100 14% NS [10] 1st 400 NR NS [5] 2nd 532 6.8% TSPLIB [12] 1st 1000 NR NS [16] 2nd 1000 NR NS [15] 1st 2392 5% TSPLIB [17] 2nd 2392 9% TSPLIB [2] 1st 10000 NR NS [4] 1st 11849 17.4% TSPLIB Clustered Traveling Salesman Divide problem into clusters using ART in O(n) Use Lin-Kernighan algorithm for global tour Use Lin-Kernighan algorithm for local tours Merge local tours in O(n) time Global operations limited to O(n) time Algorithm Overview ART O(n lg n) cluster LK O(k2.2) cluster LK O(k2.2) cluster LK O(k2.2) Merge Clusters O(n) Read problem from file O(n) Result Implementation Implemented in C++ thread-safe code Uses Windows threads for parallelism Operating System-specific code isolated to one file Should be easy to port to other parallel systems #cities Tour Length 1P Time 1000 2.58E+07 0.422 2P Time Vig factor 0.281 0.7 % off 10.40% Speedup 1.50 2000 3.61E+07 1.031 0.672 0.7 10.64% 1.53 8000 7.14E+07 8.328 4.281 0.72 10.97% 1.95 10000 7.97E+07 11.359 7.297 0.75 10.57% 1.56 20000 1.12E+08 24.641 14.406 0.8 10.53% 1.71 250000 4.00E+08 315.078 209.687 0.92 11.64% 1.50 1000000 7.94E+08 1468.165 986.48 0.97 11.03% 1.49 10000000 2.52E+09 10528.7 0.98 1.27% CONCORDE 1000 2.34E+07 1.670 2000 3.26E+07 3.500 8000 6.43E+07 26.570 10000 7.20E+07 37.620 20000 1.01E+08 84.830 250000 3.58E+08 1000000 7.15E+08 9013.53 10000000 2.495E+09 43630.7 1379.540 1k 4k 8k 10k 20k 50k 85k 150k 250k 1M Even better news… Continued Scaling Results Parallelizability Memory Management BUT – To Move Beyond Clear Need for more advanced architectures – Especially to Learn from Experience Cellular Structures necessary Same with SRNs Therefore, combine them and ACDs Recurrent Nets Obviously achieve dynamic behaviors Possible similarity to adaptive systems but with fixed parameters Simultaneous recurrent nets particularly challenging, esp. architectures Generalized Maze Problem •Graph Theoretic Representation •SRN Necessary (Werbos & Pang, ’96 & ’98) •Cellular structure – scaling •Closed form now •Convergence time now •Importance of design principles Design from output backward Require for the output node: x16 = (x2 / x1)[min{x6, x5, x4, x3} + 1]. This is a known SRN! Cellular SRN Structure Complete +1 S Output J = (x2/x1) * sum = x16(a,b) / * Current Node inputs Product Nodes Neighbor node inputs Feedback inputs (Occurs at each node (a,b) in maze.) Analyze worst-case convergence WCT = N2 - 2N + N - 3 = N2 - N - 3. Note that this is convergence in J steps. Also true for N x N maze by simple induction proof. Conclusions Power networks inherit the full range of “systems of systems” issues. These are amenable to computational intelligence solutions: – – – Detection Robust Control Reconfigurability Combinatorial Optimization