PROBABILISTIC SAFETY ANALYTICS FOR UAS INTEGRATED RISK MODELING James T. Luxhøj, Ph.D. Industrial and Systems Engineering Rutgers University The Mid-Atlantic Symposium on Aerospace, Unmanned Systems and Rotorcraft Villanova University April 10, 2014
Outline UAS System Safety and Hazard Identification Probabilistic Safety Risk Analytics Concepts of the safety risk modeling approach. Notional UAS Pipeline Inspection Scenario Concluding Remarks 2
Hazard Classification and Analysis System (HCAS) Components of the Hazard Taxonomy: Decomposes the UAS domain. Identifies the main sources or clusters of hazards for UAS. HCAS is comprehensive, but not necessarily exhaustive. Source: Luxhøj and Oztekin, 2009 3
Hazards related to UAS UAS Hazard Classification and Analysis System (HCAS) version 4.2 Aircraft Aerodynamics Airframe Payload Propulsion Avionics Hardware and Software Sensors / Antennas Communication Link Onboard Emergency Recovery Detect, Sense and Avoid Other Aircraft Systems Control Station Classification Mobile Fixed Multiple Combinations Hardware and Software Communications Link Data Link Framework Infrastructure Signals Organizational Human Factors Aircraft Design Organization Control Station Design Organization Regulatory Agency Certification Licensing Oversight AIRMEN Individual Human Factors Pilot Maintenance Technician Service and Support Personnel Organizational HF Operator Training Supervision Regulatory Agency Certification Licensing Oversight Individual Licensing Pilot Maintenance UAS Hazards Service and Support Personnel OPERATIONS Source: adapted from Luxhøj and Oztekin, 2009 Environment Hazards Airmen Hazards Flight Operations Flight Planning Phase of Flight Emergency Recovery Type of Operations Line of Sight / Beyond Line of Sight VFR / IFR Operational Control Instrument Procedures and Navigational Charts Continued Airworthiness UAV Control Station Maintenance Source Communication Interface ATC Communications Radio Data Transmission Visual Airspace Established Temporary Personnel (including Oversight Personnel and ATC) Organizational Human Factors Operator Regulatory Agency Certification Oversight ENVIRONMENT Terrain Electromagnetic Activity Weather (includes wind) Particulates FOD Wildlife Bird Strike Animals Obstacles Others Traffic External Influences International Regulatory Differences Airports (i.e., takeoff/landing areas) Navigation Network National Security Operations Hazards 4
Analytics: Bayesian Belief Networks (BBNs) Bayes Theorem: P(X 2 X 1 ) = P(X 1 X 2 )P(X 2 ) / P(X 1 ) Decision Nodes (i.e., Mitigations) X 1 X 3 X 4 D2 D1 X 2 X 5 X 6 D3 Directed Causal Link (i.e., with underlying Conditional Probability Table (CPT) indicates influence strength ) X 7 UE Chance Nodes (i.e., Causal Factors) The approach uses qualitative, probabilistic reasoning about the interactions of risk factors (chance nodes) and mitigations (decision nodes) to make inferences. Source: Luxhøj et al., 2012 5
BBN Components Chance Nodes: These are the Random Variables (i.e., the hazard causal factors - could be discrete or continuous). Each node has states (usually binary but could be more than two). Decision Nodes: These are the Mitigations or Controls. Directed Causal Links: Depict the direction of the causality. Where do the Conditional Probability Tables (CPTs) come from? - Multiple disparate data sources: - histograms, reliability models, fault and/or event trees - simulations - Knowledge Elicitation (KE) sessions with subject matter experts (SMEs) Source: https://www.metavr.com/casestudies/insitu_uas.html 6
Aviation System Risk Model (ASRM) Risk Modeling Steps Describe Case- Based Scenario Analytical Approach Identify Hazards (HCAS) Construct Influence Diagram Causal Structure Build Belief Network Insert Mitigations/ Value Functions Assess Relative Safety Risk Reduction Conditioning Context M1 V1 V2 Analytic Generalization M2 M3 V3 Source: Adapted from Luxhøj, 2003 7
A Notional Scenario Pipeline Inspection Monitoring Scenario: This UAS flight involves a trans-continental gas pipeline inspection monitoring. The UAS launches from a remote location airspace and follows a preprogrammed flight path. The UAS is to fly toward the pipeline, intercept, and then fly along the pipeline. The UAS is equipped with infrared (IR) sensors and electro-optical (EO) sensors. The Operator is a UAS Company that selects the UA, flight profile and operations team. Develop a causal narrative from scenario by exploring what ifs. What if there are local radio frequencies (RF)/power levels that interfere with the continuous connectivity required of the communication and control links? What if there is a General Aviation (GA) piloted aircraft in the vicinity of the airport? What if there is a loss of data link from the Ground Control Station (GCS) to the UAS? What if there are strong wind gusts (> 40 knots) that contribute to the loss of separation between the UAS and the manned aircraft? What if the Automatic Dependent Surveillance-Broadcast or ADS-B Out transmission from the UAS is disrupted by RF interference? (Note: ADS-B will replace radar.) 8
4.3 ENV Wind gusts M1: NextGen Enhanced 4D weather cube wind predictor 4.2 ENV Electromagnetic activity 1.1.9 VEH-UAS While flying in autonomous mode back to recovery point, UAS veers off course M2: Advanced EMI testing 3.2.3 OPS Main Source deficient 3.2.2 OPS GCS Main improper UAS Pipeline Scenario 1.2.2 VEH GCS locked 1.1.7 VEH-UAS Data link transmission disruption to GCS 2.1.1 AIR- GA pilot Inexperienced Aeronautical DM & struggles to maintain stability of the aircraft M7: GCS/UAS Link Software Design Upgrade 3.3 OPS ATC Comms./ transmission disruption 1.2.3 VEH-UAS Data link transmission disruption from GCS M6: Virtual Environment (VE) with predictive graphics displays 1.1.5 VEH- ADSB-OUT on UAS fails 4.8 ENV Other traffic in Class E airspace (near airport) M4: Mixed or Hybrid UAS control 1.2.1.1 VEH- UAS pilot fails to regain control of UAS due to signal latency M5: NextGen Enhanced DSA Technology M3: GA Sense and Avoid Technology 2.1.1 AIR -GA pilot fails to see & avoid visually or with ADSB-IN UAS/GA in-flight collision 1.0 Vehicle 2.0 Airmen 3.0 Operations 4.0 Environment 23 9
HUGIN Model with Conditional Probability Table (CPT) 0.01 10 10
HUGIN BBN Software Tool 0.0357 Note: HUGIN output is in percentages Baseline Scenario Probability = 0.000357 (3.57 x 10-4 ) *Consider exposure per 10-4 or 10-5 flight hours so risk/flight hour in the range of 10-8 or 10-9. 11
Probability Elicitation: Degree of Belief (DoB) Approach The purpose of computing is insight, not numbers. - Richard Wesley Hamming 1 Probability Ang & Buttery (1997) Verbal Descriptor 0.9999 extremely likely (i.e. almost certain) 0.9 very likely 0.7 likely 0.5 indeterminate 0.1 probable (i.e. credible) 0.01 unlikely 0.001 very unlikely 0.0001 extremely unlikely 0 12
Hazard Clusters Likelihood Multiplier Baseline Scenario Undesired Event (UE) Probability = 0.000357 (3.57E-4) 600.0000 560.7 500.0000 400.0000 300.0000 200.0000 299.5 195.6 Airmen Vehicle Operations Environment 100.0000 14.0 0.0000 Airmen Vehicle Operations Environment 13
Specific Causal Factors 600.0000 Likelihood Multiplier Baseline Scenario Undesired Event (UE) Probability = 0.000357 (3.57E-4) 500.0000 400.0000 300.0000 200.0000 100.0000 0.0000 14
Object-Oriented Bayesian Networks (OOBNs) OOBN Modeling Approach System of Systems (SoS) Key Properties: -Abstraction - Inheritance -Encapsulation Sub-net S2 Instance nodes Output node Sub-net S1 Mishap UE Output node Top-Level Model 15
4.3 ENV Wind gusts Sub-net M1: NextGen Enhanced 4D weather cube wind predictor 4.2 ENV Electromagnetic activity 1.1.9 VEH-UAS While flying in autonomous mode back to recovery point, UAS veers off course M2: Advanced EMI testing 3.2.3 OPS Main Source deficient 3.2.2 OPS GCS Main improper UAS Pipeline Scenario 1.2.2 VEH GCS locked 1.1.7 VEH-UAS Data link transmission disruption to GCS 2.1.1 AIR- GA pilot Inexperienced Aeronautical DM & struggles to maintain stability of the aircraft M7: GCS/UAS Link Software Design Upgrade 3.3 OPS ATC Comms./ transmission disruption 1.2.3 VEH-UAS Data link transmission disruption from GCS M6: Virtual Environment (VE) with predictive graphics displays 1.1.5 VEH- ADSB-OUT on UAS fails 4.8 ENV Other traffic in Class E airspace (near airport) M4: Mixed or Hybrid UAS control 1.2.1.1 VEH- UAS pilot fails to regain control of UAS due to signal latency M5: NextGen Enhanced DSA Technology Sub-net M3: GA Sense and Avoid Technology 2.1.1 AIR -GA pilot fails to see & avoid visually or with ADSB-IN UAS/GA in-flight collision 1.0 Vehicle 2.0 Airmen 3.0 Operations 4.0 Environment 16 23
Concluding Remarks Just as UAS technology is advancing, the analytical methods for probabilistic safety risk modeling need to similarly advance. BBNs facilitate the modeling and uncertainty investigation of the complex interactions of the UAS, Airmen, Operations and the Environment for an integrated safety risk assessment. OOBNs offer the potential of modular network development with reusable and portable sub-nets. The modeling approach can assist in vulnerability discovery (i.e., recognize new risks and system-level precursors) where mitigations may not yet exist. 17