Validation Test Plan - Airborne

Transcription

1 EUROPEAN AIRPORT MOVEMENT MANAGEMENT BY A-SMGCS, Part 2 Contract No. TREN/04/FP6AE/S / Marcus Biella DLR Document No: 2-D6.1.6 Version No Classification: Public Number of pages: 193 Project Funded by European Commission, DG TREN The Sixth Framework Programme Strengthening the competitiveness Contract No. TREN/04/FP6AE/S / Project Manager M. Röder Deutsches Zentrum für Luft und Raumfahrt Lilienthalplatz 7, D Braunschweig, Germany Phone: +49 (0) , Fax: +49 (0) fp6-emma@dlr.de Web page: , - All rights reserved - EMMA Project Partners The reproduction, distribution and utilization of this document as well as the communication of its contents to other without explicit authorization is prohibited. This document and the information contained herein is the property of Deutsches Zentrum für Luft- und Raumfahrt and the EMMA project partners. Offenders will be held liable for the payment of damages. All rights reserved in the event of the grant of a patent, utility model or design. The results and findings described in this document have been elaborated under a contract awarded by the European Commission.

2 Distribution List Member Type No. Name POC Distributed 1 Web Internet Intranet 1 DLR Joern Jakobi 2 AENA Guilio Maira 3 AIF Marianne Moller 4 SELEX Giuliano D'Auria 5 ANS_CR Jan Kubicek 8 DSNA Philippe Montebello 9 ENAV Antonio Nuzzo 10 NLR Jürgen Teutsch 11 PAS Alan Gilbert 12 TATM Corinne Heinrich Contractor 13 THAV Marc Fabreguettes 15 AUEB Konstantinos G. Zografos 16 CSL Prague Airport Libor Kurzweil 17 DAS Joachim Bader 18 DFS Klaus-Ruediger Täglich 19 EEC Stéphane Dubuisson 20 ERA Jan Hrabanek 21 ETG (FAV) Thomas Wittig 23 SICTA Mariacarmela Supino 24 TUD Carole Urvoy 25 SOF Lionel Bernard-Peyre Sub-Contractor CSA Karel Muendel Körte Max Körte Customer EC Doris Schroecker Additional EUROCONTROL Bengt Collin 1 Please insert an X, when the PoC of a company receives this document. Do not use the date of issue! Save date: Public Page 2

3 Document Control Sheet 2-SP6 Project Manager Marcus Biella Responsible Author Marcus Biella DLR Additional Authors Joern Jakobi Christian Drege Francois Michel Thomas Ludwig Ronny Friebel Thomas Wittig Subject / Title of Document: Related Task('s): 2-WP6.1, 2-WP6.3 Deliverable No. 2-D6.1.6 Save Date of File: Document Version: 1.00 Reference / File Name 2-D616_VO-TP_V1.0.doc Number of Pages 193 Dissemination Level Public DLR TUD THAV DLR FAV FAV Save date: Public Page 3

4 Change Control List (Change Log) Date Release Changed Items/Chapters Comment Draft: Template provided by DLR Chapter 4 and 5 Input from THAV Chapter Chapter 2 Chapter 6 Input from TUD Input from DLR (derived from 2-D613) Input from FAV (derived from 2-D661a by DLR) Chapter , Additions from TUD Chapter 6 Reworked by FAV No EC comments. Approved Final Version 1.0 Save date: Public Page 4

5 Table of Contents Distribution List... 2 Document Control Sheet... 3 Change Control List (Change Log)... 4 Table of Contents Introduction Document Context Project overview Validation Approach Methodology EMMA 2-WP6.6 Context Document Purpose Scope of Document Acronyms Ground Traffic Display & TAXI-CPDLC Trials (DLR) Validation Aims, Objectives, and Hypotheses Validation Aims Validation Objectives, Hypotheses, Indicators, Measurement Tools The test environment The validation platforms Validation team roles The experimental system TAXI-CPDLC GTD Experimental design Experimental constraints Time schedule Selected Design Scenarios specifications Measurements specification Training requirements Conduct of the study General daily test schedule Detailed Test Schedule (local time) Trials Accomplishment (proposal) Analysis Methods Surface Movement Awareness and Alerting System SMAAS (TUD) Validation Aims, Objectives, and Hypotheses Validation Aims Validation Objectives and Hypotheses Measurements The test environment (TUD) The validation platforms Validation team roles The Baseline System The experimental system: Surface Movement Awareness and Alerting System Experimental Design Experimental constraints Time schedule Selected Design Scenario specifications Training requirements Conduct of the study Save date: Public Page 5

6 3.6.1 Research Simulator Navigation Test Vehicle Analysis Methods Head Up Navigation (THAV) Validation Aims, Objectives, and Hypotheses Validation Aims Validation Objectives Hypotheses, Indicators, Measurement Tools The test environment in Thales Avionics The validation platform Validation team roles The Baseline System The experimental system The experimental system: Description of the HUD/SGS function prototype Navigation information Situation Awareness information Guidance symbol Aircraft parameters Experimental Design Experimental constraints Time schedule Selected Design Scenarios specifications Measurements in RTS Training requirements Conduct of the study Analysis Methods TAXI-CPDLC: Display of taxi clearance (THAV) Validation Aims, Objectives, and Hypotheses Validation Aims Validation Objectives Hypotheses, Indicators, Measurement Tools The test environment in Thales Avionics The validation platform Validation team roles The Baseline System The experimental system Global description of the CPDLC and ground clearance display functions prototype The CPDLC HMI Experimental Design Experimental constraints Time schedule Selected Design Scenarios specifications Measurements in RTS Training requirements Conduct of the study Analysis Methods SMA and ADS-B trials in GA (FAV) Validation Aims The test environment Validation platform Validation team roles The experimental system Save date: Public Page 6

7 6.4 Experimental Design Time Schedule Selected Design...Error! Bookmark not defined. 7 Annex I References List of Figures List of Tables Annex II Biographical Questionnaire (DLR trials) System Usability Scale ASSA Flight Crew Questionnaire I.S.A. Questionnaire EMMA2_QE2-OF_Pilot: Operational Feasibility Questionnaire for Pilots EMMA2_QE2-OI_Pilot: Operational Improvements Questionnaire for Pilots TUD Questionnaires THAV Debriefing Questionnaire for HUD/SGS and Taxi CPDLC evaluation Checklist for EMMA2 (QE2) Save date: Public Page 7

8 1 Introduction This document is positioned within the framework of activities for the European airport Movement Management by A-SMGCS, part 2 (EMMA2) project. Sub-project 2-SP6 deals with the validation activities to be carried out within the project. The first chapter of this document contains a description of the project context and the purpose of this document. 1.1 Document Context Project overview The project is named European Airport Movement Management by A-SMGCS, phase 2 with the acronym EMMA2. The duration of the project is 36 months, as a follow-up of the EMMA project (25 months). The project is organised in six different subprojects, which are co-ordinated by five different partners. There are three ground-related subprojects and one onboard-related subproject. Based on an advanced operational concept, three A-SMGCS (with focus on higher services) will be implemented at three European airports: Prague- Ruzynĕ, Toulouse-Blagnac and Milano-Malpensa. The systems are to be tested operationally (i.e. with live traffic). In addition validation activities are also performed at Paris CDG using the implemented A-SMGCS services. The three ground-related subprojects and the onboard-related subproject are autonomous and independent, but they are inter-linked with the concept and validation subprojects to guarantee that the different systems are based on a common A-SMGCS interoperable air-ground co-operation concept and that all are validated with the same criteria. On-site trials and real time simulations are to ensure the assessment of its operational feasibility and operational improvements Validation Approach During the proposal phase of EMMA2, it was decided to use the Master European Validation Plan (MAEVA) approach to validation as the basis for EMMA2 Verification Activities. With the start of EMMA2 it was decided to use the European - Operational Concept Validation Methodology (E-OCVM, Version 2) [3] for all ground validation activities and for the on-board validation activities for the green team (= DLR, TUD, FAV, DAS). On-board validation activities for the blue team (=AIF, THAV) will follow their own / internal methodology. A correlation between EMMA2 objectives and Airbus objectives is presented in 2- D6.1.1b. Airbus internal assessment methodology is based on Airbus Human Factors know-how and certification guidelines Methodology To develop the specific airborne Validation Test Plans, E-OCVM - which is especially designed for this kind of exercise - is applied. It establishes a uniform framework for the validation of ATM concepts such as A-SMGCS. This Save date: Public Page 8

9 methodology is helpful to provide guidelines along the entire validation process. This methodology allows asking the good questions related to validation and presents concrete examples of applications of the methodology. Its step-by-step approach helps the validation team to address the validation activity in an exhaustive way. The E-OCVM approach consists of six steps (and 24 activities) as outlined below (Table 1-1). It should be noted that E-OCVM only considers validation, not verification. Instead of the E-OCVM terms programme validation objectives (from activity 1.4) and exercise validation objectives (from activity 2.2) the more conventionally used terms High-Level Objectives (HLO) and Low-Level Objectives (LLO) are used in EMMA2. This document is positioned within the framework of activities for the European airport Movement Management by A-SMGCS, part 2 (EMMA2) project. Sub-project 2-SP6 deals with the validation activities to be carried out within the project. Save date: Public Page 9

10 Step Activity Description 2-D6.1.1a & b sections Step 0 State 0.1 Understand the problem 4.1, 2-D6.1.1a Concept and 0.2 Understand the proposed solution(s) 4.2, 2-D6.1.1a Assumptions 1.1 Identify the stakeholders, their needs, and involvement 5.1, 2-D6.1.1a 1.2 Identify the existing information, 5.2, 2-D6.1.1a including current and target levels of maturity 1.3 Describe validation expectations and 5.3, 2-D6.1.1a Step 1 Set outline cases outcomes, products, what Validation will success look like Strategy 1.4 Identify programme validation objectives 5.4, 2-D6.1.1a in key performance areas 1.5 Establish initial validation requirements, 5.5, 2-D6.1.1a and draft validation strategy 1.6 Select validation tools or techniques 5.5, 2-D6.1.1a 1.7 Define validation strategy 5.6, 2-D6.1.1a 2.1 Identify stakeholders acceptance criteria 3.1, 2-D6.1.1b Step 2 Determine the Experimental Needs and performance requirements 2.2 Identify project and exercise validation 3.2, 2-D6.1.1b objectives 3.4, 2-D Refine validation strategy 3.3, 2-D6.1.1b 3.5, 2-D Identify indicators and metrics 3.4, 2-D6.1.1b 3.5; 4.5, 2-D Specify validation scenarios 3.5, 2-D6.1.1b 4.3, 2-D Produce validation exercise plan , 2-D6.1.1b 4.4, 2-D Prepare the platform or facility 3.9, 2-D6.1.1b , 2-D Conduct pre-exercise testing and training 3.10, 2-D6.1.1b 4.4.5, 2-D Conduct validation experiment 4, 2-D6.1.1b 4.6, 2-D6.1.3 Step 3 Conduct the Experiment 3.2 Assess for unexpected effects or behaviours Step 4 Determine the Results Step 5 Information for Dissemination to Stakeholders Table 1-1: 4.1 Perform analysis specified in the analysis plan 4, 2-D6.1.1b Guidance for validation reports in: 5.1, 2-D6.1.1b 4.2 Prepare analysis contributions 5.2, 2-D6.1.1b 4.3 Prepare validation report 5.2, 2-D6.1.1b 5.1 Disseminate information to stakeholders 6.1, 2-D6.1.1b and decision makers 5.2 Draw conclusions and decide on actions 6.2, 2-D6.1.1b feedback to validation strategy. E-OCVM sub-steps and traceability in EMMA2 deliverables EMMA 2-WP6.6 Context 2-WP6.6 of EMMA2 focuses on onboard validation activities for the airborne platforms, established under Sub-Project 2-SP2 of EMMA2. Here, three validation platforms will be used: Demonstration Save date: Public Page 10

11 Real-time simulations, to simulate safety-critical events, validate A-SMGCS procedures, and measure operational improvements in a realistic environment Operational trials, to validate requirements and procedures in the real operational environment and show some potential operational improvements. 1.2 Document Purpose This document is the deliverable 2-D6.1.6, the Validation Test Plan for higher A-SMGCS airborne services. Its purpose is to: 1. Identify the validation aims, objectives and hypotheses for the tests 2. Plan and prepare the validation exercises 3. Provide a high-level description of how the validation activities will be conducted. 1.3 Scope of Document Generally, the document follows the E-OCVM guidelines. Apart from the aims and objectives, it describes the experimental factors that determine the experiment design, the necessary metrics and measurements, the hypotheses that can be accepted or rejected based on the measurements made, and the complete test environment. Furthermore, the scenario specifications are given and requirements for participants and training of participants are determined. Finally, the conduct of the experiments will be described and the envisioned analysis methods will be outlined. The document is divided into eight chapters. Chapter 1 is this introduction. It describes the background, purpose and scope of the document, the document structure and context. Chapter 2 The test plan for Ground Traffic Display & TAXI-CPDLC trials by DLR Chapter 3 The test plan for Surface Movement Awareness and Alerting System trials by TUD Chapter 4 The test plan for Head Up Navigation trials by THAV Chapter 5 The test plan for TAXI-CPDLC (Display of Taxi Clearance) trials by THAV Chapter 6 The test plan for by SMA/TIS-B in GA trials by FAV Chapter 7 is an annex containing lists of references, figures, and tables used in the document. Chapter 8 is an annex containing the questionnaires and test sheets to be used for the validation experiments. 1.4 Acronyms Acronym A/C ACAS ADS-B AIF AIS AMDB ANS_CR ANSP Meaning Aircraft Airborne Collision Avoidance System Automatic Dependent Surveillance Broadcast Airbus France S.A.S. Aeronautical Information Service Airport Map Data Base Air Navigation Services of the Czech Republic Air Navigation Service Provider Save date: Public Page 11

12 Acronym ARINC A-SMGCS ASSA ATC ATCO ATM ATMOS ATN ATS ATSA-SURF ATTAS CAF CAVOK CCD CDD CDG CDTI Meaning Aeronautical Radio Incorporated Advanced Surface Movement Guidance and Control System Airport Surface Situation Awareness Questionnaire Air Traffic Control Air Traffic Controller Air Traffic Management Air Traffic Management and Operations Simulator Aeronautical Telecommunication Network Apron & Tower Simulator Enhanced Traffic Situational Awareness on the Airport Surface Advanced Technology Testing Aircraft System Clearance Awareness Function Clouds and Visibility OK Crew Control Device Clearance Delivery Dispatcher Paris airport Charles De Gaulle Cockpit Display of Traffic Information CM1 Pilot in command 1 CM2 Pilot in command 2 CPDLC CPU Controller-Pilot Data Link Communication Central Processing Unit CRM Crew Resource Management () CSA DAS DCDU DLR DM DMAN EDDF EDFE EFIS EFS EIS EMM Czech Airlines Diehl Aerospace Digital Cockpit Display Unit Deutsches Zentrum für Luft- und Raumfahrt e.v. Downlink Message Departure Management Frankfurt Airport Egelsbach Airport Electronic Flight Instrument System Electronic Flight Strips Electronic Instrument System Electronic Moving Map EMMA2 European Airport Movement Management by A-SMGCS, Part 2 Save date: Public Page 12

13 Acronym ENG E-OCVM F/O FAV FCU FMS GA GEC GECO GPS/IRS GPS/WAAS GTD HF HLO HMI HIS HUD HUD/SGS ICAO IMM ISA ISDN KJFK LCD LFBO LFPG LKPR LLO LVO LVP MCDU ND NOTAM OAF OF Meaning Engine Display European - Operational Concept Validation Methodology First Officer Funkwerk Avionics Flight Control Unit Flight Management System General Aviation Ground Executive Controller Generic Experimental Cockpit Global Positioning System / Inertial Reference System, Global Positioning System / Wide Area Augmentation System Ground Traffic Display Human Factors High-Level Objectives Human Machine Interface Horizontal Situation Indicator Head-Up Display Head Up Display / Surface Guidance System International Civil Aviation Organisation Integrated Moving Map Instantaneous Self Assessment Integrated Services Digital Network New York Kennedy airport Liquid Crystal Display Toulouse Blagnac airport Paris Charles de Gaulle airport Prague airport Low-Level Objectives Low Visibility Operations Low Visibility Procedures Multipurpose Control and Display Unit Navigation Display Notice To Airmen Operational Awareness Function Operational Feasibility Save date: Public Page 13

14 Acronym OI OST PAS PFD QE2 RAD/NAV RP RTS RW SMA SMAAS SP Meaning Operational Improvements On site trials Park Air Systems AS Primary Flight Display Questionnaire for EMMA2 Radio Navigation Route Planner Real-time simulations Runway Surface Movement Alerts Surface Movement Awareness and Alerting System Sub-Project SPOR A-SMGCS Services, Procedures and Operational Requirements (EMMA2 2- D111) SQB STBY SUS TA/RA TAWS TAXI-CPDLC TCP/IP TDS SDK TEC THAV TIS-B TUD TWY UTC UM VFW VHF WILCO WP XPDR squitter beacon Stand By System Usability Scale (Questionnaire) Traffic Advisory/Resolution Advisory Terrain Awareness and Warning System Taxi Controller Pilot Data Link Communication Transmission Control Protocol / Internet Protocol Taxi Driver System Software Development Kit Tower Executive Controller Thales Avionics Traffic Information System Broadcast Technische Universität Darmstadt Taxiway Coordinated Universal Time Uplink Message Vereinigte Flugtechnische Werke Very high frequency Will Command Work-Package Transponder Save date: Public Page 14

15 2 Ground Traffic Display & TAXI-CPDLC Trials (DLR) DLR s activities in 2-WP6.6 on-board validation go in parallel with 2-WP6.3 trials. To have a complete overview of these activities a full description for both ground and on-board trials can be found in the document 2-D6.1.3 as well. Two validation platforms will be used real-time simulations (RTS), and on site trials (OST), to validate requirements and procedures in the real operational environment and show some potential operational improvements. The series of tests starts with real-time simulations at DLR-Braunschweig s Generic Experimental Cockpit (GECO), concentrating on Ground Traffic Display (GTD), and on-board Taxi Controller Pilot Data Link Communication (TAXI CPDLC). Here the whole system will be validated with and without the higher A-SMGCS services mentioned above by measuring operational feasibility and operational improvements (focusing on safety and Human Factors issues). It should be borne in mind that even though simulation re-creates a realistic environment, and permits the safe testing of safety-critical events and repetitive testing of rare events, at the end operational trials must be performed to prove the operational feasibility of new standards and procedures. Therefore the real-time simulations mentioned above are a preparatory step for the on site trials at Prague-Ruzynĕ Airport as well, when the test aircraft ATTAS is aimed at validating the higher A- SMGCS services under real operational conditions. A human in the loop real-time simulation involves the participation of pilots performing their operational tasks in a realistic environment. Two cockpit positions (CM1, CM2) will be simulated in RTS in order to test the hypotheses related to EMMA2 evaluation objectives. The 2-WP6.3 & 2-WP6.6 experiments aim at evaluating: Services / Functions User Provider To be tested in trials TAXI-CPDLC Pilots / ATCO DLR, PAS, Airtel ATN 2 RTS (a.k.a. GTD Pilots DLR RTS2 EFS ATCO PAS RTS1 for 2- OST DMAN ATCO DLR RTS1 WP6.3 trials) RP ATCO PAS The EMMA2 experiment objective is to perform an evaluation of the advanced functionalities / higher A-SMGCS services. The validation aims are broken down and further analysed in section PAS provides the Ground Application (EFS), Airtel ATN the communication stack Save date: Public Page 15

16 2.1 Validation Aims, Objectives, and Hypotheses Validation Aims The purpose of this section is to clarify what has to be achieved from the EMMA2 validation exercises in 2-WP6.6. It contains the general aims in accordance with 2-D6.1.1a&b Validation Test Plan and Generic Experimental and Test Plan [4]. These documents are in accordance with the European Operational Concept Validation Methodology (E-OCVM) [3]. The basic aim of the EMMA2 SP6 is the validation of higher A-SMGCS services as described in the ICAO Manual [4] and further refined in the 2-D1.1.1 A-SMGCS Services, Procedures and Operational Requirements (SPOR) [5]. EMMA 2-WP6.3 aims to validate the higher A-SMGCS services at Prague-Ruzynĕ airport; EMMA 2-WP6.6 aims to validate the higher A-SMGCS on-board services. Two areas of validation activities have been considered: The Operational Feasibility area refers to the definition of the operational use of equipment and procedures, in accordance with the performances assessed in the previous stage. It answers the question: Given the performances of the equipment, is it usable and acceptable? The Operational Improvements area refers to the evaluation of the operational improvements, in terms of Safety, and Human Factors using the equipment and the procedures defined in the previous stage. It answers the question: Given the accepted A-SMGCS equipment and procedures, how is ATM improved? Validation Objectives, Hypotheses, Indicators, Measurement Tools In general, it can be expected that the validation exercises will demonstrate the Operational Feasibility of the ATM operational concept and that the concept provides a solution to the specific ATM problem and leads to Operational Improvements when comparing it to current lower A-SMGCS services, both for airports and for the airborne side, and for different airport operating conditions. RTS RTS will focus on the operational feasibility of higher A-SMGCS services. RTS-platforms serve as a perfect validation platform to evoke safety critical events and to tune the system alerts to the needs of the pilots. In addition to this main goal operational improvements in terms of safety and human factor issues gains shall be proved. Also for this purpose the RTS is a well-suitable means. OST On-site validation activities will concentrate on the measurement of the technical performance and showing the operational feasibility of the whole system. Measuring operational improvements in the field are very difficult or even impossible. Frequently users and the system are not certified to use it fully operational. Furthermore, a valid baseline with ceteris paribus condition compared to the experimental condition (with A-SMGCS) does not exist at all. Weather, traffic amount, runway in use, crews, etc. change permanently and improvement effects of the A-SMGCS are shadowed then. However, in the field it has to be shown that the overall system meets the technical performance and operational requirements. When this can be proven, operational improvements, which are measured in the RTS, can be transferred to the real environment Validation Objectives Validation activities for higher onboard A_SMGCS services base on the EMMA2 SPOR document Save date: Public Page 16

17 [5]. This document describes the services of an A-SMGCS including their potential procedures and operational requirements. The ICAO Manual [4] does not define procedures and levels of implementation for higher onboard A_SMGCS services. The procedures that will be used to test the operational feasibility of are the ones defined in the SPOR [5] High-Level validation objectives Operational Feasibility These operational feasibility tests aim at assessing the user s acceptance of the EMMA2 operational procedures and the fulfilment of requirements [6]. It is expected, at the end of this stage, that the operational significance of the system is confirmed, for each set of visibility conditions, using defined procedures derived from EMMA2 deliverable SPOR [5]. This will support the promotion of adapted procedures for the use of the higher A-SMGCS services Operational Improvement To be fully validated the system must show that new procedures contribute to an operational improvement. There are two areas of interest to measure these operational improvements in on-board validation trials: Safety, Behaviour and Working Performance, and referring to the corresponding sections in the ICAO manual on A-SMGCS [4] and the SPOR [5] Low-Level validation objectives Since it is not possible to assess the above-stated HLOs directly, it is required to decompose the objectives into measurable indicators. Thus, for each category of HLOs, it is necessary to identify indicators that can be measured either objectively or subjectively. Low-Level objectives (LLOs) provide the decomposition of a high-level objective into a set of indicators that can be measured using a known technique. In 2-D6.1.1b Generic Experimental and Test Plan [6] both High and Low-Level Objectives (HLO, resp. LLO) for the validation areas have been identified. On-board relevant objectives are presented in table 2-1. Area High-Level Objectives Low-Level Objective Operational Feasibility Operational Improvements Verification of EMMA2 Operational Requirements and Procedures [HLO1] Increase of Safety [HLO2] Suitability of Behaviour and Working Performance Fulfilment of EMMA2 Operational Requirements and Procedures [LLO_1.1] Faster identification of safety hazards [LLO2.1] Increment of safety perception by users [LLO2.2] Appropriate level of user s workload [LLO4.1] Save date: Public Page 17

18 Area High-Level Objectives Low-Level Objective [HLO4] Improved situational awareness of users [LLO4.2] Less human errors [LLO4.3] Table 2-1: High and Low-Level Validation Objectives in DLR s EMMA2 on-board trials Hypotheses Hypotheses are derived either from HLOs or LLOs. They are formulated in such a way that they can be tested statistically. The indicators used in order to assess the operational improvement may differ for the various levels of A-SMGCS implementation. Furthermore, they may also differ for the various validation platforms (used in RTS vs. OST). A complete list of types of indicators used for the various high-level objectives for EMMA2 is described in the 2-D6.1.1b [6] Operational Feasibility Hypotheses RTS & OST To assess whether a system is operationally feasible, the pilots opinion plays a central role here. If the user expresses his/her acceptance to a service, a new procedure or a single system element and underlines their usability, it can be assumed that the operational feasibility is very high and evidentially proofed. Therefore the hypotheses in the area of Operational Feasibility are stated as following: Identifier OF-H0 OF-H1 Hypothesis The users opinion does not agree to the operational feasibility aspects of a specific item. The users opinion agrees to the operational feasibility aspects of a specific item. Each questionnaire item is tested against the null hypothesis that pilot answers neither agree nor disagree to a questionnaire item. When this null-hypothesis can be rejected with an error probability of 0.05 it is statistically proven that a questionnaire item is true or not true. With this result the connected operational requirement or procedure can easily be verified. Following result pattern are conceivable: Technical tests (performed in 2-SP2-5) and operational feasibility tests (performed in 2-SP6) verify an operational requirement. A SPOR requirement cannot be technically verified due to insufficient system performance (output of technical tests) but operators confirm the operational feasibility of it, then it can be assumed that the requirements could be relaxed. If the requirements cannot be satisfied due to insufficient system performance, and the operators feel that indeed the system has poor operational significance, the relevant higher A- SMGCS has to be improved Operational Improvement Hypotheses RTS For each of the hypotheses listed in this section, H1 refers to the alternative hypothesis, which will be Save date: Public Page 18

19 accepted if the H0 hypothesis is rejected Safety-related Hypotheses In order to examine the safety, subjective measurements will be conducted. The following hypotheses will be tested: S1-H0 S1-H1 Hypothesis An increment of safety while using [higher services of A-SMGCS] is not perceived by the users. An increment of safety while using [higher services of A-SMGCS] is perceived by the users. S2-H0 S2-H1 Hypothesis Situation awareness of users is not improved while using [higher services of A-SMGCS]. Situation awareness of users is improved while using [higher services of A-SMGCS]. Regarding safety, the expected result is that the self assessed safety is higher with the introduction of higher A-SMGCS services and related procedures compared with the baseline system. Regarding situation awareness, the expected result is that the self assessed situation awareness is improved with the introduction of higher A-SMGCS services and related procedures compared with the baseline system. Therefore an increase in safety can be expected Behaviour and Working Performance-related Hypotheses In order to examine the behaviour and working performance, subjective measurements will be conducted. The following hypotheses will be tested: HF-H0 HF-H1 Hypothesis An increment of workload while using [higher services of A-SMGCS] is perceived by the users. An increment of workload while using [higher services of A-SMGCS] is not perceived by the users. The expected result is that the self assessed workload is lower with the introduction of higher A-SMGCS services and related procedures compared with the baseline system. In addition to that situation awareness will be tackled in this section as well (cf. S-2H0 / S-2H1) Indicators and Measurement Tools Table 2-2 provides a synthesis of the measurements that are envisaged to be taken during the A- SMGCS evaluations describing the high- and low-level objectives, the indicators, and the used measurement tools. Main measurement tools are Subjective measurements The questionnaire for EMMA2 (QE2) for pilots Save date: Public Page 19

20 o with subscale QE-OF dealing with operational feasibility and o subscale QE-OI dealing with operational improvements It consists of items that refer to five main subjects: general, surveillance, routing / planning, TAXI CPDLC and HMI. The Id refers to the item number within the questionnaire. All items are derived from the operational requirements of the SPOR [5]. The complete questionnaire can be found in the annex. In addition to that, standard questionnaire SUS will be used to measure usability. To measure operators workload, instantaneous post measurements are considered. ISA has already been applied in lots of studies in the past. To measure operators situation awareness, questionnaires ASSA, and ISA will be applied. They have already been used in lots of studies in the past. Area HLO LLO Indicator Metric Operational Feasibility 68 items of the QE-OF Questionnaire [IND_1.1.1] Operational Improvements Verification of EMMA2 Operational Requirements and Procedures [HLO_1] Increase of Safety [HLO_2] Suitability of Behaviour and Working Performance [HLO_4] Fulfilment of EMMA2 Operational Requirements and Procedures [LLO_1.1] Situational awareness according to the operator will be preserved or improved [LLO_2.2] Appropriate level of user s workload [LLO_4.1] 10 items of the SUS [IND_1.1.2] Mid-run questionnaire I.S.A. for Situation Awareness [IND_2.2.1] Post-trial self assessed Situation Awareness [IND_2.2.2] Mid-run questionnaire I.S.A. for workload [IND_4.1.1] 6 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_ ] 5 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_ ] Mid-run self assessed situational awareness (after specific events) -11 point Likert scale - ANOVA of mean per test run are compared [M_ ] Questionnaire ASSA (15 items) a 5 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_ ] Mid-run self assessed workload (after specific events) -5 point Likert scale - ANOVA of mean per test run are compared [M_ ] Save date: Public Page 20

21 Area HLO LLO Indicator Metric Improved situational awareness [LLO_4.2] Mid-run questionnaire I.S.A. for Situation Awareness [IND_4.2.1] Mid-run self assessed situational awareness (after specific events) -11 point Likert scale - ANOVA of mean per test run are compared Less human errors [LLO_4.3] Post-trial self assessed Situation Awareness [IND_4.2.2] Self assessment of human errors [IND_4.3.1] [M_ ] Questionnaire ASSA (15 items) a 5 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_ ] Debriefing [M_ ] Table 2-2: High-level objectives, low-level objectives, indicators and metrics Save date: Public Page 21

22 2.2 The test environment The validation platforms Generic Experimental Cockpit (GECO) at DLR-Braunschweig RTS DLR s Generic Experimental Cockpit (GECO) is designed as a fixed-base flight simulator based on the Airbus A 320 architecture using the flight dynamics of the VFW614 (in accordance to DLR s test aircraft ATTAS) in a fly by wire version. It is capable for testing and evaluating new on-board systems giving a maximum of flexibility. Figure 2-1: DLR s Generic Experimental Cockpit (GECO) The simulator is equipped with a collimated outside view and widespread hardware equipment: Two seated-flightdeck including Primary Flight Display (PFD) und Navigation Display (NAV) (providing HSI during in flight and EMM during taxiing) at both sides for CM1 and CM2; and Engine Display (ENG) Several components of the Flight Management System (FMS; with RAD/NAV page in use during EMMA2 trials) including Flight Control Unit (FCU) and Multipurpose Control and Display Unit (MCDU) HMI-Interfaces (e.g. trackball, touchpad, touchscreen) Head-Up Display (HUD) High-Resolution Airport Environment Save date: Public Page 22

23 Figure 2-2: PFD, NAV, ENG and FCU in GECO The GECO is part of an integrated network; it can be used in co-operation with other simulation platforms e.g. Apron & Tower Simulator (ATS) or Air Traffic Management and Operations Simulator (ATMOS). Further equipment can be connected via ISDN or internet. The communication between the GECO and ATS is realised via the TCP/IP protocol. Experimental conditions and weather parameters will be coordinated and controlled by the instructor. Save date: Public Page 23

24 Figure 2-3: Architecture of Cockpit & Tower Simulation, Pseudo Pilots and DMAN Save date: Public Page 24

25 Advanced Technologies Testing Aircraft System (ATTAS) OST The Advanced Technologies Testing Aircraft System (ATTAS) aircraft is a VFW 614 twin-engine jet transport aircraft modified for research purposes (see Figure 2-4). It has been equipped with a full flyby-wire system and LCD displays for the test pilot. Additional equipment for EMMA2 was installed in shape of an ADS-B / TIS-B receiver as well as a VLD-2 data link hardware (by SELEX). The pilot display was driven by an EIS graphic card. For this purpose hardware equipment by Diehl Aerospace was installed into the aircraft. For the pilot interface for VLD-2 the CDTI by Funkwerk Avionics (cf. chapter ) was installed in the center pedestal of the ATTAS. Figure 2-4: ATTAS VFW614 during EMMA2 on-site trials in PRG ATTAS is equipped with EMM, Ground traffic display (fed by TIS-B), Traffic Conflict Detection (TCD), and TAXI-CPDLC (CDTI + Graphical presentation on the EMM), ADS-B out/in. The DLR HMI is installed on the Diehl display computer and connected via a so-called data pool to the aircraft experimental system. This experimental system provides the HMI with the relevant information for displaying the taxi guidance system (for instance aircraft state vector). Received data messages via VDL-2 were also distributed via the data pool to the pilot HMI (for generating a green line) as wells as to the interface system CDTI. The traffic information was available via data pool and the type of traffic information (TIS-B resp. ADS-B) was selectable during the exercise. Save date: Public Page 25

26 Figure 2-5: Test pilots side in ATTAS VFW614 during EMMA2 on-site trials in PRG An additional walkie-talkie was provided by Airport Prague. Figure 2-5 shows the test pilots side of the cockpit with the displays. The EMMA2 taxi trials will be carried out from the front cockpit. The whole experiments can be controlled from an operator station in the cabin. All parameters were recorded Validation team roles RTS Following roles have been identified: - Pilots: drive and position GECO at defined places and speeds on the movement area. - Pseudo-Controller: as approach ATCO; furthermore responsible for revising deliberately TAXI-CPDLC clearances. - Observers: experienced in on-board operations, observe and note down the pilots behaviour in special situations. Furthermore: - Simulation Supervisor: responsible for briefing/debriefing of the pseudo-pilots for special behaviour during the simulation runs; correct start of the whole simulation system; balancing pseudo-pilot workload during the exercises by distributing aircraft responsibility. - Experiment Supervisor: defines, organises and conducts the validation tests; ensures that all results are recorded. - System Administrator: ensures correct operation of the A-SMGCS test-bed equipment during the tests and that malfunctioning items of equipment are replaced expediently; ensures that the A-SMGCS equipment is appropriately configured for each test, which may include Save date: Public Page 26

27 degrading some parts or simulating meteorological conditions; has in-depth knowledge of computer and network technology and is experienced with the A-SMGCS test-bed set-up. - Technical Support Engineer: responsible for repairing all deficiencies and/or defects identified during testing; ensures that all electronic data are recorded. OST The ATTAS will take part in the on-site trials. Each trial will be conducted by a crew. 3 DLR test pilots and 1 CSA captain are selected for the trials. 2.3 The experimental system Following tools will be used and tested during the validation phases: - GTD (developed by German Aerospace Center, DLR) - TAXI-CPDLC (developed by German Aerospace Center, DLR, Park Air Systems, PAS, and Airtel ATN) TAXI-CPDLC Description of functions The TAXI-CPDLC service provides the reception of pilot clearance requests and the transmission of clearances via data link to the respective aircraft. The TAXI-CPDLC service will support only non time-critical clearances, i.e. start-up, pushback, taxi and handover. Time-critical clearances like crossing, line-up and take-off are not supported. Correspondingly the service will be available for CDD and GEC. The TEC operates via voice communication as before. The TAXI-CPDLC service is operated by the controller via the EFS. A green CPDLC marker at a flight strip indicates that the respective aircraft will send clearance requests and receive clearances via data link. When the pilot of an aircraft requests a clearance via data link, the clearance button on the respective flight strip will highlight. When the controller clicks the clearance button at the flight strip at the time when the clearance shall be issued, the clearance will be automatically transmitted to the aircraft via data link and the button turns to blue. The Taxi Route to be transmitted can be seen in the EFS routing field. Voice communication via R/T is always the back-up. Voice communication shall be used if any additional information exchange is necessary or in case of safety- or time critical situations. As input device the FAV s product CDTI-2000 will be used with additional features. The CDTI-2000 is a multi-functional display system that visualises various information. Its main purpose is to enhance the crew's situational awareness. For this purpose, interfaces to several external sensors and other avionics equipment are integrated. The data received is transformed into various display formats and shown to the pilot via an integrated Liquid Crystal Display (LCD). The following sensor information can be fed into the system: - GPS data such as position, speed, altitude, time, and satellite status - Altitude data from encoding altimeters - Traffic Information from TIS-B or ADS-B equipment - Communication Messages from VHF or Satcom transponder equipment Save date: Public Page 27

28 The sensor information can be combined with map and terrain data stored inside the CDTI-2000 to show the received sensor data in a more comprehensible form to the pilot. This includes functions such as a Moving Map mode, where the position is shown in relation to a digital map or a digital terrain database. Figure 2-6: CDTI by Funkwerk Avionics For the purpose in EMMA2 the CDTI was reprogrammed to be used as CPDLC interface for the pilot. The following menus are available: 1. Send menu: Figure 2-7: Send menu on CDTI 2. Reply menu: Save date: Public Page 28

29 Figure 2-8: Reply menu on CDTI 3. Log menu: Figure 2-9: Log menu on CDTI Taxi route clearances received via data link will be graphically presented on the ND. Different modes can be displayed (North-up, track-up, arc-mode, perspective mode) via the mode selector in the EFIS. Different zoom stages can be selected via the range selector of the EFIS. The taxi routes will presented by a different coloured line in the middle of the expected or suggested and cleared taxiway by the controller. The colour depends on the status of the clearance. At the start of an outbound leg the preliminary taxi route will be given by the CDD via data link. At the start of an inbound leg (a/c on final) a preliminary taxi route will also be given but by the TEC. This route will be displayed with the help of a magenta dashed line in the middle of the preliminary taxiway. In both cases this line will turn into complete yellow when the ground controller will approve this preliminary route. After accepting this route by PNF s WILCO (via the CDTI) the line will turn into a complete green line. After touch down the TEC has two different procedures depending on crossing a RWY: For an inbound taxi route including a RWY crossing the pilot will receive the cleared taxi route by the TEC via voice and after the a/c leaves the TEC s responsibility the PNF has Save date: Public Page 29

30 to change to the ground frequency and will get the ground taxi route after making an initial call. For an inbound taxi route not including a RWY crossing TEC will instruct the pilot to call ground to receive the cleared taxi route. In both cases the communication between pilot and tower controller will only be via voice. In the first case the magenta dashed line will appear before crossing the RWY (TEC is not equipped with TAXI-CPDLC in RTS2). After crossing the runway and calling ground (initial call) a data link massage will appear and the PNF has to answer the WILCO function of the CDTI. Before pushing this button the proposed taxi route will be displayed in a yellow drawn line. After PNF s acceptance by pushing WILCO the line will be changed into a green one. Now this route is the accepted (by the pilot) and cleared (by the controller) taxi route. In the second case after frequency change to ground and after initial call to the ground frequency a data link message will appear and this suggested taxi route (now in a full yellow line) can be accepted by the PNF. If he will accept via the WILCO of the CDTI the yellow line will turn into green. If he won t comply he has to inform the ground controller via voice. Figure Figure 2-15 show TAXI-CPDLC clearances on the EMM. Save date: Public Page 30

31 Figure 2-10: TAXI-CPDLC expected clearance for TWY H (NAV mode) Figure 2-11: TAXI-CPDLC expected clearance for TWY H, A, and RWY 24 (PLAN mode) Save date: Public Page 31

32 Figure 2-12: TAXI-CPDLC clearance for TWY H, A approved by ATCO (NAV mode) Figure 2-13: TAXI-CPDLC clearance for TWY H, A approved by ATCO (PLAN mode) Save date: Public Page 32

33 Figure 2-14: TAXI-CPDLC clearance for TWY H, A after PNF s wilco (NAV mode) Figure 2-15: TAXI-CPDLC clearance for TWY H, A after PNF s wilco (PLAN mode) Save date: Public Page 33

34 Standard TAXI-CPDLC: Procedures Outbound Given below is a TAXI-CPDLC example for an outbound flight for RWY06. The aircraft is parked at apron north (Gate 34) RWY06 has been assigned for departure The scheduled taxi route will be H F E, with crossing the RWY 13/31 in between A-SMGCS provides expected route information by TAXI-CPDLC on pilot s request, after the departure clearance has been issued CDD issues start up-clearance via TAXI-CPDLC CDD transfers the strip to GEC, and the system transmits the CONTACT message by TAXI- CPDLC PNF contacts GEC by voice GEC issues pushback clearance by TAXI-CPDLC GEC issues taxi clearance up to RWY 13/31 by TAXI-CPDLC GEC transfers the strip to TEC, and the system transmits the CONTACT message by TAXI- CPDLC. PNF contacts TEC by voice TEC issues crossing RWY 13/31 clearance, taxi to RWY 06 clearance, line-up and take-off clearance by voice Table 2-3 shows the communication between ATCO and PNF in detail. (Both GECO Staff columns can be neglected here as they refer only to the scenarios which will be used in RTS2.) DM Pilot non flying GECO staff UM ATC GECO staff Taxi Route Information: Aircraft Status: Parking at Apron North, Stand 34, Departure Clearance received, before Start-up A-SMGCS: TAXI-CPDLC 132 REQUEST TAXI ROUTING INFORMATION 243 EXPECT ROUTING TO HOLDINGPOINT E RWY 06 VIA TWY H F E Send expected Routing to GECO 3 ROGER Start-Up: Aircraft Status: Ready for Start-up CDD: 25 REQUEST START-UP CLEARANCE 239 START-UP APPROVED 0 WILCO Save date: Public Page 34

35 DM Pilot non flying GECO staff UM ATC GECO staff Handover to GEC: Aircraft Status: Performing or having performed Start-up CDD: 117 CONTACT LKPR GROUND WILCO Push-Back Aircraft Status: Initial call to GEC made by voice, ready for pushback GEC: 25 REQUEST PUSHBACK CLEARANCE 239 PUSHBACK APPROVED 0 WILCO Taxi Clearance: Aircraft Status: Pushback finished, ready for taxiing GEC: Note: The Taxi Clearance is a multielement clearance transmitted in one go. 25 REQUEST TAXI CLEARANCE Send Request to ATS TAXI TO HOLDINGPOINT E RWY 06 VIA TWY H F HOLD SHORT OF RWY 13 NEXT EXPECT VIA TWY F E 0 WILCO Handover to TEC: Aircraft Status: Reaching RWY 13/31 GEC: 117 CONTACT LKPR TOWER WILCO Crossing of Runway 13/31, Taxiing to Runway 06, Line-up and Take-off: Aircraft Status: Initial call to TEC made by voice, TEC: voice Request crossing RWY 13/31 Cleared to cross RWY 13/31 and proceed to holding point RWY 06 via Foxtrott and Echo Cleared to cross RWY 13/31 and Save date: Public Page 35

36 DM Pilot non flying GECO staff UM ATC GECO staff proceed to holding point RWY06 via Foxtrott and Echo Ready for Departure on Runway 06 Lining-Up Runway 06 Cleared for take- Off Runway 06 Handover to DEP: (optionally) Aircraft Status: Climb-out CONTACT DEPARTURE Line-Up Runway 06 Cleared for Take- Off Runway 06 A-SMGCS: voice CONTACT DEPARTURE Table 2-3: TAXI-CPDLC standard procedures for an outbound flight Standard TAXI-CPDLC: Procedures Inbound The aircraft approaches Prague Airport on RWY 06 The assigned stand is on apron north (Gate 34) The scheduled taxi route will be L G A-SMGCS provides expected route information by TAXI-CPDLC before final approach. TEC issues landing clearance by voice. TEC issues handover to GEC by voice. GEC issues taxi clearance by TAXI-CPDLC Table 2-4 shows the communication between ATCO and PNF in detail. (Both GECO Staff columns can be neglected here as they refer only to the scenarios which will be used in RTS2.) Save date: Public Page 36

37 DM Pilot non flying GECO staff UM ATC GECO staff Taxi Route Information: Aircraft Status: Before Final Approach for RWY REQUEST TAXI ROUTING INFORMATION 3 ROGER Landing clearance: Aircraft Status: Approaching RWY 24 TOWER, ILS RWY 06 established, request landing clearance Cleared to land RWY 06 Handover to GEC: Aircraft Status: After landing on RWY 06 Contact Ground on 121,900 Taxi Clearance: Aircraft Status: Initial call to GEC made by voice. 0 WILCO GROUND, on your frequency Leaving Frequency (optionally via data link): Aircraft Status: Parked on stand. 136 ON-BLOCKED A-SMGCS: 243 EXPECT ROUTING TO STAND 34 VIA TWY L G 241 TEC: voice Wind knots, cleared to land RWY 06 TEC: voice Contact LKPR Ground on 121,900 GEC: Note: Follow data link TAXI TO STAND 34 VIA TAXIWAY L G GEC: No acknowledgement by GEC via TAXI- CPDLC required Send expected Routing to GECO GEC will issue taxi clearance without pilot s request. Save date: Public Page 37

38 Table 2-4: TAXI-CPDLC standard procedures for an inbound flight Advanced TAXI-CPDLC: Procedures Outbound The aircraft is parked at Apron North (Gate 34). Runway 24 has been assigned for departure The scheduled taxi route will be H Z A-SMGCS provides expected route information by TAXI-CPDLC on pilot s request, after the departure clearance has been issued CDD issues start up-clearance via TAXI-CPDLC CDD transfers the strip to GEC, and the system transmits the CONTACT message by TAXI-CPDLC. PNF contacts GEC by voice. GEC issues pushback clearance by TAXI-CPDLC GEC decides for a different taxi route via H and A and selects this route at the routing function. GEC issues revised taxi clearance by TAXI-CPDLC. GEC transfers the strip to TEC, and the system transmits the CONTACT message by TAXI-CPDLC. Pilot contacts TEC by voice TEC issues line-up and take-off clearance by voice Table 2-5 shows the communication between ATCO and PNF in detail. (Both GECO Staff columns can be neglected here as they refer only to the scenarios which will be used in RTS2.) DM Pilot non flying GECO staff UM ATC GECO staff Taxi Route Information: Aircraft Status: Parking at Apron North, Stand 34, Departure Clearance received, before Start-up A-SMGCS: TAXI-CPDLC 132 REQUEST TAXI ROUTING INFORMATION 243 EXPECT ROUTING TO HOLDINGPOINT Z RWY 24 VIA TWY H Z Send expected Routing to GECO 3 ROGER Save date: Public Page 38

39 DM Pilot non flying GECO staff UM ATC GECO staff Start-Up: Aircraft Status: Ready for Start-up CDD: 25 REQUEST START-UP CLEARANCE 239 START-UP APPROVED 0 WILCO Handover to GEC: Aircraft Status: Performing or having performed Start-up CDD: 117 CONTACT LKPR GROUND WILCO Push-Back Aircraft Status: Initial call to GEC made by voice, ready for pushback GEC: 25 REQUEST PUSHBACK CLEARANCE 239 PUSHBACK APPROVED 0 WILCO Taxi Clearance: Aircraft Status: Pushback finished, ready for taxiing GEC: Note: The Taxi Clearance is a multielement clearance transmitted in one go. 25 REQUEST TAXI CLEARANCE 253 REVISE TAXI TO HOLDINGPOINT A RWY 24 VIA TWY H A 0 WILCO Handover to TEC: Aircraft Status: Reaching RWY 24 Holding Point A GEC: 117 CONTACT LKPR TOWER WILCO Line-up and Take-off: Aircraft Status: Initial call to TEC made by voice, TEC: voice Ready for Departure on Runway 24 Save date: Public Page 39

40 DM Pilot non flying GECO staff UM ATC GECO staff Lining-Up Runway 24 Cleared for take- Off Runway 24 Handover to DEP: (optionally) Aircraft Status: Climb-out CONTACT DEPARTURE Line-Up Runway 24 Cleared for Take- Off Runway 24 A-SMGCS: voice CONTACT DEPARTURE Table 2-5: TAXI-CPDLC advanced procedures for an outbound flight Advanced TAXI-CPDLC: Procedures Inbound The aircraft approaches Prague Airport on RWY 24 The assigned stand is on apron north (Gate 31). The scheduled taxi route will be D F G with crossing of RWY 13/31 in between A-SMGCS provides expected route information by TAXI-CPDLC before final approach TEC issues landing clearance by voice TEC issues taxi clearance, crossing-rwy13/31-clearance and handover to GEC by voice The assigned stand is changed from Gate 31 to Gate 34 and thus a different taxi route via F G has to be cleared. GEC issues revised taxi clearance by TAXI-CPDLC Table 2-6 shows the communication between ATCO and PNF in detail. (Both GECO Staff columns can be neglected here as they refer only to the scenarios which will be used in RTS2.) Save date: Public Page 40

41 DM Pilot non flying GECO staff UM ATC GECO staff Taxi Route Information: Aircraft Status: Before Final Approach for RWY REQUEST TAXI ROUTING INFORMATION 3 ROGER Landing clearance: Aircraft Status: Approaching RWY 24 TOWER, ILS RWY 24 established, request landing clearance Cleared to land RWY 24 Taxi Clearance up to RWY 13/31, Crossing of RWY 13/31 and Handover to GEC: Aircraft Status: After landing on RWY 24. Taxi to RWY 13/31 via TWY D F cleared to cross RWY 13/31 and contact Ground on 121,900 after crossing Taxi Clearance: Aircraft Status: RWY 13/31 crossed, initial call to GEC made by voice. GROUND, on your frequency A-SMGCS: 243 EXPECT ROUTING TO STAND 31 VIA TWY D F G TEC: voice Wind knots, cleared to land RWY 24 TEC: voice Taxi to RWY 13/31 via TWY D F cross RWY 13/31 and contact Ground on 121,900 after crossing GEC: Note: Follow data link Send expected Routing to GECO Assigned stand on flight strip has been changed from 31 to 34. GEC will issue taxi clearance without pilot s request. Save date: Public Page REVISETAXI TO STAND 34 VIA TAXIWAY F G

42 DM Pilot non flying GECO staff UM ATC GECO staff 0 WILCO Leaving Frequency (optionally via data link): Aircraft Status: Parked on stand. 136 ON-BLOCKED GEC: No acknowledgement by GEC via TAXI- CPDLC required Table 2-6: TAXI-CPDLC advanced procedures for an inbound flight Additions description of function for on-site trials OST During on-site trials, the TAXI-CPDLC service is operated via the EFS in the EMMA2 test bed and via the CDTI in the ATTTAS cockpit. A green LOG ON marker at a flight strip indicates that the respective aircraft is logged on to communicate via TAXI-CPDLC. Voice communication via R/T is always the back-up. Voice communication shall be used if any additional information exchange is necessary or in case of safety- or time critical situations. The EMMA2 TAXI-CPDLC implementation does not include the delivery of the departure clearance. The ATC button is to be pressed but so far no message is sent. The departure clearance has to be issued by voice via R/T as nowadays. It is aimed to have the ATTAS on stand 17 in the south apron, performing an aerodrome circling, landing and taxiing back to its final parking position Procedures for On-site trials In this chapter the communication between ATCOs and pilots is shown in detail. Voice communication is written in italic, TAXI-CPDLC in capital letters. As the runway constellation can not be predicted for the four days, two scenarios have been prepared: one for RWY24, one for RWY06. Outbound 24 Pilot by voice Pilot by TAXI CPDLC Regular ATCO by voice EMMA2 ATCO by TAXI-CPDLC CDD Position Guten Tag Prague Delivery, D-ADAM is ready for startup. Cleared to, following data link REQUEST START-UP CLEARANCE Guten Tag D- ADAM, you are cleared to destination, stand by for start up and following data link Save date: Public Page 42

43 Outbound 24 GEC Position ready for taxiing Pilot by voice Guten Tag Prague Ground, D-ADAM is on your frequency. Following data link, D-ADAM. Pilot by TAXI CPDLC WILCO WILCO REQUEST TAXI CLEARANCE WILCO Aircraft Status: Reaching RWY 24 on TWY Alpha TEC Position Ruzyne Tower, D- ADAM on your frequency. WILCO Regular ATCO by voice Guten Tag D- ADAM, follow data link. Guten Tag D- ADAM, D-ADAM. All the remaining control is performed by the regular TEC using voice EMMA2 ATCO by TAXI-CPDLC START-UP APPROVED CONTACT LKPR GROUND TAXI TO HOLDINGPOINT A RWY 24 VIA TWY P L H A CONTACT LKPR TOWER Table 2-7: TAXI-CPDLC procedures for an outbound 24 flight during on-site trials After that D-ADAM is taking off, performs an aerodrome cycling, and lands again on RWY24. Inbound 24 Pilot by voice Pilot by TAXI CPDLC Regular ATCO by voice EMMA2 ATCO by TAXI-CPDLC TEC Position D-ADAM, Guten Tag Tower Ruzyne, on Final Runway 24. Continue approach, D-ADAM Guten Tag D- ADAM, continue approach. Save date: Public Page 43

44 Inbound 24 Pilot by voice Pilot by TAXI CPDLC Regular ATCO by voice EMMA2 ATCO by TAXI-CPDLC D-ADAM, cleared to land on Runway 24, wind calm Cleared to land runway 24, D- ADAM Aircraft Status: D-ADAM landed and vacated via D, cleared to cross RWY13/31 and handed over to Ground GEC Position Guten Tag Prague Ground, D-ADAM is on your frequency. Following data link, D-ADAM. REQUEST TAXI WILCO Guten Tag D- ADAM, follow data link. TAXI TO STAND S17 VIA TWY F L P Table 2-8: TAXI-CPDLC procedures for an inbound 24 flight during on-site trials Outbound 06 Pilot by voice Pilot by TAXI CPDLC Regular ATCO by voice EMMA2 ATCO by TAXI-CPDLC CDD Position Guten Tag Prague Delivery, D-ADAM is ready for startup. Cleared to, following data link GEC Position Aircraft Status: ready for taxiing Guten Tag Prague Ground, D-ADAM is on your frequency. REQUEST START-UP CLEARANCE WILCO WILCO REQUEST TAXI CLEARANCE Guten Tag D- ADAM, you are cleared to destination, stand by for start up and following data link Guten Tag D- ADAM, follow data START-UP APPROVED CONTACT LKPR GROUND Save date: Public Page 44

45 Outbound 06 Pilot by voice Following data link, D-ADAM. WILCO Aircraft Status: Reaching RWY 13/31 on TWY F TEC Position Ruzyne Tower, D- ADAM on your frequency. Pilot by TAXI CPDLC WILCO Regular ATCO by voice link. Guten Tag D- ADAM, D-ADAM. All the remaining control is performed by the regular TEC using voice EMMA2 ATCO by TAXI-CPDLC TAXI TO HOLDINGPOINT F RWY 06 VIA TWY P L F HOLD SHORT OF RWY 13 NEXT EXPECT VIA TWY F CONTACT LKPR TOWER Table 2-9: TAXI-CPDLC procedures for an outbound 06 flight during on-site trials After that D-ADAM is taking off, performs an aerodrome cycling, and lands again on RWY06. Inbound 06 Pilot by voice Pilot by TAXI CPDLC Regular ATCO by voice EMMA2 ATCO by TAXI-CPDLC TEC Position D-ADAM, cleared to land on Runway 06, wind calm Cleared to land runway 06, D- ADAM Aircraft Status: D-ADAM landed and vacated via L, handed over to Ground GEC Position Guten Tag Prague Ground, D-ADAM is on your frequency. Following data link, D-ADAM. REQUEST TAXI WILCO Guten Tag D- ADAM, follow data link. TAXI TO STAND S17 VIA TWY L P Table 2-10: TAXI-CPDLC procedures for an inbound 06 flight during on-site trials Save date: Public Page 45

46 Working positions equipment On-board equipment The features of the experimental system are: Electronic Moving Map With additional service: Ground Traffic Display o simulated TIS-B airport and ADS-B/TIS-B equipped aircraft With additional service: TAXI-CPDLC o Routing Information (displays the Taxi route with different colours depending on the status) o Input device: the CDTI provided by FAV as substitute for a DCDU (Digital Cockpit Display Unit) for CPDLC o CPDLC Communication via ATN Protocol Figure 2-16: On-board Equipment EMM and CDTI (DLR Real Time Simulation) GTD The Ground Traffic Display provides the flight crew with surrounding traffic information (e.g. aircraft and vehicles) in addition to the EMM on the NAV. The traffic information is displayed on the navigation display in the cockpit Description of functions RTS2 Save date: Public Page 46

47 TIS-B data will be simulated in RTS2 with a frequency of 10Hz. Labels are therefore detectable and are linked to the relevant aircraft. OST TIS-B data will be updated with a frequency of 1Hz Procedures Departure Flights The display of traffic on the apron, manoeuvring area and runways will have considerable relevance to flight operations. Especially during LVO a much more accurate, relevant and consistent picture will be provided to individual cockpit crews by visual analysing the actual traffic at the airport. Normal Visibility Conditions CM1 and CM2 will crosscheck any conflict information obtained by voice communication, Ground Traffic Display function and/or own visual observation. The CM1 and CM2 will include the present traffic situation in their pre-departure briefing. Low Visibility Conditions Limited vision to identify potential conflicts will be counterbalanced with the electronic picture: shadow objects and aircraft/vehicles will be visible here. The impact on safety and to some lesser extent to efficiency will be tremendous; conflict avoidance alerting is expected to be a further step. Relevant Operations: Proposed procedures might be as follows: On the gate / parking position: CM2 select transponder STBY Before pushback, engine start or taxi: CM2 select transponder XPDR (and AUTO if available) CM1 and 2 each for himself assesses traffic situation CM1 make operational decision if any CM2 assesses traffic near/behind the aircraft as shown on the EMM and when relevant callout such traffic Taxiing out: Monitor the surround traffic as shown on the EMM and if relevant callout (CM2). Especially the capability to monitor traffic behind the aircraft when pushing back is considered as particularly relevant from a pilot perspective. Lining up: CM2 or CM1 select transponder TA/RA Arrival Flights Normal Visibility Conditions It is assumed that the Ground Traffic Display will be available in flight before the approach (It could be part of the approach check list). With such service the flight crew will be able to better plan and coordinate the taxiing task. Low Visibility Conditions Equivalent to departure flight (see above) Save date: Public Page 47

48 Relevant Operations After Landing: when the runway is vacated, CM2 select transponder XPDR (and AUTO is available) Taxiing in: CM2 monitor the surrounding traffic as shown on the EMM and if relevant callout such traffic At the gate / parking positions: CM2 select transponder STBY Pseudo-controller position equipment RTS The Pseudo-controller position consists of: Simulation Traffic 20 display with 1600x1200 resolution Keyboard and mouse Voice communication capability for all simulated frequencies Operational environment Pilots positions RTS During each validation trial both cockpit positions CM1 and CM2 will be simulated simultaneously. Subjects will change between the different roles in order to increase the statistical power of results. OST During each validation trial both cockpit positions CM1 and CM2 will be used simultaneously. Subjects will change between the different roles in order to increase the power of feedback in the debriefing Technical environment RTS The validation exercises will use the A-SMGCS test-bed components at the Apron & Tower Simulator at DLR-Braunschweig, established under 2-SP3 of the EMMA2 Project, and the on-board A-SMGCS components, established under 2-SP2 (green team) of the EMMA2 Project, and the Generic Experimental Cockpit at DLR-Braunschweig. OST The validation exercises will use the A-SMGCS test-bed components at Prague-Ruzynĕ airport, established under 2-SP3 of the EMMA2 Project, and. the on-board A-SMGCS components, established under 2-SP2 (green team) of the EMMA2 Project, equipped in DLR s test aircraft ATTAS Facilities RTS Following simulation platforms will be used: Save date: Public Page 48

49 - GECO (DLR Facilities Braunschweig, Avionic Hall Building 117) o With Pseudo ATCO located behind GECO - ATS (DLR Facilities Braunschweig, Building 104) o Including supervisor located next to ATS o With additional Pseudo Pilots Room (Building 117) OST - Test Vehicles o DLR ATTAS, VFW614, callsign D-ADAM, 18./19. Nov 2008 & 27. Nov 2008 o TUD Navigation Test Vehicle (only 18./19. Nov 2008) o Airport Prague Van (27. Nov 2008) - Airport Prague o ATTAS parking position during the trials will be S17 on APRON SOUTH (see Figure 2-17) o Tower test bed beneath the actual Visual Control Room Figure 2-17: ATTAS parking position during the OST: S Experimental design Experimental constraints RTS It is important to assess experimental constraints for the real-time simulations, in order to consider their impact on the evaluation objectives. Three main experimental constraints have been identified in the scope of the real time simulations. An overview of these constraints are providing in the following. The first constraint is the limited sample size. There will be only eight pilots providing measurements for the test design. This will have a significant impact on the internal validity of the results. Small effects that result from systematic variation of the treatment variables are hard to detect and to prove. Secondly, the DLR-Braunschweig RTS platform used for the real-time simulations replicates as far as possible the real environment, but some differences remain. It is important to keep these differences between simulated and real environment in mind (e.g. the use of a fixed-base flight simulator) when the interpretation of results is done. External validity, that describes the amount of ability to generalise Save date: Public Page 49

50 the present results to the real world, might be impaired. Thirdly, the real time simulation will not cover all possible cases of operational scenarios. Indeed, the Prague taxiing scenarios are representative, but an exhaustive approach cannot be conducted on the basis of this environment alone. Lastly, due to the dynamic character of the scenario, non-nominal events (e.g. conflict situations) cannot be guaranteed to be identical from test run to test run or even cannot be guaranteed to occur at all. In the test report, such events have to be mentioned and the test results have to be interpreted referring to such deviations Time schedule RTS - April 2007 pre-tests with GECO and ATS - May 2008 start of integration at ATS - May 2008 set up of scenarios Training / Briefing and Start Test Runs RTS2, Pt Training / Briefing and Start Test Runs RTS2, Pt.2 OST Trials were conducted & 19. November & 27. November 2008 (= 4 days of testing) Selected Design RTS The experimental design is based on the use of real experiments. In real experiments, the same scenarios are used for the Baseline System and the A-SMGCS set-up in order to achieve ceteris paribus conditions. In this way, results from the Baseline system can be directly compared with the A-SMGCS within a traffic scenario. For TAXI-CPDLC different scenarios will be used. Therefore no direct experimental comparison can be made. Nevertheless comparisons will be drawn and considered as well and the results are used to check the operational feasibility initially since the maturity level was not to be expected to be V3 (in accordance to E-OCVM Error! Reference source not found., and to give first hints (!) for operational improvements. The independent variable (IV) for on-board activities is therefore defined as follows: [1] IV SYS the system version (EMM vs. higher A-SMGCS services) with four levels of the treatment A-SMGCS Services : Level 1 Baseline (EMM), Level 2 GTD, (EMM + GTD) [Level 3 TAXI-CPDLC standard (EMM + GTD + standard TAXI-CPDLC)] [Level 4 TAXI-CPDLC advanced (EMM + GTD + advanced TAXI-CPDLC)] The independent variable is a treatment factor that is operated by the experimenter and is supposed to cause expected effects. It is described and argued in the table 3-1. Save date: Public Page 50

51 The second independent variable is the role of the pilot in the cockpit. He will act in half of the scenarios as CM1; these scenarios are later repeated to have more statistical power, and then he will act as CM2 (and vice versa for the other half of the scenarios). [2] IV ROLE subjects role with two levels (CM1, resp. CM2). Independent Variables (IV) IV SYS Description System Version= Technical System varied by different functions and their provided services Levels This variable has four different technical system versions corresponding to an A-SMGCS (= EMM) and A-SMGCS higher services: Level 1 EMM (baseline) Level 2 EMM + GTD Level 3 EMM + GTD + standard TAXI-CPDLC Level 4 EMM + GTD + advanced TAXI-CPDLC IV ROLE Role of Subject This variable has two different levels. Pilots will act as: CM1 CM2 Table 2-11: Independent variable and levels in RTS Simulation exercises plan RTS SYS 1 EMM SYS 2 EMM + GTD SYS 3 EMM + GTD + TAXI-CPDLC standard SYS 4 EMM + GTD + TAXI-CPDLC advanced CM CM Table 2-12: Combination of Experimental Factors Eight pilots will be organised in four crews that will perform eight test runs each. The simulation trials go with a complete within-subject experimental design. Each of the eight pilots will have to perform eight test runs (one trial per cell in the table above; resp. four trials as CM1, four trials as CM2) Scenarios specifications RTS A description of the scenarios for the GECO is listed in table 2-13 and EXE04 will be a standard scenario for TAXI-CPDLC; EXE02 will be an advanced TAXI-CPDLC scenario. Different versions of EXE06 will be a baseline resp. GTD scenarios. Save date: Public Page 51

52 Start Gate 34 EXE 02 EXE 04 EXE 06 13NM before RWY24 Gate 34 13NM before RWY06 Gate 34 13NM before RWY24 Gate S6 Destination RWY24 Gate 34 RWY06 Gate 34 RWY24 Gate S6 RWY24 Taxi Route Visibility Wind Time of day TAXI- CPDLC (standard) TAXI- CPDLC (advanced) H B Expected taxi-out route TWY Z revised to TWY A D F G H F E CAT I Calm L G H A D F L P Day Day Day n.a. Expected taxi-in route revised after crossing by receiving actual taxi clearance granted by the GEC: preliminary Gate 31 revised to Gate 34 Expected Taxi Route (congruent to actual clearance) Duration ~ 20 ~ 20 ~ 60 n.a. n.a. n.a. P L H A Table 2-13: Scenario Description Overview RTS (Part 1, June 2008) Save date: Public Page 52

53 Start Gate 18 EXE 02 EXE 04 EXE 06 13NM before RWY24 Gate 18 13NM before RWY06 Gate 55 13NM before RWY24 Gate S6 Destination RWY24 Gate 34 RWY06 Gate 34 RWY24 Gate S6 RWY24 Taxi Route Visibility Wind Time of day TAXI- CPDLC (standard) TAXI- CPDLC (advanced) H B Expected taxi-out route TWY Z revised to TWY A n.a. D F G Expected taxi-in route revised after crossing by receiving actual taxi clearance granted by the GEC: preliminary Stand 34 revised to Gate 18 H F E CAT I Calm Day L G Expected Taxi Route (congruent to actual clearance) H1 H A Duration ~ 20 ~ 20 ~ 60 n.a. D F L P n.a. n.a. R L H A Table 2-14: Scenario Description Overview RTS (Part 2, October 2008) Taxi route are depicted in red in Figure 2-18 Figure 2-24; preliminary taxi routes are coloured in blue. Save date: Public Page 53

54 Figure 2-18: Taxi Route in EXE02, Pt. 1 outbound (Gate 34 RWY24) Figure 2-19: Taxi Route in EXE02, Pt. 2 inbound (RWY24 Gate 34) Save date: Public Page 54

55 Figure 2-20: Taxi Route in EXE04, Pt. 1 outbound (Gate 34 RWY06) Figure 2-21: Taxi Route in EXE04, Pt. 2 inbound (RWY06 Gate 34) Save date: Public Page 55

56 Figure 2-22: Taxi Route in EXE06, Pt. 1 outbound (Gate 34 RWY24) Figure 2-23: Taxi Route in EXE06, Pt. 2 inbound (RWY24 Gate S6) Save date: Public Page 56

57 Figure 2-24: Taxi Route in EXE06, Pt. 3 outbound (Gate S6 RWY24) The GECO will appear as an Airbus A320 with a DLH-callsign in the ATS. Callsigns will change from trial to trial to make the GECO less extraordinary for the ATCOs (see table 2-15). Exercise Callsign Action SID/STAR ETA/ETD Stand EXE02 DLH72C Departure MEDOV 1A 00:10 34 EXE02 DLH73C Arrival on final 00:28 34 EXE04 DLH471 Departure TABEM 2D 00:12 34 EXE04 DLH472 Arrival on final 00:28 34 EXE06_01 DLH621 Departure MEDOV 1A 00:11 34 EXE06_01 DLH622 Arrival on final 00:28 S6 EXE06_01 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_02 DLH621 Departure MEDOV 1A 00:10 34 EXE06_02 DLH622 Arrival on final 00:28 S6 EXE06_02 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_03 DLH621 Departure MEDOV 1A 00:10 34 EXE06_03 DLH622 Arrival on final 00:28 S6 EXE06_03 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_04 DLH621 Departure MEDOV 1A 00:10 34 EXE06_04 DLH622 Arrival on final 00:28 S6 EXE06_04 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_05 DLH621 Departure MEDOV 1A 00:10 34 EXE06_05 DLH622 Arrival on final 00:28 S6 Save date: Public Page 57

58 EXE06_05 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_06 DLH621 Departure MEDOV 1A 00:10 34 EXE06_06 DLH622 Arrival on final 00:28 S6 EXE06_06 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_07 DLH621 Departure MEDOV 1A 00:10 34 EXE06_07 DLH622 Arrival On final 00:28 S6 EXE06_07 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_08 DLH621 Departure MEDOV 1A 00:10 34 EXE06_08 DLH622 Arrival On final 00:28 S6 EXE06_08 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_09 DLH621 Departure MEDOV 1A 00:10 34 EXE06_08 DLH622 Arrival On final 00:28 S6 EXE06_09 DLH623 Departure MEDOV 1A 01:04 S6 Table 2-15: List of GECO call signs for every test run Measurements specification The following sections describe how the indicators outlined in chapter will be assessed. Data are collected from two sources: firstly, data is automatically logged during the simulation exercises; secondly, data is gathered from the participating subjects (ATCOs, pilots). The latter category can be further broken down into data obtained during the exercise (observational data, subjective ratings), and data obtained after the exercise (subjective questionnaires, debriefing interviews) System Recordings A complete system recording of all sessions is available for the on-board data Subjective Data Mid-run measurements ISA Measurements RTS Pilots workload and situation awareness is measured twice in each segment of a scenario (e.g twice during an inbound segment) after certain events (e.g. shortly after crossing a RWY) Post-run measurements RTS; OST EMMA2 QE-OF & QE-OI The tailor-made questionnaires created for the use in EMMA2. Relevant scales of the debriefing Save date: Public Page 58

59 questionnaire Questionnaire for EMMA2 (QE2) will be completed after the last test run. Addressed are issues of general operational requirements, acceptance and usability of new HMI and new procedures. The complete questionnaire for pilots can be found in the Annex. This data will be analysed at a statistical level ASSA The Airport Surface Situation Awareness questionnaire will be completed after the last test run by the pilots. The questionnaire can be found in Annex II. It consists of twelve questions SUS The standard usability scale is completed after the last test run by the pilots Debriefing RTS; OST Debriefing Questionnaires Relevant scales of the debriefing questionnaire Questionnaire for EMMA2 (QE2) will be completed after the last test run. Addressed are issues of general operational requirements, acceptance and usability of new HMI and new procedures. QE2 can be found in Annex A. This data will be analysed at a statistical level Debriefing Interviews Relevant scales of the debriefing questionnaire Questionnaire for EMMA2 (QE2) will be completed. 2.5 Training requirements RTS The week before starting of the simulation is reserved for pre-experiments. Two active ANS_CR controllers, all pseudo-pilots, DLR test pilots, technicians, and the complete validation test team will test the complete simulation exercise. This week is used to tune the simulation system and the used traffic scenarios, and to train all involved participants: Pseudo-pilots, interacting with the controllers and responsible for initialising of all movements (aircraft, vehicles, tows) in the simulation traffic scenarios, are trained three weeks in advance. The training starts with a familiarisation of the overall airport environment (arrival/departure routes, taxiways, gates/remote stands, etc.) and the training of standard phraseologies. In a second phase, the pre-experiments, those pseudo-pilots are trained in a complete real time simulation scenario using typical Prague-Ruzynĕ airport traffic scenarios, procedures, and real ANS_CR controllers. The complete training will take two weeks in total. The validation test team, including observers and test coordinators, will also use the week of preexperiments, getting trained with the course of typical test runs, test tools (questionnaires, observation sheets), briefing and debriefing. OST Three DLR pilots are trained by both by the DLR engineers and human factor team members. The additional CSA pilot has already deep knowledge from the previous real-time simulations one month prior to the on-site trials. Save date: Public Page 59

60 2.6 Conduct of the study General daily test schedule between 0900 and 1500 ATTAS and TUD van try to find a low traffic period to perform TIS- B and TCD trials o those trials are independent on the EFS/DMAN test bed trials 2 ATCOs per day are needed from 1030 until 1630 o EFS/DMAN tests DMAN trials should be conducted in time of a day with departure peak traffic departure peak traffic is generally: to adapt to the ATCOs daily availability the DMAN trials could also be shifted to meet the morning or afternoon peaks o DMAN/EFS Debriefing o Lunch o TAXI-CPDLC from EMMA2 test bed with ATTAS (case study) o Debriefing ATCOs and Pilots (2 DLR Pilots + 1 CSA Pilot) Detailed Test Schedule (local time) Pilots Tu, We, Th, TCD/TIS-B CPDLC TCD/TIS-B CPDLC TCD/TIS-B CPDLC DLR #1 x x x (x) x x DLR #2 x x x x DLR #3 x x CSA #1 x x Table 2-16: Pilots Test Schedule Trials Accomplishment (proposal) ATTAS / TUD Van test Ground Traffic Display (GTD via TIS-B) and TCD (between 1,5 hour in between of ) o In a very low or zero traffic scenario, the ATTAS and the TUD van causing conflict situation to check the operational feasibility of GTD (via TIS-B) and TCD o Debriefing with Pilots o details in the 2-D616 TAXI-CPDLC trials ( ) o the ATTAS is controlled by voice via the regular ATCOs o in parallel the ATTAS communicates via TAXI-CPDLC with the EMMA2 ATCOs in the test bed the respective clearances / messages o Three EMMA2 ATCOs would operate the EFS in the test bed o Procedure would be, as tested in simulation (see below) o ATTAS performs an outbound / inbound scenario (2 aerodrome circling, ca. 80min) (4 IFR flight plans needed 1 hour before (two outbound, two inbound)) Save date: Public Page 60

61 o Operational feasibility in the field can be answered after the trials by ATCOs and Pilots (45min) o responsibilities: During the Tests, the regular GEC/CDD controllers have the ATTAS (D- ADAM) under control, but the actual ATCO (CDD/GEC) would only perform the initial call with the ATTAS pilots, are briefed about the silent START-UP, PUSHBACK, and TAXI clearance given by TAXI-CPDLC from the EMMA2 ATCOs, stay in contact with EMMA2 ATCOs, monitor the control of the ATTAS and intervene by R/T in case of critical situations. 2.7 Analysis Methods Questionnaire results will be statistically analysed by non-parametric binominal tests. Mid-run self assessed situational awareness will be stastically analysed by analysis of variance. Results of the debriefing will be reported in EMMA2 deliverable 2-D661 Airborne Validation Results Part A. Save date: Public Page 61

62 ATS GECO Validation Area Traffic Scenario Day No. Treatment TEC GEC CDD CM1 CM2 Treatment Mo, 6/23 Tu, 6/24 We, 6/25 1 Training (EFS/DMAN) Training Training EXE04 RWY06 2 Training(TAXI-CPDLC) Training TAXI-CPDLC Training EXE02 RWY24 3 Training(TAXI-CPDLC) TAXI-CPDLC standard Op. Feasibility EXE04 RWY06 4 Baseline Baseline Op. Improvement EXE06 1 RWY24 5 EFS GTD Op. Improvement EXE06 5 RWY24 6 Baseline Baseline Op. Improvement EXE06 2 RWY24 7 EFS GTD Op. Improvement EXE06 6 RWY24 8 TAXI-CPDLC + Routing TAXI-CPDLC standard 9 TAXI-CPDLC + Routing TAXI-CPDLC revised expected taxi-out: H Z H A 10 TAXI-CPDLC + Routing TAXI-CPDLC revised actual taxi-out H Z H A, taxi-in Op. Feasibility Op. Feasibility Op. Feasibility EXE04 RWY06 EXE02 RWY24 EXE02 RWY24 11 Without Tower Simulation 3 4 Training Training EXE04 RWY06 12 Baseline Training Op. Improvement EXE06 3 RWY24 13 EFS Baseline Op. Improvement EXE06 4 RWY24 14 EFS+DMAN Baseline Op. Improvement EXE06 7 RWY24 15 EFS+DMAN GTD Op. Improvement EXE06 8 RWY24 16 EFS+DMAN GTD Op. Improvement EXE06 9 RWY24 Save date: Public Page 62

63 ATS GECO Validation Area Traffic Scenario Day No. Treatment TEC GEC CDD CM1 CM2 Treatment Th, 6/26 Mo, 10/13 Tu, 10/14 We, 10/15 17 TAXI-CPDLC + Routing TAXI-CPDLC standard 18 TAXI-CPDLC + Routing TAXI-CPDLC standard 19 TAXI-CPDLC + Routing TAXI-CPDLC rev. exp. taxi-out: H Z H A 20 TAXI-CPDLC + Routing TAXI-CPDLC revised actual taxi-out H Z H A, taxi-in Op. Feasibility Op. Feasibility Op. Feasibility Op. Feasibility EXE04 RWY06 EXE04 RWY06 EXE02 RWY24 EXE02 RWY24 21 Training (EFS/DMAN) Training Training EXE04 RWY06 22 Training(TAXI-CPDLC) Training TAXI-CPDLC Training EXE02 RWY24 23 Training(TAXI-CPDLC) TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 24 Baseline Baseline Op. Improvement EXE06 1 RWY24 25 EFS GTD Op. Improvement EXE06 5 RWY24 26 Baseline Baseline Op. Improvement EXE06 2 RWY24 27 EFS GTD Op. Improvement EXE06 6 RWY24 28 TAXI-CPDLC + Routing TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 29 TAXI-CPDLC + Routing TAXI-CPDLC revised exp. taxi-out route: G A H A 30 TAXI-CPDLC + Routing TAXI-CPDLC revis. actual taxi-out H1 H Z A H1 H A, in Op. Feasibility Op. Feasibility EXE02 RWY24 EXE02 RWY24 31 Without Tower Simulation 7 8 Training Training EXE04 RWY06 32 Baseline Training Op. Improvement EXE06 3 RWY24 Save date: Public Page 63

64 ATS GECO Validation Area Traffic Scenario Day No. Treatment TEC GEC CDD CM1 CM2 Treatment Th, 10/16 33 EFS Baseline Op. Improvement EXE06 4 RWY24 34 EFS+DMAN Baseline Op. Improvement EXE06 7 RWY24 35 EFS+DMAN GTD Op. Improvement EXE06 8 RWY24 36 EFS+DMAN GTD Op. Improvement EXE06 9 RWY24 37 TAXI-CPDLC + Routing TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 38 TAXI-CPDLC + Routing TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 39 TAXI-CPDLC + Routing TAXI-CPDLC revised expected taxi-out route: H1 H Z A H1 H A 40 TAXI-CPDLC + Routing TAXI-CPDLC revised actual taxi-out H Z A H A, taxi-in TAXI-CPDLC + Routing TAXI-CPDLC revised actual taxi-out H Z H A, in Op. Feasibility Op. Feasibility Op. Feasibility EXE02 RWY24 EXE02 RWY24 EXE04 RWY06 Table 2-17: Sequence of Simulator Test Runs in DLR RTS Save date: Public Page 64

65 3 Surface Movement Awareness and Alerting System SMAAS (TUD) 3.1 Validation Aims, Objectives, and Hypotheses Validation Aims The overall aim of the SMAAS validation is to assess its potential to prevent and mitigate hazardous situations, incidents and accidents, particularly Runway Incursions, in the airport environment, and whether SMAAS thus contributes to enhanced safety Validation Objectives and Hypotheses Developing new aircraft systems requires to deal with several questions according to the following general objectives that should be fulfilled to guarantee a good system. These general objectives, which can be defined as general validation objectives, are: Operational Feasibility o Human Machine Interface o User acceptability Operational Improvements o Safety o Behaviour and Working Performance Workload Situation awareness Most of the above stated general objectives are extremely difficult to assess in a validation exercise. It is almost impossible to identify metrics that would allow a direct assessment of safety. Therefore it is required to decompose the general objectives into measurable indicators. These indicators can be measured objectively or subjectively, by defining hypothesises, with the known techniques General Objectives Introducing a new display system in the cockpit with all of its information could seduce the pilots to increase their head down time, due to the overload of information presented by the system while the pilots needs undue time to recognize the important information. Objective O1: Verify that the introduction of SMAAS does not tend to result in more Head down time for the pilots. Hypothesis H1-1: The SMAAS does not cause excessive extra head down time for the crew. With the introduction of the SMAA System in the cockpit displaying most of the important information for the crew directly on the ND, the crew could start trusting the system too much. Mainly the display of traffic information is dependent on the availability of corresponding equipment in the other aircraft (ADS-B) or on the airport (TIS-B). The crew must be aware at all times that the SMAAS only gives out the information available to the cockpit and therefore not trust the system completely. Objective O2: Assess the confidence the pilots have in the SMAAS. Hypothesis H2-1: The crew maintains a good level of confidence in the System the crew trusts what it sees on the display but stays aware of the limits of the System. Save date: Public Page 65

66 The increase in situation awareness comes along with the increase in safety, because crew failures will be minimized. To ensure that the SMAAS increases situation awareness with its presentation of surface restrictions as well the associated alerts, Objective 3 was defined. Objective O3: Verify that the presentation of surface restrictions as well as alerts in addition with the Airport Moving Map will increase situation awareness. Hypothesis H3-1: Presenting closed runways and other pertinent runway or airport restrictions from NOTAM or similar sources is an essential feature to increase situational awareness on an airport moving map. The alerting features associated are operationally relevant. Similarly, to ensure that the SMAAS increases situation awareness with its presentation of traffic information as well as the associated alerts the Traffic Conflict Detection function, Objective 4 was defined. Objective O4: Verify that the presentation of traffic information as well as the associated alerts in addition with the Airport Moving Map will increase situation awareness. Hypothesis H4-1: Presenting traffic information is an essential feature to increase situational awareness on an airport moving map. The alerting features associated are operationally relevant. Objective 5 was defined to verify that the overall design principles that should be used in the developing of aircraft systems will be adhered. Objective O5: Verify that the introduction of SMAAS follows the overall design principles used in the cockpit. Hypothesis H5-1: The HMI is consistent with existing standards/implementations. Taxiing along the airport belongs to the phases with the most workload for the pilots. Introducing a new system that will increase their workload in this phase is not desirable. Because it is unlikely to develop an information system that will have no effect on the workload it is desirable to develop a system with just a small increase of workload. Objective O6: Verify that the introduction of SMAAS does not lead to any undue increase of workload. Hypothesis H6-1: The introduction SMAAS does not lead to any undue increase of workload Ground to Air Database Upload Objectives Any evaluation of the Ground-to-Air database Upload function is very complex, since the function touches several stakeholders, such as AIS and data integrators, airlines and their ground personnel as well as flight crews. From a 2-SP2 perspective, the focus should be on the flight crew and airline operations, i.e. the potential role of the dispatcher. The key issue is how to integrate Ground-to-Air Database Upload in the airline workflow, i.e. from the moment the data arrive from the AIS/ANSP or data integrator, via the compilation of the epib and the upload of the information to the aircraft to the data handling and data representation on the flight deck. There are at least three levels that have to be addressed in the evaluation. The first level is the operational concept level. We need to establish whether the operational concept proposed is acceptable to the stakeholders, whether it covers all essential aspects and whether it can be integrated into the airline workflow without major problems. The second level is related to the first, but focuses on the flight deck and includes human factors aspects. It studies the impact of the function on the flight crew, ranging from workload aspects concerning the handling of and interaction with the data by the flight crew to HMI aspects such as an appropriate representation of the data. The third level concerns technical feasibility, using criteria such as available bandwidth, upload times and DO-200 compatibility of processes and formats. The objectives are therefore as follows: Onboard Human Factors: MO1: Verify that the introduction of the Ground-to-Air Database Upload function into the cockpit does not lead to any undue increase of workload in critical flight phases Save date: Public Page 66

67 MO2: Assess the usability and appropriateness of the crew interface to the data provided by Ground-to-Air Database Upload function MO3: Assess the HMI used to represent the data provided by the Ground-to-Air Database Upload for usability, intuitiveness and situational awareness Objective MO1: Verify that the introduction of the Ground-to-Air Database Upload function into the cockpit does not lead to any undue increase of workload in critical flight phases Hypothesis 1-1: The Ground-to-Air Database Upload function does not cause excessive extra workload for the crew. The difference in perceived workload between a baseline scenario without the Ground-to-Air Database Upload function and a scenario with the new function will be assessed by directly asking the flight crews about their impression in a questionnaire. Objective MO2: Assess the usability and appropriateness of the crew interface to the data provided by Ground-to-Air Database Upload function Hypothesis 2-1: The use of the MCDU for the interface to upload, review and amend information where necessary is acceptable. This hypothesis concerning the use of the MCDU for the crew interface will be assessed by a questionnaire. Hypothesis 2-2: Having the possibility to manually enter short-term changes from the initial information is necessary. The necessity of being able to manually enter short-term changes will be assessed by comparing the Ground-to-Air Database Upload function without this possibility, and with this possibility. In both cases, a short-term change will take place after the complete upload of the database. To this effect, a post-run interview will be used, e.g. Which runway is now closed?, along with a questionnaire. Objective MO3: Assess the HMI used to represent the data provided by the Ground-to-Air Database Upload for usability, intuitiveness and situational awareness Hypothesis 3-1: The HMI is consistent with existing standards/implementations. This hypothesis on the consistency of the HMI with existing standards/implementations will be assessed by directly asking pilots about their impression in a questionnaire. Hypothesis 3-2: Presenting closed runways and other pertinent runway or airport restrictions from NOTAM or similar sources is an essential feature to increase situational awareness on an airport moving map. The necessity of presenting closed runways and other restrictions on the airport moving map can be assessed by comparing the airport moving map only (baseline) with the airport moving map presenting the data from the Ground-to-Air Database Upload function. To this effect, a post-run interview will be used, e.g. Which runways were closed in the scenario?, along with a questionnaire Measurements The measurement tools used in SMAAS experimentations are: Observations Debriefing Questionnaire Debriefing Interview After simulator trials and field tests, users will have the opportunity to reflect about their experience gained in the trials. The test assessment crew makes sure that these data will be collected and analysed. Data is either obtained during the exercise or after the exercise. During the scenarios the pilots comments are documented by one of the engineers. After the exercise, data are collected by means of a debriefing questionnaire and a debriefing interview. Save date: Public Page 67

68 Debriefing Questionnaire The debriefing questionnaires are listed in Annex II Debriefing Interview The aim is to go deeply into the key points noticed in the observations or the questionnaire. It allows bringing out the operational improvement of the system through different human factor subject: - System usability - Confidence in the system Situation awareness 3.2 The test environment (TUD) The validation platforms During the trials two validation platforms should be used, to assess the Surface Movement Awareness and Alerting System. The first platform is the fixed base research simulator at TUD and the second one is TUD s Navigation Test Vehicle Fixed Base Research Simulator The fixed-base Research Flight Simulator of TUD s Institute of Flight Systems and Automatic Control is a highly modular and configurable research simulator, featuring a sophisticated collimated visual system consisting of a three-channel retro-projection with a viewing frustum of 180 horizontally and ±20 vertically. Pilots perceive the resulting image, which seems to be located at infinity, through a mirror. Consequently, refocusing occurs whenever pilots change their view from head-down activities inside the cockpit to the outside visual, and this approach guarantees an excellent approximation of reality for human factors evaluations []. The cockpit is not an exact reproduction of the flight deck of a specific aircraft, but deliberately kept at the more generic level of a modern glass cockpit with two flight crew members. The inside dimensions of the cockpit correspond to those of a modern widebody aircraft of the Airbus A330/340 family. While the flight simulation employs the flight mechanical model of an Airbus A300 B4, a flyby-wire flight control system with sidesticks was chosen because it represents the state of the art and allows an unobstructed view of the flight guidance displays. The cockpit features all primary and secondary controls; in some cases actual aircraft parts, such as an original A320 Flight Control Unit (FCU) and genuine thrust levers, contribute to enhance the level of immersion and realism. In other cases, as for the MCDU, parts obtained from a commercial supplier of flight simulation equipment have been used, and even some developments of the Institute were installed. As an example, the simulator features active side-sticks, which can also be operated in a standard mode. Flight guidance and system displays are presented on large 15 LCD screens, which can either, be arranged in portrait or landscape mode. Cockpit Configuration for the experiment The First Officer (CM-2) crew station will be used for evaluation, mainly because this position is already equipped with a MCDU hardware mock-up. The two LCD displays for PFD and ND are arranged in portrait configuration, thus resembling the A380 display arrangement. The outer screen is used to display a standard Airbus style PFD. The inner screen will feature a conventional Airbus style ND supplemented by the Airport Moving Map and additional HMI features as described subsequently. Display control will be achieved via the conventional EFIS Control Panel, which features a workaround for the ranges below 10 nm: When selecting the constraints (CSTR) button, pilots can use the QNH selector to control AMM range Navigation Test vehicle TUD s Navigation Test Vehicle, a Volkswagen LT 28 van, is operated by the Institute of Flight Systems and Automatic Control and was originally conceived as a mobile test platform for the development and validation of navigation sensors and equipment. It features an array of high-quality Save date: Public Page 68

69 sensors suited for precise navigation and a measurement data acquisition and logging system. A postprocessing software allows to generate highly precise reference trajectories with a CEP95 of 0,44 m based on sensor logs. Figure 3-1: TUD Van during tests on closed RWY04/22 on Prague airport, February 6th, 2006 Furthermore, as a by-product, the high-precision navigation solution also provides the prerequisites for the verification and validation of equipment used on aircraft, particularly in the domain of cockpit displays for surface movement, either in the form of software prototypes or real hardware. Consequently, TUD s Navigation Test Vehicle can be used to simulate a taxiing aircraft and enables low-cost live field tests of such onboard functions in a real airport environment, and thus an assessment of the potential impact of factors such as database and navigation accuracy (e.g. IRS drift and GPS outages or multi-path effects) or the quality of data link date (e.g. ADS-B) Navigation System A Honeywell H764 inertial reference system forms the core of the van s precision navigation system, which is supplemented by an array of various GPS receivers ranging from off-the-shelf to high accuracy measurement models. The sensor mounting positions were measured within centimetre accuracy to correct errors caused by different sensor locations. An overview to the sensor equipment is given in Table 3-1. A Wiener filter is used for the implementation of an optimum sensor data fusion combining inputs from the H764 and a Novatel RT20 GPS receiver with D-GPS capabilities. To provide differential correction data to the onboard receivers, a portable reference station can be installed at points of which the exact position is known. The range of the D-GPS broadcasts depends on the surrounding environment and can vary between several dozens and some hundred meters, provided there is a free line of sight between rover- and reference station. As a backup, D-GPS can be obtained from the ALF service, which is available within 600 km of Frankfurt/Main, Germany. For the EMMA2 trials, Save date: Public Page 69

70 Figure 3-2: Antenna mounting plate on the roof of TUD's Navigation Test Vehicle however, no D-GPS correction data were used, because the shakedown trials had shown that a sufficient real-time navigation precision in the order of a few meters could be achieved with GPS only Equipment In the back of TUD s Navigation Test Vehicle, one row of seats has been removed to make room for two custom-built, interconnected racks that hold the hardware and computer equipment. Two inverters in the back are used to convert the 12V DC supplied by the vehicle s electric bus to 250V/50 Hz AC required to drive the PC equipment. Save date: Public Page 70

71 Figure 3-3: Hardware and computer racks in the back of TUD s Navigation Test Vehicle In its standard configuration, TUD s Navigation Test Vehicle is equipped with at least two PCs interconnected by Ethernet. The Navigation PC is responsible for collecting sensor data and the realtime calculation of the GPS/IRS sensor data fusion. It is equipped with both a MIL-1553 and an ARINC 429 card to interface with common aircraft avionics. The other PC is used for multiple purposes, such as data logging, acquiring traffic data and calculating a traffic data fusion or hosting software prototypes of onboard functions such as an Airport Moving Map. Space permitting, further PCs can be added and connected to the vehicle s Ethernet network. ADS-B live traffic can be received using a Filser RT60 ADS-B receiver (purchased via Eurotelematik) that sends out traffic data via Ethernet. The Filser RT60 software was updated during the EMMA2 trial preparation to be able to receive and handle also TIS-B data information. The passenger seat served as a very simple cockpit mock-up for the pilot evaluations and was therefore fitted with a 15" off-the-shelf LCD display (conventional PC monitor) and a Logitech trackball as a basic Crew Control Device (CCD), both provisionally mounted by the help of Velcro tape. Although the LCD used is far from the ruggedised, high-contrast AMLCD screens used in aircraft cockpits, the display readability was very good even in back light conditions. Another limitation of the screen used is that its diagonal is a factor of 1.5 larger than the 6" x 8" screens currently used in the Airbus A380. Save date: Public Page 71

72 Figure 3-4: LCD screen installation in front of passenger seat The test van has the permissions necessary to operate on the movement area of Frankfurt airport (EDDF). For evaluations on the manoeuvring area, the van was equipped with a turning light, and licensed drivers with the required radio permissions were kindly supplied by Frankfurt and Prague airport authorities. In Prague, additionally, an autonomous ADS-B transponder requiring only power supply was installed on the van s roof. Sensor/ HW Manufacturer Technical Data Remarks H764 (INU) Honeywell Position <1.0 [nmi/hr] CEP velocity <1.0 [m/s] RMS Heading 0.1 [deg] RMS Pitch, Roll 0.05 [deg] RMS RT20 L1 (GPS) Novatel 12 channel L1 20 cm CEP50 (RT2 DGPS) RT2 L1/L2 (GPS) Novatel 12 CH L1/L2 2 cm CEP50 (RT2 DGPS) SK8 (GPS) Trimble 8 CH L1 2 m CEP50 (DGPS) GPS 35 Garmin 12 CH L1 5 m RMS (DGPS) Ashtech G12 Thales 12 channel L1 40cm CEP50 (DGPS) reference station RT2, Hermes Novatel, Phillips L1/L2 RTCM Type 59N INU medium accuracy High accuracy High accuracy Low cost receiver Low cost receiver GA certified reference point required. Save date: Public Page 72

73 modem (DGPS, VHF link) RTCA Type 7 VHF Modem: 459,57Mhz Odometer Bosch 56 pulse/round ABS controller ALF receiver (DGPS ) MS5535 (Barometer/ Thermometer) DeTex RTCM-104 v ,5 khz Intersema -40 C to +125 C 1-12 bar RT60 Filser 1090 Mhz, Enhanced Squitter Available within 600 km radius of Frankfurt Parport PC Interface ADS-B/TIS-B receiver Table 3-1: Sensor equipment of TUD s Navigation Test Vehicle Validation team roles The validation will take place at TU Darmstadt at the Fixed Base Research Simulator and at Prague Airport with the Navigation Test Vehicle. Both evaluation trials will have the same requirements for the personnel, for the evaluation and for the assessment crew, so that the following description is valid for the trials at the research simulator and for the Prague trials Evaluation Crew Pilots from major airlines, who are in active service, should participate in the evaluations. Their age and flight hours should be heterogeneous to achieve statistically examinable measurements. Even the airlines they work for should be different, to see results for different internal procedures or habits Assessment Crew The core assessment team for all the trials consisted of two TUD research scientists. These two research scientist are systems engineers who participate in the functional and HMI design of the Surface Movement Awareness and Alerting System (SMAAS) and who are also responsible for the creation and extension of the questionnaires used in the experiment. They act as the experiment leaders, do the briefing as well as the familiarisation, take down pilots comments and assist pilots in filling out the questionnaires. At the Prague Airport Trials two more research scientist from TUD will join the assessment crew. One of them will mainly be responsible for the operation of the navigation equipment in the Navigation Test Vehicle. The other one will drive the Navigation test vehicle at Prague Airport The Baseline System Not applicable. Save date: Public Page 73

74 3.3 The experimental system: Surface Movement Awareness and Alerting System Figure 3-5: Elements of the Surface Movement Awareness & Alerting System (SMAAS) The Surface Movement Awareness and Alerting System (SMAAS) aims at improving safety and efficiency of aircraft movements on the aerodrome surface, during takeoff and on final approach. The main purpose of this system is the avoidance of Runway Incursions by: Preventing the own aircraft, by enhanced situational awareness and alerts, from entering, crossing, taking off or landing on runways without a corresponding clearance, and providing traffic awareness and giving timely alerts if other vehicles or aircraft infringe the protection zone of a runway that is used or has been cleared for own-ship operations. Generally, the SMAAS supports the aircrew during all ground operations, takeoff and final approach by providing enhanced situational awareness and, if necessary, timely alerts in case of potentially dangerous situations. This functionality is also envisaged as a potential enhancement of the ASAS Package 1 application ATSA-SURF. The Surface Movement Awareness and Alerting System (SMAAS) consists of two complementary parts, one dedicated to situational awareness and the other one dedicated to alerting. The core element of the SMAAS is an airport moving map based on an ED-99A compliant aerodrome database, which is intended to provide the crew with enhanced positional awareness to avoid disorientation on the airfield, a common precursor of Runway Incursions. The basic airport moving map can be enhanced by three further situational awareness functions. By displaying traffic on the ground and in the takeoff or landing phases in relation to the airport moving map, Cockpit Display of Traffic Information (CDTI) functionality can be added to the basic airport moving map to increase traffic awareness. In addition to this, the Operational Awareness Function (OAF) processes and presents operationally relevant information such as runways in use, runway closures, whether Low Visibility Procedures (LVP) are in force and other information typically contained in ATIS transmissions (voice or digital) or (x)notam. Furthermore, the FMS runway is highlighted on the airport moving map to remind the crew of the FMS settings, which is intended as another measure to prevent a takeoff from the wrong runway. Save date: Public Page 74

75 Last but not least, the Clearance Awareness Function (CAF), mainly by presenting the assigned taxi route, raises the crew s clearance awareness. While it is obviously preferable to obtain the data required for OAF and CAF in a machine-readable format and via data link, there are provisions to enter these data manually using the MCDU, the MFD or a dedicated hardware control panel to ensure that the system can work independently of ground infrastructure. Only if the pure display of all this information fails to prevent a hazardous situation, the second part, the Surface Movement Alerting subsystem, which builds on the same information as the awareness part, comes into play. It can be subdivided into two integral parts, a preventive and a reactive one. The first goal of the Surface Movement Alerting system is to ensure that ownship does not cause a Runway Incursion, i.e. Preventive Surface Movement Alerting. To achieve this, the alerting part is armed using the same airport, operational and clearance data as the awareness part of the SMAAS. In contrast to systems such as the RAAS, this enables specific alerting tailored to the particular operational situation, which is in itself a prerequisite for preventive alerts up to Master Warning level (Level 3). Without operational and clearance information, it would, for example, be impossible to alert the flight crew specifically when they enter a runway that is completely closed due to heavy construction, or when they enter or try to take off from a runway without the appropriate clearances. In parallel to this rigorous ownship surveillance, relevant surrounding traffic will continuously be monitored to alert the crew if a Runway Incursion caused by others poses a significant collision hazard (Reactive Surface Movement Alerting). Nonetheless, the main idea behind the SMAAS is not to create additional alerts on the flight deck. Rather, the intention is to enable the crew, by means of improved situational awareness, to avoid potential conflicts proactively at a strategic or pre-tactical level, and to provide alerts only for last resort conflict avoidance. After all, tactical alerts from safety net functions such as ACAS or TAWS raise the crew work-load dramatically if they occur, and thus the alerting part of the SMAAS has to be seen as a backup or safety net function in situations where the awareness functions are not sufficient to avoid a hazardous situation. 3.4 Experimental Design Experimental constraints This section is dedicated to a survey of limitations of the testbed, i.e. the SMAAS prototype and the display, and the simulator environment as well as the navigation test vehicle. It is essential to capture these limitations, since they provide the context in which the evaluation results have to be seen Experimental conditions The main operational limitation of both experiments is that a single pilot configuration is used, which, apart from excluding Crew Resource Management (CRM) issues, drastically reduces the realism with respect to workload and procedures. In Prague the pilots are seated on the passenger seat of the Navigation Test Vehicle so that their workloads will tend to focus more on the observing of the display like pilots non flying normally do, while the pilots in Frankfurt in the Research Simulator have to take the pilot flying and the pilot non flying part which means that they have to observe the displays and the surrounding traffic provided by the simulator on its visual system. For both, the evaluation at the Research Simulator and the evaluations with the Navigation Test Vehicle at Prague airport, the small number of traffic is a constraint for these evaluations e.g. due to the usage of an area at Prague airport with an operationally closed Runway which just has sporadic traffic. Because an MCDU unit is missing at the Navigation Test Vehicle, the interaction with the MCDU will not be possible during the trials at Prague. Save date: Public Page 75

76 Furthermore, the realism of the ATC environment was very limited due to the fact that ATC instructions were only given for ownship, resulting in a total absence of all radio communications of ATC with simulated other traffic. However, an accurate simulation of the ATC environment would have required at least one pseudocontroller and several pseudo-pilots reading back the clearances issued to the simulated other traffic. On the one hand, this lowered pilot workload with respect to monitoring a certain frequency, but on the other hand, this also resulted in a complete loss of the party line effect, with a potentially detrimental impact on pilot situational awareness. At any rate, in the context of single pilot operations, the added negative impact of an incomplete ATC environment can be regarded as small. Of course, the fact that no real controllers were used also decreased the realism of the simulation, but the associated effect is believed to be negligible compared to the other limitations in the ATC setup. The SMAAS is based on an Airport Moving Map, so that the actuality of the AMDB, underlying the AMM is important for the proper visual picture the pilot gets from this system. The Revision dates of the AMDB used for the trials are: Prague: January 2006 Frankfurt: February 2006 Paris: October Material constraints This section summarizes the technical limitations of the evaluation setup, i.e. constraints resulting from the hard- and software employed. For each of the items below, the impact the constraint may have had on the evaluation is estimated and classified Flight Simulator Missing radio management panels. There are no radio management panels installed in the cockpit, which means that the transfer to the next control authority in an R/T environment, either on ground or in the air, cannot be simulated in a realistic fashion. Particularly in the enroute phase, however, the associated frequency changes are a major source of crew workload. Missing FANS DCDU. There is no FANS DCDU, either as software mock-up or hardware, available for the evaluation. Consequently, HMI aspects related to the CPDLC application itself will not be covered. Display size. The size of the displays (15 ) in the simulator cockpit is significantly larger than in current airliners. The 6 x 8 displays of the Airbus A380 have an equivalent diagonal of 10. Only business jets like the Gulfstream G550 feature similarly large displays (14 ) Traffic Simulation The density of the simulated traffic will be comparatively low and definitely below the usual density at daytime. Apart from that, there were the following other limitations: Single aircraft type. All simulated traffic will be visualized using an Airbus A320 model in the Lufthansa corporate livery as default, because the converter from the TUD simulation environment to the visual system is not yet capable of automatically selecting the correct airline livery from the airline code contained in the callsign. Limited number of aircraft models. The simulator visual can only use 19 distinct aircraft models, i.e. there is a limitation to 19 airline/aircraft combinations, while the overall number of Save date: Public Page 76

77 other aircraft that can be displayed is only limited by performance. Currently, the visual system is limited to 10 other aircraft due to performance limitations of the computers rendering the simulator visual. While the limitation in the number of models can be compensated easily by an intelligent combination of a larger number of hub airline aircraft of similar type (e.g. Lufthansa/Air France A320s at Frankfurt and Paris, respectively), which then leaves sufficient margin for a realistic number of other airlines, the current performance limitation to ten other aircraft is a significant limitation, since it prevents an assessment of high traffic densities where display clutter issues might become relevant for the CDTI 3. Missing ground vehicles. Due to technical constraints of both the traffic simulation and the simulator visual, a simulation of ground vehicles and aircraft under tow was not possible, which mainly decreased operational realism on the apron and is therefore believed to be a minor limitation Navigation Test Vehicle Display size. The size of the displays (17 ) in the Navigation Test Vehicle is significantly larger than in current airliners. The 6 x 8 displays of the Airbus A380 have an equivalent diagonal of 10. Only business jets like the Gulfstream G550 feature similarly large displays (14 ). Reflections. The display isn t protected from reflections by a glare shield like in COTS airplanes. That could lead to reflections on the display which wouldn t occur in an airplane. Interaction. On this Testbed there will be no interaction the pilot is familiar with in an airplane. He will be able to interact with the display using a conventional keyboard (e.g. zoom in, zoom out) but he isn t able to interact with the Navigation Test Vehicle itself Time schedule The campaign is composed of two sessions. In the first session the system will be validated at the fixed base research simulator, during 30 th June 2008 and 20 th July Individual dates and times will be appointed. In the second session the system will be validated at Prague Airport from 25 th August 2008 to 29 th August For the second session Figure 3-6 shows the validation plan. Figure 3-6: Time schedule Selected Design The condition of evaluation days is always the same: the pilot shall always play the same scenarios. Scenario description specifies the experimental condition of each scenario. 3 A potential mitigation for the performance limitation could consist of an intelligent traffic data converter for the visual taking into account viewing direction/field of view and visibility. Save date: Public Page 77

78 3.4.4 Scenario specifications For the trials several scenarios were developed, to match the different conditions of the test facilities. Therefore airports with a high complexity were chosen, because the challenges the crew faces at airports with low complexity are usually relatively minor. Therefore, at any rate, airports with high complexity are expected to be the places where the benefits of the new onboard functions will become most apparent. Regarding the MMS functionality, several pilots eventually confirmed this by commenting that the application would be extremely helpful at congested hub airports like New York s Kennedy Airport (KJFK). The choices were made to include Frankfurt (EDDF) and Paris (LFPG) airports as virtual test sites for the simulator trials. Prague airport (LKPR), as official EMMA test site, was chosen for the real world trials because it is the closest and most complex of the three EMMA 2 airports. Taking into account the various factors determining airport complexity, it can be concluded that both Frankfurt (EDDF) and Paris (LFPG) airports fulfil several of the criteria each and can thus be regarded as complex airports, while Prague (LKPR) is certainly near the threshold from high to medium complexity. Because we expect that most of the evaluation crews are mainly Lufthansa pilots, Frankfurt airport will be even though it is a complex airport easier to handle for them than Paris airport. So the Paris scenarios are mainly the complex scenarios for them. While it is easy, to select traffic in the simulator it is not this easy to control it on the live trial site. The description about the traffic used during the trials will be mentioned in each scenario paragraph if it is relevant for the scenario. Also the weather is a controllable parameter while performing the trials in the simulator. However being on a live trial site it is not possible to control weather conditions. Visibility Condition 1 or Visibility Condition 2 as defined by the ICAO European Manual on A- SMGCS [4] will be used for the simulator trials, except in the last scenario (SMA 05). A more detailed description will follow in the appropriate paragraph Research Simulator For the fixed base research simulator, 7 different scenarios will be used. They are described in the following paragraphs FAM 01 This scenario was developed, to get the pilot familiar with the simulator and the Airport Moving Map Display. Figure 3-7: Scenario FAM 01 For this reason no traffic was provided, while the pilot had to taxi out from Gate A16 to Runway 25R at Frankfurt Airport (EDDF), which is shown in Figure FAM 02 In this scenario a first alert will be provided. The pilot has to taxi from Gate A16 to Runway 07R. At the transit from Taxiway A to Taxiway W, which is the assigned taxi route, the pilot is asked, to stay Save date: Public Page 78

79 on Taxiway A and head to Runway 18, which has been marked as a closed Runway, slowly. After that he has to resume the assigned taxi route (Figure 3-8 ). Figure 3-8: Scenario FAM 02 For this scenario no traffic will occur at the airport SMA 01 The assigned Route is from Gate A16 to Runway 25L. (Figure 3-9) Runway 18 is completely closed, while Runway 25R, which must be crossed by the pilot is in use only as Taxiway. Neither a highlighting nor the alerting system is enabled. Figure 3-9: Scenario SMA SMA 02 This scenario (Figure 3-10) is the same as the previous scenario (SMA 01), except, that the highlighting of active or closed Runways and the alerting system is enabled. During the taxiing, the pilot gets the information, that Runway 25R is not longer in use just as a Taxiway but can be used as a regular Runway. The pilot should use this information, to interact with MCDU to unlock the Taxiway restriction from Runway 25R. After that he should proceed along his assigned route Save date: Public Page 79

80 Figure 3-10: Scenario SMA SMA 03 This scenario is related to scenario SMA 01. The crew had to taxi from Gate Z4 to Runway 09L, while Figure 3-11: Scenario SMA 03 Runway 09R and Runway 08L/26L are completely closed. Neither a highlighting nor the alerting system is enabled SMA 04 A similar scenario to SMA 03, except the active state of the highlighting and alerting system. Like in SMA 03 the pilot should interact with the MCDU to unlock the complete closure of Runway 09R (Figure 3-12). Save date: Public Page 80

81 Figure 3-12: Scenario SMA 04 and SMA SMA 05 In this scenario, the pilot has to taxi from Gate Z4 to Runway 09R. The visibility is set to IFR conditions. At Runway 09R the crew will be instructed, to line up after the other aircraft. This aircraft will not be shown on the Airport Moving Map Display, while other traffic is shown. The aircraft, being number one for takeoff, starts to Takeoff, rejects it and remains with damaged tyres on the Runway. Meanwhile the pilot got his Takeoff clearance. He is not able to visually see the aircraft that stands on the Runway Navigation test vehicle The scenarios were divided in two packages of scenarios. One package was developed to provide scenarios with the General Aviation Aircraft available and the other simulating traffic with two ground vehicles. For the scenarios with the ground vehicles, the south east located area is available for use, containing runway 04/22 and the surrounding taxiways M and P. TUD Van simulates the own aircraft, with the flight crew for the trials. A second vehicle simulates other traffic. For the scenarios with the General Aviation Airplane, the airplane initiates a landing procedure on a Runway, where Runway 04 should be used, with the TUD van standing on this Runway 22. Save date: Public Page 81

82 Figure 3-13: Overview of Prague airport (LKPR) EMMA-VAN-01: Take Off RWY 04 Traffic On/Approaching RWY This scenario describes a takeoff from the active Runway 22. TUD Van stands cleared for takeoff on Runway 22. Meanwhile a vehicle simulating other traffic taxis over Taxiway P, crosses the stop bar to Runway 22 and rolls onto the Runway (Figure 3-14). Save date: Public Page 82

83 Figure 3-14: EMMA VAN-01 Take Off RWY 04 Traffic On/Approaching RWY EMMA-VAN-02: Take Off RWY 04 without clearance This scenario describes a takeoff from Runway 22. TUD Van received the line up clearance on Runway 22. The crew starts taking off from Runway 22. Figure 3-15: EMMA-VAN-02 Take Off RWY 04 without clearance EMMA-VAN-03: Collision Hazard with other Traffic on Taxiway This scenarios describes a taxiing situation, where the own airplane rolls in front of a second airplane. TUD van rolls along Taxiway M to Taxiway P via Taxiway L. The vehicle simulating the following airplane rolls along the same Taxiways and starts getting closer during the scenario. Save date: Public Page 83

84 Figure 3-16: EMMA-VAN-03 Collision Hazard with other Traffic on Taxiway EMMA-VAN-04: Collision with other Traffic on Taxiway In this scenario the own airplane taxi along Taxiway L from East to West and is cleared to cross Runway 04. The intruder rolls along Taxiway R from East to West. At the intersection of Taxiway R and Taxiway L the intruder causes a collision hazard Figure 3-17: EMMA-VAN-04 Collision with other Traffic on Taxiway EMMA-VAN-05: Crossing RWY 04 without clearance In this scenario TUD Van rolls on Taxiway L towards Runway 04. The crew has not received any clearance. The Crew crosses Runway 04 (Figure 3-18). Save date: Public Page 84

85 Figure 3-18: EMMA-VAN-05 Crossing RWY 04 without clearance EMMA GA-01 Takeoff of RWY 13, Traffic on Approach RWY 31 In this scenario the own airplane stand ready for takeoff on Runway 13, while a other plane is approaching Runway 31 from air (Figure 3-19). Figure 3-19: EMMA GA-01 Takeoff of RWY 13, Traffic on Approach RWY Training requirements For both, the fixed base research simulator evaluations and the evaluations at Prague Airport a computer based presentation of the SMAAS system is performed. After that depending on the test site an introduction to the test equipment will be given. Save date: Public Page 85

86 At the research simulator a short introduction into the use of the simulator and the SMAAS system is given. This introduction includes the first scenario described in to become familiar with the simulator and the system interaction. At Prague Airport just an introduction to the SMAAS system is necessary, because the pilots don t have to operate a simulator or a vehicle. 3.6 Conduct of the study Research Simulator The study is composed of several evaluation days. During an evaluation day seven scenarios are tested. The typical plan of an evaluation day is the following, where T0 is an individual start time for each pilot. One pilot per day should participate in the evaluation. T0 to T0 + 30min: Presentation - Briefing T0 + 35min to T0 + 55min: Training T0 + 60min to T min: System evaluation EDDF scenarios T to T min: Lunch T min to T min: System evaluation LFPG scenarios T to T min: Debriefing Navigation Test Vehicle The Navigation Test Vehicle study is composed of two evaluation days. During an evaluation five scenarios are tested. The typical plan of an evaluation day is the following, where T0 is an individual start time for each pilot. Two pilots per day should participate in the evaluation. T0 to T0 + 30min: Presentation Briefing T0 + 30min to T0 + 40min: Training T0 + 40min to T min: System evaluation T min to T min: Debriefing 3.7 Analysis Methods The data obtained from the different pilots are gathered and analysed by the assessment team in order to point out the main results. The opinions shared by the majority of the pilots will be underlined. General themes and preoccupations will be extracted trough the analysis made. More individual opinions and observations concerning only a few pilots will be analysed in their specific context. An arithmetic mean is computed for the results of the questionnaires. We anticipate a small number of evaluation pilots, the results can only be used to see a general trend. Save date: Public Page 86

87 4 Head Up Navigation (THAV) This section is about the validation of display part of the HUD/SGS service. That is the surface guidance by means of a Head-Up display device. The validation of HUD/SGS consists in one campaign of 5 evaluation days. Fro each evaluation day, a pilot is invited to play 4 scenarios on a THAV cockpit simulator implementing the HUD/SGS service. 4.1 Validation Aims, Objectives, and Hypotheses The purpose of this section is to clarify what is to be achieved from the EMMA2 validation exercises in 2-WP6.2 for the HUD/SGS service. It contains the general aims in accordance with 2-D6.1.1b Generic Experimental and Test Plan [6]. The basic aim of the EMMA2 Project is the verification of higher A-SMGCS services as described in the ICAO Manual [4] and further refined in the SPOR [5]. Two areas of validation activities have been considered. The Operational Feasibility area refers to the definition of the operational use of equipment and procedures, in accordance with the performances assessed in the previous stage. It answers the question: Given the performances of the equipment, is it usable and acceptable? The Operational Improvements area refers to the evaluation of the operational improvements, in terms of Safety, Capacity, Efficiency, Human Factors, and Environmental Impact using the equipment and the procedures defined in the previous stage. It answers the question: Given the accepted HUD/SGS equipment and procedures, how is guidance improved? Validation Aims The validation aims at assessing the operational feasibility and the operational improvement of the onboard SGS/HUD service as defined in the 2-D1.1.1 SPOR document. Operational feasibility assessment is achieved by means of RT Simulation on a cockpit simulator: the crew will use the improved service during taxi-in and taxi-out procedure in condition as closest as possible from the reality. The RT simulation is also used to state operational improvement. Regarding validation constraints (training time and simulator conditions) and level of maturity of the service, observation and questionnaire have been selected to collect results from the RT simulations Validation Objectives Validation activities deal with verifying operational feasibility and operational improvement of the SGS/HUD service Operational Feasibility Operational feasibility is proved against operational requirements dedicated to the HUD/SGS service in the framework of procedure defined in the EMMA2 2-D1.1.1 SPOR [5]. High-level Objective 1 Table 4-1: Verification of EMMA2 Operational Requirements and Procedures High-level objectives in the area operational feasibility in EMMA2 Save date: Public Page 87

88 Operational Improvement Once the operational feasibility of the HUD/SGS service has been proved, the validation will take the improvements into account. Two fields are addressed: safety and working performance. Usage of the HUD/SGS service may have impact in terms of Capacity, efficiency and environment. However, at this step of service development, only primary impacts shall be considered. For the same reasons, the operational improvement will not be metered by quantifying differences between experiments in a baseline system and a improved system but through questionnaire and debriefing with the pilots. High-level Objective 2 High-level Objective 4 Increase of Safety Suitability of Behaviour and Working Performance Table 4-2: High-level objectives in the area operational improvements in EMMA Low-Level validation objectives The following table details the derivation of high-level objective into low-level objectives. Concerning Operational feasibility, only low-level Objective attached to operational requirements dedicated to HUD/SGS service are taken into account. Concerning Operational improvement, the RT simulation in a cockpit simulators and the maturity level of the service, safety perception, workload and situational awareness are in the scope of the evaluation. Area High Level Objectives Low Level Objective Operational Feasibility Operational Improvements Verification of EMMA2 Operational Requirements and Procedures [HLO1] Increase of Safety [HLO2] Verification of the guidance requirements [LLO1.8] Verification of the aircraft onboard requirements [LLO1.9] General usability of new services [LLO1.12] Increment of safety perception by users [LLO2.2] Suitability of Behaviour and Working Performance [HLO4] Appropriate level of user s workload [LLO4.1] Improved situational awareness of users [LLO4.2] Table 4-3: The low-level objectives related to high-level objectives and areas of activity Save date: Public Page 88

89 4.1.3 Hypotheses, Indicators, Measurement Tools Hypotheses Operational Feasibility Hypotheses Identifier OF-H0 OF-H1 Hypothesis The pilots opinion does not agree to the operational feasibility aspects of HUD/SGS service The pilots opinion agrees to the operational feasibility aspects of HUD/SGS service Operational Improvement Hypotheses Safety-related Hypotheses In order to examine the safety, subjective measurements will be conducted. The following hypotheses will be tested: S1-H0 S1-H1 Hypothesis An increment of safety while using HUD/SGS is not perceived by the pilots An increment of safety while using HUD/SGS is perceived by the pilots Behaviour and Working Performance Hypotheses S2-H0 S2-H1 Hypothesis Situation awareness of pilots is not improved while using HUD/SGS Situation awareness of pilots is improved while using HUD/SGS HF-H0 HF-H1 Hypothesis An increment of workload while using HUD/SGS is not perceived by the pilots An increment of workload while using HUD/SGS is perceived by the pilots Indicators and Measurement Tool The measurement tools used in HUD/SGS experimentations are: Observations Debriefing Questionnaire Debriefing Interview After simulator trials and field / flight tests, users will have the opportunity to reflect about their experience gained in the trials. The test coordinator makes sure that these data will be collected and analysed. Save date: Public Page 89

90 4.2 The test environment in Thales Avionics The validation platform Within the scope of the EMMA Phase 2 project, a cockpit simulator facility has been used for the assessment of the Head Up Display / Surface Guidance System (HUD/SGS). This cockpit simulator is mainly used to assess new operational cockpit concepts or new functions in dynamic scenarios. It allows gathering operational inputs in term of utility of new functions and usability of the HMI. For evaluation needs, the network of desktop computers of the cockpit simulator supports the following applications: A traffic generator, A flight dynamics manager, Landscape generators, A Primary Flight Display + engines application, Airport Navigation functions (IMM), The HUD/SGS function, CPDLC applications for the Non Flying Pilot (PNF) and the virtual controller, A sound system to simulate the cabin noise. Save date: Public Page 90

91 There is a permanent communication between all these applications through an adapted and efficient data link network Cockpit simulator The cockpit simulator contains an aircraft model based upon a commercial airliner. It is composed of a network of desktop computers and figures a set of aircraft systems HMI as controls and displays. It enables to perform dynamic evaluations of one or two pilots. For the purpose of the verification of the functions described above in EMMA 2 project, the pilots (Pilot Flying (PF) and PNF) use the following equipment of the cockpit: Displays A Primary Flight Display (PF), A Navigation Display (IMM) presenting the Airport Navigation function (PF) A LCD Head-Up Display displaying Surface Guidance function (PF), An IMM + CPDLC HMI display (PNF) Controls equipment Thrust levers, Pedals, Steering Hand wheels Figure 4-1: Cockpit simulator The external equipment A traffic simulator which provides, according to the chosen scenario: Save date: Public Page 91

92 o The departure airport, o The departure A/C location on the airport, o The weather condition (visibility), o The date of the scenario, o And real time traffic information (traffic location, category, GS, heading ). A CPDLC HMI dedicated to the pseudo-controller which allows to communicate by data link with the pilots. The set of possible message is the following (in the column direction down means from controller to aircraft and Up from aircraft to controller): Name Direction Comment TAXI TO destination UP Clearance specification TAXI TO destination VIA path UP Clearance specification EXPECT ROUTING TO dest VIA path UP Expect routing specification PUSH BACK APPROVED FACING direction UP START UP APPROVED UP Start up approved Push back approved facing a direction REVERT TO VOICE UP Request to revert to voice REQUEST TAXI CLEARANCE DOWN Request a taxi clearance REQUEST ROUTING DOWN Request an expect routing REQUEST STARTUP DOWN Request a start up clearance REQUEST PUSH BACK DOWV Request a push back clearance WILCO ROGER STAND BY UNDO DOWN/UP Protocol messages Moreover, the cockpit simulator presents a realistic landscape of the airport where is located the simulation in accordance with the airport database used for the evaluation session Airport Navigation application The prototype of the Airport Navigation application used for this activity covers the following functionalities: Airport moving map function presenting the own-ship and the traffic situation on the airport, with interactive menu allowing managing zooming, display mode, airport selection. Path management function allowing : Observing the expect routing and the clearance sent by the CPDLC application Or to create interactively its own path according to the controller information. The path and the taxi clearance are displayed on the moving map, allowing the pilot to manage the navigation of the aircraft on the airport platform according to traffic information. Save date: Public Page 92

93 HUD/SGS function prototype The HUD/SGS function implemented on this prototype fulfils the requirements of both Human factors HMI requirements and Functional Requirements Document for HUD/SGS function, taking into account the users needs expressed by pilots during the needs collection activity. The HUD/SGS function provides the pilot taxiing the airplane with all the necessary elements for navigation and situation awareness to ensure his mission keeping eyes-out. These elements must allow the pilot whatever the weather conditions may be to: a) Follow the assigned path and associated taxi clearances b) Maintain every time the airplane on the pavement c) Be able to report own-ship position to the control as requested d) Ensure the security of the airplane Validation team roles The assessment team was composed of four people: A manager to drive the evaluation, Three software engineers to handle the simulators One Human Factors (HF) specialist. Figure 4-2 shows the validation team role and the pilot role. PF (Pilot) PNF (Manager) Observer (Software Engineer) Operator (Software Engineer) Observer (HF specialist) Pseudo-controler + Supervisor (Software Engineer) Figure 4-2: Team roles Research Personnel The research personnel is composed of three software engineers. The first one is in charge of maintaining the simulator in operational conditions: launch the simulators and to keep it in working order. The second one has two roles: pseudo-controller and supervisor of the scenario. The last one has an observer position together with the Human factor specialist. Save date: Public Page 93

94 Experimental Participants Num Background (Flight test, Airlines Training) Nation ality Native Languag e Sex (F/M) Age class (Ex.: 40-44) Hierarchical (Captain/ F/O) Total amount of flight hours Current A/C type Former A/C type 1 Flight test F F M Captain 2 Airline F F M Captain B777 A330, A340, B747, B737, C160, N2501 noratlas 3 Airline F F M F/ B A320 4 Airline F F M F/O ATR 42/72 BeechCraft 90 5 Airline F F M Captain 8000 A320 A Pseudo operators One of the software engineers has both a pseudo-controller position and a supervisor position. Figure 4-3: Screenshot of Pseudo-operator Display Save date: Public Page 94

95 The pseudo-controller position consists of: Managing the communication via data-link and via voice with the crew, that is, to send clearances and expect routing and to receive acknowledgements and requirements. A specific HMI is used to write data-link message and to indicate when a message is received by data-link. Figure 4-4: Screenshot of Supervisor Display The Simulation Supervisor position consists of: Scheduling the current scenario Managing the traffic around the ownship. The supervisor and pseudo-controller positions are merged since it is important to coordinate traffic management and clearances Observers Having explained the protocol to the pilot, the HF specialist observed attentively his behavior and comments. At the end of the test he interviewed him. The third software engineer took down notes all along the assessment to prepare the report with the HF specialist The Baseline System Not applicable Pilot Roles The pilot is always the PF and the CM1 (as described in SPOR 2-D1.1.1). He is in charge of taxiing the aircraft. The manager takes the PNF and CM2 role CM1 (as described in SPOR 2-D1.1.1). He is mainly in charge of communications. PF Tasks: Save date: Public Page 95

96 Taxiing the aircraft Cross-checking clearances PNF Tasks: Supervising the PF Ensuring communication (data-link and R/T) with pseudo-controler Checking clearances Paper work Other assisting task on command of the PF Baseline System Procedures Not applicable The experimental system The HUD/SGS function is an avionic function that is designed to support the aircrew of an aircraft during taxi operations. The concept, for the Surface Guidance Function on HUD, is that it provides, using adapted symbology, tactical support to the pilot taxiing the airplane, as a complement to other parts of the A-SMGCS that provide general surface situation awareness information Within the scope of the EMMA 2 Project, the HUD/SGS function is a concept-demonstrator intended for use only in situations where the aircrew has independent means of verifying the support provided. The HUD/SGS function has to provide to the pilot taxiing the airplane all the necessary elements for navigation and situation awareness to ensure his mission keeping eyes-out. These elements must allow the pilot whatever the weather conditions may be to: Follow the assigned path and associated taxi clearances Maintain every time the airplane on the pavement Be able to report own-ship position to the control as requested Ensure the security of the airplane Save date: Public Page 96

97 Figure 4-5: Experimental system 4.3 The experimental system: Description of the HUD/SGS function prototype The prototype of the HUD/SGS application presents the following items: Navigation information according to the airport ground path (expect routing) and the associated clearance provided by controller or CPDLC. Guidance cues according to the aircraft situation on the airport platform and the path to follow. Situation awareness according to the global aircraft position and its precise location on the pavement. Aircraft parameters (Ground Speed, heading, altitude). Navigation information According to the information provided by controller and the CPDLC, and using available data from airport database, the HUD/SGS function provides the following navigation items: Save date: Public Page 97

98 [2] [1] Figure 4-6: Surface Guidance symbology on straight line Taxi Route, (trajectory to be followed during taxi operation on the airport) Taxiway centre line ([1], Figure 4-6) is a 3D symbology generated from the description, in the airport database, of the taxi lines of every segment composing the path to be followed. Taxiway borders ([2], Figure 4-6) representing the limit envelope of the taxiways are positioned on both sides of the Taxiway centre line, indicating to the pilot the safety limits of the taxiway beyond which he has to avoid going not to put wheels outside the pavement. These borders are represented by little discs which are located regularly along the way. This regularity allows pilots to estimate distances. Stops (stop bar or clearance limit) [1] Figure 4-7: Stop bar This information is presented on the Head-Up display ([1], Figure 4-7) using 3D symbology, representing a barrier located at the place where the plane has to stop. The barrier appears to a Save date: Public Page 98

99 sufficient distance of the plane so that the pilot can have the information enough early to stop the airplane and in case the curved trajectory would not allow seeing the barrier, textual information ([1], Figure 4-8) indicating the remaining distance until stop is also presented in the HUD. [3] [1] [5] [4] [2] Figure 4-8: Turn and clearance information Turn Indication To inform the pilot approaching a turn different information are presented on the HUD: A symbol representing a curved arrow is displayed when the aircraft is approaching a turn. This arrow gives the direction of the turn and the name of the taxiway after the turn. ([2], Figure 4-8) Textual information is displayed above this symbol with the remaining distance to the beginning of the turn ([3], Figure 4-8) Situation Awareness information Aircraft situation A textual information with the name of the taxiway element where is the airplane is presented on the HUD ([5], Figure 4-8). Runway intersection When the A/C is approaching a runway, the name of the runway is displayed on both taxiway side on the HUD and a line associated to the runway border is displayed on the 3D view. Taxiway intersection Save date: Public Page 99

100 When the A/C is approaching an intersection with taxiways, all segment taxiway names connected are displayed on the HUD, and a line segment is displayed in the 3D view to represent the situation of the taxiway segment in connection ([4], Figure 4-8). Stop panel To inform the pilot when he approaches one clearance stop (less than 150 m), a symbol 2D representing a panel STOP with the remaining distance and the name of stop is presented on the HUD ([1], fig3-4). The text STOP flashes when the plane is unless 50 m apart from stop. Runway crossing When the plane is approaching the runway or is crossing the runway textual information appear: RUNWAY AHEAD Figure 4-9: Runway ahead information ON RUNWWAY The information ON RUNWAY is presented as soon as the A/C nose crosses the limit of the stop bar and as long as the tail of the plane is on the runway. Top sight view (2D) When the A/C is approaching very near the turn or is in the turn, certain elements of the symbology are not any more visible or usable. To remedy this problem a 2D top sight representing the plane on the taxiway, the wheels, the taxi line and the limits of the taxiway, is displayed in the HUD. Save date: Public Page 100

101 [1] Figure 4-10: The top sight view Guidance symbol Figure 4-11: Guidance cue To help the pilot to follow the optimised trajectory in the turns and to avoid putting wheels outside the taxiway, a guidance symbol ([1], Figure 4-10, Figure 4-11) indicating the heading to be followed is presented in the HUD. The heading value is computed taking into account the relative position of the plane on the taxiway, the track followed by the plane and the track associated to the taxi line. Actually, the pilot has to follow the square symbol controlling the round aircraft symbol. The arrow above the square symbol indicates that the square symbol is going to move to the left. The pilot has to prepare his manoeuvre Aircraft parameters Some A/C parameters are displayed on the HUD presenting the following information: Ground speed ([1], Figure 4-12) Heading ([2], Figure 4-12) Altitude ([3], Figure 4-12) Save date: Public Page 101

102 [1]] [2] [3] Figure 4-12: GS, heading and altitude information Experimental System Procedures The procedures applicable for the experimental validation exercises are those described in each scenario (see section 4.4.4) Influencing Factors Three factors can influence on board requirements for HUD/SGS services. These are: Visibility conditions Traffic density Aerodrome layout. Visibility Conditions: The ICAO Manual [4] proposes the following breakdown: Visibility Condition 1: Visibility sufficient for the pilot to taxi and to avoid collision with other traffic on taxiways and at intersections by visual reference, and for personnel of control units to exercise control over all traffic on the basis of visual surveillance. Visibility Condition 2: Visibility sufficient for the pilot to taxi and to avoid collision with other traffic on taxiways and at intersections by visual reference, but insufficient for personnel of control units to exercise control over all traffic on the basis of visual surveillance. Visibility Condition 3: Visibility sufficient for the pilot to taxi but insufficient for the pilot to avoid collision with other traffic on taxiways and at intersections by visual reference, and insufficient for personnel of control units to exercise control over all traffic on the basis of visual surveillance. For taxiing this is normally taken as visibilities equivalent to a Runway Visual Range (RVR) of less than 400 m but more than 75 m; Visibility Condition 4: Visibility insufficient for the pilot to taxi by visual guidance only. This is normally taken as a RVR of 75 m or less. Scenarios in Visibility condition 1 and in visibility condition 4 are tested during the RT simulations. Traffic Density: Traffic density is measured from the mean busy hour independent of visibility conditions. Traffic density is divided into three categories: Light (L): No more than 15 movements per runway or typically less than 20 total aerodrome movements per hour Medium (M): 16 to 25 movements per runway or typically between 20 to 35 total aerodrome movements per hour Heavy (H): 26 or more movements per runway or typically more than 35 total aerodrome movements per hour. In simulation, traffic density is Light on the overall airport. However, all the traffic is gathered around the ownship, this is the reason why, the traffic could be considered as Medium in the pilot point of Save date: Public Page 102

103 view. Aerodrome Layout: For aerodrome layout, the ICAO Manual [4] proposes the following three levels: Basic (B): An aerodrome with one runway with one taxiway to one apron area Simple (S): An aerodrome with one runway, having more than one taxiway to one or more apron areas Complex (C): An aerodrome with more than one runway, having many taxiways to one or more apron areas. RT simulation take place in Charles-De-Gaulle Airport (ICAO code LFPC) which is a complex aerodrome. 4.4 Experimental Design Experimental constraints Time schedule The following Gantt chart gives a rough overview about the main steps of this validation campaign. The campaign is composed of five evaluation days. Figure 4-13: Gantt Chart of the campaign Selected Design The condition of evaluation days is always the same: the pilot shall always play the same scenarios. Scenario description specifies the experimental condition of each scenario Scenarios specifications The scenarios are built to cover a large range of standards situations on the airport. Four scenarios have been implemented: 1. Taxi out from a gate to a runway threshold by nominal visibility conditions. 2. Taxi in after landing from a runway exit line to a gate with a crossing runway situation. 3. Taxi out from a gate to a runway threshold by bad visibility conditions and higher density of traffic. 4. Taxi in from a runway threshold to a parking after canceling of takeoff (not planned taxiing) Next paragraphs give details on these scenarios Common general environment parameters The pilot is on an Airbus A380 aircraft, all scenarios are situated on Paris Charles de Gaulle airport (LFPG). Save date: Public Page 103

104 Scenario 1: Taxi out Experimentation environment Taxi out from a gate to a runway threshold by nominal visibility conditions. Objective of evaluation: Evaluate the impact of the system on a standard taxi. Take system in hands. Airport: LFPG (Paris CDG) Own ship aircraft: Airbus A380 Taxi route: Departure from Terminal 2 D, gate D12 Taxi to west Via taxiways G, F, U, UC3 Taxi up to holding point WA(1) Take off on runway 08L Environment: Visibility: CAVOK Time: Middle of journey Ground traffic: 8 planes, among which 2 are on the own ship trajectory Objective for operator: Do the taxiing by using the CPDLC system and the moving map to visualize the trajectory and the clearances. Respect holding point consigns, the dialog procedures and the taxi limit speeds. Trajectory: Save date: Public Page 104

105 Other aircraft: Ref Type of vehicle Name Remarks Way102 Medium F-ABCA Shall cross the way of the own ship after the first clearance. Way103 Heavy F-DCBB Shall cross the way of the own ship after the second clearance Way104 Service Service4 Way105 Small F-EEFD Way106 Medium F-QZAE Is near the own ship at the beginning of the scenario Way107 Helicopter F-GHUF Way108 Small F-DFGG Way109 Small F-PJUH Detailed scenario Scenario for operator Actor Description Comm type ATC Sent before startup: «Expect routing runway 08L, holding point DL WA(1) via taxiways stand D12, twy G, F, U, UC3» Scenario Beginning of scenario 1 Pilot Checklist before startup Pilot «Request startup» DL ATC «Startup approved» DL Pilot «WILCO» DL Pilot «Request pushback» DL ATC «Pushback approved to west» DL Pilot «WILCO» DL Pilot Push back Scenario Start of the simulated pushback Pilot Start of engines and checklist before taxi Pilot «request taxi» DL ATC «Taxi to runway 08L, holding point WA(1) via taxiways G, F Hold DL short F» Pilot «WILCO» DL Pilot Taxi up to taxiway F and stop Scenario Start aircraft A when own ship is at the beginning of the first bend. ATC When aircraft A enters the parking: «Taxi to runway 08L, holding DL point WA(1) via taxiways F, U Hold short U» Pilot «WILCO» DL Scenario Start aircraft B when own ship is near the second parking entrance. Pilot Taxi up to taxiway U et stoppe ATC Behind aircraft B, «Taxi to runway 08L, holding point WA(1) via DL taxiways U, UC3» Pilot «WILCO» DL Save date: Public Page 105

106 Pilot Taxi up to the holding point WA(1) and stop ATC «Revert to voice» DL Pilot «WILCO» DL ATC «AT001, Contact tower on Bye.» VHF Pilot «AT001, contacting tower on Bye» VHF Pilot scenario This chapter describes the explanations given to the pilot before beginning of scenario Situation The aircraft is an Airbus A380 called «AT001» The mission is a commercial flight with passengers, departure from Paris CDG (LFPG), destination to Toulouse Blagnac (LFBO). We are situated at Paris CDG, on parking D gate D12. We are on the aircraft. The engines are off. Doors are closed and lock, ready to go. The weather is clear. Objective: The aircraft shall leave the gate and taxi for a take of on runway 08L by respecting ATC consigns Communications Communications shall be via CPDLC system every time it is possible. Otherwise, communications shall be via VHF. VHF communications are in French or in English, according to pilot preferences Scenario Type Description Beginning of scenario 1 Action Installation on the cockpit, checklist before startup Comm DL «Request startup» Wait for evt Wait for ATC answer Comm DL «WILCO» Comm DL «Request pushback» Wait for evt Wait for ATC answer Comm DL «WILCO» Action Push back (simulated by operator) Action Checklist before taxi Comm DL «request taxi» Wait for evt Wait for ATC answer Comm DL «WILCO» Action Follow the ATC consigns and answer when necessary. End of the scenario when arrived to the objective of taxi (indicated by operator) Scenario 2: Taxi in Experimentation environment Taxi in after landing from a runway exit line to a gate with a crossing runway situation. Objective of evaluation: Evaluate the impact of the system on a standard taxi in operation with runway crossing. Airport: LFPG (Paris CDG) Save date: Public Page 106

107 Own ship aircraft: Airbus A380 Taxi route: Landing on runway 08R Leaving runway by exit line V6 Via taxiway S6 Holding point S6 before crossing runway 08L-26R Via taxiways S6, RT16, R, P Taxi up to gate E14 (Terminal 2E) Environment: Visibility: CAVOK Time: Middle of journey Ground traffic: 8 aircraft, among which 2 are on the own ship trajectory Objective for operator: Do the taxiing by using the CPDLC system and the moving map to visualize the trajectory and the clearances. Respect holding point consigns, the dialog procedures and the taxi limit speeds. Trajectory: Other aircraft: Ref Type of vehicle Name Remarks Way202 Heavy F-DDFI Shall cross the way of the own ship after the first clearance Way203 Small F-SDFJ Shall cross the way of the own ship after the second clearance Way102 Medium F-ABCA Way103 Heavy F-DCBB Way105 Small F-EEFD Way106 Medium F-QZAE Way109 Small F-PJUH Save date: Public Page 107

108 Detailed scenario Scenario for operator Actor Description Comm type ATC Sent before top of descent: «Expect routing parking E via S6, RT16, R, RP4, P» DL Scenario Beginning of scenario 2 Pilot «Runway is free» VHF ATC «Taxi to holding point S6, hold short S6» VHF Pilot «WILCO» VHF Pilot Taxi up to holding point S6 and stop ATC «Cross runway 08L-32R and contact ground frequency on 121.6» VHF Pilot «WILCO» VHF Pilot Cross runway and stop Pilot «Runway is free» VHF ATC «AT001 hello, Revert to Datalink» VHF Pilot «WILCO» VHF ATC «Taxi to gate E14 via R hold short RP7» DL Pilot «WILCO» DL Pilot Taxi up to taxiway RP7 and stop Scenario Start aircraft B when own ship is near the taxiway RP7 ATC Behind aircraft B, «Taxi to gate E14 via R hold short RP4» DL Pilot «WILCO» DL Pilot Taxi up to taxiway P and stop Scenario Start aircraft C when own ship arrives on bend near RP4 ATC Behind aircraft C, «Taxi to gate E14 via P» DL Pilot «WILCO» DL Pilot Taxi up to la gate E14 Pilot Turn off engines and communications Pilot scenario This chapter describes the explanations given to the pilot before beginning of scenario Situation The aircraft is an Airbus A380 called «AT001» The mission is a commercial flight with passengers, coming from Toulouse Blagnac (LFBO), arriving at Paris CDG (LFPG). Aircraft has landed on runway 08R at Paris CDG. We are going to take exit line V6. The weather is clear. Objective: The aircraft shall taxi up to his parking place by respecting ATC consigns Communications Communications shall be via CPDLC system every time it is possible. Otherwise, communications shall be via VHF. VHF communications are in French or in English, according to pilot preferences Scenario Type Description Démarrage scenario 2 Comm VHF «Runway free» Save date: Public Page 108

109 Wait for evt Follow the ATC consigns and answer when necessary When arrived on the parking, end of scenario Scenario 3: low visibility Experimentation environment Taxi out from a gate to a runway threshold by bad visibility conditions and higher density of traffic. Objective of evaluation: evaluate the usability of the system for taxiing with bad visibility conditions. Airport: LFPG (Paris CDG) Own ship aircraft: Airbus A380 Taxi route: Departure from Terminal 2 D, gate D12 Taxi to west Via taxiways G, N, B, DB, D, BD4 Taxi up to holding point Y3 (1) Take off on runway 09R Environment: Horizontal visibility: 0.2 NM Time: Middle of journey Ground traffic: 8 aircraft, among which 4 are on the own ship trajectory Objective for operator: Do the taxiing by using the CPDLC system and the moving map to visualize the trajectory and the clearances. Respect holding point consigns, the dialog procedures and the taxi limit speeds. Trajectory: Other aircraft: Ref Type of vehicle Name Remarks Save date: Public Page 109

110 Way302 Light F-EHGK Used to simulate a queue before take of (first aircraft) Way303 Small F-TUJL Used to simulate a queue before take of (second aircraft) Way304 Small F-BFGM Shall cross the way of the own ship after the second clearance. Way305 Heavy F-GFJN Followed by the own ship after the third clearance. Way306 Medium F-BNVO Way106 Medium F-QZAE Is near the own ship at the beginning of the scenario Way108 Small F-DFGG Way109 Small F-PJUH Detailed scenario Scenario for operator Actor Description Comm type ATC «Expect routing runway 09R, holding point Y3(1) via taxiways D12, DL G, N, B, D» Scenario Beginning of scenario 3 Pilot «Request taxi» DL ATC «Taxi to runway 09R, holding point Y3(1) via taxiways G Hold short DL F» Pilot «WILCO» DL Pilot Taxi up to taxiway F and stop ATC When own ship arrives near taxiway F, «Taxi to runway 09R, holding DL point Y3(1) via taxiways N Hold short holding point N(3)» Pilot «WILCO» DL Pilot Taxi up to holding point N(3) and stop Scenario Start aircraft A when own ship is on the middle of strait line ATC After aircraft A, «Taxi to runway 09R, holding point Y3(1) via DL taxiways N, B Hold short B» Pilot «WILCO» DL Pilot Taxi up to taxiway B and stop Scenario Start aircraft B when own ship is near holding point ATC After aircraft B, «Taxi to runway 09R, holding point Y3(1) via DL taxiway B, DB, D, BD4» Pilot «WILCO» DL Pilot Taxi up to holding point Y3(1) and stop Scenario Start aircraft C after own ship stopped in the queue. Scenario Start aircraft D after own ship stopped in the queue. ATC «Revert to voice» DL Pilot «WILCO» DL ATC «AT001, Contact tower on Bye.» VHF Pilot «AT001, contacting tower on Bye» VHF Save date: Public Page 110

111 Pilot scenario This chapter describes the explanations given to the pilot before beginning of scenario Situation The aircraft is an Airbus A380 called «AT001» The mission is a commercial flight with passengers, departure from Paris CDG (LFPG), destination to Toulouse Blagnac (LFBO). We are situated at Paris CDG, on parking D gate D12. We are on the aircraft, after pushback. Doors are closed, engines on. We received the «expect routing» information before pushback. Aircraft is ready to taxi. Visibility is low (horizontal visibility 150 ft) Objective: The aircraft shall leave the gate and taxi for a take of on runway 09R by respecting ATC consigns Communications Communications shall be via CPDLC system every time it is possible. Otherwise, communications shall be via VHF. VHF communications are in French or in English, according to pilot preferences Scenario Type Description Beginning of scenario 3 Action Installation on the cockpit, checklist before taxi Comm DL «request taxi» Wait for evt Wait for ATC answer Comm DL «WILCO» Action Follow the ATC consigns and answer when necessary. End of the scenario when arrived to the objective of taxi (indicated by operator) Scenario 4: Taxiing without CPDLC Experimentation environment Taxi in from a runway threshold to a parking after canceling of takeoff (not planned taxiing). Objective of evaluation: Evaluate the usability of the system without CPDLC system, and standard VHF dialog with ATC. Airport: LFPG (Paris CDG) Own ship aircraft: Airbus A380 Taxi route: Enter on runway 09R by holding point Y3 Leaving runway by taxiway K1 Via taxiways BD5, B, BM4, M, A, NA2, FN1 Taxi up to parking S27 (Terminal A) Environment: Visibility: CAVOK Time: Middle of journey Ground traffic: 8 planes, none on the trajectory Save date: Public Page 111

112 Objective for operator: Do the taxiing by using the CPDLC system and the moving map to visualize the trajectory and the clearances. Respect holding point consigns, the dialog procedures and the taxi limit speeds. Trajectory: Pilot scenario Situation The aircraft is an Airbus A380 called «AT001» The mission is a commercial flight with passengers, departure from Paris CDG (LFPG), destination to Toulouse Blagnac (LFBO). We are situated at Paris CDG, on holding point Y3 before entering runway 09R. The weather is clear. After a failure of flaps, we take the decision to go back to the parking. Objective: The aircraft shall go back to a stand by respecting ATC consigns Communications Communications shall be via VHF. VHF communications are in French or in English, according to pilot preferences Scenario Type Comm VHF Description Beginning of scenario 3 «AT001, because of technical failure, we cancel the flight, we go back to stand» Save date: Public Page 112

113 Wait for evt Action Wait for ATC answer Follow the ATC consigns and answer when necessary. End of the scenario when arrived to the objective of taxi (indicated by operator) Table 4-4: Non-nominal Events Measurements in RTS Data is either obtained during the exercise or after the exercise. During the exercise, data are collected by the HF expert and the software engineer on Observer Test Sheet. After the exercise, data are collected by means of a debriefing questionnaire and a debriefing interview. Observer Test Sheet The aim is to observe cognitive activity of the pilot: understanding of the system, situation awareness, identification of means and aims to achieve the task with respect to the experimental system Scenario Pilot Actions Check of information after message reception - Identification of the aircraft position in the airport - Identification of the final destination - Identification of the route steps - Identification of obstacles (traffic,..) System Interaction: - Manual edition of complementary information - Usage of arc rose, plan mode - Selection of airports - Usage of airport data find tool - Piloting the aircraft on the ground Table 4-5: Observer Test Sheet Debriefing Questionnaire The debriefing questionnaire is detailed in Annex 8.8. Save date: Public Page 113

114 Debriefing Interview The aim is to go deeply into the key points noticed in the observations or the questionnaire. It allows bringing out the operational improvement of the system through different human factor subject: - System usability - Confidence in the system - Situation awareness The debriefing discussion relies on the following canvas: Subject 1 System Usability: I. Do you consider that the system brings you something from one you know? II. III. Are informations from datalink easily understandable? Is interpretation of the symbology difficult in particular conditions (conjunction of several events)? Subject 2 Situation awareness: 1. What kind of information do you use to understand the situation? a. To place you in the airport? b. To place your final destination? c. To place your current clearance? 2. What kind of information do you use to taxi your aircraft? a. To understand the CPDLC message? b. To identify the taxiway of your path? c. To identify the next holding point? 4.5 Training requirements Before exercise, a power-point presentation of the system is performed. The presentation recalls: - the context of the study, - the objective of the simulation - the cockpit simulator - the experimental system - the human factor experiments Then, on the cockpit simulator, the pilot becomes acquainted with the system by a short taxi on the airport. This step stops as soon as the pilot feels at ease with the experimental system. 4.6 Conduct of the study The study is composed of five evaluation days. During an evaluation days four scenarios are tested. The typical planning of an evaluation day is the following: Save date: Public Page 114

115 09.00 am to am: Presentation - Briefing am to am: Training am to am: Pause am to am: System evaluation (1 st run) pm to pm: Debriefing pm to pm: Lunch am to am: System evaluation (2 nd run) pm to pm: Debriefing Usually, two scenarios are tested during a run. A full scenario test run can be broken up into 3 phases: [0-5 ] Warm-up: transitory phase used to launch the scenario, provide necessary instructions to pilot, explain conditions and prepare the transfer to exercise phase [5-35 ] Exercise phase: the pseudo-controller sends instructions to the pilot as described in the scenario. Pilot uses the experimental system to taxi the aircraft according to the controller instructions. [35-37 ] Break. 4.7 Analysis Methods The data obtained from the different pilots are gathered and analysed by the assessment team in order to point out the main results. The opinions shared by the majority of the pilot will be underlined. General themes and preoccupations meet will be extracted trough the analysis made. More individual opinions and observations concerning only few pilots will be analysed in their specific context. An arithmetic mean is computed for the results of the questionnaire. Due to the small population studied, the results can only be used to see a general trend, the figure obtained is not relevant. The data recorded for the trajectories will enable to draw on a graph the position of the wheels during the taxiing. In a first time, it is not foreseen to analyse in details the trajectories recorded. Save date: Public Page 115

116 5 TAXI-CPDLC: Display of taxi clearance (THAV) This section is about the validation of display part of the TAXI-CPDLC service. That is the display of taxi clearance and expected routing on a moving map. In the remainder of this chapter, this function is called TAXI CLEARANCE DISPLAY. The validation of TAXI CLEARANCE DISPLAY is managed by THAV on the same environment as the HUD/SGS validation (see section 4). This is the reason why the content of many sub-sections in this chapter is identical to the HUD/SGS chapter and will refer to the HUD/SGS chapter (marked see HUD/SGS ). The validation of TAXI CLEARANCE DISPLAY consists in two campaigns of 5 evaluation days called RTS1 and RTS2. In RTS1, the TAXI CLEARANCE DISPLAY is evaluated alone. In RTS2, TAXI CLEARANCE DISPLAY is evaluated together with HUD/SGS service. Fro each evaluation day, a pilot is invited to play 4 scenarios on a THAV cockpit simulator implementing the TAXI CLEARANCE DISPLAY function. 5.1 Validation Aims, Objectives, and Hypotheses The purpose of this section is to clarify what is to be achieved from the EMMA2 validation exercises in 2-WP6.2 for the TAXI CLEARANCE DISPLAY service. It contains the general aims in accordance with 2-D6.1.1b Generic Experimental and Test Plan [6]. The basic aim of the EMMA2 Project is the verification of higher A-SMGCS services as described in the ICAO Manual [4] and further refined in the SPOR [5]. Two areas of validation activities have been considered. The Operational Feasibility area refers to the definition of the operational use of equipment and procedures, in accordance with the performances assessed in the previous stage. It answers the question: Given the performances of the equipment, is it usable and acceptable? The Operational Improvements area refers to the evaluation of the operational improvements, in terms of Safety, Capacity, Efficiency, Human Factors, and Environmental Impact using the equipment and the procedures defined in the previous stage. It answers the question: Given the accepted HUD/SGS equipment and procedures, how is guidance improved? Validation Aims The validation aims at assessing the operational feasibility and the operational improvement of the onboard TAXI CLEARANCE DISPLAY service as defined in the 2-D1.1.1 SPOR document. Operational feasibility assessment is achieved by means of RT Simulation on a cockpit simulator: the crew will use the improved service during taxi-in and taxi-out procedure in condition as closest as possible from the reality. The RT simulation is also used to state operational improvement. Regarding validation constraints (training time and simulator conditions) and level of maturity of the service, observation and questionnaire have been selected to collect results from the RT simulations Validation Objectives Validation activities deal with verifying operational feasibility and operational improvement of the TAXI CLEARANCE DISPLAY service. Save date: Public Page 116

117 Operational Feasibility Operational feasibility is proved against operational requirements dedicated to the TAXI CLEARANCE DISPLAY service in the framework of procedure defined in the EMMA2 2-D1.1.1 SPOR [5]. High-level Objective 1 Table 5-1: Verification of EMMA2 Operational Requirements and Procedures High-level objectives in the area operational feasibility in EMMA Operational Improvement Once the operational feasibility of the TAXI CLEARANCE DISPLAY service has been proved, the validation will take the improvements into account. Two fields are addressed: safety and working performance. Usage of the TAXI CLEARANCE DISPLAY service may have impact in terms of Capacity, efficiency and environment. However, at this step of service development, only primary impacts shall be considered. For the same reasons, the operational improvement will not be metered by quantifying differences between experiments in a baseline system and a improved system but through questionnaire and debriefing with the pilots. High-level Objective 2 High-level Objective 4 Increase of Safety Suitability of Behaviour and Working Performance Table 5-2: High-level objectives in the area operational improvements in EMMA Low-Level validation objectives The following table details the derivation of high-level objective into low-level objectives. Concerning Operational feasibility, only low-level Objective attached to operational requirements dedicated to TAXI CLEARANCE DISPLAY service are taken into account. Concerning Operational improvement, the RT simulation in a cockpit simulators and the maturity level of the service, safety perception, workload and situational awareness are in the scope of the evaluation. Area High Level Objectives Low Level Objective Operational Feasibility Operational Improvements Verification of EMMA2 Operational Requirements and Procedures [HLO1] Increase of Safety [HLO2] Verification of the guidance requirements [LLO1.8] Verification of the aircraft onboard requirements [LLO1.9] General usability of new services [LLO1.12] Increment of safety perception by users [LLO2.2] Save date: Public Page 117

118 Area High Level Objectives Low Level Objective Suitability of Behaviour and Working Performance [HLO4] Appropriate level of user s workload [LLO4.1] Improved situational awareness of users [LLO4.2] Table 5-3: The low-level objectives related to high-level objectives and areas of activity Hypotheses, Indicators, Measurement Tools Hypotheses Operational Feasibility Hypotheses Identifier OF-H0 OF-H1 Hypothesis The pilots opinion does not agree to the operational feasibility aspects of TAXI CLEARANCE DISPLAY service The pilots opinion agrees to the operational feasibility aspects of TAXI CLEARANCE DISPLAY service Operational Improvement Hypotheses Safety-related Hypotheses In order to examine the safety, subjective measurements will be conducted. The following hypotheses will be tested: S1-H0 S1-H1 Hypothesis An increment of safety while using TAXI CLEARANCE DISPLAY is not perceived by the pilots An increment of safety while using TAXI CLEARANCE DISPLAY is perceived by the pilots Behaviour and Working Performance Hypotheses S2-H0 S2-H1 Hypothesis Situation awareness of pilots is not improved while using TAXI CLEARANCE DISPLAY Situation awareness of pilots is improved while using TAXI CLEARANCE DISPLAY HF-H0 HF-H1 Hypothesis An increment of workload while using TAXI CLEARANCE DISPLAY is not perceived by the pilots An increment of workload while using TAXI CLEARANCE DISPLAY is perceived Save date: Public Page 118

119 by the pilots Indicators and Measurement Tool The measurement tools used in TAXI CLEARANCE DISPLAY experimentations are: Observations Debriefing Questionnaire Debriefing Interview After simulator trials and field / flight tests, users will have the opportunity to reflect about their experience gained in the trials. The test coordinator makes sure that these data will be collected and analysed. 5.2 The test environment in Thales Avionics The validation platform See chapter Airport Navigation application The prototype of the Airport Navigation application used for this activity covers the following functionalities: Airport moving map function presenting the own-ship and the traffic situation on the airport, with interactive menu allowing managing zooming, display mode, airport selection. Path management function allowing: Observing the expect routing and the clearance sent by the CPDLC application Or to create interactively its own path according to the controller information. The path and the taxi clearance are displayed on the moving map, allowing the pilot to manage the navigation of the aircraft on the airport platform according to traffic information Validation team roles See chapter Research Personnel See chapter Experimental Participants The experimental participants for RTS1 campaign are: Save date: Public Page 119

120 Num Background (Flight test, Airlines Training) Nation ality Native Languag e Sex (F/M) Age class (Ex.: 40-44) Hierarchical (Captain/ F/O) Total amount of flight hours Current A/C type Former A/C type 1 Flight test F F M Captain NC NC NC 2 Airline F F M Captain B777 A330, A340, B747, B737, C160, N2501 noratlas 3 Airline F F M F/ B A320 4 Airline F F M F/O 4500 A320 5 Airline F F M F/O 8000 A320 A300 The experimental Participants for RTS2 campaign are: Num Background (Flight test, Airlines Training) Nation ality Native Languag e Sex (F/M) Age class (Ex.: 40-44) Hierarchical (Captain/ F/O) Total amount of flight hours Current A/C type Former A/C type 6 Flight test F F M Captain NC NC NC 7 Airline F F M Captain B777 A330, A340, B747, B737, C160, N2501 noratlas 8 Airline F F M F/ B A320 9 Airline F F M F/O ATR 42/72 BeechCraft Airline F F M Captain 8000 A320 A300 Note: some pilots have participated to RTS1 and RTS Pseudo operators See chapter Observers See chapter The Baseline System Not applicable Pilot Roles See chapter Baseline System Procedures Not applicable. Save date: Public Page 120

121 5.3 The experimental system [1] Figure 5-1: Experimental system The Controller Pilot Data Link Communications (CPDLC) and ground clearance display function are avionic functions that are intended to support the flight crew of an aircraft during surface movement operations. These applications allow receiving, validating and visualizing navigation information sent by the control tower. The expected benefits of this kind of communication are: To reduce the radio frequencies channels congestion and, consequently, in the long run, to increase the traffic on the airport, To help prevent dangerous errors in surface navigation such as runway incursion or take-off from a taxiway or the wrong runway respecting the controller instructions, To reduce pilot workload, To contribute to reduce taxiing delays, especially on complex airfields Global description of the CPDLC and ground clearance display functions prototype The CPDLC application The CPDLC application prototype provides the flight crew a dedicated HMI accessible to both pilots on the ND (Figure 5-2 D-LINK upper menu cf. red frame) and PFD FO displays ([1] fig3-1). Save date: Public Page 121

122 Figure 5-2: CPDLC menu on ND display This function permits the pilots: To receive the controller navigation information, To control and validate them, To reach permanently these data. The ground clearance display function To remain independent from this CPDLC application, a ground clearance edition function is supplied to pilots. This function is accessible in the IMM upper menus (Figure 5-3 Manual option). Save date: Public Page 122

123 Figure 5-3: Ground clearance display menu This application enables pilots to edit his own expect routing and to set the right clearance interactively on the map The CPDLC HMI The CPDLC HMI allows pilots: To receive navigation controller instructions with the UPLINK menu, To send navigations requests. The UPLINK menu When the pilots receive a controller instruction they hear a sound signal and a green flag with a textual information are displayed at the foot of the IMM interface (Figure 5-4 blue frame). Save date: Public Page 123

124 Figure 5-4: The controller instruction A specific menu permits to quickly manage this information (Figure 5-5). Save date: Public Page 124

125 [1] [2] Figure 5-5: The CPDLC UPLINK HMI This menu displays the current controller instruction ([1] Figure 5-5) and furnishes the pilots with interactive buttons to treat the information. The controller sends messages like: Expect routing, Start up, Pushback and Taxi clearances And communication instructions (Revert to voice, frequencies specification). In this example (Figure 5-5), an expected routing is sent to the A/C. Then, the pilot, or co pilot, can LOAD the trajectory. At that time this one is represented on the IMM ([1] Figure 5-6). Save date: Public Page 125

126 [1] Figure 5-6: Expect routing loaded Thanks to this clear graphical display, the pilot can validate the instruction sending ROGER to the control tower. When the validation is made, the expect routing appears on the Integrated Moving Map and, with an adapted symbology, on the HUD. That is the same mechanism when the pilots receive taxi clearances (magenta way Figure 5-7). The only difference is that the pilot sends WILCO to specify to the controller that he is going to do the action required. Save date: Public Page 126

127 [1] Figure 5-7: Taxi clearance The destination will be always displayed on the map, representing by a white target ([1] Figure 5-7). When this target is no longer visible a white arrow indicates the direction of the destination point ([1] Figure 5-8). [1] Figure 5-8: Destination direction arrow Save date: Public Page 127

128 Furthermore, a really useful function is that the application memorizes the different navigation information sent by the controller. They are easily accessible at any time by pressing the up and down arrows on the HMI ([2], Figure 5-5) Experimental System Procedures The procedures applicable for the experimental validation exercises are those described in each scenario Influencing Factors See chapter Experimental Design Experimental constraints Material constraints The material constraints have been analysed before the evaluation. The most important limits identified for the material constraints are: The display angle of the simulator is only 90 and does not enable the pilot to have a vision on the right side of the cockpit, which is useful in the reality to help acquiring references during sharp right turns in taxiing. The lighting of the taxiways is not operative on the flight simulator. This lighting is used for taxiing in bad weather conditions or at nighttimes. The taxi model of the aircraft is not very realistic; the piloting is a little different from a real aircraft. There is no simulation of the effect of a wheel going in the grass and getting bogged down. There is no simulation of interference with obstacle or other aircraft. The cockpit is not symmetric, as the FO position has limited commands: he can only manage communication from datalink and check aircraft situation on an Integration Moving Map Experimental conditions The following elements are limits related to experimental conditions of the evaluation: The simulator has a crew configuration, but only the PF is under evaluation, the PNF is not a real pilot but a member of the evaluation team. Except for traffic, the effective risks of taxi operations are not simulated (no effect if a wheel gets out of the taxiway, no obstacles simulated) and in these conditions, the possible stress of that flight phase is very limited. The general workload of a taxi phase has not been reproduced for the evaluation (A/C preparation for take off, clearance reception, etc ) Time schedule The following Gantt chart gives a rough overview about the main steps of this validation campaign. The campaign is composed of two session of 5 evaluation days called RTS1 and RTS2. In the first session, the system is validated alone and in the second session the system is validated together with the HUD/SGS service. Save date: Public Page 128

129 Figure 5-9: Gantt Chart of the RTS1 campaign Figure 5-10: Gantt Chart of the RTS2 campaign Selected Design The condition of evaluation days is always the same: the pilot shall always play the same scenarios. Scenario description specifies the experimental condition of each scenario Scenarios specifications See chapter Measurements in RTS See chapter Training requirements See chapter Conduct of the study The study is composed of tow session of five evaluation days. During an evaluation days four scenarios are tested. The typical planning of an evaluation day is the following: am to am: Presentation - Briefing am to am: Training am to am: Pause am to am: System evaluation (1 st run) pm to pm: Debriefing pm to pm: Lunch am to am: System evaluation (2 nd run) pm to pm: Debriefing Usually, two scenarios are tested during a run. Save date: Public Page 129

130 A full scenario test run can be broken up into 3 phases: [0-5 ] Warm-up: transitory phase used to launch the scenario, provide necessary instructions to pilot, explain conditions and prepare the transfer to exercise phase [5-35 ] Exercise phase: the pseudo-controller sends instructions to the pilot as described in the scenario. Pilot uses the experimental system to taxi the aircraft according to the controller instructions. [35-37 ] Break. 5.7 Analysis Methods See chapter 4.7. Save date: Public Page 130

131 6 SMA and ADS-B trials in GA (FAV) 6.1 Validation Aims One area of validation activities has been considered. The Operational Feasibility area refers to the definition of the operational use of equipment and procedures, in accordance with the performances assessed in the previous stage. It answers the question: Given the performances of the equipment (here: SMA and ADS-B), is it usable and acceptable? 6.2 The test environment Validation platform For the GA trials a Piper Archer 2 (Piper PA28-181) is deployed (Figure 6-1). The aircraft is owned by a flight club based in Egelsbach (EDFE). It will be chartered and flown to Prague (LKPR) for one day (optimal case). Figure 6-1: Piper Archer 2 (Piper PA28-181) in PRG For ADS-B In the aircraft is equipped with a Selex 1090ES Receiver and the CDTI-2000 as moving map device. ADS-B Out is realized with a TRT-800A Mode-S Transponder and a GPS1201 which is a GPS/WAAS Sensor of Free Flight Systems. The respective equipment components can be found in the Figure 6-2. Save date: Public Page 131

132 The SMA algorithm for the GA aircraft is integrated in the CDTI-2000, however to support this functionality additional components where required. The picture below shows the installed components and their relations. Figure 6-2: Experimental System in FAV s GA trials The following table describes the equipment components and their role for the EMMA2 on site trials Component Manufacturer Description TRT-800A Funkwerk Avionics Class 1 Mode-S transponder with ADS-B Out capability GPS1201 Free Flight Systems 12 channel GPS/WAAS Sensor compliant to FAA TSO C145a provides integrity by Fault Detection and Exclusion (FDE) mechanism 1090ES_RX Selex Communications 1090 MHz ADS-B Receiver CDTI-2000 Funkwerk Avionics Cockpit Display including Airport Moving Map, Traffic Display, Conflict Detection and Surface Movement Alerting algorithm SAE 5-35 Sandia Barometric Altitude Encoder to support ownstate altitude determination TNL Approach Trimble Original Equipage of the aircraft used for ownstate positioning It should be noted that the transponder broadcasts the ADS-B messages using a single antenna at the bottom of the aircraft. The ground vehicles will be equipped with a SQB unit developed by ERA. The SQB units comprise a GPS receiver as well as a 1090MHZES transmitter and can easily be attached on the vehicles roof. Save date: Public Page 132

133 6.2.2 Validation team roles The Team consists of one pilot and one research scientist. The pilot is a flight trainer employed by the flight club. The pilot s tasks comprise the control of the aircraft and the communication with tower control. The research scientist observes the behaviour of equipment and pilot during the different iterations of the scenarios. 6.3 The Experimental System - CDTI-2000 The CDTI enhances the pilot s situational awareness by providing positional awareness with the Airport Moving Map and traffic awareness by displaying ADS-B/TIS-B traffic items in relation to the own position. For the on-site trials it did not support OAF (Operational Awareness) and CAF (Clearance Awareness), since CPDLC is beyond any operational considerations for GA aircraft. Hence, the CDTI-2000 will not have information about the taxi clearance (e.g. via CPDLC), the Surface Movement Alert (SMA) functions are restricted to the following three features: Detection of other aircraft on runway during final approach and landing Detection of other aircraft on runway during take-off roll Alerting upon entering a runway The SMA algorithm is based on the own state determination of the CDTI and the traffic data store. In order to identify a traffic item as an intruder the CDTI firstly determines the own flight phase as an approach or take-off. Detection of Runway Incursion during Approach and Landing The CDTI detects the final approach and landing phase when all of the following conditions are true: The ground speed is above 40 knots. The aircraft altitude is no more than 1000 feet above the airport elevation. The aircraft magnetic track differs by no more than 15 from the direction of a runway whose threshold is within 5 NM from own aircraft position. That runway is now determined to be the assigned landing runway. The CDTI determines all traffic items of the internal traffic store that are within 500 feet of the airport elevation and whose lateral position is within any polygon of the assigned landing runway. Note: such traffic items are said to "occupy" that runway. If any such traffic item that is determined to occupy the assigned landing runway is less than 2 NM away from the current own aircraft position, this traffic item is marked as a conflicting target and a warning is displayed on the CDTI s screen. Detection of Runway Incursion during Take-Off The CDTI detects the take-off phase when all of the following conditions are true: The aircraft altitude is no more than 1000 feet above the airport elevation The aircraft magnetic track differs by no more than 2 from the direction of a runway whose threshold is within 5 NM from own aircraft position. The aircraft is located within any polygon of that runway. The ground speed has risen from below 10 knots to above 20 knots. That runway is now determined to be the assigned take-off runway. The CDTI determines all traffic items of the internal traffic store that are within 500 feet of the airport elevation and whose lateral position is within any polygon of the assigned take-off runway. Note: such traffic items are said to "occupy" that runway. Save date: Public Page 133

134 If any such traffic item that is determined to occupy the assigned take-off runway is less than 2 NM away from the current own aircraft position, this traffic item is marked as a conflicting target and a warning is displayed on the CDTI s screen. Detection of Runway Entrance Whenever the own aircraft position passes a holding position (stopbar) in the direction toward the runway, the CDTI-2000 determines this as entering the respective runway. Alerting Any traffic item that is found to be a conflicting target either during approach and landing or during take-off is highlighted with red colour (for both the target symbol and the data tag). The CDTI-2000 automatically increases the display range so that all conflicting targets are displayed. While there is at least one conflicting target, there is an indication below the own aircraft symbol in the form of a white frame with black background that contains in red colour the text "RUNWAY INCURSION!" Upon the event of entering a runway (passing a holding position), the CDTI-2000 indicates that event for 5 seconds with an indication below the own aircraft symbol in the form of a white frame with black background that contains in yellow colour the text "ENTERING RWY XXX" (with XXX = runway designator). 6.4 Experimental Design Experimental Constraints Experimental Conditions During the trials it is foreseen to perform in principle two different scenarios. 1. An aircraft approaching a runway blocked by a vehicle. 2. An aircraft taking-off, but again with blocking vehicle on the present runway. Initially the approach scenario was planned to happen on the closed Runway 22 with the intention to not disturb the normal traffic. The initial planning for the Take-Off scenario was to start the aircraft on an operating runway (RWY13) or alternatively also on the closed RWY22 and locate the ground vehicle in a blocking way either by crossing RWY13 on RWY22 or with the other option by crossing RWY22 on a taxiway. After discussion with the local ATC and the airport operator the scenarios were adjusted in terms of impact on safety and traffic capacity. It is against controller rules to allow an approach or take-off on a closed runway; furthermore the blocking of an operational runway needs to be avoided. The finally proposed scenario outline for the approach is to simulate a landing on RWY 13, while the ground vehicle (TUD Van) crosses RWY 13 on RWY22. The aircraft would finish the approach with a goaround and land afterwards. In the case of take-off, the scenario is limited to a simulation only. Since the airborne conflict detection algorithm identifies the take-off phase by a certain increase of the ground speed Material Constraints Single antenna (bottom)..the ADS-B Out messages are broadcasted by an antenna mounted at the bottom of the aircraft, which might lead to a loss of signals due to shadowing effects. During the trials this happens only one time, while simulating a take-off. This effect was boosted by the unfavourable slope of the runway (RWY04/LKPR). Save date: Public Page 134

135 SMA restrictions for GA The SMA algorithm does not consider any intentions, neither of the target nor of the own aircraft. For example it does not warn the pilot if he moves the aircraft towards a runway currently approached by another aircraft. Missing CPDLC: Since for GA no CPDLC is considered, no informations on taxi clearances or blocked runways are available Time Schedule The trials are foreseen to happen in week 35. In an optimal case the trials would last one day depending on the weather conditions. However two days are foreseen to cope with weather uncertainties. The aircraft will start on the 27 th August 08 at the earliest possible time (08:00 CET) in Egelsbach (EDFE) Scenarios specifications It is foreseen to perform two different scenarios. 1. An aircraft approaching a runway blocked by a vehicle. 2. An aircraft taking-off, but again with blocking vehicle on the present runway Approach Scenario EMMA-VAN-GA-01 Runway Incursion #1 / GA Final Approach Remarks / ATC Location Van Crew Task Prague Airport (LKPR) Roll along Taxiway L from West to East and cross RWY 22. Display Configuration: Airport Moving Map on ND GA Crew Task Final Approach on RWY 13. Scenario Script The GA crew is in final approach on RWY 13. The van crew rolls on RWY 22 towards RWY 13 and crosses the runway while GA is at 1000ft. CPDLC is not available. As soon as the alert is triggered in the GA, the crew executes a Go Around. As soon as the alert is triggered in the Van, the crew exits the runway. If no alert has come when the GA is at 700ft, the GA crew executes a go around. Subjective Measurements Objective Measurements Post-run and debriefing questionnaires. Post-run and debriefing pilot comments. Logging of Alerts from the Van and the GA. Table 6-1: Scenario 1: FAV & TUD OST Save date: Public Page 135

136 Figure 6-3: Approach Scenario - Entry positions (FAV aircraft and TUD Van) Take-Off Scenario EMMA-VAN-GA-02 Runway Incursion #3 / GA in Take Off mode Remarks / ATC Location Van Crew Task GA Crew Task Prague Airport (LKPR) Roll along Taxiway M from West to East and cross RWY 04. Simulate Take-Off on RWY04 by accelerating to a ground speed above 20kts. Display Configuration: Airport Moving Map on ND Scenario Script Subjective Measurements Objective Measurements The GA crew starts Take Off from RWY 04 after having received the clearance from ATC. The van crew rolls on Taxiway M towards RWY 04 and crosses the runway. As soon as the alert is triggered in the GA, the crew aborts take off. As soon as the alert is triggered in the Van, the crew exits the runway. Post-run and debriefing questionnaires. Post-run and debriefing pilot comments. Logging of Alerts from the Van and the GA. If no alert has come when the GA is at 40kts, the GA crew aborts take off. Table 6-2: Scenario 2: FAV & TUD OST Save date: Public Page 136

137 Figure 6-4: Take-Off Scenario - Entry positions (FAV aircraft and TUD Van) No further planning is made, it is foreseen to have a briefing with pilots, drivers, coordinators and tower controllers before the start of activities. This meeting is deemed to consider operational restriction etc Safety Net for On-Site Testing For the approach scenario an abort point is defined in discussions with Prague ATC. That abort point (see Figure 6-5) is chosen to be at the runway intersection with taxiway P (Papa). This intersection is defined and shall never be transgressed by the aircraft, meaning the cusp of the turn being well in front or in any case not beyond an imagined line (red line) through that point. Figure 6-5: Abort point in OST (red line) Save date: Public Page 137

138 The Take-Off scenario only requires an acceleration to a speed above 20kts, in order to simulate a take-off and trigger an alert. However, during the test while moving on the south-western part of the closed runway 22/04 no crossing of the operating RWY31/13 is allowed Measurements During the life trials the function of the SMA algorithm shall be approved, the scenarios were set up to fulfil the single conditions height above runway, speed and direction as well as distance to target vehicle. As a baseline for the post-trial evaluation the logging function of the CDTI-2000 will be applied. The log file shall contain records of ADS-B target positions as well as records of the own state. Furthermore the CDTI will record the triggering time of runway alerts. All the records in the log are accompanied by a timestamp derived by the CDTI system clock which itself is synchronised with the GPS time derived by the Trimble 2000 GPS device. No questionnaire will be completed, but debriefing comments by the pilot need to be collected. Results of the debriefing will be reported in EMMA2 deliverable 2-D661 Airborne Validation Results Part A. 6.5 Training requirements No training activities are foreseen either for the observation crew or for the pilot. 6.6 Conduct of the Study T0 Flight to Prague (LKPR) T0+2 hours Meeting with all involved parties to clarify operational considerations T0+4,5 hours Approach Scenario (3 iterations planned) T0+6 hours Take Off Scenario (number of iterations not fixed) T0+7 hours Flight back to Egelsbach (EDFE) In order to consider weather impacts the trial date could be shifted by one day or splitted into two consecutive days if convenient. 6.7 Analysis Methods The CDTI will log the traffic data including the ownship and target positions combined with GPS timestamps, it will also log the time at which a runway incursion is triggered. Those data enable a comparative analysis of for example: Who is the first being aware of a conflict. During a discussion after the trials the pilot s opinion and comments are collected. It should be noted that only one Pilot is participating in the trials, hence the statement can only be seen as a subjective point of view. Save date: Public Page 138

139 7 Annex I 7.1 References [3] EUROCONTROL, EUROPEAN Operational Concept Validation Methodology (E-OCVM) 2.0, Brussels, March [4] ICAO, Advanced Surface Movement Guidance and Control Systems (A-SMGCS) Manual, Doc 9830 AN/452, First Edition, [5] EMMA2, A-SMGCS Services, Procedures and Operational Requirements, 2-D1.1.1_SPOR, Version 0.13, 2008, [6] EMMA2, Generic Experimental and Test Plan, 2-D6.1.1b_GETP, Version 0.05, 2008, [7] EMMA, Validation Test Plan PRG, European Commission, DG TREN, The Sixth Framework Programme, D613, Version , [8] EMMA2, Ground System Requirements for High-Level A-SMGCS at Prague Airport, 2- D3.1.1_GSR, Version 0.09, [9] C. Vernaleken, Autonomous and Air-Ground Cooperative Onboard Systems for Surface Movement Incident Prevention, PhD thesis (in preparation). [10] C. Vernaleken, L. Mihalic, M. Güttler and U. Klingauf, A fresh look at runway incursions: onboard surface movement awareness & alerting system based on SVS, Proc. SPIE, Enhanced and Synthetic Vision, Vol. 6226, Orlando [11] P. Wipplinger, Untersuchung des Pilotenverhaltens bei HALS/DTOP-Anflügen, Dissertation, Technische Universität Darmstadt List of Figures Figure 2-1: DLR s Generic Experimental Cockpit (GECO) Figure 2-2: PFD, NAV, ENG and FCU in GECO Figure 2-3: Architecture of Cockpit & Tower Simulation, Pseudo Pilots and DMAN Figure 2-4: ATTAS VFW614 during EMMA2 on-site trials in PRG Figure 2-5: Test pilots side in ATTAS VFW614 during EMMA2 on-site trials in PRG Figure 2-6: CDTI by Funkwerk Avionics Figure 2-7: Send menu on CDTI Figure 2-8: Reply menu on CDTI Figure 2-9: Log menu on CDTI Figure 2-10: TAXI-CPDLC expected clearance for TWY H (NAV mode) Figure 2-11: TAXI-CPDLC expected clearance for TWY H, A, and RWY 24 (PLAN mode) Figure 2-12: TAXI-CPDLC clearance for TWY H, A approved by ATCO (NAV mode) Figure 2-13: TAXI-CPDLC clearance for TWY H, A approved by ATCO (PLAN mode) Figure 2-14: TAXI-CPDLC clearance for TWY H, A after PNF s wilco (NAV mode) Figure 2-15: TAXI-CPDLC clearance for TWY H, A after PNF s wilco (PLAN mode) Figure 2-16: On-board Equipment EMM and CDTI (DLR Real Time Simulation) Figure 2-17: ATTAS parking position during the OST: S Figure 2-18: Taxi Route in EXE02, Pt. 1 outbound (Gate 34 RWY24) Figure 2-19: Taxi Route in EXE02, Pt. 2 inbound (RWY24 Gate 34) Figure 2-20: Taxi Route in EXE04, Pt. 1 outbound (Gate 34 RWY06) Figure 2-21: Taxi Route in EXE04, Pt. 2 inbound (RWY06 Gate 34) Figure 2-22: Taxi Route in EXE06, Pt. 1 outbound (Gate 34 RWY24) Figure 2-23: Taxi Route in EXE06, Pt. 2 inbound (RWY24 Gate S6) Figure 2-24: Taxi Route in EXE06, Pt. 3 outbound (Gate S6 RWY24) Figure 3-1: TUD Van during tests on closed RWY04/22 on Prague airport, February 6th, Figure 3-2: Antenna mounting plate on the roof of TUD's Navigation Test Vehicle Save date: Public Page 139

140 Figure 3-3: Hardware and computer racks in the back of TUD s Navigation Test Vehicle Figure 3-4: LCD screen installation in front of passenger seat Figure 3-5: Elements of the Surface Movement Awareness & Alerting System (SMAAS) Figure 3-6: Time schedule Figure 3-7: Scenario FAM Figure 3-8: Scenario FAM Figure 3-9: Scenario SMA Figure 3-10: Scenario SMA Figure 3-11: Scenario SMA Figure 3-12: Scenario SMA 04 and SMA Figure 3-13: Overview of Prague airport (LKPR) Figure 3-14: EMMA VAN-01 Take Off RWY 04 Traffic On/Approaching RWY Figure 3-15: EMMA-VAN-02 Take Off RWY 04 without clearance Figure 3-16: EMMA-VAN-03 Collision Hazard with other Traffic on Taxiway Figure 3-17: EMMA-VAN-04 Collision with other Traffic on Taxiway Figure 3-18: EMMA-VAN-05 Crossing RWY 04 without clearance Figure 3-19: EMMA GA-01 Takeoff of RWY 13, Traffic on Approach RWY Figure 4-1: Cockpit simulator Figure 4-2: Team roles Figure 4-3: Screenshot of Pseudo-operator Display Figure 4-4: Screenshot of Supervisor Display Figure 4-5: Experimental system Figure 4-6: Surface Guidance symbology on straight line Figure 4-7: Stop bar Figure 4-8: Turn and clearance information Figure 4-9: Runway ahead information Figure 4-10: The top sight view Figure 4-11: Guidance cue Figure 4-12: GS, heading and altitude information Figure 4-13: Gantt Chart of the campaign Figure 5-1: Experimental system Figure 5-2: CPDLC menu on ND display Figure 5-3: Ground clearance display menu Figure 5-4: The controller instruction Figure 5-5: The CPDLC UPLINK HMI Figure 5-6: Expect routing loaded Figure 5-7: Taxi clearance Figure 5-8: Destination direction arrow Figure 5-9: Gantt Chart of the RTS1 campaign Figure 5-10: Gantt Chart of the RTS2 campaign Figure 6-1: Piper Archer 2 (Piper PA28-181) in PRG Figure 6-2: Experimental System in FAV s GA trials Figure 6-3: Approach Scenario - Entry positions (FAV aircraft and TUD Van) Figure 6-4: Take-Off Scenario - Entry positions (FAV aircraft and TUD Van) Figure 6-5: Abort point in OST (red line) List of Tables Table 1-1: E-OCVM sub-steps and traceability in EMMA2 deliverables Table 2-1: High and Low-Level Validation Objectives in DLR s EMMA2 on-board trials Table 2-2: High-level objectives, low-level objectives, indicators and metrics Table 2-3: TAXI-CPDLC standard procedures for an outbound flight Table 2-4: TAXI-CPDLC standard procedures for an inbound flight Table 2-5: TAXI-CPDLC advanced procedures for an outbound flight Save date: Public Page 140

141 Table 2-6: TAXI-CPDLC advanced procedures for an inbound flight Table 2-7: TAXI-CPDLC procedures for an outbound 24 flight during on-site trials Table 2-8: TAXI-CPDLC procedures for an inbound 24 flight during on-site trials Table 2-9: TAXI-CPDLC procedures for an outbound 06 flight during on-site trials Table 2-10: TAXI-CPDLC procedures for an inbound 06 flight during on-site trials Table 2-11: Independent variable and levels in RTS Table 2-12: Combination of Experimental Factors Table 2-13: Scenario Description Overview RTS (Part 1, June 2008) Table 2-14: Scenario Description Overview RTS (Part 2, October 2008) Table 2-15: List of GECO call signs for every test run Table 2-16: Pilots Test Schedule Table 2-17: Sequence of Simulator Test Runs in DLR RTS Table 3-1: Sensor equipment of TUD s Navigation Test Vehicle Table 4-1: High-level objectives in the area operational feasibility in EMMA Table 4-2: High-level objectives in the area operational improvements in EMMA Table 4-3: The low-level objectives related to high-level objectives and areas of activity Table 4-4: Non-nominal Events Table 4-5: Observer Test Sheet Table 5-1: High-level objectives in the area operational feasibility in EMMA Table 5-2: High-level objectives in the area operational improvements in EMMA Table 5-3: The low-level objectives related to high-level objectives and areas of activity Table 6-1: Scenario 1: FAV & TUD OST Table 6-2: Scenario 2: FAV & TUD OST Save date: Public Page 141

142 8 Annex II 8.1 Biographical Questionnaire (DLR trials) Pilot pre-trial questionnaire Name:.. Company:.. Background (Flight test, Airlines Training) Nationality Native Language Left/right handler (L/R) Sex (F/M) Age class (ex : 40-44) Hierarchical (Captain/ F/O) Amount of flight hours Application knowledge Aircraft currently operated: Aircraft previously operated: Base Airport:.. Have you operated to/from PRG before 2008?.. Have you conducted research simulation before? If yes, please give details below: Have you any experience of using ground guidance Moving Map System (MMS) or higher A-SMGCS services? If yes, please give details below: In your experience, what issues/problems need to be covered by a ground guidance Moving Map System (MMS) or higher A-SMGCS services? Save date: Public Page 142

143 8.2 System Usability Scale Strongly disagree Strongly agree 1. I think that I would like to use this system frequently I found the system unnecessarily complex I thought the system was easy to use I think that I would need the support of a technical person to be able to use this system I found the various functions in this system were well integrated I thought there was too much inconsistency in this system I would imagine that most people would learn to use this system very quickly. 8. I found the system very difficult to use I felt very confident using the system I needed to learn a lot of things before I could get going with the system If you have any additional comments, please add them here: Save date: Public Page 143

144 8.3 ASSA Flight Crew Questionnaire Airport Surface Situational Awareness ASSA Strongly disagree Strongly agree 1. Display use did not increase head down time Display clutter was not a problem Aided in surface traffic awareness Aided in surface visual traffic acquisition Increased time for checklist. 6. Increased time for crew duties. 7. Surface map easy to bring up. 8. Ownship symbol easy to see Accurately shows my position More info should be displayed Useful info provided Easy to get orientated to MMS Display Aid in determining position of traffic Aid in understanding ATC communication Aid in locating traffic Save date: Public Page 144

145 8.4 I.S.A. Questionnaire WORKLOAD Excessive High Comfortable Relaxed Under-utilised Save date: Public Page 145

146 8.5 EMMA2_QE2-OF_Pilot: Operational Feasibility Questionnaire for Pilots Introduction: Date: Subject Number: Company: Age: Gender: Pilot Experience: A-SMGCS Experience: Simulation Facility / Test Airport: Test Phase: Below you find questions/statements for which we are interested in your personal opinion how far you can agree to or not (answers from 1 Strongly disagree to 6 Strongly agree ). Your answers will help us to decide if the system design, system performances or new procedures have met your demands. There are two additional columns where you can make a cross: If you feel that the content of a statement is of no value for you NOT IMPORTANT or you were not affected by this item during the test: NOT AFFECTED, e.g. you are asked to the alert performance but you wasn t affected by such an alert. Please make a cross in those columns then. Please refer your answers to the experiences you gained while you were using the new EMMA2 cockpit services (Ground Traffic Display, Traffic Conflict Detection, Head-Up Navigation, TAXI-CPDLC, etc.) which are called new cockpit services or new display in the statements below. If you want to provide additional comments/explanations you can use the whole row. Your data will be kept confidential. Thank you in advance. Save date: Public Page 146

147 ID 2-G 3-G 5-G 7-G Questions / Statements When working with the new cockpit services, procedures, responsibilities and functions are clearly defined. Please answer for each particular service: Strongly disagree Disagree Slightly disagree Slightly agree Agree Strongly agree Not important Not affected a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC I think the new cockpit services were well integrated into the existing systems. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC The new cockpit services are capable of being used appropriately when operating within the movement area. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC Even in case of a failure of an element of a new cockpit services, the failure effect was such that the status was always in the "safe" condition. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC EMMA2 OR GEN_Serv-06 GEN_Serv-10 GEN_Serv-13 GEN_Serv-16 9-G The new cockpit services enable me to interface and function efficiently. Please answer for each particular service: GEN_Serv-22 Save date: Public Page 147

148 11-G 12-G 14-G 15-G 16-G a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC The design of the new cockpit services excludes failures that result in erroneous data for operationally significant time periods. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC The new cockpit services have the ability to provide continuous validation of data and timely alerts to the user when the system must not be used for the intended operation. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC The new cockpit services provide a continuous service. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC Any unscheduled break in operations was sufficiently short or rare as not to affect the safety of aircraft. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC I was able to detect significant failures and could initiate remedial action to restore the service or provide a reduced level of service. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC GEN_Serv-24 GEN_Serv-25 GEN_Serv-27 GEN_Serv-28 GEN_Serv-29 Save date: Public Page 148

149 17-G 19-G The new cockpit services allowed me for a reversion to adequate back-up procedures if failures in excess of the operationally significant period occur. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display c) TAXI-CPDLC The new cockpit services are capable of supporting operations of aircraft and vehicles within their minimum and maximum speeds and any heading. Please answer for each particular service: a) Ground Traffic Display b) Traffic Conflict Display GEN_Serv-32 GEN_Perf-01 c) TAXI-CPDLC SURV_Serv-01 1-S 4 The ground traffic display provided accurate position information on all movements within the movement area that I could work safely and efficiently. 2-S The ground traffic display provided identification and labelling of the surrounding traffic that I could work safely and efficiently SURV_Serv-02 3-S The ground traffic display was coping with moving and static aircraft and vehicles that I could work safely and efficiently SURV_Serv-03 4-S The ground traffic display was sufficiently capable of updating surveillance data of the surrounding traffic that I could work safely and efficiently SURV_Serv-04 4 In a simulation environment it is not possible to address the operational feasibility of a surveillance service since the performance of the service itself is simulated. Save date: Public Page 149

150 5-S (While I was using it) The ground traffic display was unaffected by operationally significant effects such as adverse weather and topographical conditions SURV_Serv-05 1-A In general, the traffic conflict detection service supported me to identify conflicts in time ALERT_Serv-01 ALERT_Serv-08 2-A The traffic conflict detection service supported me to identify incursions onto runways within sufficient time to enable the appropriate remedial action ALERT_Serv-02 ALERT_Serv-07 6-A The traffic conflict detection service supported me to identify additional conflicts/infringements like route deviation alerts ALERT_Serv-06 ALERT_Serv-07 7-A I appreciated that the conflict information was displayed continuously while the conflict is present ALERT_Serv-09 8-A The conflict information was unambiguously displayed ALERT_Serv A I found it useful that the ground boundary includes at least the runway strip and that its width is defined according to ILS holding positions (CAT I and CAT II/III) ALERT_Serv-13 Save date: Public Page 150

151 10-R The used terminology or symbology to represent a taxi route was easy to interpret ROUT_Serv-12 2-H The new display maintained a balance between human and machine tasks HMI_Serv-02a 3-H The new display permitted me to retain the power to make decisions as to those functions for which I was responsible HMI_Serv-02b 4-H The new display provided a convenient mix of visual, audio and tactile inputs and responses HMI_Serv-02c 5-H Input devices were functionally simple - involving me in a minimum number of input actions HMI_Serv-03 7-H The new display allowed me to operate the new services in a safe and efficient manner HMI_Serv-07 8-H The new display was harmonised where possible with existing cockpits components HMI_Serv-08 Save date: Public Page 151

152 9-H The new display design took into account my working environment under various operational conditions, means, the new display was adaptable to the various circumstances HMI_Serv H The new display allowed me to utilise the display capabilities (e.g. range scale selection, brightness, map overlays) HMI_Serv H The new display provided me with the complete airport traffic situation, allowing a rapid situation assessment HMI_Serv H I was presented with a clear 'picture' to easily locate and identify aircraft HMI_Serv H Only the minimum traffic information was permanently displayed HMI_Serv H In case of an alert, I was provided with clear and visible indication as soon as the conflict emerged HMI_Serv H Conflict information was unambiguously displayed HMI_Serv-23 Save date: Public Page 152

153 18-H I was provided with clear and visible indication when I was deviating from my cleared taxi route HMI_Serv H All relevant surrounding traffic were presented in clear and pre-defined formats that helped me operating my a/c in a safe and efficient manner HMI_Serv H Depending on my operational needs, the new display was highly configurable with regard to layout, size, shape, fonts, colours and interaction capability HMI_Serv H TAXI-CPDLC assisted me in handover operations HMI_Serv H I was reliably presented with a means to input TAXI CPDLC messages HMI_Serv H It was possible to easily correct a mistaken action HMI_Serv H I was provided with a quick and efficient means to modify a previous ATC assigned route after receiving a revised taxi clearance HMI_Serv-61 Save date: Public Page 153

154 39-H I could always easily intervene with the new display to set additional constraints unknown to the new onboard function (e.g. closed taxiway) HMI_Serv H The indication of stop bars was well integrated into the new display HMI_Serv H When the crossing was operated by TAXI- CPDLC, the status of the displayed stop bar was always in accordance with ATCO clearances HMI_Serv H The presentation of other surrounding movements was not delayed to an extent where it is no longer operationally acceptable HMI_Perf H The response time of the HMI was adequate to allow making inputs without having to wait unduly for the system to process and validate the input HMI_Perf-05 2-T When needed, it was always possible to switch back from data link communication to voice communications in a safe and efficient manner TAXI- CPDLC_Serv-02 3-T I was provided with an effective humanmachine interface to permit efficient TAXI- CPDLC communication with ATC TAXI- CPDLC_Serv- 06a ATCO Save date: Public Page 154

155 6-T In the event of an unexpected termination of a data link application I was sufficiently notified of the failure TAXI- CPDLC_Serv-09 8-T I was always provided with the capability to adequately respond to TAXI-CPDLC messages and to issue requests, as appropriate TAXI- CPDLC_Serv T TAXI-CPDLC messages were appropriately displayed and stored in a manner that permitted timely and convenient retrieval when such actions were necessary TAXI- CPDLC_Serv T I was always under the control of only one ATC unit at a time TAXI- CPDLC_Serv T Each TAXI-CPDLC message transmission was followed by a positive technical acknowledgement, which informed me that the message has completely been transmitted and is available on the ATCO s display TAXI- CPDLC_Serv T The time I needed to look outside was not considerably impaired by operating TAXI- CPDLC Anticipated constraint, SPOR T The amount of inputs to operate TAXI- CPDLC was reasonably low Anticipated constraint, SPOR 3.7 Save date: Public Page 155

156 16-T The total time required for selecting a TAXI- CPDLC message, transmission of the message, reading and interpretation of the message was adequate to communicate in a safe and efficient manner ICAO Doc9694 PART IV, T ATCO s TAXI-CPDLC response time was sufficient short to work in a safe and efficient manner Anticipated constraint, SPOR T The handling of other aircraft by TAXI- CPDLC did not lead to loss of important information, which is usually provided by a R/T partyline effect Anticipated constraint, SPOR T The mix of TAXI-CPDLC and voice communication with different clearances did not lead to confusion and safety critical communication errors Anticipated constraint, SPOR T The TAXI-CPDLC messages were easy to understand and could be handled in a safe and efficient way. If not please write down which one should be improved and how: EMMA2 message set Save date: Public Page 156

157 23-T It was easy to recognise an incoming TAXI- CPDLC message EMMA2 SPOR 3.7.3f 25-T Receiving TAXI-CPDLC taxi route information in advance of the real taxi route clearance is an appropriate procedure to provide me with important preliminary information EMMA2 SPOR 3.7.3j 26-T Receiving TAXI-CPDLC taxi route information in advance of the real taxi route clearance did not exceed my available mental and time resources EMMA2 SPOR 3.7.3j 27-T TAXI-CPDLC communication while the aircraft was taxiing could be performed in a safe and efficient way EMMA2 SPOR 3.7.3m 28-T The frequency of the next control position can be transmitted silently by a TAXI-CPDLC message, but the initial call from the pilot at the next control position should be retained by voice EMMA2 SPOR 3.7.3p Save date: Public Page 157

158 8.6 EMMA2_QE2-OI_Pilot: Operational Improvements Questionnaire for Pilots Introduction: Date: Subject Number: Company: Age: Gender: Pilot Experience: A-SMGCS Experience: Simulation Facility / Test Airport: Test Phase: Below you find statements for which we are interested in your personal opinion how far you can agree to or do not (answers from 1 Strongly disagree to 6 Strongly agree ). Your answers will help us to decide if the new services are able to provide operational improvements in terms of supporting you in aerodrome operations. Please refer your answers to the experiences you gained while you were using the new EMMA2 cockpit services (Ground Traffic Display, Traffic Conflict Detection, TAXI-CPDLC). If you want to provide additional comments/explanations you can use the whole row. Your data will be kept confidential. Thank you in advance. Save date: Public Page 158

159 ID Statements Strongly disagree Disagree Slightly disagree Slightly agree Agree Strongly agree 1P-OI The communication via TAXI-CPDLC would reduce potentially safety-critical misunderstandings between ATCO and Flight Crew compared to using radio communication only. Comments: EMMA SPOR 1.5 2P-OI The display of graphical taxi clearances on the Electronic Moving Map (EMM) would enable me to follow an assigned taxi route more safely. Comments: EMMA SPOR 1.5 3P-OI The display of graphical taxi clearances on the Electronic Moving Map (EMM) would enable me to follow an assigned taxi route more efficiently. Comments: EMMA SPOR 1.5 4P-OI The receipt of only textual taxi clearances (without a graphical display) via TAXI-CPDLC would also enable me to follow an assigned taxi route more safely. Comments: EMMA SPOR 1.5 Save date: Public Page 159

160 5P-OI The receipt of only textual taxi clearances (without a graphical display) via TAXI-CPDLC would also enable me to follow an assigned taxi route more efficiently. Comments: EMMA SPOR 1.5 6P-OI The indication of the surrounding traffic is an additional information source to navigate and manage the aircraft speed more safely. Comments: EMMA SPOR 1.5 7P-OI The indication of the surrounding traffic is an additional information sources to navigate and manage the aircraft speed more efficiently. Comments: EMMA SPOR 1.5 8P-OI Particularly in high workload situations, a Traffic Conflict Detection (TCD) service would clearly indicate and better draw my attention to a potential safety critical situation. Comments: EMMA SPOR 1.5 [LLO5.3] Save date: Public Page 160

161 8.7 TUD Questionnaires In the following the TUD Questionnaires are listed. For both trials the questionnaires are similar except the following differences. In the pilot Intake Questionnaire the familiarity with the current airport should be rated. That means that the questions that will be presented to the pilot matches the current airports while in this document the Prague Questionnaires are listed. Also the Cooper Harper Rating for MCDU was not included for the Prague trials because there was no interaction with Ground to Air Database Upload functionality. Also the MCDU Airport menu questions should not be answered by the pilots in this trails. But for Prague an additional question is added at the Surface Traffic Display chapter. This question asks about the time, the traffic appears. It is necessary, because in the real life system latencies could occur. Save date: Public Page 161

162 EMMA2 - Airport Moving Map Questionnaire Please share your opinion on the following aspects of the Airport Moving Map with us! Please check one box for each question only, and give a brief rationale for your choice! Overall Impression 1. The Airport Moving Map helps to increase situational awareness. Referring to your own experience, does it improve the monitoring task and/or your perception of the aircraft s current and future position? strongly disagree strongly agree 2. The Airport Moving Map gives me support I miss with current systems. strongly disagree strongly agree 3. When using the Airport Moving Map, I perceive the level of safety as high. Do you feel safe with respect to surface navigation? strongly disagree strongly agree 4. The Airport Moving Map is easy to handle. strongly disagree strongly agree 5. The Airport Moving Map keeps me in the loop. strongly disagree strongly agree 6. The Airport Moving Map provides me with the right information at the right time. strongly disagree strongly agree Save date: Public Page 162

163 Information if any which I would like to have, but that is missing: 7. All information I need to accomplish ATC instructions is available. strongly disagree strongly agree Information if any which I would like to have that is missing: 8. With the Airport Moving Map, it is not possible to get lost on an airfield any more. strongly disagree strongly agree Layout, Accessibility and Symbology 9. The display colours chosen for the Airport Moving Map are satisfactory. strongly disagree strongly agree What if anything would you improve? 10. The different colour codes used are easy to interpret. strongly disagree strongly agree Was there any colour coding that was difficult to interpret or confusing? 11. Information is arranged conveniently. strongly disagree strongly agree 12. The used symbols are easy to interpret. strongly disagree strongly agree Save date: Public Page 163

164 Was there any symbol that was difficult to interpret or confusing? 13. Sometimes information I do not need is displayed. strongly disagree strongly agree Which information if any is not useful in your opinion? 14. Different types of information are easy to find. strongly disagree strongly agree 15. The concept of interaction with the Airport Moving Map is satisfactory. strongly disagree strongly agree 16. The amount of information on the Airport Moving Map is too large. strongly disagree strongly agree Which information is essential, what if anything could be removed? 17. Labels, terms and abbreviations chosen in the Airport Moving Map are easy to interpret. strongly disagree strongly agree Was there any label or abbreviation that was difficult to interpret or confusing? 18. Textual information is very well readable (fonts, font size). strongly disagree strongly agree Save date: Public Page 164

165 Integration Aspects 19. This type of display should be integrated into the Navigation Display. strongly disagree strongly agree 20. It is relevant to have different display modes (PLAN, ARC, ROSE). strongly disagree strongly agree 21. The number of selectable ranges is sufficient. strongly disagree strongly agree 22. The size of the display is acceptable. strongly disagree strongly agree 23. The ND is overloaded with information if the Airport Moving Map is added. strongly disagree strongly agree Your suggestions Are there any thoughts or comments regarding the airport moving map you would like to share? Save date: Public Page 165

166 Save date: Public Page 166

167 EMMA2 a) SMAAS; b) MCDU Save date: Public Page 167