Demonstration Report. Abstract. Document information. Project Manager. Edition

Transcription

1 Demonstration Report Document information Project Title Project Number Project Manager Deliverable Name FREE Solutions LSD ENAV Edition Task contributors Demonstration Report ENAV S.p.A., Air France, Alitalia, DFS, DSNA, EUROCONTROL (Network Manager), Lufthansa, MATS, Ryanair, Skyguide, Swiss Airlines, HOP!, KLM, SICTA, Technosky. Abstract This document describes the Demonstration report of the FREE Solutions project that involves ENAV, DSNA, MATS, Skyguide as ANSP; Air France, Alitalia, DFS, Lufthansa, Ryanair, Swiss Airlines, HOP!, KLM as Airspace Users and Eurocontrol as Network Manager. The FREE Solutions project intended to demonstrate the feasibility and support implementation of a user-preferred route that is, as far as practicable, the optimized route allowed in a certain period of time, in a well-defined portion of airspace, defining ad-hoc new City-Pairs routes, Direct Routings and Free Routing operations, when and where possible, exploiting some Flexible Use of Airspace operations and guaranteeing the minimum level of safety required for ATC operations.

2 Project Number LSD Edition Authoring & Approval Prepared By - Authors of the document. Name & Company Position & Title Date Davide DEL VECCHIO/ENAV WP6 Manager 02/06/2016 Paolo VERI/ENAV WP6 Contributor 02/06/2016 Valérie NEYNS/DSNA T6.1 Contributor 02/06/2016 Christopher BRAIN/EUROCONTROL NM WP 4 Manager/T4.1 Leader 02/06/2016 Valentina CEDRINI/ENAV T4.2 Leader 02/06/2016 Giulia Lotti/ENAV T4.3 Leader 02/06/2016 Daniela Di Febo/ALITALIA T5.4 Leader 02/06/2016 Thierry LAURENT/DSNA T6.1 Leader 02/06/2016 Cristiana Abbate/ENAV T6.2 Leader 02/06/2016 Reviewed By - Reviewers internal to the project. Name & Company Position & Title Date Jaqueline Coquel/AIRFRANCE Point of Contact 17/06/2016 Carlo Pacenti/ALITALIA Point of Contact 17/06/2016 Jérôme DUFOSSEZ/DSNA Point of Contact 17/06/2016 Project Management Team 17/06/2016 Enrico LUCINI/ENAV Member Christopher BRAIN/EUROCONTROL NM Point of Contact 17/06/2016 Vincent Van De Nes/KLM Point of Contact 17/06/2016 Andreas HERZOG-HAVEL/LUFTHANSA Point of Contact 17/06/2016 Robert Sant/MATS Point of Contact 17/06/2016 Paolo Iacobelli/RYANAIR Point of Contact 17/06/2016 Pascal LATRON/SKYGUIDE Point of Contact 17/06/2016 Andreas TRISKA/SWISS Point of Contact 17/06/2016 Reviewed By - Other SESAR projects, Airspace Users, staff association, military, Industrial Support, other organisations. Name & Company Position & Title Date Florence SERDOT-OMER/DSNA OFA Coordinator 06/07/2016 Kris DELCOURTE/EUROCONTROL OFA Coordinator 06/07/2016 Fabio GRASSO/EASA ATM/ANS Regulations Officer 06/07/2016 Approved for submission to the SJU By - Representatives of the company involved in the project. Name & Company Position & Title Date Giancarlo Ferrara/ENAV Project Manager 21/07/2016 Laurnet RENOU/AIRFRANCE Project Management Board 21/07/2016 Member Carlo Pacenti/ALITALIA Project Management Board 21/07/2016 Member Morten Grandt/DFS Project Management Board 21/07/2016 Member Jérôme DUFOSSEZ/DSNA Project Management Board 21/07/2016 Member Christopher BRAIN/EUROCONTROL NM Project Management Board 21/07/2016 Member 2 of 213

3 Frank Nagel/LUFTHANSA Robert Sant/MATS Singh CHOORAH/RYANAIR Pascal LATRON/SKYGUIDE Andreas TRISKA/SWISS Rejected By - Representatives of the company involved in the project. Project Management Board Member Project Management Board Member Project Management Board Member Project Management Board Member Project Management Board Member Name & Company Position & Title Date 21/07/ /07/ /07/ /07/ /07/2016 <Name / Company> <Position / Title> <DD/MM/YYYY> Rational for rejection None. Document History Edition Date Status Author Justification /04/2015 Draft /07/2015 Draft /10/2015 Draft Davide Vecchio Davide Vecchio Davide Vecchio Del Del Del Initial document version with Demo Plan information and initial raw data collected from live trials Consolidation with analysed live trial data and integration of results per KPAs Integration of additional information related to bunch 1 (ENV/CAP, SAF, HP methodologies ; list of performed COMM activities; main conclusions and recommendations) /12/2015 Draft Thierry Laurent Review of the document /12/2015 Draft Valérie Neyns /01/2016 Draft Valérie Neyns /01/2016 Draft Valérie Neyns /02/ /03/2016 First iteration of the report First draft of Bunch 2 report Valérie Neyns Valérie Neyns /03/2016 Draft Bunch 2 Valérie Neyns Document sent for review to ENAV Document sent for review to the Consortium members Integration of comments and minor modification about bunch 1 results Approval of all the comments and modifications for the bunch 1 part. Document sent for HP and safety Contribution Integration of contribution Document sent for internal review and further information 3 of 213

4 /04/2016 Draft Bunch 2 Valérie Neyns /04/2016 Draft Bunch 2 Valérie Neyns /04/ /05/2016 Second iteration of the report Bunch 2 First draft of Bunch 3 report Valérie Neyns Valérie Neyns /06/2016 Draft Bunch 3 Valérie Neyns /06/ /06/ /07/ /07/2016 Bunch 3 consolidated Final draft Bunch 3 Consolidated Final Version Final Version for Approval /09/2016 Final Version Valérie Neyns Valérie Neyns Valérie Neyns Davide Vecchio Valérie Neyns Davide Vecchio Davide Vecchio Del Del Del Integration of HP and safety contributions Integration of ENAV inputs Integration of NM inputs Document sent for internal review Integration of comments and minor modification about bunch 2 results Approval of all the comments and modifications for the bunch 2 part. Document sent for HP and Safety Contribution Integration of different contributions (ENAV) Document sent for internal review and further contributions Integration of the comments from the internal review. Fine tuning of general sections Corrections from the external review Fine Tuning of the document Document update after SJU review Intellectual Property Rights (foreground) This deliverable consists of SJU foreground of 213

5 Project Number LSD Edition Table of Contents EXECUTIVE SUMMARY INTRODUCTION PURPOSE OF THE DOCUMENT INTENDED READERSHIP STRUCTURE OF THE DOCUMENT GLOSSARY OF TERMS ACRONYMS AND TERMINOLOGY CONTEXT OF THE DEMONSTRATIONS SCOPE OF THE DEMONSTRATION AND COMPLEMENTARITY WITH THE SESAR PROGRAMME PROGRAMME MANAGEMENT ORGANISATION Consortium Structure Main Roles and responsibilities Project Management Panels Escalation Process Participants and Contact list WORK BREAKDOWN STRUCTURE DELIVERABLES RISK MANAGEMENT EXECUTION OF DEMONSTRATION EXERCISES EXERCISES PREPARATION Dates selection Analysis of environmental and efficiency benefits Safety Analysis Human Performance Methodology Guidelines to all the experiment participants Operational Staff Briefings Questionnaires EXERCISES EXECUTION DEVIATIONS FROM THE PLANNED ACTIVITIES EXERCISES RESULTS SUMMARY OF EXERCISES RESULTS CHOICE OF METRICS AND INDICATORS SUMMARY OF ASSUMPTIONS Impact on Safety, Capacity and Human Factors Description of assessment methodology Results impacting regulation and standardisation initiatives ANALYSIS OF EXERCISES RESULTS Unexpected Behaviours/Results CONFIDENCE IN RESULTS OF DEMONSTRATION EXERCISES Quality of Demonstration Exercises Results Significance of Demonstration Exercises Results DEMONSTRATION EXERCISES REPORTS DEMONSTRATION EXE-LSD REPORT Exercise Scope Conduct of Demonstration Exercise EXE-LSD Exercise Results Conclusions and recommendations DEMONSTRATION EXE-LSD REPORT of 213

6 Project Number LSD Edition Exercise Scope Conduct of Demonstration Exercise EXE-LSD Exercise Results Conclusions and recommendations DEMONSTRATION EXE-LSD REPORT Exercise Scope Conduct of Demonstration Exercise EXE-LSD Exercise Results Conclusions and recommendations DEMONSTRATION EXE-LSD REPORT Exercise Scope Conduct of Demonstration Exercise EXE-LSD Exercise Results Conclusions and recommendations DEMONSTRATION EXE-LSD REPORT Exercise Scope Conduct of Demonstration Exercise EXE-LSD Exercise Results Conclusions and recommendations SUMMARY OF THE COMMUNICATION ACTIVITIES NEXT STEPS CONCLUSIONS RECOMMENDATIONS REFERENCES APPLICABLE DOCUMENTS REFERENCE DOCUMENTS List of tables Table 1: Exercise EXE-LSD overview Table 2: Exercise EXE-LSD overview Table 3: Exercise EXE-LSD overview Table 4: Exercise EXE-LSD overview Table 5: Exercise EXE-LSD overview Table 6: FREE solutions activities and its link to SESAR OFAs Table 7: Escalation Process Table 8: Project Team Directory Table 9: Formal Deliverables Table 10: Indicators and Metrics for assessing expected benefits NEST Table 11: Exercises execution/analysis dates Table 12: Summary of Demonstration EXE-LSD &002 Results Table 13: Summary of Demonstration EXE-LSD Table 14: Summary of Demonstration EXE-LSD Table 15: Summary of Demonstration EXE-LSD Table 16: Summary of metrics and indicators Table 17: Demonstration Assumptions Table 18: Demonstration Scenarios vs Safety Assessment Scenarios Table 19: Hazard Identification results Table 20: EXE-LSD Demo Objectives Table 21: The list of city pairs routes optimization to be executed Table 22: EXE-LSD City Pairs List - weekends Table 23: City Pairs Routes Table 24: Safety related questions Table 25: Success Criteria Environment-Fuel Efficiency and Predictability OBJ-LSD Table 26: Environment-Fuel Efficiency Results - Metrics and Measures Weekend City Pair of 213

7 Project Number LSD Edition Table 27: Success Criteria - Capacity OBJ-LSD Table 28: Capacity Results - Metrics and Measures Weekend City Pair Table 29: Success Criteria - Predictability OBJ-LSD Table 30: Predictability and ATC Planning Results - Metrics and Measures Weekend City Pair Table 31: Success Criteria Safety OBJ-LSD Table 32: Safety Results - Metrics and Measures City Pair Weekend Table 33: ANSPs Questionnaires - City Pairs Table 34: Success Criteria Role and responsibilities - OBJ-LSD Table 35: Success Criteria Operating Methods and Procedures - OBJ-LSD Table 36: Success Criteria Controller Performance - OBJ-LSD Table 37: Success Criteria - Changes to Tasks and roles OBJ-LSD Table 38: Success Criteria - Roles and responsibilities OBJ-LSD Table 39: Success Criteria - Operating Methods and Procedures OBJ-LSD Table 40: Success criteria - Pilot performance OBJ-LSD Table 41: Success Criteria: Changes to Tasks and Roles OBJ-LSD Table 42: City Pairs delivered benefits Table 43: EXE-LSD Demo Objectives Table 44: EXE-LSD City Pairs List - weekdays Table 45: DRs potential benefits evaluations Table 46: The list of Direct Routings executed Table 47: Safety related questions - DR Table 48: Success Criteria Environment Fuel Efficiency OBJ-LSD Table 49: Environment-Fuel Efficiency Results - Metrics and Measures Weekends Direct Routings 109 Table 50: Success Criteria - Capacity OBJ-LSD Table 51: Capacity Results - Metrics and Measures Weekend Direct Routings Table 52: Success Criteria Predictability and ATC Planning OBJ-LSD Table 53: Predictability and ATC Planning Results - Metrics and Measures Weekend Direct Routings Table 54: Success Criteria - Safety OBJ-LSD Table 55: Safety Results - Metrics and Measures Direct Routings Weekends Table 56: ANSPs Questionnaires - Direct Routings Table 57: Success Criteria Role and responsibilities - OBJ-LSD Table 58: Success Criteria Operating Methods and Procedures - OBJ-LSD Table 59: Success Criteria Controller Performance - OBJ-LSD Table 60: Success Criteria - Changes to Tasks and roles OBJ-LSD Table 61: Success Criteria - Changes to tasks and roles (2) OBJ-LSD Table 62: AUs Questionnaires - Direct Routings Table 63: Success Criteria - Roles and Responsibilities OBJ-LSD Table 64: Success Criteria - Operating Methods and Procedures OBJ-LSD Table 65: Success Criteria - Pilot Performance OBJ-LSD Table 66: Success Criteria - Changes to Tasks and Roles (2) OBJ-LSD Table 67: Success Criteria - OBJ-LSD Table 68: DRs Weekend delivered benefits Table 69: EXE-LSD Demo Objectives Table 70: Selected DRs weekdays Table 71: DRs weekdays potential benefits evaluations Table 72: Success Criteria Environment-Fuel Efficiency OBJ-LSD Table 73: Environment-Fuel Efficiency Results - Metrics and Measures Weekdays Direct Routings 146 Table 74: Success Criteria - Capacity OBJ-LSD Table 75: Capacity Results - Metrics and Measures Weekdays Direct Routings Table 76: Success Criteria Predictability and ATC Planning OBJ-LSD Table 77: Predictability and ATC Planning Results - Metrics and Measures Weekend Direct Routings Table 78: Success Criteria Civil-Military Cooperation and Coordination OBJ-LSD Table 79: Success Criteria OBJ-LSD Table 80: Civil-Military Cooperation and Coordination - Metrics and Measures Weekdays Direct Routings Table 81: DR Weekdays and Final Total Table 82: EXE-LSD Demo Objectives of 213

8 Project Number LSD Edition Table 83: List of skyguide - New DCTs Table 84: List of Marseille ACC D and Z sectors - DRA DCTs Table 85: List of ENAV - Italy E(Entry) and X(Exit) FRA points Table 86: List of Malta E(entry), X(exit) and I(intermediate) FRA points Table 87: Safety related questions - FRA Table 88: Success Criteria Environment Fuel Efficiency OBJ-LSD Table 89: Environment - Fuel efficiency results - Metrics and Measures Free Routing Table 90: Success Criteria - Capacity OBJ-LSD Table 91: Capacity Results - metrics and measures Free Routing Table 92: Success Criteria - Predictability OBJ-LSD Table 93: Predictability Results - Metrics and Measures Free Routing Table 94: Success Criteria - Safety OBJ-LSD Free Routing Weekends Table 95: Safety Results - Metrics and Measures Free Routing Weekends Table 96: ANSPs Questionnaires - Free Routings Table 97: Success Criteria Role and responsibilities - OBJ-LSD Table 98: Success Criteria Operating Methods and Procedures - OBJ-LSD Table 99: Success Criteria Controller Performance - OBJ-LSD Table 100: Success Criteria - Changes to Tasks and roles OBJ-LSD Table 101: AUs Questionnaires - Free Routings Table 102: Success Criteria - OBJ-LSD Table 103: EXE-LSD Results per Day Table 104: EXE-LSD Overall results Table 105: Communication Points of Contact Table 106: Planned communications activities Table 107: Actual communications activities - city pairs, DRs & FRA Table 108: Overall results of the FREE Solutions demonstrations Table 109: Free Solutions contribution to SESAR Solutions List of figures Figure 1: FREE Solutions Consortium Figure 2: FREE Solution Project Organisation Chart Figure 3: FREE Solution Project Management Panels Composition Figure 4: Work Breakdown Structure Figure 5: Work-package inter-relationships Figure 6: Risk Management Process Figure 7: ATM Master Plan Target Figure 8: Trial Flights Compare to Reference Flights Figure 9: Normalised Flights Figure 10: City Pair Comparison Elements Figure 11: GSN Top Level Claim Figure 12: GSN CP Scenario Figure 13: GSN CP Hazard Identification Figure 14: Participating ACCs - City Pairs Figure 15: City Pairs potential benefits evaluations Figure 16: Flight Planning Process Figure 17: ARG1.2.2 ANSP Equipment Figure 18: ARG ANSP - Operational and flight procedures Figure 19: ARG ANSP 1/ Figure 20: ARG ANSP 2/ Figure 21: ARG AU - Equipment Figure 22: ARG AU - Operational and flight procedures (1/2) Figure 23: ARG AU - Operational and flight procedures (2/2) Figure 24: ARG AU - People related issues Figure 25: ARG AU - Workload Figure 26: ARG AU - Working methods Figure 27: Groundside HP objective HP Figure 28: Groundside HP objective HP Figure 29: Groundside HP objective HP009 Workload Overall of 213

9 Project Number LSD Edition Figure 30: Groundside HP objective HP009 - Workload Figure 31: Groundside HP objective HP009 Communications Changes Figure 32: Groundside HP objective HP009 Situational Awareness Overall Figure 33: Groundside HP objective HP009 Situational Awareness Figure 34: Groundside HP objective HP009 Task Time Execution Figure 35: Groundside HP objective HP009 Communications & coordinations Figure 36: Groundside HP objective HP009 Trust Figure 37: Groundside HP objective HP009 Operative Acceptability Figure 38: Groundside HP objective HP009 Potential Errors Figure 39: Groundside HP objective HP009 EC Potential Errors (a) Figure 40: Airside HP Objective HP Figure 41: Airside HP Objective HP Figure 42: Airside HP Objective HP010 (workload) Figure 43: Airside HP Objective HP010 (tasks time execution) Figure 44: Airside HP Objective HP010 (communications load) Figure 45: Airside HP Objective HP010 (potential errors) Figure 46: Airside HP Objective HP Figure 47: Airside HP Objective HP014 (potential errors) Figure 48: Participating ACCs and City Pair Weekday Direct Route Figure 49: EGLL-LIRF potential conflicting traffic Figure 50: DRs during weekend Figure 51: EXE-LSD Demo Objectives Figure 52: Example of routes involved in traffic filtering Figure 53: ARG2.2.2 ANSP Equipment Figure 54: ARG2.2.2 ANSP Airspace configuration Figure 55: ARG2.2.3 ANSP Operational and flight procedures Figure 56: ARG2.2.5 ANSP People related issues Figure 57: ARG AU Equipment Figure 58: ARG AU Operational and flight procedures (1/3) Figure 59: ARG AU Operational and flight procedures (3/3) Figure 60: ARG AU Operational and flight procedures (2/3) Figure 61: ARG AU - People related issues Figure 62: ARG AU Workload Figure 63: ARG AU - Working methods Figure 64: Groundside HP objective HP Figure 65: Groundside HP objective HP Figure 66: Groundside HP Objective HP038 - Workload Overall Figure 67: Groundside HP Objective HP038 Three main dimensions of workload Figure 68: Groundside HP objective HP038 - Communications Changes Figure 69: Groundside HP objective HP038 - Situational Awareness Overall Figure 70: Groundside HP objective HP038 - Situational Awareness Figure 71: Groundside HP Objective HP038 - Task Time Execution Figure 72: Groundside HP objective HP038 - Communications & Coordination Figure 73: Groundside HP Objective HP038 - Trust Figure 74: Groundside HP objective HP038 - Operative Acceptability Figure 75: Groundside HP Objective HP038 - Potential errors a Figure 76: Groundside HP Objective HP038 - Potential errors b Figure 77: Airside HP Objective HP033 Roles and Responsibilities Figure 78: Airside HP Objective HP037 Operating Methods and procedures Figure 79: Airside HP Objective HP039 Workload Figure 80: Airside HP Objective HP039 Typology of workload Figure 81: Airside HP Objective HP039 - Task time execution Figure 82: Airside HP Objective HP039 - ATC communications Figure 83: Airside HP Objective HP039 - Potential errors Figure 84: Airside HP Objective HP043 Tasks and Roles Figure 85: DRs weekdays - SCN-LSD Figure 86: Examples of segregated airspace being released by EUUP during trial period Figure 87: Top 20 Delay Locations for en-route Delays in November Figure 88: EXE-LSD Involved ACCS of 213

10 Project Number LSD Edition Figure 89: Skyguide - Overview - new DCTs Figure 90: Marseille ACC D and Z sectors - Overview DRA DCTs Figure 91: ENAV Italy: Overview E(Entry) and X(Exit) FRA points Figure 92: ENAV Italy: Overview I(intermediate) FRA points Figure 93: ENAV Italy: Overview IA(I-Intermediate and A-arrival) FRA points Figure 94: Malta Overview E(entry),X(exit) and I(intermediate) - FRA points Figure 95: FRA Generic Examples (Exit - Entry) Figure 96: ARG3.2.2 ANSP Equipment Figure 97: ARG3.2.3 ANSP Airspace Configuration Figure 98: ARG ANSP - Operational and flight procedures Figure 99: ARG ANSP - People related issues Figure 100: ARG AU - Equipment Figure 101: ARG AU Operational and flight procedures - Communications Figure 102: ARG AU Operational and flight procedures - Fuel Figure 103: ARG AU - People related issues - Workload Figure 104: ARG AU Working Methods Figure 105: ARG AU Environmental and meteorological conditions Figure 106: Groundside HP objective HP Figure 107: Groundside HP objective HP Figure 108: Groundside HP Objective HP067 Three main dimensions of workload Figure 109: Groundside HP objective HP067 - Communications Changes Figure 110: Groundside HP objective HP067 - Situational Awareness Figure 111: Groundside HP Objective HP067 - Task Time Execution Figure 112: Groundside HP Objective HP067 - Trust Figure 113: Groundside HP objective HP067 - Operative Acceptability Figure 114: Groundside HP Objective HP067 - Potential errors a Figure 115: Groundside HP Objective HP067 - Potential errors b Figure 116: Examples of Communications Activities - City Pairs Figure 117: Examples of Communications Activities RTS Figure 118: Examples of Communications Activities Free Routing Figure 119: Examples of Communications Activities Free Routing (2) Figure 120: Examples of Communications Activities Leaflet Figure 121: Examples of Communications Activities Brochure Figure 122: Examples of Communications Activities Final Event of 213

11 Executive summary The FREE Solutions project intended to demonstrate the feasibility and supported implementation of a user-preferred route that is, as far as practicable, the optimized route allowed in a certain period of time, in a well-defined portion of airspace, defining ad-hoc new City-Pairs routes, Direct Routings and Free Routing operations. The main objective of this Large Scale demonstration was through a set of live trials activities, assessed the operational impact, de-risking the implementation of the main concepts addressed towards the deployment phase (i.e. City-pairs, DRs, FRA, FUA). Very positive results were obtained from the City-Pairs routes experimentation, comprising live trials during weekends (EXE-LSD ) and live trials during weekdays (EXE-LSD ). 120 flights flew the selected city-pairs. Thanks to these new routings, 391 NM, 81 min, 3823 kg of Fuel, kg of CO2 and 53 kg of NOx could be saved. Based on these results, the second set of live trials (DRs) aimed to demonstrate benefits in terms of flight punctuality and improved ATC planning. Very positive results were obtained from the Direct routing experimentation, comprising live trials during weekends (EXE-LSD ) and live trials during weekdays (EXE-LSD ). A total of 760 flights flew the proposed DRs. Thanks to these new routings, 3767NM, 720 min, kg of Fuel, kg of CO2 and 778 kg of NOx could be saved. These results represented an important step towards the Free Routing Experimentation. Positive results were obtained from the Free Routing experimentation, consisting in live trials during weekends (EXE-LSD ). 149 flights flew the proposed FRs. Thanks to these new routings, 324 NM, 46 min, 3701 kg of Fuel, kg of CO2 and 63 kg of NOx could be saved. Based on all these results, a set of recommendations are proposed concerning the future implementation of DRA & FRA. 11 of 213

12 Introduction 1.1 Purpose of the document This document provides the Demonstration report for the FREE Solutions project. It describes the results of demonstration exercises defined in the related FREE Solutions Demonstration Plan Second Consolidated Delivery, Edition 2.0, 20/02/2015 [1] and how they have been conducted. 1.2 Intended readership This document is meant to bring useful information to the overall European ATM community about the findings of the Demonstration. Any stakeholder interested in the deployment of Free Routing operations and Flexible Use of Airspace should find relevant material. Particularly: ANSPs to measure potential improvements in the network structure, and to identify possible difficulties for implementation AUs to be aware of what can be expected from Network improvements National Authorities and EC to identify limitations due to current operating rules and necessity for regulatory evolutions SESAR JU and EC to evaluate the feasibility of large scale deployment of the Free Route concept, cost-effectiveness of the implementation, confirm or adjust deployment schedule. The Operational Focus Areas (OFA) Free Routing and Airspace Management and AFUA. 1.3 Structure of the document The document describes the framework of the demonstrations and provides an overview of management approach. The preparation of demonstration exercises and the results are also described. In summary, the document structure follows the template provided by SESAR Joint Undertaking: Section 1 introduces the document; Section 2 recalls context and scope of the demonstrations with reference to their complementarity with the overall SESAR programme and PCP activities; Section 3 provides an overview of the project management approach of FREE Solutions, including the work and resource breakdowns, project panels, roles and responsibilities, risks management; Section 4 details the preparatory activities carried out to execute live trials, reports on actual execution dates and summaries main deviations compared to planned activities; Section 5 summarizes the results of each exercise; Section 6 describes in detail the undertaken large scale demonstrations activities for City Pairs Optimization, Direct Routings and Free Routing; Section 7 describes the communications activities performed by each project partner; Section 8 summarises conclusions and recommendations; Section 9 provides the reader with the main references and applicable documents. 12 of 213

13 Glossary of terms DEFINITION SOURCE Free Route Airspace Direct Routing Direct Routing Airspace A specified airspace within which users may freely plan a route between a defined entry point and a defined exit point, with the possibility to route via intermediate (published or unpublished) way points, without reference to the ATS route network, subject to airspace availability. Within this airspace, flights remain subject to air traffic control. Direct routing is established within direct routing airspace with the aim of providing airspace users additional flight planning route options on a larger scale across FIRs such that overall planned leg distances are reduced in comparison with the fixed route network and are fully optimised due to FUA specifically within high & very high-density workload environments and / or involving cross ACC/FIR boundaries Airspace defined laterally and vertically with a set of entry/exit conditions where published direct routings are available within this airspace, flights remain subject to air traffic control. ERNIP Part 1 European Airspace Design Methodology Edition /06/2013 ATM Masterplan (DS12) Free Route Task Force SJU Conclusions 28/02/2014 City Pair Flexible Use of Airspace (FUA) A connection from departure airport to destination airport Airspace considered as one continuum in which all users requirements (civil and military) have to be accommodated to the maximum extent possible. EUROCONTRO L (EC) Regulation N 2150/ Acronyms and Terminology Term ACC AIP AIRAC AMC ANSP AOI AOR Definition Area Control Centre Aeronautical Information Publication Aeronautical Information Regulation and Control Airspace Management Cell Air Navigation Service Provider Area Of Interest Area of Responsibility 13 of 213

14 Term ATC ATCO ATFCM ATS AU AUP CFSP CONOPS CP CPF DR DRA EC EOBT E-OCVM ERNIP Definition Air Traffic Control Air Traffic Controller Air Traffic Flow and Capacity Management Air Traffic Services Airspace User Airspace Use Plan Computer Flight plan Service Provider Concept of Operations City Pair Correlated Positions of A Flight Direct Routing Direct Routing Airspace Executive Controller Estimated Off-Block Time European Operational Concept Validation Methodology European Route Network Improvement Plan EVEN Direction of IFR cruising levels within true track margin of FAB FABEC FBZ FHA FIR FLOS FMS FPL FR FRA Functional Airspace Blocks FAB Europe Central Flight planning Buffer Zones Functional Hazard Assessment Flight Information Region Flight Level Orientation System Flight Management System Flight Plan Free Routing Free Routing Airspace 14 of 213

15 Term FUA GSN HF HP ICAO IFPS KPA KPI NM Definition Flexible Use of Airspace Goal Structuring Notation Human Factors Human Performance International Civil Aviation Organization Initial Integrated Flight Plan Processing System Key Performance Area Key Performance Indicator Network Manager ODD Direction of IFR cruising levels within true track margin of OFA OLDI OSED PC PCP PF PjM PM PMB PMO PMP PoC RAD RIO RTS SAM SES Operational Focus Areas Online Data Interchange Operational Services and Environment Description Planning Controller Pilot Common Project Pilot Flying Project Management Pilot Monitoring Project Management Board Project Management Office Project Management Plan Point of Contact Route Availability Document Risks Issues Opportunities Real Time Simulation Safety Assessment Methodology Single European Sky 15 of 213

16 Term SESAR SESAR Programme SID SJU STAR TC TL UPRs UUP WBS WP WPM Definition Single European Sky ATM Research Programme The programme which defines the Research and Development activities and Projects for the SJU. Standard Instrument Departure SESAR Joint Undertaking (Agency of the European Commission) Standard Terminal Arrival Route Task Contributor Task Leader Users Preferred Routes Updated (Airspace) Use Plan Work Breakdown Structure Work Package Work Package Manager 16 of 213

17 Context of the Demonstrations FREE Solutions Large Scale Demonstration is a common project including 7 airspace Users (AUs), 5 Air Navigation Service Providers (ANSPs) and Network Manager (NM), whose main objective was to assess different aspects of the Free Route concepts, both Direct Routing along with FUA and initial Free Routing in a multi FAB portion of airspace. The project focused on the design of operational solutions throughout the flight plan life cycle, from a better planning to Airspace Users preferred tactical routes definition including appropriate information exchanges and ensuring that the required safety level is maintained or improved. It also assessed, through a set of live trials activities, the operational impacts de-risking the implementation of the main concepts addressed towards the deployment phase (i.e. City-pairs, DRs, FRA, FUA). These objectives are identified in the SESAR programme, and addressed by the Pilot Common Projects. The proposed implementation follows two major steps. The first one is the development of Direct Routing, the second one is the introduction of Free Routing. Direct Routing is defined as the shortest connection close to the great circle between 2 published waypoints consisting of a succession of Direct Segments and/or ATS route segments (OFA OSED). Free Routing corresponds to the ability of the airspace user to plan and re-plan a route according to the user-defined segments within significant blocks of Free Route Airspace (FRA) (i.e. multiple FIR AOIs or FABs) where airspace reservations are managed in accordance with FUA principles. Free Routing User defined segments are segments of a great circle connecting any combination of two user-defined or published waypoints, within low to medium traffic complexity environments. 2.1 Scope of the demonstration and complementarity with the SESAR Programme Demonstration Exercise ID and Title Leading organization Demonstration exercise objectives OFA addressed Applicable Operational Context Related projects in the SESAR Programme Demonstration Technique EXE-LSD City Pairs Weekends ENAV OBJ-LSD , OBJ-LSD , OBJ-LSD OFA SCN-LSD : This scenario refers to Normal day of Operations during weekends, which means any period of time starting from Friday night at 23:00 UTC up to Monday morning at 07:00 UTC 1. Scenario is applicable from end of SID to beginning of STAR, so it is limited to En- Route phase and to the consortium airspace. Aircraft will fly in accordance with existing operational procedures in controlled airspace. In the same way, for involved ACCs, operations will be ensured through existing working procedures. P04.02, P04.03 Live Trial 1 This slot is reduced up to 06:00 UTC in the Summer 17 of 213

18 Number of trials Table 1: Exercise EXE-LSD overview Demonstration Exercise ID and Title Leading organization Demonstration exercise objectives OFA addressed Applicable Operational Context Related projects in the SESAR Programme Demonstration Technique Number of trials EXE-LSD City Pairs Weekdays ENAV OBJ-LSD , OBJ-LSD , OBJ-LSD , OBJ-LSD OFA SCN-LSD : This scenario refers to Normal day of Operations during week days, which means any period of time starting from Monday morning at 07:00 UTC up to Friday night at 23:00 UTC. Scenario is applicable from end of SID to beginning of STAR, so it is limited to En- Route phase and to the consortium airspace. Aircraft will fly in accordance with existing operational procedures in controlled airspace. In the same way, operations will be ensured through existing working procedures for involved ACCs. P04.02, P04.03, P Live Trial Table 2: Exercise EXE-LSD overview Demonstration Exercise ID and Title Leading organization Demonstration exercise objectives OFA addressed Applicable Operational Context EXE-LSD Direct routing Weekends ENAV OBJ-LSD , OBJ-LSD , OBJ-LSD OFA SCN-LSD : This scenario refers to Normal day of Operations (Autumn 2015 season) during weekends, which means any period of time starting from Friday night at 23:00 UTC up to Monday morning at 07:00 UTC 2. Scenario is applicable to En-Route phase and to the consortium airspace. It is related to the execution of cross-accs DR routings. 2 This slot is reduced up to 06:00 UTC in the Summer 18 of 213

19 Related projects in the SESAR Programme Demonstration Technique Number of trials Aircraft will fly in accordance with existing operational procedures in controlled airspace. In the same way, operations will be ensured through existing working procedures for involved ACCs. Direct Routing options are designed from a network perspective and developed based on NM analyses identifying significant flows on network level. Those identified DRs are not limited to consortium flights only but are published and open to all traffic. P04.02, P04.03, P Live Trial Table 3: Exercise EXE-LSD overview Demonstration Exercise ID and Title Leading organization Demonstration exercise objectives OFA addressed Applicable Operational Context EXE-LSD Direct routing Weekdays ENAV OBJ-LSD , OBJ-LSD , OBJ-LSD , OBJ-LSD , OBJ-LSD OFA SCN-LSD : This scenario refers to Normal day of Operations (Autumn 2015 season) during week days, which means any period of time starting from Monday morning at 07:00 UTC up to Friday night at 23:00 UTC. Scenario is applicable to En-Route phase and to the consortium airspace. It is related to the execution of cross-accs DR routings. Aircraft will fly in accordance with existing operational procedures in controlled airspace. In the same way, for involved ACCs operations will be ensured through existing working procedures. This scenario is applicable to DRs interfering with reserved/restricted airspace structures. Their availability will be the results of coordination with national AMCs, military Approve Agencies and NM, properly notified to airspace users through the AUP and UUP process for any update of the airspace status. The coordination and notification process will focus on the airspace activations and associated restrictions to inhibit flight planning through military training areas (as opposed to today s common practice to work with route availability rather than airspace availability). When using airspace activations in AUP/UUP, 19 of 213

20 Related projects in the SESAR Programme Demonstration Technique Number of trials ANSPs and AMCs have to ensure that the publication of those airspaces used for dynamic activation via AUP/UUP includes the complete airspace volume relevant for flight planning (including flight planning buffer zones, FBZ, if any). The Network Manager ensures a timely collection and distribution of the airspace status updates sent by participating ANSPs. Based on this information AUs participating in the trial update their flight plans in order to take advantage of current airspace availability (or to avoid airspace recently activated). P04.02, P04.03, P Live Trial Table 4: Exercise EXE-LSD overview Demonstration Exercise ID and Title Leading organization Demonstration exercise objectives OFA addressed Applicable Operational Context EXE-LSD Free Routing during week-end ENAV OBJ-LSD , OBJ-LSD , OBJ-LSD OFA SCN-LSD : A Free Routing area is designed within the following airspace: o o o 2 Marseille ACC Sectors (i.e. D/Z); Geneva/Zurich ACCs; Roma/Milano/Padova/Brindisi ACCs; o Malta ACC. In this FRA area (Italian and Maltese airspace more information are provided in the section for Aix and Skyguide Airspace) AUs may freely plan a route between a defined entry point and a defined exit point, with the possibility to route via intermediate (published) way points. Within this airspace, flights remain subject to air traffic control. It is expected a stepwise approach in the execution of the exercises comprised within this scenario. Initially Entry-Exit planning is allowed only at specific times and only above FL 365. Final target is to run this scenario for an entire week end, which means any period of time starting from Friday night at 23:00 UTC up to Monday morning at 07:00 UTC. Scenario is applicable to En-Route phase and to the consortium airspace. Flight planning is filed and accepted by all ANSP authorities as well as NM. 20 of 213

21 Related projects in the SESAR Programme Demonstration Technique Number of trials P04.02, P04.03 Live Trial Table 5: Exercise EXE-LSD overview FREE solutions project is part of SESAR programme which aims at modernising Air Traffic Management in Europe and at representing the technological pillar of the Single European Sky. The Common Projects are the fundamental pillars for SESAR deployment. They identify those ATM functionalities that are considered to be essential contributors to the improvement of the European ATM system performance and that are mature for implementation and demonstrate that they have a global positive business case for the European ATM network, in particular if deployed in synchronisation. Common projects take the form of Commission Implementing Regulations that make the deployment of the ATM functionalities included in Common projects mandatory within a given timeframe and geographical scope. Common projects define What needs to be deployed, Where and When it needs to be deployed and by Whom. In this context, Free Solutions activities are undertaken within two, out of six, ATM Functionalities identified in the Pilot Common Project. They are AF#3 - Flexible Airspace Management and Free Route and AF#4 - Network Collaborative Management. The former is expected to enable a more efficient use of airspace, thus providing significant benefits linked to fuel consumption and delay reduction, while the latter is expected to improve the quality and the timeliness of the network information shared by all ATM stakeholders, thus ensuring significant benefits in terms of Air Navigation Services productivity gains and delay cost savings. Additionally, FREE Solutions project aims at contributing to the overall satisfaction of the performance targets, in each relevant key performance area, set by SESAR WPB4.1. FREE Solutions project is part of a series of different activities whose aim is to increasingly address and mature Free Route concept. As shown in the table below, this set of FREE solutions exercises are linked with other on-going projects in the framework of SESAR programme and contribute to higher level of maturity of the listed Operational Improvement element. OFA ID Operational Focus Area name Relevant OI Steps ref. (coming from the Integrated Roadmap) Free Routing AOM-0500: Direct Routing for flights both in cruise and vertically evolving for cross ACC borders and in high & very high complexity environments. AOM-0501: Free Routing for Flights both in cruise and vertically evolving within low to medium complexity environments Airspace Management and AFUA AOM-0202-A: Automated Support for strategic, pre-tactical and tactical Civil-Military Coordination in Airspace Management (ASM) AOM-0206-A: Flexible Military Airspace Structures in Step Table 6: FREE solutions activities and its link to SESAR OFAs 21 of 213

22 Programme management The Programme was managed accordingly to the Demonstration Plan. No significant change occurred 3.1 Organisation The Project was managed in compliance with the SJU Execution Guidelines. The Project Manager ensured the correct execution of all project processes. This section is mainly focused on the management process applied internally in order to achieve the required level of coordination with external stakeholders/customers of the project. The Project was managed through a set of roles and corresponding responsibilities entrusted to key people selected on purpose by each FREE Solutions Consortium member. Clear roles and responsibilities definition were essential to FREE Solutions project success Consortium Structure The Project Consortium was composed by the following members: - ANSPs: ENAV S.p.A. (project coordinator), DFS, DSNA, MATS and Skyguide - Airlines: Air France, Alitalia, Lufthansa, Ryanair and Swiss Airlines - EUROCONTROL/Network Manager HOP! and KLM were participating in this demonstrator as third parties of Air France; SICTA and Technosky as affiliates of ENAV Figure 1: FREE Solutions Consortium Each of the participating companies signed a Mandate through which: authorised ENAV to act as the Project Coordinator on behalf of all Consortium members committed to provide the required resources to fulfil the scope and pursued the project scope and objectives as described in the Technical Offer. Furthermore, the Consortium Members have provided the full batch of legal, economic and financial capacity evidences (included in the Administrative offer of the proposal) in line with the SESAR JU procedure. ENAV as a project Coordinator signed the Grant Agreement (ref: SJU/LC/0292 CTR) with SESAR JU on 19 th September 2014 to which all Consortium partners have accessed through signing the Annex 4 Accession Form. The consortium has been formally established by signing the Consortium Agreement on 22 nd January Main Roles and responsibilities FREE Solutions project relied on senior governance and management of the relationship between the Members involved in Project, Work Packages and Task levels to ensure an effective 22 of 213

23 Project Number LSD Edition coordination/consultation approach and a comprehensive project oversight. In this respect the overall Project has been structured in one transversal Activity, Project Management and five working activities. The following [Figure 2] depicts the Project Breakdown Structure: Figure 2: FREE Solution Project Organisation Chart The Project Manager - with its project management office (PMO) and WP Managers - had the overall authority and responsibility for managing and executing the FREE Solutions project in line with the Demonstration Plan (V2). The Project Manager worked with the required resources and support to perform project planning. All project and subsidiary management plans were reviewed and approved by the project Sponsor as required. The following main roles and tasks distribution/delegation were used as a reference for the project coordination and management approach: Project Manager (PjM) The Project Manager has been appointed by the Coordinator and is responsible for the day to day interface management and coordination of the overall Project, as well as constituting the Consortium operational and technical interface with the European Institutions and with the FREE Solutions stakeholders. The PjM is the Point-of-Contact for all issues related to the FREE Solutions project. The PjM is responsible for all technical, quality and schedule aspects (at the moment costs are not included in the scope) of the tasks defined in the project s Work Plan. In line with SESAR JU requirements, PjM s main responsibilities are listed below: Prepare and implement the Project plan o plan deliverables taking into account dependencies between the Project tasks; o define the Project organisation (resources, milestones), Implement the SJU methodologies within the Project; Provide a comprehensive oversight of the Project and manage the operational relationship between the Members involved at WP and task level; Identify, trackand manage risks, issues and opportunities at the Project level. Integrate RIOs guidelines from the Programme Management Office; Organise and chair Project progress reviews and meetings; Prepare Project Control Gates by making sure that all required information are available to all participants; Participate in the Project Control Gates with the aim to ensure the appropriate link of the Project with the SJU; Ensure proper and timely communication of information, within the Project and to the SJU; Submit Change requests (if required); Support the organisation and management of the technical contribution of experts assigned 23 of 213

24 by the Partners at WP and task level; Project Manager is also in charge of preparation and submission of some reporting activities such as: informal monthly project progress report, quarterly project progress reports to SJU Project Officer. 510 Project Management Office (PMO) In order to ensure the project is conducted in line with the objectives above presented PjM will be supported by PMO, composed of: Deputy project manager Quality and risk manager Communication manager (as covered by relevant Task in WP 6) Project secretariat 511 Project Point of Contact (PoC) Each Consortium member identified a figure and eventually the alternate to supervise / coordinate the Project internally within his/her Organisation in order to contribute to the project itself for all project steering matters (e.g. project progress validation, deliverable formal approval etc.). He/She has the authority to fully represent his/her own company for all communication and decision needs related to the contracted work within the project and is also responsible for project progress approval. He/She is responsible for the validation of decisions with impacts on contracted activities decisions at Project level. 512 Quality and Risk Manager (QRM) The Quality and Risk manager supports the Project Manager in his/her duties with particular focus on the Quality Management and Risks, Issues and Opportunities management tasks. He/She is also the Demonstrator Technical Engineer focusing on technical aspects of the demonstration preparation activities. His/Her main responsibilities are listed below: Apply SJU guidelines as appropriate for FREE Solutions in order to enable the implementation of a common quality system at project level, ensure proper and effective implementation of the quality control and quality assurance, Ensure the project and its deliverables are compliant with the general principles and to the expected level of Quality, Support Project Manager in ensuring proper and effective, risk-free and with required level of quality, technical preparation of the demonstration activities. 513 Communication Manager (CM) The Communication manager supports the Project Manager in his/her duties focusing on the communication tasks such as: elaboration of the detailed Communication plan and to effectively conduct all the required communication activities to ensure an high visibility to the project activities and foster the dissemination and buy-in of its results. coordination with the appropriate Units of the members, managerial support for the organisation of the Project Meetings and any relevant planned event. 24 of 213

25 514 WP Manager (WPM) In order to ensure the appropriate coordination, information sharing and management of the project including the overall project execution the work is allocated per WPs. For each WP, a WP Manager must be identified and he/she will directly report to the PjM. WPMs are responsible for the progress of the activities within each stream, coordinates the daily subactivities with the PjM and reports to the PjM the work progress and any problem. The WPM plays a fundamental role in: ensuring the coherence at WP level of relevant tasks progresses and achievements supporting the process of re-aligning the project with its objectives and with the needs of the Programme, reviewing plans, reviewing Risks Issues and Opportunities raised at tasks level and supporting the definition of treatment actions, checking that the assigned resources are committed and adequate in number according to the activities to be performed within the WP identifying, collecting and following-up all intra/ inter WP collaboration and coordination needs raised from/ to the WP s tasks performing comprehensive management coordination (bottom-up and top-down) within its scope of supervision providing (periodically, on request when necessary) all information about the status of the tasks, especially in case of deviation from the project Work Plan coordinating the work with the other WPM and the Task Leader(s) identified within the WP. 515 Task Leader (TL) Within the WP activities, work is allocated per task. For each task, a TL must be identified and he/she will directly report to the related WPM. The role/responsibilities of the TL are the following: to ensure the timely delivery of the task according to the project Work Plan to ensure the highest cost efficiency by minimising unnecessary travel (i.e. by promoting the use of WebEx and teleconference facilities). to ensure that the deliverables are in line with the format and quality standard defined by the QRM and SJU methodologies. to provide (periodically, on request and/or when necessary) all information about the status of the task, especially in case of deviation to the project Work Plan to coordinate the work with the Task contributors assigned to the activity (clearly identifying a task work plan). 516 Task Contributor (TC) Within the WP activities, work is allocated per task. For each task, Task Contributors must be identified for each partner involved. They will be coordinated by the relevant Task Leader. TC are responsible for the technical implementation of the initiatives falling under their respective tasks. The role/responsibilities of the TCs are the following: to ensure the timely delivery of the task contribution according to the task work plan developed by Task Leader and agreed by WP Manager to coordinate the work with the Task Contributors assigned to the activity. to ensure the highest cost efficiency by minimising unnecessary travel (i.e. by promoting the use of WebEx and teleconference facilities) to ensure that the deliverables are in line with the format and quality standard defined by the 25 of 213

26 QRM and SJU methodologies. to provide (periodically, on request when necessary) all information about the status of the task, especially in case of deviation to the project Work Plan Project Management Panels Based on the map of roles and responsibilities detailed in previous chapter [Figure 2], the project management and coordination was undertaken through ad-hoc management structure/panels. Main panel was the Project Management Board (PMB) - in charge of the executive and technical management of the project [Figure 3] Figure 3: FREE Solution Project Management Panels Composition The FREE Solutions Project Management Board was formed by the Project Manager (PjM), who is a chairman of the PMB, the Project Management Office members (PMO), the Project Points of Contact (PoCs) appointed from each organization, the WP Managers (WPM). In addition, if deemed necessary, and according to the specific issue(s) of the agenda, Task leaders and/or contributors were invited to participate to the Project Management Board. The FREE Solutions Management Board met regularly, in conjunction with one of the main project milestones. The PMB supported the PjM in the project implementation and monitoring of all the project activities, their progress, the milestones achieved, the effort spent or to be spent in future periods. In case of any problem, PMB were asked to contribute in defining the possible mitigation actions or contingency plans to be applied, and, when deemed necessary, to look for an arbitration among project parties. Project Management Board (PMB) Members Project Manager (Chairperson) Project PoCs PMO WP Managers Roles The main roles/responsibilities of the PMB are the following: Supporting the PjM in the project implementation and monitoring of all the project activities, their progress, the milestones achieved, the effort spent or to be spent in future periods. Being responsible for the overall technical and operational content of the FREE Solutions project at work-package level. The PMB will approve each main contractual deliverable prior to its delivery to the customer. 26 of 213

27 Being responsible for the management of required (and agreed) changes to the management documents, WPs and Task descriptions, planning. Mechanism Meet quarterly one week before the reporting periods cut-off dates, or whenever deemed necessary Webex (or F2F if felt necessary) Escalation Process Risks, issues, disputes and conflicts are potential threats to the project outcomes (e.g. budget, delivery time, quality or scope). If they are not managed effectively they may have a negative impact on the project outcomes. Risks and issues are threats that need to be managed in order to minimize the impact on the project outcomes. Escalation is a critical management technique followed by FREE Solutions partners to facilitate the communication threats to the project outcomes, to assign accountability for resolution of threats, to aid in mitigation actions for potential risks, and to resolve issues which the TL and/or WPM cannot resolve. The Escalation process was used to ensure critical issues are raised soon enough to prevent undesirable impacts to the FREE Solutions project and to guarantee that the appropriate parties are informed and involved in critical decision-making process. The Project Manager, Sponsor and Partners strove always to make decisions and addressed issues at the lowest possible level (i.e. task level). In any case, it was important that any risk, issue, dispute, conflict, or discrepancy regarding the work plan were resolved in a way that was conducive to maintain the project schedule and preventing any on-going difficulties. In order to ensure a full alignment with the project schedule and prompt problem solving, FREE Solutions used a standard escalation model that is briefly described below. Based on the coordination and management activities described in this document and in line also with the SJU processes defined in the SESAR PMP, the process of escalation for any dispute, risk, issue and/or conflict was handled according to the following model: Escalation level Task Leader WP Manager PjM Description If an agreement within /or with other task(s) cannot be achieved, TL(s) has to escalate the issue to the relevant WPM(s). All issues should begin with a priority level equal to Low in /are order to properly follow the escalation process at task(s) level. The TL(s) is responsible for re-evaluating the priority of each new issue. The TL(s) determine(s) the level of impact, action plan and address all possible resolution alternatives. Along with the issues owner(s), the TL(s) will monitor the resolution progress. If the issue cannot be resolved at task(s) level, the TL(s) will change the priority to Medium. At this point, the WPM(s) will take over the responsibility, with the support of the involved TL(s), to properly address the resolution. The WPM(s) is responsible for re-evaluating the priority of each new issue with a status of Medium. Along with TL(s), the WPM(s) will monitor the progress of the issue. If the issue cannot be resolved at WP(s) level, the WPM(s) will change the priority to High. At this point, the WPM(s) will escalate the issue to the PjM. The PjM is responsible for re-evaluating the priority of each new issue with a status of High. Along with the involved WPM(s)/TL(s), the PjM will monitor the progress of the issue. If the issue cannot be resolved at the PjM level, he will change the priority to Critical. At this point, the PjM will 27 of 213

28 escalate the issue to the PMB. PMB The PjM will escalate all issues with a status of Critical to the PMB that will guide and provide issue resolutions approving (if so required) an ad-hoc contingency plan Table 7: Escalation Process Each of the escalated issue and its solution were reported within the Project Progress Report in order to provide the required visibility to the SJU as well as facilitate their monitoring and lessons learnt Participants and Contact list The Project Team directory is reported in the table hereafter. Title Company Name Project Sponsor SJU Serge Bagieu Project Manager ENAV Giancarlo Ferrara Project Point of Contact ENAV Claudio Cannavicci Project Point of Contact Alitalia Carlo Pacenti Project Point of Contact DFS Morten Grandt Project Point of Contact DSNA Julie Ibalot (until the 30 th of September) Project Point of Contact ECTL/NM Dimitris Apsouris Project Point of Contact Air France Laurent Renou Jérôme Dufossez (from the 1 st of October) Project Point of Contact Lufthansa Andreas Herzog-Havel Project Point of Contact MATS Robert Sant Project Point of Contact Ryanair Choorah Singh Project Point of Contact Skyguide Pascal Latron Project Point of Contact Swiss Andreas Triska Quality and Risk Manager ENAV Luigi Brucculeri Communication Manager ENAV Cristiana Abbate WP2 Manager ENAV Davide Del Vecchio WP3 Manager DSNA Julie Ibalot (until the 30 th of September) Jérôme Dufossez (from the 1 st of October) 28 of 213

29 WP4 Manager ECTL/NM Chris Brain WP5 Manager DSNA Julie Ibalot (until the 30 th of September) WP6 Manager ENAV Davide Del Vecchio T2.1 Leader DFS Morten Grandt T2.2 Leader ENAV Maurizio Mancini T2.3 Leader ENAV Davide Del Vecchio T3.1 Leader ENAV Enrico Lucini T3.2 Leader ENAV Enrico Lucini Jérôme Dufossez (from the 1 st of October) T3.3 Leader DSNA Julie Ibalot (until the 30 th of September) T3.4 Leader Skyguide Lukas Vasatko T4.1 Leader ECTL/NM Chris Brain T4.2 Leader ENAV Maurizio Mancini T4.3 Leader ENAV Nicoletta Lombardo Jérôme Dufossez (from the 1 st of October) T5.1 Leader RYANAIR/ENAV Paolo Iacobelli/Paolo Veri T5.2 Leader Skyguide Pascal Latron T5.3 Leader ENAV Michele Mastrodonato T5.4 Leader Alitalia Daniela Di Febo T6.1 Leader DSNA Thierry Laurent T6.2 Leader ENAV Cristiana Abbate Table 8: Project Team Directory Project Contact List is available: on extranet in the folder SESAR Joint Undertaking Programme > Large Scale Demonstration > LSD Free Solutions > Other Documentation on egroupware: in the folder FREE Solutions 3.2 Work Breakdown Structure The project Work Breakdown Structure covered all the required activities while allowing for clear identification of the leaderships and competences to be provided by each Consortium partner. 29 of 213

30 Project Number LSD Edition The figure below summarizes the highest level of the WBS decompositions into 6 WPs and 18 Tasks, including their ownerships Figure 4: Work Breakdown Structure WPs inter-relationships are shown in following picture: of 213

31 Project Number LSD Edition Figure 5: Work-package inter-relationships The defined CONOPS, requirements and procedures were also improved through an iterative refining process based on the evidences and results from WP4 and WP5 activities. In particular, WP4 performance assessment assisted the design phase. Then, it planned and executed assessment of live trials. The Demonstration Plan and the reference requirements were used in input to WP3 to define more in detail the operational solutions, scenarios and procedures for each Project Operational Solution/Application (i.e. City Pair optimization, Direct Routing, Free Routing). Using the defined procedures and reference scenarios, the initial performance assessment were performed and demo trials were prepared (T5.1) going through the Dress Rehearsal and Training activities (T5.2) to come then to the real Flight Trials execution activities (T5.3). In line with the defined, Demonstration Plan objectives and on the operational solution design output (i.e. WP2 and 3), flight trials data were collected and analysed (T5.4 in cooperation with WP4). Once the results were analysed, this final report was drawn up (T6.1) with the description of the different project activities and the results achieved. 3.3 Deliverables Deliverable name Demonstration Plan (D01) Demonstration Plan (D02) Date 03/11/2014 First Delivery 20/02/2015 Second Consolidated Delivery Demonstration Report (D03) 31/08/ Risk Management Table 9: Formal Deliverables Project Manager was expected to report formally on the RIO activities performed, maintaining RIO registers at least on a monthly basis. For ensuring it, the FREE Solutions Project Manager supported by PMO, was informed on escalated risks, issues and opportunities from task leaders according to the escalation process in order to process them and together with the Project Management Board to decide on the adequate treatment actions to be taken. Monitoring of risks, issues and opportunity were discussed during the PMB meetings. The main objective of the risk management plan was to increase probability and impact of positive events. Therefore, it was very important to share and endorse a common approach to risk management processes by defining how to face, plan and execute risk management along all the project lifecycle. The Consortium built a Risk Management Plan in compliance and integration with recognized SJU Risk Management methodology. The following figure depicts the process flow which will be followed for the risk management of the present project. 31 of 213

32 Figure 6: Risk Management Process Each WP Manager reported the risks encountered in the task to the Project Manager. This enabled synthesis of the main potential risks to be systematically elaborated and detailed in the quarterly Project Progress Reports. The main risk exposures were reviewed by the Management Board to which the Project Manager presented possible actions to mitigate them. In particular, FREE Solutions Risk Management Plan was built along three main sections: Risk Identification Risk Analysis and Evaluation Risk Treatment The updated list of identified risks is available on the SJU extranet in the dedicated section of the project. 32 of 213

33 Execution of Demonstration Exercises 4.1 Exercises Preparation Dates selection In general, flight trials dates selection took into account the following considerations: - It was generally agreed to avoid period of peak traffic and so, Summer, Easter and Christmas periods were excluded; - In accordance with airspace users and NM it was decided to avoid week days and week-ends after implementation of an AIRAC cycle, as well as change of airlines season schedule; - Other specificities were taken in due consideration, particularly the avoidance of ski-flows periods during Winter, which significantly affects Swiss and French airspace. All the motivations above led to the selection of specific flight trials dates Analysis of environmental and efficiency benefits To prepare each demonstration exercise, the benefits expected from the scenarios were evaluated in terms of environment and efficiency. The evaluation methodology consisted in comparing historical filed flight plans and proposed new flight plans based on the planned network improvements. The comparisons were performed using the DDR and NEST tools. The former provided, through a web application, a simple and comprehensive interface allowing generation and download of future traffic for planning purposes and past traffic for post-ops analysis of traffic trends and statistics. The latter, provided the possibility of airspace structure design and development, capacity planning and post operations analysis, strategic traffic flow organisation, scenario preparation for simulations and studies at local or network level. It permitted assessing the following indicators and metrics, listed in the table below: KPA KPI Metric Flight Efficiency Length saved Nautical miles Time saved Minutes Environmental Sustainability Fuel burn saved Tons CO2 saved Kg NOx saved Kg Table 10: Indicators and Metrics for assessing expected benefits NEST Safety Analysis Each scenario was analysed evaluating the acceptability of the risks introduced by the change according to EU Reg. 1035/2011 and Eurocontrol Safety Assessment Methodology taking also into account ANSPs, AUs and NM perspectives. The compliance of the Safety activities with the regulatory and the methodological frameworks was guaranteed by structuring a Safety Case encompassing the Safety Assessment Scenarios. The results of the Safety Case for all scenarios are collected in the FREE Solutions Safety Case [4] Human Performance Methodology An HP Case was developed and updated during the whole FREE Solutions lifecycle, i.e. operational solutions design, implementation/demonstration, evaluation. A structured approach was applied, compliant with the Human Performance Assessment Process and guidelines developed within SESAR framework by the transversal work package WP16. Contents 33 of 213

34 Project Number LSD Edition referred to Arguments and HF activities actually processed in the project are provided in the related document HP Assessment Plan Due to the fact that operational concepts investigated in this project may affect both air and ground operators, a coupled air & ground evaluation was performed in order to investigate if and how the interactions between air and ground could affect performance. A set of agreed synchronized steps was executed in order to support the coordination and cooperation among HP teams for the data collection during flight trials execution: o o o Identify shared understanding of HP issues from air and ground HP assessment processes, focusing on the interoperability issues. Define interoperability impact in terms of exercise objectives and prioritize the objectives allocating them per typology of exercise and per severity of concerned interoperability topic. Define a set of indicators and metrics together with the related success criteria to ensure the human performance interoperability objectives are adequately assessed. Where possible and appropriate, the same indicators and metrics for air and ground are planned to be used to assess the air/ground interoperability objectives. The mixed HP team based on the involvement of ANSPs and AUs Human Factors experts worked jointly for the development of HF activities and the subsequent collection of data Guidelines to all the experiment participants For each demonstration exercise, a general guideline document was produced. The objective of this document was to provide a general description of the FREE Solutions Project, a detailed description of the first set of flight trials planned, as well as guidelines and advices for a correct execution of the trials. The document was intended to be read by operational and administrative personnel involved in the project, for the preparation, execution and the post-flight evaluation of the flight trials. The document was divided in sections covering the following aspects: General description Solution assessment Route description and procedures Questionnaires for operational staff Swim line diagram Points of contact Operational Staff Briefings Leaflets were prepared to clarify scope, ambitions, needs of the project, and describe any further operational procedure required. They were made available to Pilots, Flight Dispatchers, ATCOs and Flow Managers involved in the trial. No need for specific further training was identified Questionnaires The activities performed in the context of WP4 to produce the foreseen HP and SAF cases allowed preparation of post trials questionnaires, split between air and ground sides, which have been used to collect operational experts' feedback. Human Performance and safety were assessed through post flight trial questionnaire. It aims to collect ATCOs and pilots opinions on what they experienced in the FREE Solutions flight trial they had just completed. The questionnaire was made up of six sections, focused on: - Workload - Situational awareness - Teamwork and Operations - Safety - Change management 34 of 213

35 Project Number LSD Edition Trust in concept For each statement, there was a set of possible ratings. They had to select the one that most accurately reflected their thoughts and activities. 4.2 Exercises Execution 719 Exercise ID EXE-LSD EXE-LSD EXE-LSD EXE-LSD EXE-LSD Exercise Title City Weekends Pairs City Pairs Weekdays Direct Routings Weekends Direct Routings Weekdays Actual Exercise execution start date 14/03/ /03/ /03/2015 Actual Exercise execution end date 15/03/ /03/ /03/2015 Actual Exercise start analysis date Actual Exercise end date 13/04/ /09/ /04/ /04/ /04/ /09/ /11/ /11/ /11/ /11/ /11/ /04/ /11/ /11/ /11/ /04/2016 Free Routing 05/03/ /03/ /04/ /06/2016 Table 11: Exercises execution/analysis dates Deviations from the planned activities For more information about deviations from the planned activities for each exercise, please refer to sections , , , , of 213

36 Exercises Results 5.1 Summary of Exercises Results EXE-LSD EXE-LSD Demonstration Objective Title Demonstration Objective ID OBJ Success Criterion Exercise Results Demonst ration Objective Status Advantages of using new routes/corridors in terms of efficiency/environ ment sustainability Input to airspace community Efficiency: Positive impact on fuel saving and CO2 Capacity: No negative local and network impact Predictability and Planning: Positive impact on flight punctuality and on ATC planning Human Performanc e: No negative impact associate workload and situational awareness on Production of a list of recommend Length saved: 391 NM Time Saved: 81 minutes Fuel burned saved: 3823 CO2 saved: NOx saved: 53 Eligibility:118 Acceptance: 113 Eligibility and acceptance: 5 ATFCM delay: none attributed to bunch 1. No negative effects. Sector occupancy: not significant enough ACC throughput: not significant enough Sector workload: not significant enough Conflicts: not significant enough Actual duration vs planned duration: not demonstrated. To be reviewed during DRs trials Mental workload; no major change for both pilots and controllers. Mental and physical demand slightly above average. Situational awareness: slightly above average. No major change Not produced OK OK NOK OK NOK 36 of 213

37 EXE-LSD EXE-LSD Demonstration Objective Title Demonstration Objective ID OBJ Success Criterion Exercise Results Demonst ration Objective Status Optimization of the cost for AUs and for ANSP Acceptability/Tol erability of introduced changes ations for the FAB community Formal feeding of OFA and OFA with FREE Solutions results Production of a list of recommend ations to update/impr ove existing Not produced regulations/s tandards in support of Free Routing operations Reduction in cost per single flight (Fuel/Tons and Time/Minute s) Optimized airspace Not produced allocation from AMCs Reduction as far as reasonably practicable Fully achieved of every negative impact Safety remain at or better than Fully achieved its current level of OFA involved as external reviewer since planning of demonstration activities Time Saved: 81 minutes Fuel burned saved: 3823 OK NOK OK NOK OK OK 37 of 213

38 EXE-LSD EXE-LSD Demonstration Objective Title Demonstration Objective ID OBJ Success Criterion Exercise Results Demonst ration Objective Status 726 Safety Analyse of all the critical issues related to Fully achieved the operations managemen t All the evidences requested Fully achieved by NSA provided Table 12: Summary of Demonstration EXE-LSD &002 Results EXE-LSD OK OK Demonstration Objective Title Advantages of using new DR in terms of efficiency/environ ment sustainability Demonstration Objective ID 002 OBJ Success Criterion Efficiency: Positive impact on fuel saving and CO2 emissions Capacity: no negative local impact and network impact Exercise Results Length saved: 2171 NM Time Saved: 469 minutes Fuel burned saved: 21367kg CO2 saved: kg NOx saved: 416 kg ATFCM delay: none attributed to bunch 2. Sector occupancy: not significant enough ACC throughput: not significant enough Sector workload: not significant enough Conflicts: not significant enough The overall amount of traffic does not have to be considered as very high as trial period was not a peak period (in line with the agreed criteria for the selection of Demonstr ation Objective Status OK OK 38 of 213

39 trials dates) Input to airspace community Acceptability/Tole rability of introduced changes Predictability and ATC planning: positive Actual duration vs planned impact on duration: not demonstrated flight punctuality and on ATC planning Human performance Mental workload remained at : No proper level without negative negative effects for both pilots and impact on controllers. associate Situational awareness: No major workload changes and in some cases even and augmented situational awareness Production of a list of Detailed recommendations are recommend provided in the dedicated ations for sections of this document as the FAB results of live trials activities community Formal feeding of Provision of a list of OFA recommendations, participation 3 and OFA to joint meetings and active OFAs involvement as external with FREE document reviewers Solutions results Production of a list of recommend ations to update/impr No specific need to ove existing change/improve existing regulations/s regulations/standards highlighted tandards in support of Free Routing operations Reduction as far as No negative effects on reasonably operations, according to the practicable argument analysis of every negative impact NOK OK OK OK OK OK 727 Table 13: Summary of Demonstration EXE-LSD EXE-LSD of 213

40 Demonstration Objective Title Advantages of using new DR in terms of efficiency/environ ment sustainability Input to airspace community Demonstration Objective ID OBJ Success Criterion Efficiency: Positive impact on fuel saving and CO2 emissions Capacity: no negative local impact and network impact Exercise Results Length saved: 1596 NM Time Saved: 251 minutes Fuel burned saved: 5683kg CO2 saved: kg NOx saved: 362kg ATFCM delay: none attributed to bunch 2. Sector occupancy: not significant enough ACC throughput: not significant enough Sector workload: not significant enough Conflicts: not significant enough The overall amount of traffic does not have to be considered as very high as trial period was not a peak period (in line with the agreed criteria for the selection of trials dates) Demonstr ation Objective Status OK OK Predictability and ATC planning: positive Actual duration vs planned impact on NOK duration: not demonstrated flight punctuality and on ATC planning Human performance : No negative impact on Merged with EXE-LSD OK associate HP results workload and situational awareness Production of a list of Detailed recommendations are recommend provided in the dedicated OK ations for sections of this document as the FAB results of live trials activities community Formal feeding of Provision of a list of OFA recommendations, participation 3 and OFA to joint meetings and active OK OFAs involvement as external with FREE document reviewers Solutions 40 of 213

41 results Possibility to correctly calculate, file & process flight plans based on airspace availability Optimization of the cost for AUs and for ANSP Acceptability/Tole rability of introduced changes Production of a list of recommend ations to update/impr No specific need to ove existing change/improve existing regulations/s regulations/standards highlighted tandards in support of Free Routing operations Communicat ion of airspace Achieved volumes to be avoided The Majority of flight plans pass IFPS checking Achieved to a minor extent than automaticall expected: some flights were y, or files the affected by unexpected FPL in behaviour reported in the accordance devoted section to the availability of identified DRs Reduction in cost per single flight (Fuel/Tons and Time/Minute s) Optimized airspace allocation from AMCs Reduction as far as reasonably practicable of every negative impact Safety remain at or better than its current level of Safety Analyse of all the critical Time Saved: 487 minutes Fuel burned saved: 22379kg Achieved Merged with EXE-LSD Safety results OK OK OK OK OK OK 41 of 213

42 728 issues related to the operations managemen t All the evidences requested Fully achieved by NSA provided Table 14: Summary of Demonstration EXE-LSD EXE-LSD OK Demonstration Objective Title Feasibility of FR operations on Cross Border Areas Input to airspace community Demonstration Objective ID OBJ Success Criterion Efficiency: Positive impact on fuel saving and CO2 Capacity: No negative local and network impact Predictability and ATC planning: Positive impact on flight punctuality and on ATC planning Human Performanc e: No negative impact associate workload and situational awareness on Production of a list of recommend ations the for FAB Exercise Results Length saved: 324 NM Time Saved: 46 minutes Fuel burned saved: 3701kg CO2 saved: kg NOx saved: 63 kg ATFCM delay: none attributed to bunch3. Sector occupancy: not significant enough ACC throughput: not significant enough Sector workload: not significant enough Conflicts: not significant enough Actual duration vs planned duration: not demonstrated Mental workload remained at proper level without negative effects for both pilots and controllers. Situational awareness: No major changes Detailed recommendations are provided in the dedicated sections of this document as results of live trials activities Demonstr ation Objective Status OK OK NOK OK OK 42 of 213

43 community Acceptability/Tole rability of introduced changes 007 Formal feeding of Provision of a list of OFA recommendations, participation 3 and OFA to joint meetings and active OFAs involvement as external with FREE document reviewers Solutions results Production of a list of recommend ations to update/impr No specific need to ove existing change/improve existing regulations/s regulations/standards highlighted tandards in support of Free Routing operations Reduction as far as No negative effects on reasonably operations, according to the practicable argument analysis of every negative impact Safety remain at or better than Fully achieved its current level of Safety Analyse of all the critical issues related to Fully achieved the operations managemen t All the evidences requested Fully achieved by NSA provided OK OK OK OK OK OK 729 Table 15: Summary of Demonstration EXE-LSD Choice of metrics and indicators 43 of 213

44 Objective ID & Title KPA KPI Metric OBJ-LSD To demonstrate the advantages of using new routes/corridors on most penalized citypairs in terms of efficiency/environment sustainability OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized medium-long haul traffic in terms of efficiency/environment sustainability OBJ-LSD To demonstrate the feasibility of FR operations on Cross Border Areas OBJ-LSD To give input to airspace community interested in DR/FR operations OBJ-LSD To demonstrate the possibility to correctly calculate, file & process flight plans based on airspace availability rather than route availability OBJ-LSD To demonstrate that an airspace management tailored on AU needs along with an improved cooperation with AMC can contribute to optimize the cost for airlines as well for air navigation services OBJ-LSD To demonstrate that the Environment- Fuel Efficiency Airspace Capacity En Route Predictability and ATC planning Human Performance a. Length saved b. Time saved c. Fuel burned saved d. CO2 saved e. NOx saved f. Eligibility flights planned g. Acceptance - flights flown h. Eligibility and acceptance flights planned against flights flown a. ATFCM delay b. Sector occupancy c. ACC throughput d. Sector workload e. Conflicts Actual time duration vs planned time duration Mental Workload Situational awareness a. Nautical miles b. Minutes c. Tons d. Delta Kg e. Delta Kg f. Number of flights g. Number of flights h. Delta number of flights a. Minutes b. Number of aircraft./hour c. Number of aircraft/hour d. Delta % points e. Number of potential conflicts/secto r Delta minutes Scores (questionnaires) N/A - General Recommendations List of recommendations N/A - Civil-Military Cooperation and Coordination Updates of EAUP by EUUP Availability of TSA/TRA to GAT per day Average AO reaction time interval to re-file FPL following UUP before EOBT Number of updates Hours and minutes Hours and Minutes Safety Residual Fuel at arrival Scores (questionnaires) 44 of 213

45 changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably Safe Summary of Assumptions Table 16: Summary of metrics and indicators The four assumptions identified in the FREE Solutions Demonstration Plan delivered to the SJU [1] were still valid at the time of exercises execution. In addition, during the preparatory activities to live trials execution, two new brand assumptions were made. Identifier Title Type of Assumption Description Justification Flight Phase KPA Impacted Source Value(s) Owner Impact on Assessment ASS Trials Conditions Schedule Trials periods are identified so that to avoid period of peak traffic which may influence the ability to perform planned exercises with the required level of flexibility. This means planning of trials activities avoiding Christmas, Easter and Summer, Ski-Flows 3 period Perform Trials in average traffic conditions En-Route, other flight phases are excluded All the ones identified for this project Consortium Members N/A N/A Valid for the entire project N/A 741 Identifier Title Type of Assumption Description Justification ASS Trials Conditions Schedule Trials periods are identified so that to avoid period after implementation of AIRAC cycles and Airlines season schedule change. The purpose is to de-risk live trials activities by performing in consolidated routine operations. Perform Trials in nominal and routine traffic conditions 3 Ski-Flows is the name for the flow of traffic bringing tourists in skiing locations during winter seasons (mainly form UK towards Switzerland and Austria) which affect Swiss and French airspace 45 of 213

46 Flight Phase KPA Impacted Source Value(s) Owner Impact on Assessment En-Route, other flight phases are excluded All the ones identified for this project Consortium Members N/A N/A Valid for the entire project N/A Table 17: Demonstration Assumptions Impact on Safety, Capacity and Human Factors Nothing to be reported in this section Description of assessment methodology Environment-Fuel Efficiency Methodology In order to produce the Environment Case for the FREE Solutions demonstration it was necessary to study the effects of the solutions proposed for City Pairs in relation to the addressed KPA Error! Reference source not found.. Additionally the demonstration supported the need to address the argument which forms the basis of the case as follows: Environment Argument - Improve the performance of flight efficiency and environment sustainability (emissions reduction) across the European Network The FREE Solutions environment assessment provided evidence to support the environment case argument which is to improve the performance of flight efficiency and environment sustainability (emissions reduction) across the European Network and contribute to reaching the SESAR contribution to the targets shown in the figure below: Figure 7: ATM Master Plan Target 46 of 213

47 Project Number LSD Edition In order to quantify the effects of the improvements demonstrated in each solution, direct comparison with flights using the route used prior to the improvement, termed the Reference Route, with flights being trialled on the new improved route, termed the Solution Route, is made Figure 8: Trial Flights Compare to Reference Flights In order to mitigate the effects of so many factors affecting the absolute fuel consumption shown in Figure 8 by various colour dots, the way to handle different factors is by finding the mathematical relationship for how the different parameters affect the fuel consumption of the flight. The actual fuel consumption of the flights are then re-calculated (using these mathematical relationships) to represent the fuel consumption of a flight for a given set of conditions (mass, meteorological conditions, cruising altitude etc.) that are kept the same for all flights. This is called normalisation and is used during the analysis. After normalisation of the factors having the largest impact on fuel burn, the difference between individual flights in the two datasets has been reduced. This is illustrated in Figure 9 with more homogenous colours of the squares and the dots. The two sets of flights are now more comparable and it will be easier to detect the difference in fuel burn that we are trying to identify. Trial flights (normalised) Reference flights (normalised) Compare fuel burn Figure 9: Normalised Flights The Reference Route is defined as the route which directly relates to the improvement provided during the AIRAC cycle prior to the trial known as the Reference Period. For flight comparisons corresponding days of the week are used when comparing the traffic in the Reference Period with the Solution Period. The Solution Route is the new route, including the improvement, provided by the direct route. 47 of 213

48 Figure 10: City Pair Comparison Elements Figure 10 shows the possible comparison elements for the analysis of a City Pair. For the Environment Sustainability and Fuel Efficiency KPA the primary analysis is aimed at the comparison between the R(ref) and R(scn) in terms of the delta route length to be flown per trial flight. In addition, comparisons of the last filed Flight Plans (FPLs) of the subject flights during the Reference Period, FPL(ref), with the same flights during the Solution Period, FPL(scn), provides evidence of the planned rate of usage by AOs of the R(scn) against the R(ref). A comparison of the horizontal Correlated Positons of a Flight (CPF), in the en-route phase of flight shows actual behaviour of the subject flights both during the Reference and Solution periods, including important evidence of the actual distance flown per trial flight. This distance may be influenced by actions of aircrew, weather, ATC or ATFCM. Furthermore comparisons with CPF(scn) and R(scn) and optionally CPF(ref) and R(ref), provides input to the Predictability KPA Airspace Capacity En-Route Methodology The capacity assessment provides evidence to support a high level capacity case for the demonstration which focuses on the argument that the improvements in environment do not provide a negative impact on capacity both at local and Network level. The analysis is mainly focused on the en-route Airspace Capacity KPA. In order to quantify the effects of the trial solutions on capacity a comparison with the situation prior to the improvement being trialled are made. To this end, the demonstration supported the need to address the argument which form the basis of the case as follows: Capacity Argument - Ensure that improvements in environment do not provide a negative impact on capacity both at local and Network level The reference to be used in the capacity assessment is the traffic, airspace design, sector configurations, route network, ATC procedures, ATC working methods and FUA processes/procedures used on corresponding days in the AIRAC cycle prior to the subject AIRAC cycle of the Solution Period. The solution and corresponding exercises deals with the implementation of the demonstration scenarios identified for the EXE-LSD and EXE-LSD to handle City Pairs optimization through direct routing, where only consortium flights are allowed to participate. When performing the capacity analysis for City Pair and DR (weekday and weekend) the following aspects have been considered by comparing Reference and City Pair Solutions in three phases (conditions): Strategic -model assigned potential flights Pre-tactical - filed flight plans Operations - tactical flown trajectories 48 of 213

49 Project Number LSD Edition The following quantitative analysis has been conducted for City Pairs: Potential flights; Number of eligible (consortium city pair only) flights that will be able to file proposed solutions Realised flights; Number of effected (consortium- city pair only) flights that will accept and file proposed solutions At Most Risk ACC, in terms of accommodation of required capacity levels, the following sector analysis is performed: o Sector load; o Sector sequence; o Sector re-entry; o Sector clipping; o Conflict analysis. When significant differences in the listed parameters are observed, between the Reference and Solution, investigations are performed and later analysed in all three phases (conditions): model based, filed flight plans and flown trajectories. Generally, significant impact on capacity is not expected, particularly for City Pairs, as they take place in low traffic density periods with flights being limited in number. However, listed quantitative parameters are measured, prior to the trials, during the execution phase and documented accordingly in the post operational analyses Safety Assessment Methodology This section illustrates the methodology adopted to undertake Safety activities within FREE Solutions Project. The safety assessment of FREE Solutions has been structured in three main Safety Assessment Scenarios and aims at demonstrating that all demonstration scenarios are acceptably safe. The table below provides the correspondence between Demonstration and Safety Assessment Scenarios: Demonstration Scenarios Safety Assessment Scenarios SCN-LSD City Pairs weekend SCN-LSD City Pairs weekdays SCN-LSD Direct Routings Weekends SCN-LSD Direct Routings Weekdays SCN-LSD Free Routing City Pairs Direct Routings Free Routing Table 18: Demonstration Scenarios vs Safety Assessment Scenarios All scenarios were analysed evaluating the acceptability of the risks introduced by the change according to EU Reg. 1035/2011 and Eurocontrol Safety Assessment Methodology taking also into account ANSPs, AUs and NM perspectives. The compliance of the Safety activities with the regulatory and the methodological frameworks has been guaranteed by structuring a Safety Case encompassing all the Demonstration/Safety Assessment Scenarios. The Safety Analysis has been organized and performed according to the Goal Structuring Notation (GSN) model. 49 of 213

50 Project Number LSD Edition According to the Safety Case methodology the GSN has been organized to afford the Demonstration Objectives verification through the application of the EUROCONTROL Safety Assessment Methodology (SAM). In the pictures below the GSN structure for the City Pairs Safety Case is illustrated as an example (the other Scenarios have been managed similarly). For more details regarding the Safety Case results regarding all Demo Scenarios please refer to [4]. Arg 0 The changes introduced by FREE Solutions project are acceptably safe ST 001 The Safety of the FREE Solutions Project flight trials will be demonstrated applying the EUROCONTROL Safety Assessment Methodology taking into account all the foreseen scenarios (City Pairs, Direct Routing and Free Routing) Arg 1 The flight trials execution related to the City Pairs concept will be acceptably safe Arg 2 The flight trials execution related to the Direct Routing concept will be acceptably safe Figure 11: GSN Top Level Claim Arg 3 The flight trials execution related to the Free Routing concept will be acceptably safe Cr 01 The Safety of the operations shall remain at or better than its current level of Safety Arg 1 The flight trials execution related to the City Pairs concept will be acceptably safe Cr 03 All the evidences requested by the local NSAs will be provided Cr 02 Every negative effect on the operations shall be reduced as low as reasonably practicable (A.L.A.R.P.) A 01 All the critical issues related to the stakeholders operations will be analyzed and mitigated A 02 Each Consortium Partner will individually demonstrate that: - results are locally reached at National level - NSA s requirements are locally reached - FREE Safety Objectives have been locally reached ST 002 Determine all the hazard introduced by the change and identify the related causes, effects, existing mitigation means, Safety Objectives and all the Safety issues to be monitored during the operational phase. C 01 EUROCONTROL Safety Assessment Methodology Arg 1.1 The opearational scenario has been described Arg 1.2 The hazard have been identified Arg 1.3 The effects have been identified and classified determining their related causes and existing mitigation means Arg 1.4 Safety Objectives have been identified Arg 1.5 Safety Issues to be monitored after the implementation have been identified Arg 1.6 Safety evidences are collected Demonstra tion Plan Scenario Description Local Saf Assessment ST 004 Considering the new operational scenario, for each hazard identified, high level effects have been identified and classified according to the mitigation means. The ones related to the partners operational scenario have been specified in the Local Safety Assessment conducted by the Consortium Partners. ST 005 Considering the new operational scenario, for each High Level Hazard identified in FREE Safety Case,the related Safety Objectives have been identified. The ones related to the partners operational scenario have been specified in the Local Safety Assessment conducted by the Consortium Partners. ST 006 Considering the new operational scenario, for each High Level Hazard identified in FREE Safety Case, the related Safety Requirements have been specified. The process have been detailed according to the Partners operational scenarios in the local Safety Assessments ST 007 Considering the new operational scenario, for each Safety Issue identified in FREE Safety Case and in the related annexes, the related identified evidences are monitored in order to guarantee the acceptable level of Safety of the Operations FHA Safety Case FHA Local Saf Assessment FHA Safety Case FHA Local Saf Assessment Figure 12: GSN CP Scenario PSSA Safety Case PSSA Local Saf Assessment SSA Safety Case SSA Local Saf Assessment 50 of 213

51 Project Number LSD Edition In accordance with the SAM-FHA, the Hazard identification process was performed to verify the GSN Argument 1.2. The process was carried out in line with the strategy outlined in the picture below: Arg 1.2 The hazard have been identified ST 003 Considering the new operational context, all the criticalities which could affect the Operations Safety Level during the flight trials execution have been identified Arg Preliminary hazard have been identified Arg Equipment affected by the change has been considered Arg Airspace configuration has been considered Arg Operational and flight procedures have been considered Arg People related issues have been considered Arg Environmental and meteorological conditions have been considered Demonstration Plan FHA- Safety Case FHA- Local Safety Assessment FHA- Safety Case FHA- Local Safety Assessment FHA- Safety Case FHA- Local Safety Assessment FHA- Safety Case FHA- Local Safety Assessment HP Case Figure 13: GSN CP Hazard Identification FHA- Safety Case FHA- Local Safety Assessment FHA- Safety Case FHA- Local Safety Assessment The hazard identification process involved (by direct participation or indirect contribution to the document) several Subject Matter Experts from all the Consortium stakeholders and was performed by means of a brainstorming session to identify and verify all the criticalities that could have affected the operations during the flight trials. For each hazard effect, severities and maximum probability of occurrence were allocated applying the Worst Credible Effect principle. The hazards were identified pointing out ANSPs and AUs issues independently but taking into account also their interactions. Moreover, the analysis, focusing on the causes, effects and evidences, allowed to cover also the two other phases of the SAM, namely PSSA and SSA. In the Safety Case document, the complete list of Safety Objectives to be accomplished to guarantee the acceptance of the risks identified is reported. A set of Safety Requirements has been identified and an ad-hoc questionnaire (to be filled out after trial flights) was prepared to validate the identified hazard. As an example, Table 19 shows the hazard list as output of the brainstorming session focused on CPs. It shows that the hazard identification has covered all the arguments requested to verify the FHA phase of the Safety Assessment Methodology (Arg to Arg ). Hazard ID ANSP/AU Hazard description City Pair GSN Reference HZ 01 ANSP Inconsistencies of FPL planning, distribution and processing Arg Arg HZ 02 ANSP Technical issues (i.e. FDP, OLDI, STCA, HMI Display Inconsistencies) especially as no system modification is foreseen Arg Arg of 213

52 Hazard ID ANSP/AU Hazard description City Pair GSN Reference HZ 03 AU Invalid flight plan that could be either rejected by IFPS or be a non-correct one Arg Arg HZ 04 HZ 05 HZ 06 HZ 07 HZ 08 HZ 09 ANSP ANSP ANSP ANSP AU AU Sector overload Arg Arg Interactions between trial flights and standard flights Unexpected entry/exit point (related to the new design of the City Pairs/Direct Routing procedures) (Conflict typology) Conflicts with small angles or close to sector/fir boundaries Fuel shortage due to inadequate calculation of the necessary fuel to perform flight trials routes Arg Arg Arg Arg Arg Arg Navigation error Arg Arg HZ 10 ANSP Trial flights performing CP/DR cross active military areas on planned route Arg HZ 11 ANSP Navigation issues during flight trials execution Arg HZ 12 ANSP Discrepancy between ATCOs mental picture and filed route by aircraft Arg Arg HZ 13 AU Discrepancy between pilot mental picture and filed route by aircraft Arg Arg HZ 14/15 ANSP/AU Adverse meteorological conditions Arg Arg Table 19: Hazard Identification results During the brainstorming session, which involved all the Consortium partners SMEs by direct participation or indirect contribution to the document, the FHA (Functional Hazard Assessment) of the SAM was performed and the severities of the abovementioned hazards were identified. The details of the hazard identification, together with its related causes and effect analysis, are reported in Free Solution Safety Case 1.0 [4]. Table 19 shows that the hazard identification covered all the arguments requested to verify the FHA phase of the Safety Assessment Methodology (Arg to Arg ). Moreover, the causal and effect analysis has led the Safety analysis to cover also the other phases of the SAM by identifying the Safety Objectives to be accomplished to guarantee the acceptance of the risks identified. To verify the acceptability of the risks a set of Safety Requirements was identified and an ad-hoc questionnaire has been prepared to confirm that the hazards identified have been correctly assessed. The final version of the Safety Case which was validated and approved by all the Consortium partners and its results are presented in. 52 of 213

53 Project Number LSD Edition HP Assessment Methodology Human Performance (HP) denotes the human capability to i) successfully accomplish tasks and ii) meet job requirements [7].The capability of a human to successfully accomplish tasks depends on a number of variables that are usually investigated within the HP underlying discipline known as Human Factors (HF). In line with the technical proposal, an HP Case was developed and updated during the whole FREE Solutions lifecycle. The HP Case in general provides a structured framework to effectively accomplish declared objectives by systematically addressing the HF aspects (i.e. benefits & issues, gaps derived from past related projects) throughout the whole project life-cycle, i.e. operational solutions design, implementation/demonstration, evaluation. In FREE Solutions context, the HP Case was developed within the sub-work package T4.3 according to the aim to support the smooth transition towards Free Routing operations implementation, going through City Pairs and Long Direct Route operations plus an advanced flexible use of airspace. In order to cover this high level project HP objective, a structured approach was applied. It was compliant with the Human Performance Assessment Process and guidelines developed within SESAR framework by the transversal work package WP16 [7]. SESAR HP assessment process uses an argument and evidence approach to provide the necessary evidence that HF issues & benefits have been identified and addressed. The approach also includes details regarding how, i.e. HF methods, issues & benefits are planned to be modelled and then managed. In FREE Solutions, the HP Case was organised according to a tree-structure that has started with the Top Claim, i.e. Argument 0 and it was then decomposed till third-level arguments and related HF methods for the evidence collection. Contents referred to Arguments and HF activities actually processed in the project are provided in related document HP Assessment Plan [10]. Due to the fact that operational concepts investigated in this project may affect both air and ground operators, a coupled air & ground evaluation has been performed in order to investigate if and how the interoperability between air and ground could affect performance. In line with the major guidelines of SESAR WP16, the HP Case within FREE Solutions foresaw the integrated assessment of both ground and air operations. A set of agreed synchronized steps was executed in order to support the coordination and cooperation among HP teams for the data collection during flight trials execution. The main steps were as follows. A. Identify shared understanding of HP issues from air and ground HP assessment processes, focusing on the interoperability issues. B. Define interoperability impact in terms of exercise objectives and prioritize the objectives allocating them per typology of exercise and per severity of concerned interoperability topic. C. Define a set of indicators and metrics together with the related success criteria to ensure the human performance interoperability objectives are adequately assessed. Where possible and appropriate, the same indicators and metrics for air and ground are planned to be used to assess the air/ground interoperability objectives. The mixed HP team based on the involvement of ANSPs and AUs Human Factors experts has collaborated together for the development of HF activities and the subsequent collection of data Results impacting regulation and standardisation initiatives No potential impacts identified according to the outcomes of performed live trials exercises. 5.4 Analysis of Exercises Results Please refer to the different tables in section Unexpected Behaviours/Results For more information about unexpected behaviours/results for each exercice, please refer to the sections: 53 of 213

54 (EXE-LSD ), (EXE-LSD ), (EXE-LSD ), (EXE-LSD ) Confidence in Results of Demonstration Exercises Quality of Demonstration Exercises Results During screening of questionnaires and subsequent delivery of statistics an anomaly was observed. For some questions of filled-in templates, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve the full score (i.e. 100%) through the sum up of afferent intermediate quotes. As a final effect a margin of uncertainty is provoked and induced subsequent effects, first on the analysis of data and secondly, on the reporting. However, the quality of results is assumed to be on a at least sufficient level. For individual exercises results quality, please refer to the respective sub-chapters in chapter Significance of Demonstration Exercises Results Data collected through questionnaires were manipulated and then converted into statistics. Manipulation consisted into the aggregation of scores. Conversion implied that answers to the different choices were expressed in percentage values, (i.e. partial quotes were reported in reference to the overall amount. 54 of 213

55 Demonstration Exercises reports 6.1 Demonstration EXE-LSD Report Exercise Scope The scope of the exercise was to set-up and fly a set of DRs, during week-ends (starting at 00:00 AM of Saturday and ending at midnight of Sunday), restricted to some identified consortium flights. The overall approach for city pairs optimization aimed at stretching as much as possible existing planned company routes to exploit potential benefit, especially in terms of length reduction and consequent fuel savings and emissions reduction. To this end, the following stepped approach was taken, as described in the section 4.1 of the FREE Solutions Demonstration Plan v2.0 [1]: Proposed city pairs optimization must include as many different ACCs from the consortium as possible, in line with the overall project objectives to execute activities in a cross FAB area; there should be potential improvement/optimization being identified when comparing planned and flown trajectories; expected benefits must be assessed as being valuable; they represent quick wins to be achieved in a short term time-frame; they allow to plan a flight route 5NM shorter than the one planned before the trials, at least. On the other hand, certain city pairs were discarded depending on several factors. First, it must be taken into consideration the overall complexity of the current European Airspace, which limited the possibility to select some proposed city pairs optimization. More in details, additional factors were discovered and taken into account while performing flight trials preparatory activities, such as: Some medium/long-haul proposed city pairs optimization were not selected due to the crossing of Libyan airspace along the route direction (e.g EHAM HDAM proposed by KLM) City Pairs selection was also constrained by lack of radar coverage in some areas of the consortium airspace. This happened for those areas at the boundary of this airspace like for example the FIR border of Alger in the southern part of the consortium airspace (e.g. LIRF- SBGR, LIRF-EDAG, LIRA-GMMX) Another factor taken into account for not selecting specific city pairs was the airspace due to the presence of more than two FIRs which would have required numerous tactical precoordination activities This is the example of the following proposed city pairs optimization LIRF-DAAG/SBGR, LIRA-GMMX with Marseille and Rome FIR involved and Alger FIR potentially affected. In general, flight trials dates selection took into account the following considerations. It was agreed to avoid period of peak traffic and, so, Summer, Easter and Christmas periods were excluded. Then, in accordance with airspace users and NM it was decided to avoid week days and week-ends after implementation of a AIRAC cycles, as well as change of airlines season schedule (e.g. from winter to summer). Additionally, some specific request coming from consortium partners were taken in due consideration like, for example, the avoidance of ski-flows periods during winter, which significantly affects Swiss and French airspace. All the motivations above led to the selection of specific flight trials dates, as indicated in the section [ ] of this document. To conclude, this exercise must be considered as a step to demonstrate to the consortium members and the community that identified benefits may be quickly delivered. The preparatory activities, the outcomes and the lessons learned of this exercise are closely linked with the DRs weekend exercise EXE-LSD , whose aim is to further exploit all the benefits identified and enlarge the number of participating airlines. 55 of 213

56 Project Number LSD Edition Exercise Scenario Exercise scenario SCN-LSD referred to normal day of operations during weekends, with no significant events/deviations form routine operations. It encompassed a time window starting at 00:00 AM of Saturdays and ending at midnight of Sundays (UTC). Scenario is applicable from end of SID to beginning of STAR and is consequently limited to En-Route phase. Its geographical scope is limited to the airspace of the members of FREE Solutions consortium. Aircraft flew in accordance with existing operational procedures in controlled airspace. At the same time, operations were ensured through existing working procedures for all the ACCs involved in the trails activities Exercise Objectives Figure 14: Participating ACCs - City Pairs This exercise addressed the demonstration objectives and related KPAs listed in the table below. OBJ ID OBJ Description KPAs and TAs OBJ-LSD OBJ-LSD To demonstrate the advantages of using new routes/corridors on most penalized city-pairs in terms of efficiency/environment sustainability To give input to airspace community interested in DR/FR operations Environment - Fuel Efficiency; Human Performance; Airspace Capacity - En- Route. Predictability and ATC Planning General N/A OBJ-LSD To demonstrate that the changes introduced Safety 56 of 213

57 by each scenario foreseen by FREE Solutions Project are acceptably/tolerably safe Human Performance Table 20: EXE-LSD Demo Objectives Conduct of Demonstration Exercise EXE-LSD Exercise Preparation Route Design The project adopted an incremental preparation process that took into account the analysis of the most penalized routes of the current network together with the inputs for improvement provided by the Airspace Users involved in the project. Following the design work completed in WP3, a first set of identified City Pairs (i.e. 109 city-pairs were initially proposed by AUs) routes optimization was initially evaluated by the NM, to create more efficient routes in terms of mileage saving, fuel consumption, and environmental impact Such evaluations were performed using the DDR (Demand Data Repository) and NEST tools. The former provides, through a web application, a simple and comprehensive interface allowing generation and download of future traffic for planning purposes and past traffic for post-ops analysis of traffic trends and statistics. The latter, provides the possibility of airspace structure design and development, capacity planning and post operations analysis, strategic traffic flow organisation, scenario preparation for simulations and studies at local or network level. Actual work consisted in making a comparison between planned company route against expected city pairs optimization route for each selected flight, calculating the potential gains in terms of reduced route length (NM), reduced fuel burnt (kg), reduced CO2 (kg) Figure 15: City Pairs potential benefits evaluations After several iteration of a scenario evaluation, the results were used to support WP5 in the process of choosing the list of City Pairs routes optimization to be executed during the planned flight trials (reported in the table below). DEPARTURE DESTINATION LFPG FIMP LIPZ LSZH 57 of 213

58 LEBL EHAM EBBR EBCI EGSS EGLL EGSS EHAM LFML LFLL LFML LEVC LFMN LSZH LIRF LIRF LIRA LIRA LIRF LMML LFMN LIPZ LIPZ LFST EDDF EDDF Table 21: The list of city pairs routes optimization to be executed The outcome of this process was a list of identified city pairs per each involved airline, per each day of the trial, with specific aircraft ID and number, as reported in the table below. This result took also into account the opportunity to guarantee a minimum number of available city pairs per each participating consortium airline, like for example the addition of VANAS-DITON for Lufthansa City Pairs. The overall number of FREE Solutions eligible flights was 125, with 123 to be performed during week end and 2 during week days, as shown in the section [ ]. Saturday 14th March FT Eligible 1. EHAM LIRF AZA107 A32S 11:20 UTC Y 2. AZA109 A32S 17:05 UTC Y 3. AZA11A A32S 19:30 UTC Y 4. EGLL LIRF AZA201 A32S 06:45 UTC Y 5. AZA207 A32S 16:25 UTC Y 6. AZA209 A32S 20:00 UTC Y 7. LEVC EDDF DLH82T A32A 11:40UTC Y 8. LFMN EDDF DLH99P A319 06:15 UTC Y 9. DLH93T A321 08:55 UTC Y 10. DLH24U A319 17:35 UTC Y 11. LFLL LIPZ HOP4522 CRJ7 09:00 UTC Y 12. EHAM LIRF KLM1597 B737 05:50 UTC Y 13. KLM65L B737 08:55 UTC Y 14. KLM1607 B737 15:55 UTC Y 15. KLM69M B737 20:00 UTC Y 16. EHAM LFMN KLM1263 B737 08:45 UTC Y 17. KLM41U E190 13:15 UTC Y 18. KLM1273 E190 19:25 UTC Y 19. EBBR LIRF RYR7010 B738 09:30 UTC Y 20. RYR2983 B738 14:40 UTC Y 21. EBCI LIRA RYR382C B738 06:30 UTC Y 58 of 213

59 22. EGSS LIRA RYR43ST B738 08:40 UTC Y 23. RYR28SN B738 12:10 UTC Y 24. RYR91DG B738 16:10 UTC Y 25. RYR60UL B738 19:40 UTC Y 26. EGSS LMML RYR473L B738 16:45 UTC Y 27. LIPZ LSZH SWR1661 RJ1H 09:55 UTC Y 28. SWR1663 RJ1H/A319 14:45 UTC Y 29. LEBL LSZH SWR183Q A32S 19:50 UTC Y 30. SWR193E A32S 14:40 UTC Y Sunday 15th March 31. LFML LIPZ AFR1862 A319 11:05 UTC Y 32. EHAM LIRF AZA107 A32S 11:20 UTC Y 33. AZA109 A32S 17:05 UTC Y 34. AZA11A A32S 19:30 UTC Y 35. EGLL LIRF AZA201 A32S 06:45 UTC Y 36. AZA207 A32S 16:25 UTC Y 37. AZA209 A32S 20:00 UTC Y 38. LFML LFST HOP25ZY E145 15:15 UTC Y 39. HOP27ZY E145 19:05 UTC Y 40. EHAM LIRF KLM1597 B737 05:50 UTC Y 41. KLM65L B737 08:55 UTC Y 42. KLM1607 B737 15:55 UTC Y 43. KLM69M B737 20:00 UTC Y 44. EHAM LFMN KLM1263 B737 08:45 UTC Y 45. KLM41U E190 13:15 UTC Y 46. KLM1273 E190 19:25 UTC Y 47. EBBR LIRF RYR7010 B738 09:40 UTC Y 48. RYR2983 B738 14:40 UTC Y 49. EBCI LIRA RYR382C B738 06:30 UTC Y 50. RYR148Z B738 16:40 UTC Y 51. EGSS LIRA RYR43ST B738 08:40 UTC Y 52. RYR28SN B738 12:10 UTC Y 53. RYR91DG B738 16:10 UTC Y 54. RYR60UL B738 19:40 UTC Y 55. LIPZ LSZH SWR1661 RJ1H 09:55 UTC Y 56. LEBL LSZH SWR1953 A32S 09:45 UTC Y 57. SWR193E A32S 14:40 UTC Y 58. SWR183Q A32S 19:50 UTC Y Saturday 21st March 59. EHAM LIRF AZA107 A32S 11:20 UTC Y 60. AZA109 A32S 17:05 UTC Y 61. AZA11A A32S 19:30 UTC Y 62. EGLL LIRF AZA201 A32S 06:45 UTC Y 63. AZA207 A32S 16:25 UTC Y 64. AZA209 A32S 20:00 UTC Y 59 of 213

60 65. LEVC EDDF DLH82T A32A 11:40UTC Y 66. LFMN EDDF DLH99P A319 06:15 UTC Y 67. DLH93T A321 08:55 UTC Y 68. DLH24U A319 17:35 UTC Y 69. LFLL LIPZ HOP4522 CRJ7 09:00 UTC Y 70. LFML LFST HOP21SL E145 07:50 UTC Y 71. EHAM LIRF KLM1597 B737 05:50 UTC Y 72. KLM65L B737 08:55 UTC Y 73. KLM1607 B737 15:55 UTC Y 74. KLM69M B737 20:00 UTC Y 75. EHAM LFMN KLM1263 B737 08:45 UTC Y 76. KLM41U E190 13:15 UTC Y 77. KLM1273 E190 19:25 UTC Y 78. EBBR LIRF RYR7010 B738 09:30 UTC Y 79. RYR2983 B738 14:40 UTC Y 80. EBCI LIRA RYR382C B738 06:30 UTC Y 81. EGSS LIRA RYR43ST B738 08:40 UTC Y 82. RYR28SN B738 12:10 UTC Y 83. RYR91DG B738 16:10 UTC Y 84. RYR60UL B738 19:40 UTC Y 85. EGSS LMML RYR473L B738 16:45 UTC Y 86. LIPZ LSZH SWR1661 RJ1H 09:55 UTC Y 87. SWR1663 RJ1H/A319 14:45 UTC Y 88. LEBL LSZH SWR1951 A32S 06:30 UTC Y 89. SWR1953 A32S 09:45 UTC Y 90. SWR193E A32S 14:40 UTC Y 91. SWR183Q A32S 19:50 UTC Y Sunday 22nd March 92. LFML LIPZ AFR1862 A319 11:05 UTC Y 93. EHAM LIRF AZA107 A32S 11:20 UTC Y 94. AZA109 A32S 17:05 UTC Y 95. AZA11A A32S 19:30 UTC Y 96. EGLL LIRF AZA201 A32S 06:45 UTC Y 97. AZA207 A32S 16:25 UTC Y 98. AZA209 A32S 20:00 UTC Y 99. LEVC EDDF DLH82T A32A 11:40UTC Y 100. LFMN EDDF DLH99P A319 06:15 UTC Y 101. DLH93T A321 08:55 UTC Y 102. LFML LFST HOP25ZY E145 15:15 UTC Y 103. HOP27ZY E145 19:05 UTC Y 104. EHAM LIRF KLM1597 B737 05:50 UTC Y 105. KLM65L B737 08:55 UTC Y 106. KLM1607 B737 15:55 UTC Y 107. KLM69M B737 20:00 UTC Y 108. EHAM LFMN KLM1263 B737 08:45 UTC Y 60 of 213

61 109. KLM41U E190 13:15 UTC Y 110. KLM1273 E190 19:25 UTC Y 111. EBBR LIRF RYR7010 B738 09:40 UTC Y 112. EBCI LIRA RYR382C B738 06:30 UTC Y 113. RYR148Z B738 16:40 UTC Y 114. EGSS LIRA RYR43ST B738 08:40 UTC Y 115. RYR28SN B738 12:10 UTC Y 116. RYR91DG B738 16:10 UTC Y 117. RYR60UL B738 19:40 UTC Y 118. LIPZ LSZH SWR1661 RJ1H 09:55 UTC Y 119. LEBL LSZH SWR1953 A32S 09:45 UTC Y 120. SWR193E A32S 14:40 UTC Y 121. SWR183Q A32S 19:50 UTC Y Saturday 28th March 122. LIPZ LSZH SWR1663 RJ1H 14:45 UTC Y 123. LFLL LIPZ HOP4522 CRJ7 09:00 UTC Y Table 22: EXE-LSD City Pairs List - weekends Flight Planning The FREE Solutions FPL Route Segment shown in the table below, highlights for every City Pair, only the improved portion on the filed ATS route for the flight trial (3 rd column of the table). It was included in the field 15 of the ICAO FPL document, during flight planning. This portion of route was matched with the route connection between departure aerodrome until the first waypoint of the FREE Solution FPL route, and from the last waypoint of the FREE Solutions FPL route until the destination aerodrome. The preparatory activities led to an agreed route which was referred only to ground-ground agreement between involved ANSPs and represented a silent coordination procedure. Actually, FREE Solutions flights were cleared by ATC unit according to the agreed route without any further coordination required. To clarify, an abstract from the guidelines document prepared for live trials execution [2] is shown hereafter and is related to manual filing of flight plan for the EBBR-LIRA city pair. 1. Fill the flight planning field 15 with the complete route: 2. N0448 F370 DCT REMBA UM624 RITAX UT27 GTQ UQ343 BEGAR DCT IPLUB DCT GITOD GITOD3F (speed and level can be adapted) 3. Errors LI1A and LS1A on DCT segment will be forced. 4. Please Insert in field 18 of the FPL: RMK/FREE TRIAL 5. The flight can be subjected to normal ATFM restrictions DEPARTU RE LFPG DESTINATI ON FIMP FREE SOLUTION FPL ROUTE MOKIP-KOGAS-LURAG-TOP-IRBAK- BELIX FREE SOLUTIONS AGREED ROUTE MOKIP-KOGAS-TOP-BELIX LIPZ LSZH VIC-RESIA-KELIP VIC-RESIA_KELIP LEBL LSZH IRMAR-LAMUR-GUDAX IRMAR-LAMUR-GUDAX EHAM LIRF BEGAR-ODINA-OGLAK-XIBIL BEGAR-XIBIL EBBR LIRF BEGAR ODINA OGLAK XIBIL BEGAR-XIBIL 61 of 213

62 EBCI LIRA BEGAR-IPLUB-GITOD BEGAR-IPLUB-GITOD EGSS LIRA BEGAR IPLUB GITOD BEGAR IPLUB GITOD MOKIP-KOGAS-LURAG-TOP-BETEN- EGLL LIRF ELKAP MOKIP-KOGAS-TOP-ELKAP BEGAR-ODINA-OGLAK-XIBIL BEGAR-XIBIL EGSS LMML BEGAR-ODINA-EROVI-MARON-GZO BEGAR-MARON-GZO EHAM LFMN GILIR MEDAM GILIR MEDAM LFML LIPZ NOSTA GIXUL KOPER NOSTA GIXUL KOPER LFLL LIPZ ATMAD NITAM RIDVA KOPER ATMAD KOPER LFML LFST OMASI GILIR OMASI GILIR LEVC EDDF VANAS DITON VANAS DITON LFMN EDDF VANAS DITON VANAS DITON Table 23: City Pairs Routes Then, the figure below reports the actions needed to ensure a correct flow of information from flight planning till OPS Room. A Free Solutions flight plan is submitted to NM: if the flight plan (processed manually) is accepted, no other information is required; if the flight plan is not accepted, the flight plan is submitted again to NM with the appropriate correction. If the Airline fill a flight plan not compliant with the FREE Solutions FPL, or if the flight is cancelled, the AU Point of Contact informs the national ANSP Point of Contact about the flight trial cancellation (via and phone call), not later than 1 hour before the EOBT Figure 16: Flight Planning Process Operational Staff Briefings Based on the already mentioned guidelines document for live trials execution [2] which was tailored on the needs of the different actors involved in the trial execution (e.g. ATCO, Pilots, NM, Dispatcher), a set of briefings with operational (ground and airborne sides) was prepared and executed to provide advices for a correct execution of the trials. Actually, the following activities were undertaken: Leaflet preparation for ATCOs involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required Leaflet preparation for Pilots involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required Leaflet preparation for Flow Managers Positions involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required 62 of 213

63 Advanced sharing of ATCOs feedback questionnaires Advanced sharing of Pilots feedback questionnaires Liaison with the identified Point of contact for each ACC involved in the flight trials for setting up the relevant operational scenario. Liaison with the identified Point of contact for each airline involved in the flight trials for setting up the relevant operational scenario Questionnaires The activities performed in the context of WP4 to produce the foreseen HP and SAF cases, encompassed a brainstorming session with operational experts (ground and airborne sides) to identify, discuss and review a set of specific questions to be handled to flight crew, controllers and flight dispatcher involved during trials activities (answers to these questionnaires may be found in the following sections of this report). The outcome of this preparation process was a list of post trials questions, split between air and ground sides, which have been used to collect operational expert s feedback. The table below shows the relations between the safety related questions proposed to the operational people (i.e. ATCO and Pilots) and the hazards. Questionnaire Target Question ID Question description Hazard ID reference ANSP 4.1 ANSP 4.2 ANSP 4.3 ANSP 4.4 ANSP 4.5 ANSP 4.6 ANSP 4.7 ANSP 4.8 ANSP 4.9 ANSP 4.10 I was able to safely manage inconsistencies between FPL and route/trajectory flown by the trial flight I was able to safely manage technical issues related to HMI/OLDI/FDP I was able to safely manage the interactions between trial flights and standard flights I was able to safely manage deviations from the assigned routes due to fuel issues/active military areas/adverse meteorological conditions Have contingency procedures (if any) been adequate to cope with an unexpected event? Did you notice any change in the amount of ATC communications compared to routine operations? In case of increased coordination/communication related to flight trials activity, has the safety been guaranteed? Did you manage conflicts (potential or not) due to the new entry/exit points introduced by City Pairs routes? Did you manage conflicts (potential or not) within the FIR/sector boundaries? Did you manage conflicts (potential or not) between trial flights and active military areas? HZ-01 HZ-02 HZ-05 HZ-10 HZ-14 HZ-11 HZ-12 4 HZ-12 HZ-06 HZ-07 HZ-10 4 HZ-04 and HZ-12 have been verified by analyzing the results of the HP sections of the questionnaire related to Situational Awareness and Workload. 63 of 213

64 Questionnaire Target Question ID Question description Hazard ID reference AU 1.1 Operational - Have you been able to fly your planned FREE Solutions Route? HZ-03 AU 1.6 Operational - Have you been able to perform routine fuel checks without any problem? HZ-08 AU 1.7 Operational - Did you notice any change in the amount of ATC communications compared to routine operations? HZ-09 HZ-13 AU 1.8 Operational - Final remaining fuel has been final reserve fuel? HZ-08 AU 1.9 AU 2.1 AU 2.2 AU 2.3 AU 2.4 Operational - Were all waypoints of your planned ATS route stored in your FMS Navigational database? Safety - Have Free Solutions ad-hoc contingency procedures (if any) been adequate to cope with any unexpected events during the flight? Safety - Did you perceive that the ATCO s in contact during the flight were fully aware of the operational scenario of the FS flight trials? Safety - Did you deviate from the planned ATS route on ATC request or due to adverse meteorological conditions? Safety - If you answered YES to question 2.3, did you feel that safety was ever compromised due to such deviations? HZ-09 HZ-08 HZ-09 HZ-09 HZ-15 HZ-09 HZ-15 AU 3.1 Workload - Did you notice any differences in your workload levels? HZ-13 AU 3.4 AU 4.1 AU 4.2 AU 4.3 AU 4.4 Workload - If you answered YES to question 3.1, did you feel that, due to increased/decreased workload levels, safety was ever compromised? Working methods - Were you required to alter your routine working methods in order to fulfil your duties? Working methods - If you answered YES to question 4.1, was Free Solutions operational information, provided before the flight, exhaustive with regards to roles and responsibilities, working methods and operational requirements? Working methods - If you answered YES to question 4.1, did you feel that, due to alteration of working methods, safety was ever compromised? Working methods - Was the information provided before the flight trial sufficient to safely perform the flight? HZ-13 HZ-13 HZ-13 HZ-13 HZ Table 24: Safety related questions 64 of 213

65 Project Number LSD Edition Exercise execution The exercise was executed in the following weekends 14/03/ /03/ /03/ /03/ /03/2016 starting from 00:00 UTC of the Saturdays end ending at midnight of the Sundays Deviation from the planned activities It was decided to limit the exercise time window to Saturdays and Sundays, not taking into account the very first hours of the Mondays. In the Demonstration Plan, a duration up to 07:00 AM of the Mondays was reported. This deviation was made to limit operational (ground and airborne) commitment for trials activities, but mainly because of few potential flights interested in city pairs optimizations. Few flights (19) chose other routings, for reasons not relevant to the exercise (more favourable wind or lower total cost) Some technical issues were underlined: o o o Total cost of filed routing greater than total cost of company route Time issue (like time constraints) Meteorological issue : Not favourable wind conditions along the planned route Exercise Results Summary of Exercise Results Results per KPA This section details the results obtained by applying the methodologies described in the section of this document for each addressed KPA Environment - Fuel Efficiency The results collected and analysed for this KPA contributes to the satisfaction of the following demonstration objective OBJ-LSD , which has two dedicated success criteria environmentfuel efficiency and predictability respectively. OBJ-LSD To demonstrate the advantages of using new routes/corridors on most penalized city-pairs in terms of efficiency/environment sustainability Success Criterion Environment Sustainability and Fuel Efficiency : Reduction on fuel burn, CO2 and Nox. Reduction in time and distance flown Results Fully achieved Table 25: Success Criteria Environment-Fuel Efficiency and Predictability OBJ-LSD Efficiency can be measured through 8 different indicators (i.e. Length saved, Time saved, Fuel burned saved, CO2 saved, NOx saved, Eligibility flights planned, Acceptance - flights flown, Eligibility and acceptance flights planned against flights flown). The table hereafter summarizes the results observed and the key performance indicators used to obtain them. KPA KPI Metric Results 65 of 213

66 Environmental Sustainability and Fuel Efficiency Time saved Delta Minutes R(ref) v R(scn) -224mins CPF(ref) v CPF(scn)-81mins Distance saved Delta Nautical Miles R(ref) v R(scn) -1813nm CPF(ref) v CPF(scn)-391nm Eligibility flights planned to conduct the trial solution Acceptance flights that have flight planned the trial solution Eligibility and acceptance flights planned for trial solution against flights flight planned for trial solution Fuel burn saved enroute 5 Number of flights Number of flights Delta Number of flights Delta Kg R(ref) v R(scn)-8019kg CPF(ref) v CPF(scn)-3823 CO2 saved enroute Delta Kg R(ref) v R(scn)-25339kg CPF(ref) v CPF(scn) NOx saved enroute Delta Kg R(ref) v R(scn)-81kg CPF(ref) v CPF(scn)-53kg Table 26: Environment-Fuel Efficiency Results - Metrics and Measures Weekend City Pair For any further details on the assessment, data measurement and results refer to the detailed analysis reported in Error! Reference source not found Airspace Capacity - En-Route The results collected and analysed for this KPA contributes to the satisfaction of the following demonstration objective OBJ-LSD , which has dedicated success criterion for airspace capacity. The subsequent table summarizes the results observed and the key performance indicators used to obtain them. OBJ-LSD To demonstrate the advantages of using new routes/corridors on most penalized city flights not considered in fuel burn, CO2 and NOx calculation as aircraft types were not the same in ref and scn samples 66 of 213

67 pairs in terms of efficiency/environment sustainability Success Criterion Results Capacity: no negative local and network Partially achieved, based on volume of impact trials traffic. No negative effects Table 27: Success Criteria - Capacity OBJ-LSD Capacity can be measured through 5 different indicators (i.e. ATFCM delay, Sector occupancy, ACC throughput, sector workload, conflicts). Nevertheless, observed changes during trials could not demonstrate the advantages of using new routes/corridors on most penalized city-pairs in terms of local and network impacts, as it has been summarized in the table hereafter. KPA KPI Metric Results Airspace Capacity (enroute) ATFCM delay Minutes None attributed to trial city pair DCTs No negative effects Sector occupancy ACC throughput Number of aircraft per hour Number of aircraft per hour Change not significant enough to measure quantitatively Change not significant enough to measure quantitatively Sector workload Delta % points Change not significant enough to measure quantitatively Conflicts Number of potential conflicts per sector Change not significant enough to measure quantitatively Table 28: Capacity Results - Metrics and Measures Weekend City Pair Qualitative results regarding controller workload in relation to capacity taken from Skyguide and ENAV controller questionnaires are reported in the sections and For any further details on the assessment, data measurement and results refer to the detailed analysis reported in Error! Reference source not found Predictability and ATC planning: OBJ-LSD To demonstrate the advantages of using new routes/corridors on most penalized city-pairs in terms of efficiency/environment sustainability Success Criterion Predictability: Reduction in the variation of the distribution of actual enroute time duration vs. planned enroute time duration. Results Not achieved. An expected reduction of the delta time between reference and scenario examples, indicating a more predictable scenario, was not demonstrated in the data analysed. This may be due to wind or other influencing factors. To be reviewed in Phase 2 Table 29: Success Criteria - Predictability OBJ-LSD KPA KPI Metric Results 67 of 213

68 Predictability Actual en-route time duration vs. planned en-route time duration Delta Minutes Ref v Scn An expected reduction of the delta time between reference and scenario examples, indicating a more predictable scenario, was not demonstrated in the data analyzed. This may be due to wind or other influencing factors Table 30: Predictability and ATC Planning Results - Metrics and Measures Weekend City Pair For any further details on the assessment, data measurement and results refer to the detailed analysis reported in Error! Reference source not found Safety This section reports the main results of the safety analysis conducted for the FREE Solutions City Pairs operational scenarios. Full analysis is reported as separate document attached to this project deliverable [4]. Safety was evaluated through (i) residual fuel at arrival and (ii) scores to questionnaires. In particular, post-trial questionnaires, delivered to and compiled by a remarkable pool of ATCOs and Pilots (171 ATCOs and 117 Pilots), were structured to validate identified hazard and to capture whether they had an impact on the Safety of the operations or not. In the next two sub-sections the statistical results of the questionnaires are provided. They were organized discriminating the scope of the analysis ( Ground side and Air side ) and were presented to guarantee the traceability between the hazards and the related questionnaires answers in order to demonstrate the arguments of the Safety Case ANSPs analysis The Ground side questionnaire was structured in five sections, one of which was dedicated to assess the safety issues arisen during the trials. Some of the hazards were handled by referring also to the Workload and Situational Awareness sections of the questionnaire model since some aspects of the analysis are shared between Safety and Human Performance boundaries. This section presents the results according to the GSN Arguments structure in order to demonstrate that every aspect of the Safety assessment has been considered pointing out the connections between the arguments, the hazards and the dedicated questions of the questionnaire. ARG was not treated in the analysis as it was considered only to verify the acceptability of the preliminary hazards identified in the first version of the safety assessment. ARG Equipment affected by the change has been considered The purpose of this argument is to demonstrate that the potential failures or issues regarding the Equipment affected in organizing and executing the flight trials have been considered. 68 of 213

69 Figure 17: ARG1.2.2 ANSP Equipment The results presented in Figure 17 show that both FPL inconsistencies and technical issues were safely managed by the controllers during the trials (respectively 73% and 81% of positive answers). It is also necessary to underline that the N/A (Not Applicable) answers are not analysed unless they represent a significant percentage of the total answers (i.e. >40%). Despite the fact that, they could be reported as not happening of a specific event reported in the question, in most of the cases. Moreover, the few negative answers provided by the ATCOs clearly demonstrated that no technical issues were experienced during the trials execution (For both questions 4.1 and 4.2 just 1% of No answers have been collected). However, the few negative answers can be interpreted as triggered by the will of the operational staff to protect the trial traffic from interfering traffic as suggest during pre-trials briefing and provided guidelines. In some cases some route which required manual intervention from controllers where no longer tested during DR trials (e.g. TOP BETEN ELKAP).ARG Airspace configuration has been considered The ARG aim is to demonstrate that the potential issues related to the airspace configuration were considered. Therefore, the focus is put on situational awareness and on the level of ATCO workload during the trials execution. The results of the analysis related to the above mentioned areas are reported in the dedicated HP section [ ]. Nevertheless, it can be assumed that no particular differences arisen with respect to the baseline operations regarding the workload and situational awareness of the ATCOs participating to the trials. ARG Operational and flight procedures have been considered ARG was structured to demonstrate that the flight and operational procedures to be used during the flight trials are adequate and not demanding by a safety perspective. Six dedicated questions have been submitted to the ATCOs, whose results and their analysis are reported hereafter: 69 of 213

70 Figure 18: ARG ANSP - Operational and flight procedures Before browsing the analysis results it would be helpful to highlight that the rationale behind the group of questions 4.3, 4.4 and 4.5 is different from the one of the group composed by questions 4.8, 4.9 and The first three questions suggest that a yes answer has to be considered as auspicious from a safety perspective since it reminds to a positive impact on the Safety of the operations. On the other hand questions 4.8, 4.9 and 4.10 imply a negative impact on Safety in case of yes answer. The answers related to the question 4.3 and 4.4 highlights that no negative impact was experienced by the controllers in charge in managing both interactions between trial and standard flights and any kind of deviations from the assigned route. Question 4.5 induces some bias and uncertainty on its results since the most of the answers considered the contingency procedures as a topic not to be applicable in the scope of the city pairs flight trials and, additionally, there are no justification collected referring to this item. Furthermore, even if the negative questions amount to 11% of the total pool of answers, a correlation analysis with the other questions was made and has highlighted that all the ATCOs who provided a no answer did not report any issue on the safety of the operations. Therefore it is reasonable to affirm that this topic doesn t provide the analysis with any relevant result and hence special attention should be dedicated during the next steps of the project to obtain a better formulation of the question in order to guarantee a sound contribution to the analysis. Questions 4.8, 4.9 and 4.10 aim at identifying if the controllers experienced no issues in managing potential conflicts. The results of the questions suggest that in almost all cases no particular issues was experienced by the controllers. A remark has to be risen up considering the N/A answers provided for the question 4.10 (interactions with active military areas). The consistent number of N/A are due to the minor percentage of flights that have flown week days trials or to the traffic managed during the working shift which did not was part of consortium traffic, but interacted with military areas. On the other side the positive answers provided for the above mentioned questions are not negligible even if the analysis of the related comments provided by the controllers shows that the conflicts were, in the most cases, potential rather than effective ones. Thus, it is reasonable to affirm that the ATCOs were always in control and confident with the new concept implementation as they were informed about the potential issues prior to the trials execution. 70 of 213

71 Project Number LSD Edition The analysis of the justifications provided highlighted that the ATCOs had not perceived any particular issue regarding the traffic management due to the low level of traffic managed during the trials. ARG People related issues have been considered ARG was pointed out to determine the impact on the human perspective of the operational concept introduced by the City Pairs flight trials execution. The questionnaire sections under analysis are related to situational awareness and to the level of ATCO workload during the trials execution. Furthermore, beside the HP results, two dedicated questions (4.6 and 4.7) were foreseen in the safety section of the questionnaire to evaluate the impact of the amount of communications on the operations with respect to the ones normally experienced in daily operations. The statistical analysis results of the proposed questions are reported below: Figure 19: ARG ANSP 1/ Figure 20: ARG ANSP 2/2 By analysing the results of question 4.6 it is possible to state that in most cases no changes in the communications required were experienced by the controllers. Anyway, even if the 19% of the answers reported an increase in communications, the related results of question 4.7 show that safety was not affected by the change. 71 of 213

72 Project Number LSD Edition The results of the analysis related to the workload and situational awareness are reported in the dedicated HP section [ ], nevertheless it can be assumed that no particular differences arisen with respect to the baseline operations. ARG Environmental and meteorological conditions were considered ARG was identified to consider the impact of the environmental and meteorological conditions on the operations. Analysis focused on determination of how adverse meteorological conditions could have affected the trials by inducing involved aircrafts involved in deviating from the assigned route. As already mentioned during the analysis of the ARG 1.2.4, the evaluation of the question 4.4 confirmed that any safety issue was experienced in managing deviations from the assigned route and therefore it is possible to declare that adverse meteorological conditions did not affect the trials Airspace Users analysis The Air side questionnaire was structured to validate the results of the Safety Assessment analysis and is made of four sections. Section 4 covers the safety analysis even if there is a set of transversal questions belonging to the other sections that are also usable for the safety analysis purposes. This sub-section presents the results according to the GSN Arguments structure in order to demonstrate that every aspect of the Safety assessment was considered pointing out the connections between the arguments, the hazards and the dedicated questions of the questionnaire. ARG results are not reported as it was only considered to verify the acceptability of the preliminary hazards identified in the first version of the safety assessment. The same applies to ARG Airspace Configuration since this topic hasn t been considered as affecting the operations by AUs point of view. ARG Equipment affected by the change has been considered The argument was proposed to demonstrate that potential failure affecting the equipment during Airspace Users operations Figure 21: ARG AU - Equipment The results clearly show that no issues were experienced by the pilots due to Equipment-related issues (respectively 87% and 97% of the answers were positive by a safety perspective). Anyway it should be noted that question 1.1 was proposed to the pilots firstly to understand if none issues were in place during the trials, but focusing on the operations rather than on the equipment. It is then possible to assume that since most pilots were able to fly their planned route, it is clear that no issues linked to the flight planning occurred. 72 of 213

73 Project Number LSD Edition On the contrary even if 10% of the pilots reported that they were not able to fly their FREE Solutions Route there isn t the possibility to determine the linked causal factors since no specific justifications were provided to the answers. ARG Operational and flight procedures have been considered ARG was structured to demonstrate that the flight and operational procedures to be used during the flight trials are adequate and not demanding by a safety perspective Figure 22: ARG AU - Operational and flight procedures (1/2) Figure 23: ARG AU - Operational and flight procedures (2/2) Taking in account the results shown in the charts it is clear that no particular issue was experienced by pilots. They were comfortable with the trials execution and, even if they have experienced a deviation from the assigned route, the Safety of the operations was never compromised. 73 of 213

74 Project Number LSD Edition Fuel checks operations required during the flights have been highlighted by a restricted group of pilots since some issues took place while performing some long directs and this topic has been already kept as lesson learned for the preparation of the second bunch of the trials. Furthermore communications with the ground were not affected by the trials execution. The answers related to the proposed questions display that no particular change was experienced compared to daily operations and, as a consequence, it is possible to confirm that there was no degradation of the Safety level of the operations. ARG People related issues have been considered ARG was investigated to address the impact on human perspective of the operational concept introduced by the City Pairs flight trials execution. The rationale behind the analysis of the hazard by the pilot perspective is analogous to the one used in the Ground analysis. The results related to this argument are complemented by the HP analysis related to the Workload and Working Methods sections. Anyway a set of questions were selected to conduct the analysis by a Safety perspective and the results are shown below: Figure 24: ARG AU - People related issues Those who answered the workload section of the questionnaire pointing out an increase of the workload were also asked to determine if, in their opinion, the level of Safety was ever compromised. The results are proposed below: Figure 25: ARG AU - Workload 74 of 213

75 Project Number LSD Edition Hereafter the results related to the Working methods section of the questionnaires are presented: Figure 26: ARG AU - Working methods Almost 30% of pilots interviewed reported a decrease of workload and just 4% declared an increase. All the pilots who experienced a change in the workload level with respect to the baseline operations affirmed that the Safety of the operations was not affected at all. It can be concluded that, pilots workload was not affected by trials execution. Similarly the working methods related answers show that pilots did not perceive any change compared to daily operations and those who did not report that confirmed, anyhow, that no safety issues were in place during the trials. Pilots interviews ended with a general consideration about provided pre-trials information, which were assessed to be accurate, complete and timely. They confirmed to be well informed about trials activities and expected changes to the working methods, leading to a positive impact on the Safety of the operations. ARG Environmental and meteorological conditions have been considered ARG was foreseen to consider the impact of the environmental and meteorological conditions on the operations. Analysis focused on determination of how adverse meteorological conditions could have affected the trials by inducing involved aircrafts involved in deviating from the assigned route. A dedicated question was foreseen in the questionnaire to analyse the deviations experienced during the trials executions (questions 2.3 and 2.4) and, as already mentioned, its evaluation confirmed that no safety issue was experienced Safety Summary of Results OBJ-LSD To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably Safe Success Criterion Every negative effect on the operations shall be reduced as far as reasonably practicable Results Fully achieved no negative effects on operations, according to the argument analysis showed in the 75 of 213

76 The Safety of the operations shall remain at or better than its current level of Safety All the critical issues related to the operations management by all the stakeholders involved will be analysed All the evidences requested by NSA will be provided previous sections Fully achieved - according to the argument analysis showed in the previous sections Fully achieved - according to the argument analysis showed in the previous sections Fully achieved evidence provided to NSA and no issue reported Table 31: Success Criteria Safety OBJ-LSD KPA KPI Metric Results Safety Residual Fuel at arrival Tons It was decided to not measure and replace this measurement with ad-hoc questions for pilots Table 32: Safety Results - Metrics and Measures City Pair Weekend Additionally, during demonstration planning stage a quantitative measurement was envisaged. The aim was to measure the overall amount (tons) of residual fuel at arrival. However, during live trials preparatory activities it was agreed with involved AUs to replace this measurement with ad-hoc questions to be inserted in the post-trial questionnaire to pilots (i.e Have you been able to perform routine fuel checks without any problem? Final remaining fuel has been final reserve fuel? This decision better matched with existing AUs policies about fuel data external communications, as well as represented a more rapid and practical way to get the same piece of information HP Results Results reported here are derived from the brainstorming session executed during the preparatory phase Error! Reference source not found. and the actual execution of live trials. They are organised and proposed with respect to the two strands of analysis, groundside and airside, as shown in the following sections ANSPs Analysis During the trials execution a customized questionnaire was administered within the sectors of ACC units directly affected by the demonstration campaign. Namely, the involved ACC units were: Milan, Padua, Rome for ENAV; Geneva for Skyguide. On the ground side the amount of trial flights allowed the collection of 171 questionnaires spread out among the involved ANSPs. The table hereafter provides details on the questionnaires, i.e. amount of formats collected per each ANSP and distribution with respect to the roles (Executive/Planner controllers). ANSPs Overall EC Formats PC Formats ENAV Skyguide Table 33: ANSPs Questionnaires - City Pairs 76 of 213

77 Project Number LSD Edition Data collected through questionnaires were manipulated and then converted into statistics. Manipulation consisted into the aggregation of ENAV and Skyguide scores and the subsequent organization according to the two roles, i.e. EC and PC. Conversion implied that answers to the different choices were expressed in percentage values, i.e. partial quotes were reported in reference to the overall amount. During screening of questionnaires and subsequent delivery of statistics an anomaly was observed. For some questions of filled-in templates, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve full score (100%) through the sum up of afferent intermediate quotes. As a final effect, a margin of uncertainty is recorded and induced subsequent effects, first on the analysis of data and secondly, on the reporting. This uncertainty can be classified as not severe because affected questions were not so many (i.e. 09 questions out of 32). Furthermore, percentages of uncertainty achieved tolerable levels (i.e. around 04% with only one case of 17% as a peak). Considering the fact that questionnaires were submitted at the end of trials and their replication was not possible, these uncertainties cannot be solved. The adopted mitigation was to make reader aware of this bias highlighting these cases during the analysis report. A proper disclaimer was adopted for those questions affected by the bias and the related percentage score of uncertainty was declared. Scores are reported in the following sections. Results are organized and then proposed according to the different Human Performance objectives linked to EXE-LSD Details regarding declared HP objectives can be found on the related document HP Assessment Plan [10]. Roles and Responsibilities Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP003 To demonstrate that controller role and responsibilities (EC, PC) are clear and consistent during city pairs optimization flight trials OBJ-LSD Table 34: Success Criteria Role and responsibilities - OBJ-LSD The purpose of this objective foresees the assessment of working methods and operational info/procedures in terms of i) any eventual change with respect to current situation and ii) trend of change (i.e. improvement or deterioration), if occurred. The collected scores are almost similar between EC and PC and no significant changes are shown in-between. Global results were then used in these analyses. Relevant aspects asked to controllers are presented in the next figure (figure 27) and concerned: Changing of coordination procedures: Results were mitigated. For 53% of ATCOs, there was no change, whereas it changed for 47% and for most of them, this change was translated by an increase in coordination procedures. - Changing of shared information with other team members: it did not change for 63% of ATCOs, even if for 34% of ATCOs, a rise in shared information was perceived. Increase in coordination and information exchanges recorded through questionnaires appeared acceptable basing on free comments provided by controllers. Some of them noticed an increase of tasks to inform other units/sectors about trials. Therefore, these results can be attributed to the trials and not really to the use of city pairs in everyday life. Accuracy of information related to procedures and working methods: almost all ATCOs answered positively (94%) to this topic, against only 2% who answered negatively (4% did not answered to this question). 77 of 213

78 Figure 27: Groundside HP objective HP003 In summary, collected scores - checked with feedbacks provided by ATCOs at the end of trials - confirm that roles and responsibilities for the ATCOs (both ECs and PCs) are still clear and consistent. Operating Methods and Procedures Identifier Objective Supportive Objective Exercise Changing of coordination procedures OBJ-LSD0105-HP007 To demonstrate that controller operating methods and procedures (EC, PC) are clear and consistent during city pairs optimization flight trials OBJ-LSD Table 35: Success Criteria Operating Methods and Procedures - OBJ-LSD Purpose of this objective is to complement evidence derived through a devoted brainstorming before the trials Error! Reference source not found. with additional more tangible proof collected after the execution of the exercise. Suitability of controllers operating methods and procedures represents other important markers for the HP assessment of EXE-01. These elements, as anticipated in the previous section, are complementary to roles & responsibilities topics. A particular behaviour of operating methods and procedures could influence the other ones and vice-versa. This explains why some questions already showed during the assessment of objective HP003 are here again analysed. This objective mainly implied to check the ability to allow flight trials to fly planned and agreed routes. Positive scores were recorded for this topic. Almost all ATCOs (96.5%) were able to allow flight trials to fly their FREE Solutions planned routes (only 1% answered no and 2.5% of ATCOs did not answered to this topic) Changing of shared information Ability to allow flight trials to fly the FREE Solutions agreed route Accuracy of information related to FREES procedures Figure 28: Groundside HP objective HP007 Increase Decrease In summary, achieved scores demonstrated that the objective was positively reached. Yes No N/A Yes No 78 of 213

79 Project Number LSD Edition Controller Performance Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP009 To demonstrate that controller (EC, PC) performance is appropriate during city pairs optimization flight trials OBJ-LSD Table 36: Success Criteria Controller Performance - OBJ-LSD Evaluate controller performance means the assessment of users perception with respect to different markers/dimensions that basically are: o Workload, o o o o o o Situational awareness, Tasks time execution, Communications and coordination, Trust in the concept, Job satisfaction in adopting the new operational concept, Potential errors induced by the introduction of the proposed concept Some of these markers are interconnected among them, e.g. tolerable level of workload and ATC communications make the acceptability of new concept smoother and then allow the achievement of positive results in terms of task execution, job satisfaction and then trust in the new concept. Scores collected through questionnaires were arranged together and reported according to the markers previously introduced. Workload Figures hereafter show controllers feedback regarding level of workload experienced during the trials. Results are divided by EC and PC. Figure 29 provides a general overview of mean workload for both EC and PC positions. The histogram clearly shows that on average the PC role was the one mainly influenced in terms of workload. Comments provided by controllers through free notes sustain this trend. According to them, the FREE solutions trials introduced new conflicts with which they were not familiar, inducing this way additional effort to PC for their detection and mitigation. Furthermore the current system seemed to controllers not completely able to support PC activities due to the devoted effort request to the manual insert of FREE solutions routes. PC was in general asked for additional work in terms of more coordination procedures and communications with the other unit/sector to inform about trial. 79 of 213

80 Figure 29: Groundside HP objective HP009 Workload Overall In order to collect information about workload as objective as possible, attention was posed to the investigation of 4 dimensions: Mental demand, i.e. supplementary cognition effort devoted to the execution of FREE Solutions trial; Physical demand, i.e. extra actions/tasks like more speeches or more interactions through HMI; Frustration, i.e. potential sense of irritation/stress/annoyance; Changes in the amount of ATC communications. Figure 30 shows the mean score value for the (a) perception of additional mental workload, (b) extra actions undertaken, (c) feelings. Within a scale from 1 to 10, ATCOs had to evaluate the elements making up workload with respect to baseline conditions, where 1 means less demanding, 5 means no change and 10 means more demanding Additional mental workload Extra actions undertaken Feelings Figure 30: Groundside HP objective HP009 - Workload a. Additional mental workload caused by flight trials, i.e. supplementary cognition effort devoted to the execution of FREE Solutions trials. EC PC 80 of 213

81 Project Number LSD Edition According to Figure 30, a very limited increase of mean value was recorded for PC and EC in case of mental demand. Achieved mean score was equal to 6 for both of them. Most probably current tasks like monitoring in/out traffic, update flight plans, identify potential new conflicts due to the alternative FREE Solutions route influenced the levels of these two workload dimensions. b. Extra actions/tasks undertaken during the flight trials session, i.e. more communication (speeches) or more interactions through HMI; According to Figure 30, a single difference appeared between EC and PC: a slight increase was experienced by PCs for extra actions undertaken during the flight trials session. The current system seemed to controllers not completely able to support PC activities due to the devoted effort request to manually insert FREE solutions routes. PC was in general asked for additional work in terms of more coordination procedures and communications with the other unit/sector to inform about trial. c. Feelings of insecurity, discouragement, irritation, stress, annoyance. According to Figure 30, no change with respect to baseline conditions was noticed in case of feelings (mean levels equal to 5). In order to collect information about workload as objective as possible, the three dimensions previously assessed (i.e. mental demand, physical demand and frustration) were enriched through the assessment of ATC communication load. Figure 31 provides collected scores (in percentage) in terms of changes of ATC communications as were experienced by EC and PC roles, respectively. In both cases, most controllers declared communications unchanged. Interesting is the increase/decrease trends that were exactly opposite for EC and PC. The EC, directly affected by communications, perceived a major increase of communications with respect to a decrease (i.e. 23% and 11%, respectively). PC, not directly involved in the ATC frequency occupancy, experienced an opposite trend with a slight decrease of calls with respect to an increase (i.e. 15% of decrease and 10% of increase) EC 66 Figure 31: Groundside HP objective HP009 Communications Changes In general, scores gave back results in terms of workload that are quite aligned with initial feedbacks derived through brainstorming Error! Reference source not found.:controllers hypothesized a slight increase of workload due to extra effort devoted to tasks like monitoring, analysis of traffic situation (due to new routes) and update of flight plans. Direct feedbacks from questionnaires confirmed this hypothesis only regarding the mental demand dimension. With reference to physical demand and frustration, mean scores appeared unchanged (compared to current operations) and proper levels of performance were still assured. Achieved results allow inferring that in general the level of workload was still maintained at proper level without negative effects on the overall performance PC 75 Increase Decrease No change 81 of 213

82 Project Number LSD Edition Situational Awareness Situational awareness is the continuous extraction of environmental information, the integration of this information with previous knowledge to form a coherent mental picture and the use of that picture in directing further perception and anticipating future events. The figure below (Figure 32) provides the comparison between EC and PC positions with respect to the overall mean value of situational awareness Figure 32: Groundside HP objective HP009 Situational Awareness Overall Histogram clearly shows that - on average the PC role achieved lower level of situational awareness. Potential causes behind this trend were already provided in the previous paragraph. The rationale behind this score was assessed in a devoted set of questions that were proposed to controllers, investigating: ability to anticipate traffic behaviour; Not focusing on problem/area of the screen; No potential risk to forget important info; Ability to plan and organise activities according to own preference; Unwanted surprise due to unexpected event; No loss of time spent to search for info on the screen. For all questions, the rating scale was diversified from 0 (i.e. poor level of situational awareness) to 6 (i.e. high level of situational awareness). 82 of 213

83 Always Almost always Often Neutral Rarely Almost never EC PC Never Figure 33: Groundside HP objective HP009 Situational Awareness In both cases collected scores were more than positive. Mean figures were greater than the neutral pole by one or two units, for all the topics. Better results were achieved for the EC position where mean values overtook neutral value by two units for most of the topics. The rationale behind this trend could be found in the free notes provided by controllers. In some cases PC had to pose special attention in anticipating and solving new conflict points triggered by FREE solutions routes. Furthermore, the novelty of trials (route segment and its lack of tracks on the radar screen, need to manual insert FPLs sometimes, need of further clarifications among ATCOs, ATCOs/pilots, etc.) potentially grabbed the PC attention and could have influenced his/her perceived level of situational awareness. Task Time Execution Another marker that could influence performance is the time/effort devoted to the execution of tasks. Figure hereafter shows achieved scores with respect to four topics deemed able to influence the execution of activities, i.e. traffic complexity (if changed and with which trend); sector capacity (if changed and with which trend); conflict detection (if changed and with which trend); conflict resolution (if changed and with which trend). Unfortunately data recorded for this topic were not completely reliable. As anticipated in section , for some questions of filled-in templates scores were not marked by all controllers thus leaving relative cells empty. Three questions linked to task time execution were not signed-in by all controllers and the full score of 100% - through the sum up of intermediate quotes was not achieved bringing as a final effect a margin of uncertainty. This uncertainty can be observed for questions interested to: 0 Anticipation Not focused on problem sector capacity (06% of uncertainty for EC, 07% for PC), conflict detection (08% of uncertainty for EC, 06% for PC), No risk to forget Planification Not surprised No search for information conflict resolution (04% of uncertainty for both EC and PC). As no significant differences were observed between EC and PC s scores, they were analysed without discriminating roles. 83 of 213

84 Increase Decrease No N/A Figure 34: Groundside HP objective HP009 Task Time Execution Histogram provide positive results meaning that for most controllers the new FREE Solutions routes did not affect the time devoted to the execution of tasks. No changes were declared for all questions. The only peaks in terms of alteration of current tasks are observed with reference to change of traffic complexity and conflict detection. They are moderate, still below the midpoint. According to controllers notes, the FREE Solutions routes introduced changes in terms of tracks, then in terms of conflicts location and detection. These comments justify in a certain way the peaks of changes registered for the topic task time execution. Communications and Coordination Three ad-hoc questions were posed to controllers for the assessment of this topic, i.e. 0 Traffic complexity Sector capacity Changes in the amount of ATC communications (the same already treated for the topic workload ); Changes in the coordination procedures; Conflict detection Conflict resolution Changes in terms of information exchanged with the other team member. As no significant differences were observed in the coordination procedures and the amount of shared information between EC and PC s scores, scores were analysed without discriminating roles Increased Decreased No Coordination procedures Shared information N/A Figure 35: Groundside HP objective HP009 Communications & coordinations 84 of 213

85 Project Number LSD Edition For both roles, the management of FREE Solutions trials did not influence the current level of communications (i.e. load of ATC exchanges) and the coordination task (i.e. way to execute coordination and contents). More than 50% of the interviewed controllers did not experience changes. They managed trial flights adopting the same coordination procedures currently used. Moderate peaks - in terms of growth - were recorded for coordination procedures and information exchanged with other team members. Rationale behind these trends can be found on free comments provided by some controllers. They referred that due to the novelty of activity and the trials, they experienced a slight increase of coordination with the other sector/unit to inform about concerned trial or to clarify some information related to it. Trust in the Concept Another marker able to influence the general controller performance is the trust in the new concept/route introduced through FREE Solutions. Relevant aspects investigated during trials were: Usefulness of new routes; Reliability of new routes; Controller confidence while working with new procedures. The subsequent figure shows overall mean scores organised according to EC and PC positions. Always Almost always Often Neutral Rarely Almost never Never Usefulness Reliability Confidence EC PC Figure 36: Groundside HP objective HP009 Trust The level of confidence is slightly increased with respect to the neutral pole making ATCOs (EC and PC) often confident with new procedures related to the trials. For both EC and PC, results gave back an averaged trend quitter close to baseline conditions in case of usefulness. Similar results are observed for the reliability, in case of EC. While for the PC, the level of reliability slightly increased with respect to baseline conditions. In general, the level of trust in the new concept did not distance itself from the one already present in baseline operations (i.e. routes and related procedures). This result allows inferring that experienced trails and afferent Optimization City Pairs concept can be introduced into real operations in a smooth way. Results achieved in terms of job satisfaction and potential errors reported in the following sections provide further evidence to this statement. Job Satisfaction It addresses the human well-being, the cognitive comfort in executing the prescribed work. During trials this indicator was investigated at high level posing indirect questions, i.e.: Acceptability of the FREE Solutions agreed route considering the balance between gain on route length and workload; 85 of 213

86 Project Number LSD Edition Ability to allow flight trials to fly the FREE Solutions agreed route Figure 37: Groundside HP objective HP009 Operative Acceptability Figure 37 shows that for both EC and PC positions, very positive results of operative acceptability were registered. More than 70% of interviewees declared the proposed trials acceptable at operational level because they provided a gain greater than the effort devoted for the management. Furthermore nearly 100% of controllers experienced actual feasibility to let concerned aircraft fly agreed FREE Solutions route. Effectiveness and operative efficiency of proposed new routes can positively influence controllers performances. Good level of satisfaction is supposed to positively influence users acceptability and then transition of trials from experiment to real execution during daily operations. For the sake of completeness, some controllers triggered doubts regarding acceptability in case of high traffic density, in case of crossing significant traffic flows and - in general - the domino effects on surrounding traffic (i.e. delay manoeuvre/heading to separate aircraft). Positive results to these questions allow inferring that controllers experienced satisfaction (at operative level) in managing the trials and then in executing their job. Potential Errors The purpose of questionnaires was also to assess that new routes did not induce errors in executing operations with dangerous side effects on controller performances and then on safety level of operations. Next figures collect controllers scores referred to safety level of operations. Questions proposed to them were related to: Acceptability 5.3 risk to forget information (already treated in the section on situational awareness); reliability of proposed route (already treated in the section on trust in concept); interactions among mixed traffic (FREE Solutions and standard one); effects of extra coordination/communications on traffic management; Accuracy level of information related to procedures and working methods. Figure 38 shows that (a) coexistence of mixed aircraft did not induce errors in the management of traffic for both controllers and that (b) information related to procedures and working methods for trials management was deemed very accurate for most of interviewees Ability 2.3 Yes No N/A 86 of 213

87 Interactions among mixed traffic Accuracy level of information Figure 38: Groundside HP objective HP009 Potential Errors Question risk to forget important information is the same already introduced in the paragraph devoted to the situational awareness. It is part of the set of those questions that shall be considered in an opposite way due the nature of adopted questionnaire and the data entry applied during the analysis. This question actually has to be re-interpreted no risk to forget something important. In case of both EC and PC, a high score was registered (i.e. 5) that - during the reading of histogram - means that controllers did not feel - almost always - to forget something important. As for reliability, no significant values were recorded. Scores were neutral or slightly more, for both roles. Yes No N/A Figure 39: Groundside HP objective HP009 EC Potential Errors (a) Finally, the question referred to safety effects of extra coordination/communications on traffic management did not collect marked positions because revealed to be not applicable, according to experience trials (54% for EC and 51% for PC respectively). In general, controllers did not report cases in which trials induced EC and/or PC in performing incorrect actions or actual errors. Changes to Tasks and Roles Identifier OBJ-LSD0105-HP013 Objective To demonstrate that changes to controller tasks and role (EC, PC) are managed during city pairs optimization flight trials Supportive Exercise OBJ-LSD Objective Table 37: Success Criteria - Changes to Tasks and roles OBJ-LSD of 213

88 Project Number LSD Edition This objective is focussed on the assessment of potential changes to controllers activities and roles/responsibilities due the execution of FREE Solutions trials, additionally to the detection and mitigation of potential errors. It implies the verification of those elements, declared by controllers during brainstorming Error! Reference source not found., potentially affected by a change, i.e.: monitoring and analysis of traffic situation; flexibility to manage mixed traffic within the sector (i.e. indirectly means sector capacity); detection of potential conflicts and their resolutions; coordination and info exchanges; safety level of operations. The above elements were translated into devoted questions of questionnaire. Collected scores are reported hereafter, according to the two roles. As already noticed in the previous section (i.e. task time execution, communications and coordination, potential errors), ECs did not perceive any significant changes with respect to baseline situation for the different arguments. This tendency is clearly observable regarding variation of sector capacity and resolution of conflict. In practice, the amount of aircraft managed in the sector was the same as a normal week-end day. Techniques applied nowadays for the management of conflicts and their resolution were still applied by EC (i.e. radar vector, heading, FL change). The only concern highlighted by controllers through free comments was related to the fear of losing this quality. They were afraid of being not still able to apply current flexible solutions for conflicts resolution due to the fact that aircraft are already flying the optimum track and any eventual deviation can have negative impact on the aircraft performance. Controllers stressed the fact that for safety purposes, current conflicts resolution must be still available and applicable. Results referred to safety confirm that during trials the safety level of operations was still assured. Even the management of mixed traffic ( flying along ATS route network and Free routing) was assured and their mutual interaction was positively supervised (positive feedback from 91% of EC). In case of PC, the fact that no significant changes occurred is more visible. For all elements, i.e. from traffic complexity going through conflict detection and resolution, tasks were deemed equal to the current ones. Only in case of coordination procedures, 42% of controllers experienced a change in terms of increase. Comments given by controllers provide reasonable rationale of this trend. Some of them experienced a slight increase of coordination with other sector/unit to inform about concerned trial or to clarify some information related to it. Technical limitations of current ground systems intensified this trend. In general, recorded scores showed that no significant changes occurred at ground side. Basically, a stability of current roles and related tasks is underlined AUs Analysis During the execution phase a customized format was administered to the several flight crews involved in the trials. The 5-day experimental campaign (i.e. 14/15 & 21/22 March 2015, 10 April 2015) allowed the execution of 124 reliable flights. Theoretically it would have been implied the submission of the format to both Pilot Flying (PF) and Pilot Monitoring (PM) per each flight and the subsequent collection of 248 questionnaires. Actually, 117 questionnaires were returned and they included in an aggregated way PF and PM feedback. Questionnaires were filled-in by pilots coming from four AUs of the Consortium, i.e. AZA, DLH, RYR and SWISS. Data of questionnaires were then manipulated expressing collected scores into percentage values, i.e. share of replies to the different choices was reported in percentage with respect to the overall value of collected data. Scores are reported in the following sub-sections. Results are organized and then proposed according to the different Human Performance objectives linked to EXE-LSD of 213

89 Project Number LSD Edition Details regarding declared HP objectives can be found on the devoted document HP Assessment Plan [10]. Roles and Responsibilities Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP004 To demonstrate that pilot role and responsibilities (PF, PM) are clear and consistent during city pairs optimization flight trials OBJ-LSD Table 38: Success Criteria - Roles and responsibilities OBJ-LSD The coverage of this objective foresees the assessment of working methods and operational information/procedures in terms of i) any eventual change with respect to current situation and ii) trend of change (i.e. improvement or deterioration), if occurred. Eventual changes in these elements could affect roles and responsibilities of pilot due to their mutual dependence. Clear way of work/procedure influences the role of the actor and his/her boundary of action. Viceversa, unclear or confusing procedures may negative influence the working action of the user and compromise his/her understanding about limit of intervention. Relevant aspects asked to pilots were: routine working methods (if changed or not); Exhaustiveness of operational info preparatory for the trial. Figure 40 shows overall scores, collected for both these aspects Figure 40: Airside HP Objective HP004 Histograms referred to first topic clearly show that pilots routine working methods changed due to FREE Solutions flight trials only for a 05% of users. For the most of the pilots the working methods did not change. It induces indirect benefits to i) the quality of crew operations that is preserved at current level and ii) the saving of cognitive effort in handling activities. Histograms referred to second topic show that the operational information provided to the flight crew revealed be unchanged. For 92% of pilots the question was not applicable, nothing of significant was modified. Only for few users (i.e. 07%) there was something new in terms of working methods, roles & responsibilities and provided information. Collected scores, integrated with feedbacks provided by pilots at the end of trial, confirm that role and responsibility for both PF and PM are still clear and consistent due to the non-change of current routine working methods and the operational info. Stated changes (highlighted by very few users) were still acceptable due to the fact that they still guaranteed exhaustiveness of operative information. 89 of 213

90 Project Number LSD Edition Operating Methods and Procedures Identifier Objective Success Criterion Success Criterion Supportive Exercise Objective OBJ-LSD0105-HP008 To demonstrate that pilot operating methods and procedures (PF, PM) are clear and consistent during city pairs optimization flight trials Impact of concerned operational concept on pilot activities is clearly identified and accepted Positive feedback from pilots during the flight trials session OBJ-LSD Table 39: Success Criteria - Operating Methods and Procedures OBJ-LSD The purpose of this objective is to complement evidence derived through a devoted brainstorming before the trials Error! Reference source not found. with additional more tangible one collected after the execution of the exercise. Suitability of crew operating methods and procedures represents another important marker for the HP assessment of EXE-LDS These elements, as anticipated in the previous section, are complementary to roles & responsibilities topics. A particular behaviour of operating methods and procedures could influence the other ones and viceversa. This explains why some questions already showed during the assessment of objective HP004 are here again proposed. Assessment of PF/PM operating methods and procedures mainly implied to check: adherence to planned FREE Solutions route; operational ability to perform fuel check during the flight as requested by current procedures; eventual change to routine working methods (if changed or not); availability of proper info to safely execute the trial. Figure 41 shows overall scores, collected for all these aspects Figure 41: Airside HP Objective HP008 The figure above shows very positive results. Pilots were able to fly FREE Solutions planned routes and to comply with routine fuel checks most of the time (i.e. respectively 87% and 91%). As anticipated during HP004 description, routine working methods were rarely modified (only 05% of cases). Furthermore, information given to the crews of different airlines was sufficient to safely conduct the flight in 95% of the time. These results are aligned with preliminary impressions collected during the brainstorming sessions where no significant changes - with respect to current operations - were envisioned for the Pilot Flying. For the Pilot Monitoring, the only operating method/procedure that could have been affected during the trials execution was envisioned the intermediate fuel checks. Collected scores did not 90 of 213

91 Project Number LSD Edition confirm this issue. Current way of performing intermediate fuel check revealed to be still applicable and then suitable for pilots. In the light of i) positive feedbacks derived from heterogeneous crews and ii) substantial unchanged PF/PM operations after FREE Solutions trials execution, the objective HP008 was covered. Crew operating methods and procedures still remain clear and consistent as in current operations. Pilot Performance Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP010 To demonstrate that pilot (PF, PM) performance is appropriate during city pairs optimization flight trials OBJ-LSD Table 40: Success criteria - Pilot performance OBJ-LSD Evaluate crew performance means the assessment of users perception with respect to different markers/dimensions that basically are: o o o o o o o workload (mental, physical); trust in the concept; situational awareness; tasks time execution; job satisfaction in adopting the new operational concept; load of ATC communications; potential errors induced by the introduction of the proposed concept. Some of these markers are interconnected among them, e.g. tolerable level of workload and ATC communications make the acceptability of new concept smoother and then allow the achievement of positive results in terms of task execution, job satisfaction and then trust in the new concept. Workload & Situational Awareness Figure 42 shows pilots feedback regarding workload experienced during the trials. Attention was paid to the investigation of: potential changes in workload level; negative effects on crew performance due to workload increase; typology of workload mainly affected Figure 42: Airside HP Objective HP010 (workload) 91 of 213

92 Project Number LSD Edition Collected results show that level of workload did not change for most of the users (66%), whereas 30% of pilots experienced a reduction of effort. A very limited amount of pilots declared an increase of workload (04%). Based on the opinion of those who noticed an alteration of workload, no changes occurred in terms of overall performance (20% reported no change, 73% declared the question not applicable according to the trial just executed). Regarding typology of workload, most pilots still confirmed that the question was not applicable considering the trial just executed (76%). The mental effort revealed to be the most affected of the four typologies of workload (15%). Scores gave back results in terms of workload that are against initial feedbacks derived through brainstorming Error! Reference source not found.. During brainstorming pilots hypothesized negative effects on crew workload due to potential increase of ATC requests to change the Flight Plan. Direct feedbacks from questionnaires did not confirm this hypothesis. On the contrary, workload appeared unchanged (compared to current operations) and proper levels of crew performance were still assured. Achieved results allow inferring that the situational awareness (both individual and shared between the crew) was still maintained at proper level. Tasks time execution & Job satisfaction Another marker that could influence crew performance is the time/effort devoted to the execution of tasks. Figure hereafter shows achieved scores with respect to four on-board activities deemed by AUs easily influenced due to the new concept, i.e. execution of planned FREE Solutions route; feasibility to perform a continuous climb (if FREE Solutions route is within the climb phase); feasibility to start the descent at the Top of Descent (if FREE Solutions is within the descent phase); variation of speed due to ATC requests Figure 43: Airside HP Objective HP010 (tasks time execution) According to reported scores, very positive results were achieved in terms of route adherence to the planned one: 87% of crew declared the feasibility of this task and only 10% encountered some obstacles in flying it. Positive results were also collected in case of descent operations where 70% of pilots were able to begin the descent at the predefined TOD, the remaining share was equally split into pilots not able to comply with point of descent (15%) and pilots that according to the route could not assess the concerned task (remaining 15%). 92 of 213

93 Project Number LSD Edition In relation to CC (continuous climb) operations, in most cases pilots were not able to assess the ability to execute the task because not applicable (62%). The remaining 38% of pilots were able to assess the actual execution of CC operations: 33% positively performed the operation, whereas for the remaining 05% was not possible. Concerning potential speed variations requested by ATC, a large set of pilots did not experience the need to perform this task (74%) with subsequent positive effects on overall flight operations performance. Only very few amount of pilots (23%) experienced the ATC request. Achieved results demonstrated that flight operations were successfully executed as planned before starting the trial, with subsequent benefits to crew overall performance. Furthermore, scores allow inferring that positive performance in terms of task execution can reasonably induce affirmative pilots behaviour in adopting the new operational concept with good feedbacks in terms of job satisfaction. Load of ATC communications A further indicator able to influence the performance is the load of air/ground/air communications. The figure hereafter shows pilots perception on changes of ATC communications due to the introduction of new concept Figure 44: Airside HP Objective HP010 (communications load) Users perception was quite contrasting itself. Approximately, half part of pilots did not recognize any change (53%) whilst the other half noticed a perceptible reduction (43%). This score could be justified on the light of heterogeneous typology of FREE Solutions routes. Some of them were simple overflies with subsequent reduction of communication amount, others were mixed (i.e. part of overfly and descent into the terminal area) with probable perception that communications did not change with respect to current operations. Potential Errors The purpose of questionnaire was also to assess that new routes did not induce errors in executing operations with dangerous side effects on crew performance. Figure 45 collects pilots scores referred to safety level of operations. Questions proposed to AUs were related to: effects on safety due to changes of workload level; effects on safety due to modification of working methods; availability of proper info to safely execute the trial. 93 of 213

94 Figure 45: Airside HP Objective HP010 (potential errors) Cases of safety degradation were not registered. In most cases, questions were declared not applicable because either workload level did not change (70%) or working methods were not altered (94%). Furthermore, the information provided to the crew was deemed sufficient and appropriate for the safe execution of trial (95%). Pilots did not report cases in which new routes induced crew in performing wrong operations or errors, also because operating methods and procedures revealed to be unchanged with respect to current way of work. Changes to Tasks and Roles OBJ-LSD0105-HP014 Objective To demonstrate that changes to pilot tasks and role (PF, PM) are managed during city pairs optimization flight trials Supportive Exercise OBJ-LSD Objective Table 41: Success Criteria: Changes to Tasks and Roles OBJ-LSD This objective is the final one related to the assessment of airside HP aspects. It is devoted to the identification and - in case evaluation - of any potential changes to crew activities and roles/responsibilities due to the execution of FREE Solutions routes. This objective is very close to the ones already discussed in previous sections and therefore it represents the closure, the wrap-up of overall airside assessment. This objective implies the verification of those elements declared by pilots potentially affected by a change, i.e.: procedure of routine fuel check during the trial; routine working methods; operational information related to roles & responsibilities to be adopted during the trial; safety level of operations. Figure 46 is mainly focussed on the first three aspects reported above. 94 of 213

95 Figure 46: Airside HP Objective HP014 No significant changes occurred in airborne operations. Current way to perform routine fuel check was successfully applied in most cases (91%). Routine working methods did not change for 94% of pilots. Operative information was deemed still exhaustive and not impacting crew roles and responsibilities, then the topic appeared not applicable (92%). This represents in practice an essential stability of current way of work and related procedures. No significant changes were then experienced and reported by crews of different airlines. Furthermore, potential for pilot errors were not registered. During trials the level of safety was still assured as occurs nowadays. Figure 47, already proposed for objective HP010, confirms that cases of safety degradations did not occur during trials Figure 47: Airside HP Objective HP014 (potential errors) Pilots did not experienced errors or misunderstanding induced by the new FREE Solutions routes Results impacting regulation and standardisation initiatives Nothing to be reported in this paragraph Unexpected Behaviours/Results 95 of 213

96 Project Number LSD Edition Nothing to be reported in this paragraph Quality of Demonstration Results As it may be verified in the other sections of this chapter, the approach adopted was a combination of FPL information and actual track data, which gave as an output high quality results compared to the conventional approach used when performing RTS simulation activities. The savings in terms of distance, duration and fuel burn are consistent when measured from the NM or the AUs' perspectives. Moreover, the assessment of real trajectories confirms the actual improvement that was expected on the flight plans basis. The results are lower, but this is in line with SES established flight efficiency indicators. As trials took place during a low traffic period and the number of flight trials was too low to create a significant effect on ATC operations and controller workload, no impact on capacity was expected. However, the fact that sector occupancy was not affected brings confidence in the feasibility of the designed solution at higher traffic rates. During screening of questionnaires and subsequent delivery of statistics an anomaly was observed. For some questions of filled-in templates, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve the full score (i.e. 100%) through the sum up of afferent intermediate quotes. As a final effect a margin of uncertainty is caused and induced subsequent effects, first on the analysis of data and secondly on the reporting. However, this uncertainty can be classified as not severe because involved questions were not so many (i.e. 09 questions out of 32). Furthermore, percentages of uncertainty achieved tolerable levels (i.e. around 04% with only one case of 17% as a peak) Significance of Demonstration Results The number of flight trials is quite significant and the operational environment unchanged. This guarantees confidence in the results published Conclusions and recommendations Conclusions This section summarizes results achieved with the execution of the FREE Solutions live trials concerning City Pairs optimization through the use of direct routings. Main conclusions are drawn from the results per KPA reported in the section of this document and are listed below. CONC 1. Positive impact on fuel saving and related CO2 emissions, especially on flight efficiency, is reported by using the identified new routes for some of the most penalized city pairs; CONC 2. No ATFCM delay directly attributed to city pair trials, with no negative effects on local and network airspace capacity; CONC 3. City Pairs scenario did not introduce any negative impact on associated workload (Pilots and Controllers) and situational awareness, as: CONC 4. a. There were no particular differences compared to baseline operations, so related workload and situational awareness were not negatively impacted. Even if, in very few cases, some increase was reported for workload level, it was affirmed that safety of operations was never affected. b. roles and responsibilities were assessed clear and consistent due to the non-change of current routine working methods and procedures. The Safety of operations was maintained at the level of baseline operations, as: a. no particular change was experienced compared to daily operations from both Pilots and Controllers and, as a consequence no safety issues was reported during the trials. 96 of 213

97 Project Number LSD Edition CONC 5. b. the number of requested Air-Ground and Ground-Ground communications was not significantly affected, even if in some cases a moderate growth was recorded for what concern ground-ground coordination, mainly due to the novelty of trials activity and the consequent additional coordination required. Issues experienced during city pairs live trials were properly managed, as: a. Pre-trials information were assessed appropriate and timely as both Pilots and Controllers felt confident with concept implementation and informed about potential issues to be experienced during trials activities, as well as potential changes to their working methods. b. Deviations from planned activities (e.g. route deviations, conflicts) was always safely managed by both Controllers and Pilots who experienced it without affecting safety of operations. c. FPL inconsistencies and technical issues have been safely managed during the trials. CONC 6. Positive impact on flight punctuality and on ATC planning was not demonstrated during the city pairs live trials. Finally, introduced benefits can be summarized with the following measured values, which provide readers with the delta between the live trials operational scenario and the baseline scenario. Trial Flights Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) 120 (6) kg Recommendations Table 42: City Pairs delivered benefits Here below there is the list of proposed recommendations for the subsequent FREE Solutions live trials exercises. REC 1. Flexible use of airspace and management of reserved areas shall be fully addressed during the execution of the EXE-LSD due to the impossibility to start performing these activities during the city pairs weekdays exercise (i.e. EXE-LSD ); REC 2. The second set of live trials focusing on DRs shall investigate the possibility to demonstrate benefits in terms of flight punctuality and improved ATC planning so that to address predictability KPA; REC 3. FREE Routing trials shall allow fuel check operations through intermediate waypoints 6 This number of flights, and the related data, are the sum of 118 executed during the EXE-LSD and 2 executed during the EXE-LSD Actually, the total number of eligible FREE solutions flights before the city pairs trials was 125, but 5 of them did not plan the proposed city pairs optimizations. 97 of 213

98 Project Number LSD Edition Demonstration EXE-LSD Report Participating ACCs in the City Pair weekday trial included Padua and Zurich ACCs as shown in the figure below Exercise Scope Exercise Scenario Figure 48: Participating ACCs and City Pair Weekday Direct Route Exercise scenario SCN-LSD referred to normal day of operations during weekdays, with no significant events/deviations form routine operations. It encompassed a time window starting from Monday morning at 07:00 UTC up to Friday night at 23:00 UTC. Scenario is applicable from end of SID to beginning of STAR, so it is limited to En-Route phase and to the consortium airspace. Aircraft fly in accordance with existing operational procedures in controlled airspace. At the same time, operations will be ensured through existing working procedures for all the ACCs involved in the trails activities Exercise Objectives This exercises addresses the demonstration objectives and related KPAs listed in the table below OBJ ID OBJ Description KPAs and TAs OBJ-LSD OBJ-LSD OBJ-LSD To demonstrate the advantages of using new routes/corridors on most penalized city-pairs in terms of efficiency/environment sustainability To give input to airspace community interested in DR/FR operations To demonstrate that an airspace management tailored on AU needs along with an improved cooperation with AMC can contribute to optimize the cost for airlines as well for air Environment - Fuel Efficiency; Human Performance; Airspace Capacity - En- Route. General N/A General N/A 98 of 213

99 navigation services OBJ-LSD To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably safe Safety Table 43: EXE-LSD Demo Objectives Conduct of Demonstration Exercise EXE-LSD Exercise Preparation The table below summarizes the FREE Solutions flight trials executed during the weekday city pairs scenario (i.e. SCN-LSD ). Due to the complexity of coordination activities to be put in place with relevant military authorities and the tight time constraints for the execution of the weekdays exercise, the city pairs selection criteria were identified taking into account the following considerations: Minimum interference with military areas; Minimum impact on daily operations and related duty tasks; Cross-border optimized city pairs routes. These criteria led to the initial selection of the following city pairs: LIPZ-LSZH: This city pair represented a short haul cross border route option with no interference with Italian military areas and minor interference with Swiss ones, together with no significant impact on daily operations of involved ACCs (i.e. Padua and Zurich). Additionally, the indicated city pairs optimization represented a high cost benefit activity due to the fact that pre-analysis results showed a potential high fuel savings compared to route length. Actually the ratio between fuel saved and route length was one of the most promising of the overall city pairs optimization identified. This allowed a clear and firm request from Swiss Airlines to test the proposed route optimization as preliminary step towards operational implementation; EGLL-LIRF: This city pair represented a medium haul cross border route option with no interference with military areas (inside the consortium airspace), but a major impact on daily operations of involved ACCs, which meant interference with outbound departing flows from Rome Fiumicino heading northwest. The level of impact was not acceptable for a scenario being executed during the weekdays, that was not the case if the exercise would have taken place during the weekdays for the difference in the amount of traffic to be handled. In addition to that, the proposed city pairs optimization represented an optional route for Alitalia depending on the forecasted wind conditions during the day of trial. 99 of 213

100 Figure 49: EGLL-LIRF potential conflicting traffic So, due to the abovementioned reasons and the risk of not having the second proposed city pairs optimization flown during the trial, it was decided to discard the second option from the exercise activity and execute only the first one. Friday 10th April 1. LIPZ LSZH SWR1663 RJ1H 12:45 UTC Y 2. SWR1665 RJ1H 18:05 UTC Y Table 44: EXE-LSD City Pairs List - weekdays Exercise execution The exercise EXE-LSD was executed on Friday 10 th April Deviation from the planned activities Due to the limited amount of time versus the time required to set up and perform coordination activities with the military authorities, it was agreed to de-scope the activities of the second exercise for city pairs optimization and postpone the analysis of the flexible use of airspace during direct routing operations. In particular, data that will be collected during the EXE-LSD will be used as basis for considerations and drawing of results for this specific item, by addressing the following demonstration objectives OBJ-LSD and OBJ-LSD not addressed with the city pair live trials. Some technical issues were underlined : o Unexpected unavailability of military area interfering with trials routes (EXE- LSD ) o Further analysis while approaching the trials execution and consequent exclusion form the trails list of some flights (EXE-LSD ) - Time issue (like time constraints) 100 of 213

101 Project Number LSD Edition Exercise Results As detailed in section and due to the limited amount of collected data there are no specific results for this exercise. Collected data were merged with the outputs of EXE-LSD as explained in the footnote N 6 of this document (ref. sec ) Conclusions and recommendations As detailed in section and due to the limited amount of collected data there are no specific conclusions and recommendations drawn for this specific exercise. 6.3 Demonstration EXE-LSD Report Exercise Scope The scope of this exercise was to allow airspace users to file and execute optimized FPL routings. Those optimized routings were made available for flight trials as Direct Routings supporting specific traffic flows. The following stepped approach was taken, as described in the section 4.1 of the FREE Solutions Demonstration Plan v2.0 [1]. The analysis performed for the identification of city pairs optimization (i.e. EXE-LSD et EXE-LSD ) is considered as input for step 2 in order to identify a set of DRs. Those DRs are selected in accordance with the following criteria: selected DRs can be flown by two or more previously identified city pairs optimization; selected DRs are of interest for two or more ACCs of the consortium area; selected DRs may be composed of DRs segments linking waypoints in different ACCs within the consortium area; selected DRs allow efficiency optimization of flights vertical profiles; additional DRs not analysed during CPs trials, due to shortage of adequate time for their negotiation and agreement with interested actors. For this exercise DR possibilities were also made available to trial flights operated by AUs not participating in the FREE Solutions project and organised for the purpose of the project activities. FPL routings of trial flights have to contain at least one FREE Solutions DR as designed in the context of WP3 activities Exercise Scenario Exercise scenario SCN-LSD referred to normal day of operations during weekends, with no significant events/deviations form routine operations. It encompassed a time window starting at 23:00 UTC of Friday and ending at 07:00 UTC of Monday morning. Scenario is limited to En-Route phase. Its geographical scope was limited to the airspace of the members of FREE Solutions consortium. Aircraft fly in accordance with existing operational procedures in controlled airspace. At the same time, operations will be ensured through existing working procedures for all the ACCs involved in the trails activities. Participating ACCs in the DR trials included Marseille, Geneva, Zurich, Milan, Rome Padua, Brindisi and Malta ACCs. Direct Routing options were designed from a network perspective and developed based on NM analyses identifying significant flows on network level. Those identified DRs were not limited to consortium flights only but were published and open to all traffic. 101 of 213

102 Exercise Objectives Figure 50: DRs during weekend This exercise addresses the demonstration objectives and related KPAs listed in the table below. OBJ ID OBJ Description KPAs and TAs OBJ-LSD OBJ-LSD OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized medium-long haul traffic in terms of efficiency/environment sustainability To give input to airspace community interested in DR/FR operations To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably safe Efficiency; Human Performance; Airspace Capacity - En- Route. Predictability and ATC Planning General N/A Safety Human Performance Figure 51: EXE-LSD Demo Objectives Conduct of Demonstration Exercise EXE-LSD Exercise Preparation Route design As for the EXE-LSD and EXE-LSD , the project adopted an incremental preparation process. The most penalized routes of the current network have been analysed, taking also into account inputs from Airspace Users. After an initial evaluation by the NM (based on DDR and NEST tools), a set of Direct Routes was established for trialling in order to implement more 102 of 213

103 Project Number LSD Edition efficient routes in terms of mileage, fuel consumption, and environmental impact. The table 7 below indicates all the potential benefits for proposed weekend direct routes. Potential gains/losses Total impacted Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) flights Saturday ,56-657, , ,1-372, Sunday ,37-652, , ,03-370, Table 45: DRs potential benefits evaluations The new DRs of this second set of Flight trials has been selected taking into account the requests provided by Consortium-Airspace Users, inside Available Air Space focusing the attention on the medium-long haul trajectories, and on the most promising City-Pairs assessed after March flight trials. In summary, identified direct routings were composed by: DRs already selected and assessed during city pairs optimization; DRs considering an already existing route which would be chosen if current constraints, like for example FL restrictions, can be deleted or relaxed; DRs still under evaluation which were not available for March 2015 trials, but assessed promising by project participants. All the DCTs chosen for the trial are reported in the table below. Each DR should be planned according to the specified direction (e.g. EKTOL-NOSTO meant that the trajectory could be flown in only one direction, from EKTOL to NOSTO), except for the following Direct Routes: ELB-EKTOL, ELB-LUNAR, LUNAR-LATAN that were bidirectional (highlighted in dark grey). Direct Route Flight level Direct Route Flight level 1 ELB EKTOL FL310 FL BATOX GOTMO FL350 FL460 2 EKTOL NOSTO FL310 FL GOTMO IBENI FL350 FL460 3 ELB LUNAR FL310 FL PELOS LEVDI FL350 FL460 4 LUNAR LATAN FL310 FL ROTAR RESIA FL250 FL460 5 BEGAR ODINA FL270 FL BORMI RESIA FL200 FL460 6 ODINA OGLAK FL270 FL PUNSA IBODI FL250 FL460 7 OGLAK XIBIL FL250 FL ATMAD OTKOB FL200 FL460 8 OGLAK GIANO FL310 FL OTKOB ALBET FL200 FL460 9 OGLAK AMANO FL310 FL BEGAR RESIA FL245 FL ODINA IPLUB FL350 FL RESIA BEDEG FL310 FL IPLUB GITOD FL250 FL BEDEG EKMUR FL200 FL MORSS XATOS FL200 FL ORSUD IPLUB FL350 FL OSPOK XATOS FL200 FL ABN RIPDU FL240 FL SOTAX ELSAG FL200 FL RIPDU SOVUB FL200 FL ELSAG LEVDI FL250 FL FJR RASPA FL245 FL LEVDI ARLOS FL350 FL RASPA ERETI FL245 FL ELSAG LUNAR FL350 FL ERETI TABOT FL245 FL LUNAR GOKEL FL350 FL TINOT NIVDA FL245 FL660 7 These estimations do not take into account the following DCTs as no traffic was assigned: ELSAG-LEVDI, ELSAG-ULKEN, ULKEN-LATAN, BORMI-RESIA. 103 of 213

104 VANAS GODRA FL305 FL GODRA DITON FL305 FL ELSAG ULKEN FL350 FL NIVDA TABOT FL245 FL ULKEN LATAN FL350 FL XATOS LEVDI FL350 FL XATOS ULKEN FL350 FL460 Table 46: The list of Direct Routings executed This set of identified DRs was not limited to the consortium flights only. They were opened to all traffic flows which may benefit from them. The availability of the Direct Routes reported in the tables above were published by EUROCONTROL-NM in the RAD Section, annex Special Events FREE Solutions Flight Trials, with the exception of Direct Routes in LFMM Airspace, published via AIP SUP (i.e. [FJR-RASPA-ERETI-TABOT]; [TINOT-NIVDA-TABOT]; [OSPOK-XATOS]; [MORSS-XATOS]; [SOTAX-ELSAG]) Flight planning The guidelines document prepared for live trials execution Error! Reference source not found. provided a kind of user manual to plan a flight. The FREE Solution direct routing was included between portions of routes compliant with current ATS Route Network. Moreover, the proposed Direct Route segments could be planned in combination with other Directs of the same table, or as a single segment. Some of the Direct Routes could be combined with other Direct Routes published by other ANSPs belonging to FREE Solutions Consortium. In summary: - The field 15 of the ICAO FPL document was filled with the complete route, - The field 18 permitted to inform of the trial by inserting RMK/FREE TRIAL Example of a FREE Solutions flight plan including the FREE Solutions Direct Routes for the DCT: BEGAR-ODINA-IPLUB-GITOD (City Pair EBBR-LIRF): N0448 F370 DCT REMBA UM624 RITAX UT27 GTQ UQ343 BEGAR DCT ODINA DCT IPLUB DCT GITOD GITOD4F Operational Staff Briefings Based on the already mentioned guidelines document, a set of briefings with operational personnel (ground and airborne sides) was prepared and executed to provide advices for a correct execution of the trials. The objectives of theses briefings were to provide a general description of the FREE Solutions Project, and a detailed description of the second set of flight trials planned for November 2015, providing also guidelines and advices for a correct execution of the trials. As for the previous EXE (EXE-001 and EXE-002), the following activities were undertaken: Leaflet preparation for ATCOs involved in trials to clarify scope, ambition, needs of the project, and describing any further operational procedure required; Leaflet preparation for Pilots involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required; Leaflet preparation for Flow Managers Positions involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required Advanced sharing of ATCOs feedback questionnaires; Advanced sharing of Pilots feedback questionnaires; Liaison with the identified Point of contact for each ACC involved in the flight trials for setting up the relevant operational scenario; 104 of 213

105 Liaison with the identified Point of contact for each airline involved in the flight trials for setting up the relevant operational scenario. Different AIC were also provided to inform Aircraft Operators about new Direct Routes that were temporary implemented for a SESAR-FREE Solutions trial during weekends 21s st /22 nd of November and 28 th /29 th of November (see appendix for more details). All the direct routes were published in RAD, except for LFMM that were only published via AIP SUP Questionnaires The activities performed in the context of WP4 to produce the foreseen HP and SAF cases, encompassed a brainstorming session with operational experts (ground and airborne sides) to identify, discuss and review a set of specific questions to be handled to flight crew, controllers and flight dispatcher involved during trials activities. The outcome of this preparation process was a list of post trials questions (see appendix c for the questionnaires used), split between air and ground sides, which were used to collect operational feedbacks. Different safety related areas were concerned by the analysis: Inconsistencies of FPL planning, distribution and processing Technical issues Sector overload Interactions between flights flying DRs and other flights Resuming the assigned route after a deviation Unusual entry/exit point Crossing of Active military areas Navigation issues Adverse meteorological conditions. The table below shows the relations between the safety related questions proposed to the operational people (i.e. ATCO and Pilots) and hazards. Questionnaire Target Question ID Question description Hazard ID reference ANSP 4.1 ANSP 4.2 ANSP 4.3 ANSP 4.4 ANSP 4.5 ANSP 4.6 ANSP 4.7 Did you encounter any problems with flight plans? The amount of tactical rerouting was within acceptable limits Did you encounter any technical issues (i.e. HMI/FDP/OLDI) I was able to manage interaction between traffic flying DRs and other flights using the current network I was able to manage deviations to the FPL route due to active military areas/adverse meteorological conditions Have contingency procedures (if any) been adequate to cope with an unexpected event? Did you notice any change in the amount of ATC communications/coordination required compared to routine operations? HZ-01 HZ-06 HZ-07 HZ-02 HZ-05 HZ-15 HZ-06 HZ-02 HZ-04 HZ of 213

106 Questionnaire Target Question ID Question description Hazard ID reference ANSP 4.8 ANSP 4.9 ANSP 4.10 AU 1.1 AU 1.6 AU 1.7 AU 1.8 AU 1.9 AU 2.1 AU 2.2 AU 2.3 AU 2.4 AU 3.1 AU 3.4 AU 4.1 AU 4.2 Has the sector configuration been adequate to manage the traffic in control during the trials? Concerning the new DR scenario, did you manage conflicts within your area of responsibility or at the boundary? (entry/exit points) Did you manage unexpected interactions with active military areas? Operational - Have you been able to fly your planned FREE Solutions Route? Operational - Have you been able to perform routine fuel checks without any problem? Operational - Did you notice any change in the amount of ATC communications compared to routine operations? Operational - Final remaining fuel has been final reserve fuel? Operational - Were your planned ATS routes stored in your FMS Navigational database? Safety - Have Free Solutions ad-hoc contingency procedures (if any) been adequate to cope with any unexpected events during the flight? Safety - Did you perceive that the ATCO s in contact during the flight were fully aware of the operational scenario of the FS flight trials? Safety - Did you deviate from the planned ATS route on ATC request or due to adverse meteorological conditions? Safety - If you answered YES to question 2.3, did you feel that safety was ever compromised due to such deviations? Workload - Did you notice any differences in your workload levels? Workload - If you answered YES to question 3.1, did you feel that, due to increased/decreased workload levels, safety was ever compromised? Working methods - Were you required to alter your routine working methods in order to fulfil your duties? Working methods - If you answered YES to question 4.1, was Free Solutions operational information, provided before the flight, exhaustive with regards to roles and responsibilities, working methods and operational requirements? HZ-12 HZ-06 HZ-07 HZ-08 HZ-11 HZ-03 HZ-07 HZ-09 HZ-09 HZ-13 HZ-09 HZ-03 HZ-07 HZ-09 HZ-09 HZ-15 HZ-09 HZ-15 HZ-13 HZ-13 All AU related hazards All AU related hazards 106 of 213

107 Questionnaire Target Question ID Question description Hazard ID reference AU 4.3 AU 4.4 Working methods - If you answered YES to question 4.1, did you feel that, due to alteration of working methods, safety was ever compromised? Working methods - Was the information provided before the flight trial sufficient to safely perform the flight? All AU related hazards All AU related hazards 2274 Table 47: Safety related questions - DR Exercise execution The potential benefits of the proposed DRs were tested on two weekends: 21 st - 22 nd of November th 29 th of November 2015 starting from Friday night at 23:00 UTC to Monday morning at 07:00 UTC Deviations from the planned activities In general, Airspace Users planning system had selected the most convenient route (cheapest) depending from several factors such as: en-route wind conditions, en-route charge, route length. For these reasons, despite the availability of a shorter route, some available DRs were not selected due to unfavourable overall costs conditions. The route selection criteria were applicable for all flights, but they mainly affected long haul flights during live trials. This effect was largely expected when planning live trials activities and hence the missed selection of some available DRs cannot be seen as a deviation from planned activities, but it is anyhow reported for the sake of completeness. Actual deviations from planned activities are the following and refer to data collection and analysis. The first concern the demonstration objective OBJ-LSD for which it was decided to measure residual fuel at arrival through ad-hoc post trial questions, instead of making a quantitative measure of remaining fuel tons. This decision taken in accordance with AUs representatives during live trials preparatory activities did not affect data collection and allowed SAF experts to develop a safety case and supporting arguments as initially planned. The second is related to HP data collection. In some cases, it was not possible to discriminate between planner and executive controller roles when aggregating and analysing collected data, due to incompleteness of the provided answers. This forced HP experts to generalize the analysis and not make it specific for the assigned role and associated responsibilities Exercise Results Summary of Exercise Results Results per KPA This section details results obtained by applying the methodologies described in the section of this document Environment Fuel Efficiency The results collected and analysed for this KPA contributes to the satisfaction of the following demonstration objective OBJ-LSD of 213

108 OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized mediumlong haul traffic in terms of efficiency/environment sustainability Success Criterion Efficiency: Positive impact on fuel saving and CO2 emissions Results Fully achieved Table 48: Success Criteria Environment Fuel Efficiency OBJ-LSD Flights for comparison were taken from the Reference Weekend November However, the number of flights was not equal in number for the two weekends, so the total number of flights taken for comparison is different for each weekend. As agreed at project level during the Free Solutions WP4 progress meeting #5 held in Brussels on the 4 th February 2016, some flights were removed from the final economy scenario calculations as follows: Long range flights (oceanic, to/from Asia) such flights have a very long distance and were planned based on wind conditions. Flights with a distance between Reference track and Solution track of more than 40NM in the horizontal plane such flights had different trajectories and used different routings. As a result some flights had extra distance during the trial Figure 52: Example of routes involved in traffic filtering The table hereafter summarizes results 8 observed and KPI used. KPA KPI Metric Results Environment- Time saved Delta FPL(ref) v FPL(scn) - Fuel Efficiency Minutes 611mins Distance saved Delta Nautical Miles CPF(ref) v CPF(scn)-469 mins FPL(ref) v FPL(scn) nm CPF(ref) v CPF(scn)- 2171nm 8 Measures refer to the total values for both weekends. 108 of 213

109 Eligibility flights planned to conduct the trial solution 9 Number of flights 642 Flights following filtering of the sample that have flight planned to conduct the trial solution FPL(ref) v FPL (scn) Flights following filtering of the sample that have actually flown the trial solution CPF(ref) v CPF(scn) Fuel burn saved enroute Number of flights Number of flights Delta Kg FPL(ref) v FPL (scn) kg CPF(ref) v CPF(scn)-21367g CO2 saved en-route Delta Kg FPL(ref) v FPL (scn) kg CPF(ref) v CPF(scn) kg NOx saved en-route Delta Kg FPL(ref) v FPL (scn) -350kg CPF(ref) v CPF(scn)-416kg Table 49: Environment-Fuel Efficiency Results - Metrics and Measures Weekends Direct Routings It must be pointed out that for this specific exercise the total average saving for a single flight between FPL(Ref) and FPL(scn) was 9,47nm, while the average saving between CPF(ref) and CPF(scn) was 5,3nm. For any further details on the assessment, data measurement and results refer to the detailed analysis reported in Error! Reference source not found Airspace Capacity The results collected and analysed for this KPA contributes to the achievement of the following demonstration objective OBJ-LSD , which has dedicated success criterion for airspace capacity. OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized mediumlong haul traffic in terms of efficiency/environment sustainability Success Criterion Results 9 This number does not take into account the number of flights planned by Ryanair. Information not yet transmitted 10 Number of flights (FPL) that were not compared: 49 ( ), 38 ( ), 24 ( ) and 56 ( ). 11 Number of flights (CPF) that will be not compared: 45 (for ), 41 (for ), 26 (for ) and 27 (for ). 109 of 213

110 Capacity: no negative local and network impact Promising results. The traffic load did not permit to observe any negative effects. Table 50: Success Criteria - Capacity OBJ-LSD The subsequent table summarizes the results observed and the key performance indicators used to obtain them. KPA KPI Metric Results Airspace Capacity (enroute) ATFCM delay Minutes None attributed to DRs Sector occupancy ACC throughput Number of aircraft per hour Number of aircraft per hour Change not considered significant enough to measure quantitatively Change not considered significant enough to measure quantitatively Sector workload Delta % points Change not considered significant enough to measure quantitatively Conflicts Number of potential conflicts per sector Change not considered significant enough to measure quantitatively Table 51: Capacity Results - Metrics and Measures Weekend Direct Routings Qualitative results regarding controller workload in relation to capacity are reported in the devoted sections. For any further details on the assessment, data measurement and results refer to the detailed analysis reported in Error! Reference source not found Predictability and ATC Planning OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized mediumlong haul traffic in terms of efficiency/environment sustainability Success Criterion Predictability: Positive impact on flight punctuality and on ATC planning Results Not Achieved Table 52: Success Criteria Predictability and ATC Planning OBJ-LSD KPA KPI Metric Results Predictability Actual en-route time duration vs. planned en-route time duration Delta Minutes An expected reduction of the delta time between reference and scenario examples, indicating a more predictable scenario, was not demonstrated in the data analysed Table 53: Predictability and ATC Planning Results - Metrics and Measures Weekend Direct Routings For any further details on the assessment, data measurement and results refer to the detailed analysis reported in Error! Reference source not found of 213

111 Project Number LSD Edition Safety This section reports the main results of the safety analysis conducted for the FREE Solutions Direct Routings operational scenarios. Full analysis is reported as separate document attached to this project deliverable [4]. Safety was evaluated through: - safety assessment and safety brainstorming, - calculation of residual fuel at arrival and, - answers to post-trial questionnaires. In particular, post-trial questionnaires (213 ATCOs and 124 Pilots) were analysed aiming to validate identified hazards and to conclude if they had an impact on the Safety of the operations or not. Statistical results have been organized discriminating the scope of the analysis (Ground and Air side) and are presented to guarantee the traceability between the hazards and the related questionnaires answers in order to demonstrate the arguments of the Safety Case ANSPS analysis This section presents the results according to the GNS Arguments structure in order to demonstrate that every aspect of the Safety assessment was considered pointing out the connections between the arguments, the hazards and the dedicated questions of the questionnaire. Workload and Situational Awareness sections of the questionnaire were also taken into account being common aspects shared between Safety and Human Performance. ARG was treated in the analysis as it was considered only to verify the acceptability of the preliminary hazards identified in the first version of the safety assessment. ARG Equipment affected by the change has been considered The purpose of this argument is to demonstrate that the potential failures or issues regarding the Equipment affected in organizing and executing the flight trials were considered Figure 53: ARG2.2.2 ANSP Equipment 111 of 213

112 Project Number LSD Edition The above results presented show that ATCOs did not experience problems with flight plans or technical issues (i.e. HMI/FDP/OLDI). This situation could have caused also an increase of workload for planner air traffic controllers. It is also necessary to underline that the N/A (Not Applicable) answers are not analysed unless they represent a significant percentage of the total answers (i.e. >40%). Despite the fact that, they refer to the lack of a specific event reported in the question, in most of the cases. ARG Airspace configuration This argument aims to demonstrate that the potential issues related to the airspace configuration were considered. ATCOs were asked to evaluate the adequateness of the airspace configuration during the Direct Routing trials. This topic is mostly linked to sector overload issue with a cascade effect on the ATCOs increase of workload. Several causes of sector overload were identified: unexpected inbound flights entering the sector, inaccurate monitoring of flights and insufficient sectorization. Other analysis and results are reported in the dedicated HP section Figure 54: ARG2.2.2 ANSP Airspace configuration The collected answers demonstrate the airspace configuration was adequate enough to allow a safe management of the traffic during the trials. ARG Operational and Flight Procedures This argument was structured to demonstrate that the flight and operational procedures to be used during the flight trials are adequate and not demanding by a safety perspective. Six questions were submitted to the ATCOs, results and related analysis are reported hereafter. 112 of 213

113 Figure 55: ARG2.2.3 ANSP Operational and flight procedures According to the ATCOs answers and related justifications the following considerations were done: - N/A answers mean that the ATCOs did not experienced what they were asked to evaluate, - The majority of interviewed ATCOs did not encounter any problems in re-routing traffic. In some cases there were no conflicts, thus no re-routing was made necessary. - In case of conflicts, some of them occurred near the boundary while others in typical well known areas. - In the majority of cases, ATCOs easily managed the interactions between DR flights and other flights, - During the trials it was not needed to apply contingency procedures, - Military areas were not active at the moment of the flight trials. Unexpectedly some ATCOs declared that they had to manage unexpected interactions with military areas (less than 1%). No justifications for these answers were provided. During flight trials no military areas were active, thus the "YES" answers should be reasonably considered as mistakes. Bad weather conditions were not experienced, - Contingency procedures were not used. However, some ATCOs declared that the contingency procedures were not adequate to cope with unexpected events (less than 8%). No justifications for these answers were provided. From the analysis of free text comments provided by the same ATCOs answering "NO" the only issues encountered were related to increase of coordination and rerouting. They also registered low traffic load. Thus the "NO" answers should be reasonably considered as mistakes. However, some interactions required the use of headings/vectors which led to the indication of very few negative answers. Thus, it is reasonable to affirm that the ATCOs have been always in control and confident with the implementation of DR concept as they were informed about the potential issues before the trials execution. ARG People related issues 113 of 213

114 Project Number LSD Edition This argument was pointed out to determine the impact on the human perspective of the operational concept introduced by the Direct Routings flight trials execution. The questionnaire sections referred to situational awareness and to the level of ATCO s workload during the trials execution. One question related to the amount of the communications was included in the Safety section Figure 56: ARG2.2.5 ANSP People related issues According to the results related to the amount of communications, in most of the cases (67%) no differences with current scenario were experienced. It is also necessary to underline that the N/A (Not Applicable) answers are not analysed unless they represent a significant percentage of the total answers (i.e. >50%). Despite the fact that, they refer to the not happening of a specific event reported in the question, in most of the cases. With reference to the answers regarding the YES, ATCos were asked to provide additional information: The Yes-increased answers were justified as follows: - new interactions with traffic need more communications, - unfamiliar routing requested more coordination, - managed traffic closer to the sector border than usual, with a white radar label ( white label refers to traffic not concerning the sector) ARG Environmental and meteorological conditions The argument was identified to consider the impact of the environmental and meteorological conditions on the operations. This topic has been tackled by the question 4.5 (Arg Operational and flight procedures have been considered). In fact, adverse meteorological conditions is a hazard whose effect is trial flight needs to deviate from the assigned route to avoid adverse weather conditions As already mentioned during the analysis of the ARG 2.2.4, adverse meteorological conditions were not experienced and/or did not affect the trials Airspace Users analysis This sub-section presents the results according to the GSN Arguments structure in order to demonstrate that every aspect has been considered pointing out the connections between the arguments, the hazards and the dedicated questions of the questionnaire. ARG results are not reported as it was only considered to verify the acceptability of the preliminary hazards identified in the first version of the safety assessment. The same applies to ARG Airspace Configuration since this topic was not considered as affecting the operations by AUs point of view. 114 of 213

115 Project Number LSD Edition ARG Equipment affected by the change The purpose of this argument is to demonstrate that the potential failures or issues regarding the Equipment affected in organizing and executing the flight trials were considered. 100% 80% 60% 40% N/A NO YES 20% % 1.1 Have you been able to fly your planned FREE Solutions Route? Figure 57: ARG AU Equipment The results clearly show that no issues were experienced by the pilots due to Equipment-related issues (92% of the answers were YES ). It should be noted that question 1.1 was proposed to pilots mainly to understand if no issues were recorded during the trials but focusing on the operations rather than on the equipment. It is possible to assume that since the most of the pilots were able to fly their planned route then it is clear that no issues linked to flight planning occurred. On the contrary, even if 6% of the pilots reported that they were not able to fly their FREE Solutions Route there is no possibility to determine the causal factors since no justifications were provided to the answers. ARG Operational and Flight Procedures This argument was structured to demonstrate that the flight and operational procedures to be used during the flight trials are adequate and not demanding by a safety perspective. 115 of 213

116 Figure 58: ARG AU Operational and flight procedures (1/3) Taking into account the results shown in the charts, it is clear that no particular Safety issue was experienced by pilots. According to the answers, it looks like they were comfortable during the trials execution and, even if they experienced some deviations from the assigned route, the Safety of the operations was never compromised. Also, as integration to ground-side safety analysis, the answers provided by pilots related to ATCO s awareness perception (i.e. question 2.2) are positive Figure 59: ARG AU Operational and flight procedures (3/3) Furthermore, communications with the ground were not affected by the trials execution. The answers related to the proposed questions display that no particular difference was experienced compared to daily operations and, as a consequence, it is possible to confirm that there was no degradation of the Safety level of the operations. 116 of 213

117 Figure 60: ARG AU Operational and flight procedures (2/3) Fuel checks operations required during the flights were highlighted by a restricted group of pilots since some issues took place while performing some long directs. ARG People related issues This argument was pointed out to determine the impact on the human perspective of the operational concept introduced by the Direct Routings flight trials execution. The rationale behind the analysis of the hazard by the pilot perspective is similar to the one used in the ground-side analysis. The results related to this argument are complemented by the HP analysis related to the Workload and Working Methods sections. A set of questions was selected to perform the analysis by a Safety perspective and the results are shown below: Figure 61: ARG AU - People related issues Almost 39% of pilots interviewed reported a decrease of workload and just 3% declared an increase. All pilots who have experienced a change in the workload level with respect to the baseline operations affirmed that the Safety of the operations was not affected at all. In fact, those who have answered the workload section of the questionnaire pointing out an increase of the workload were also asked to determine if, in their opinion, the level of Safety was ever compromised. The results are shown in Figure of 213

118 2513 Project Number LSD Edition % 80% 60% 40% 20% 0% If you answered YES to question 3.1, did you feel that, due to increased/decreased workload levels, safety was ever compromised? Figure 62: ARG AU Workload It can be said that, pilots workload was not affected by trials execution. Hereafter the results related to the Working methods section of the questionnaires are presented: N/A NO YES Figure 63: ARG AU - Working methods Similarly, the working methods related answers show that pilots did not perceive any change compared to daily operations and those who did not report that confirmed, anyhow, that no safety issues were in place during the trials. Pilots interviews ended with a general consideration about provided pre-trials information, which was assessed to be accurate, complete and timely. They confirmed to be well informed about trials activities and expected changes to the working methods, leading to a positive impact on the Safety of the operations. ARG Environmental and meteorological conditions This argument was identified to consider the impact of the environmental and meteorological conditions on the operations. Analysis focused on determination of how adverse meteorological conditions could have affected the trials by inducing involved aircraft in deviating from the assigned route. 118 of 213

119 Project Number LSD Edition A dedicated question was foreseen in the questionnaire to analyse the deviations experienced during the trials executions (question 2.3) and, as already mentioned, its evaluation confirmed that no safety issue was experienced Safety Summary Results OBJ-LSD To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably Safe Success Criterion Every negative effect on the operations shall be reduced as far as reasonably practicable The Safety of the operations shall remain at or better than its current level of Safety All the critical issues related to the operations management by all the stakeholders involved will be analysed All the evidences requested by NSA will be provided Results Fully achieved no negative effects on operations, according to the argument analysis showed in the previous sections Fully achieved - according to the argument analysis showed in the previous sections Fully achieved - according to the argument analysis showed in the previous sections Fully achieved evidence provided to NSA and no issue reported Table 54: Success Criteria - Safety OBJ-LSD KPA KPI Metric Results Safety Residual Fuel at arrival Tons It was decided to not measure and replace this measurement with ad-hoc questions for pilots Table 55: Safety Results - Metrics and Measures Direct Routings Weekends Additionally, during the demonstration planning stage a quantitative measurement was envisaged. The aim was to measure the overall amount (tons) of residual fuel at arrival. However, during live trials preparatory activities it was discussed and agreed with involved AUs to replace this measurement with ad-hoc questions to be inserted in the post-trial questionnaire for pilots (i.e Have you been able to perform routine fuel checks without any problem? Final remaining fuel has been final reserve fuel?). This decision better matched with existing AUs policies about external communication regarding fuel consumption, and it represented a more rapid and practical way to get the same piece of information HP Results Results reported in this sub-section are derived from the brainstorming session executed during the preparatory phase Error! Reference source not found. and the actual execution of Direct Routes trials. They are organised and proposed with respect to the two strands of analysis: groundside and airside ANSPs Analysis As already done in previous exercises, a customized questionnaire was administrated within the different sectors of ACC units directly affected by the trials: Milan, Padua, Rome, Brindisi (ENAV) Marseille (DSNA) Geneva (Skyguide) 119 of 213

120 Project Number LSD Edition Questionnaires analysed here were provided by ENAV and Skyguide. No particular comments or remarks were provided by DSNA. On the ground side the amount of trial flights allowed the collection of 213 questionnaires spread out among the involved ANSPs. The table hereafter provides details on the questionnaires, i.e. amount of formats collected per each ANSP. In some cases, information on role performed during live trial was not provided by controllers. For this reason, it was not possible to make a difference between planner and executive roles answers. Therefore, the following sections summarize results in an aggregated way. ANSPs Overall EC Formats PC Formats Other ENAV Skyguide Table 56: ANSPs Questionnaires - Direct Routings Data collected through questionnaires were manipulated and then converted into statistics. Manipulation consisted into the aggregation of ENAV and Skyguide scores and the subsequent organization. Conversion implied that answers to the different choices were expressed in percentage values or in mean values, i.e. partial quotes were reported in reference to the overall amount. Scores (percentage or mean) are reported in the following sections. Results are organized and then proposed according to the different Human Performance objectives linked to EXE-LSD Nevertheless, for some questions, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve the full score (i.e. 100%) through the sum up of afferent intermediate quotes. A margin of uncertainty is provoked and induced subsequent effects, first on the analysis of data and secondly, on the reporting. This uncertainty can be classified as not severe because achieved tolerable levels. Details regarding declared HP objectives can be found on the devoted document HP Assessment Plan [10]. Scores collected through questionnaires are hereafter provided in an aggregated way, i.e. mean value without specifying the role (EC and PC) or the ANSP of origin Roles and Responsibilities Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP032 To demonstrate that controller role and responsibilities (EC, PC) are clear and consistent during direct routings supporting specific traffic flows flight trials OBJ-LSD Table 57: Success Criteria Role and responsibilities - OBJ-LSD The purpose of this objective foresees the assessment of working methods and operational information/procedures in terms of i) any eventual change with respect to current situation and ii) trend of change (i.e. improvement or deterioration), if occurred. Relevant aspects asked to controllers are presented in the next figure and concerned: - Changing of coordination procedures: 69% of the ATCOs did not experience variation. Whereas, only 24% declared an increase in coordination procedures. With respect to EXE-LDS , the trend is promising. During City Pairs campaign the increase in coordination was exactly twice and the no-change was less than the one achieved in this trial session. Probably the publication of new FREE Solutions route and the traffic management according to segments/routes and not according to highlighted flights (that was instead typical of EXE-LDS session) were responsible for this positive trend. 120 of 213

121 Project Number LSD Edition Variation of shared information with other team members: it did not change for 71% of ATCOs, even if for 24% a rise in shared information was perceived. These scores confirmed initial hypothesis and trend achieved for the question previously discussed. Also in this case, the overall trend was better than the one registered during City Pairs trial. - Accuracy of information related to procedures and working methods: the majority (86%) of the ATCOs estimated that the information related to procedures and working methods was accurate and complete. (For information, only 7% of the ATCOs did not answer to this question). This trend is in line with the one of EXE-LDS Figure 64: Groundside HP objective HP032 In summary, collected scores confirm that roles and responsibilities from an ATCO s perspective are still clear and consistent, mainly due to the fact that no particular changes were experienced during trials. Operating Methods and Procedures Identifier Objective Supportive Objective Changing of coordination procedures Exercise Changing of shared information Accuracy of information related to FREES procedures Increase Decrease OBJ-LSD0105-HP036 To demonstrate that controller operating methods and procedures (EC, PC) are correctly defined to allow flight trials on direct routings supporting specific traffic flows to take place OBJ-LSD Table 58: Success Criteria Operating Methods and Procedures - OBJ-LSD Purpose of this objective is to complement evidence derived through a devoted brainstorming before the trials Error! Reference source not found. with additional more tangible one collected after the execution of the exercise. This objective mainly implied to check the ability to allow flight trials to fly the planned and agreed route. Positive scores were obtained for this topic: 86% of ATCOs were able to provide planned agreed route (Only 10% answered no and 4% did not answered to this question). Yes No 121 of 213

122 Figure 65: Groundside HP objective HP036 Adding to this result i) the unchanged coordination procedures, ii) the amount of shared information among team member quite stable to the current one and iii) the accuracy of FREE Solutions procedures, can be inferred that HP objective was completely assessed and that operating methods and procedures (EC, PC) were correctly defined. Controller Performance Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP038 To demonstrate that controller (EC, PC) performance is appropriate during direct routings supporting specific traffic flows flight trials OBJ-LSD Table 59: Success Criteria Controller Performance - OBJ-LSD Evaluate controller performance means the assessment of users perception with respect to different markers/dimensions that basically are: o Workload, Ability to allow flight trials to fly the FREE Solutions agreed route Yes No N/A o o o o o o Situational awareness, Tasks time execution, Communications and coordination, Trust in the concept, Job satisfaction in adopting the new operational concept, Potential errors induced by the introduction of the proposed concept As previously explained, some of these markers are interconnected among them, e.g. tolerable level of workload and ATC communications make the acceptability of new concept smoother and then allow the achievement of positive results in terms of task execution, job satisfaction and then trust in the new concept. Scores collected through questionnaires are hereafter reported according to the markers previously introduced. Workload Figures hereafter show controllers feedback regarding level of workload experienced during the trials. Within a scale from 1 to 10, ATCOs had to evaluate the elements making up the workload with respect to baseline operations, where 1 means less demanding, 5 means no change and 10 means more demanding. 122 of 213

123 Mean Workload Overall ATCOs Figure 66: Groundside HP Objective HP038 - Workload Overall Figure here above provides a general overview of mean workload. The histogram clearly shows that the use of Direct Routings did not influence the ATCOs workload. In order to collect information about workload as objective as possible, attention was posed to the investigation of 3 dimensions: Mental demand, i.e. supplementary cognition effort devoted to the execution of FREE Solutions trial; Physical demand, i.e. extra actions/tasks like more speeches or more interactions through HMI; Frustration, i.e. potential sense of irritation/stress/annoyance Additional mental workload Extra actions undertaken Feelings Figure 67: Groundside HP Objective HP038 Three main dimensions of workload Figure 67 shows that for the 3 main dimensions assessed, there is no significant change with respect to the baseline situation, meaning that controllers did not perceive significant variations (improving or deterioration of perceived load level). With respect to EXE-LDS , in case of Direct Routes scenario, the mental effort seemed to be better because decreased of one unit (from 6 to 5), same pattern in case of physical demand and feelings (from 6 to 5). This positive trend for workload contradicts initial worries triggered during preparatory brainstorming Error! Reference source not found.. In order to collect information about workload as objective as possible, the three dimensions previously assessed (i.e. mental demand, physical demand and frustration) were enriched through the assessment of ATC communication load. 123 of 213

124 Increase 60 Decrease No change 40 N/A Figure 68: Groundside HP objective HP038 - Communications Changes Figure 68 provides collected scores in percentage in terms of changes in ATC communications. Most ATCOs declared that the number of communications were unchanged. Most of users did not experience variations in terms of communications (80%, very high). This pattern is aligned with initial feedbacks collected during brainstorming sessions. Among those who perceived a variation, the prevalence declared a decrease in communications (13%) instead of a pejorative increase (only 05%). Probably the observable reduction of communications depended on the fact that new Direct Routes were published and then agreed among all partners (ground and air sides) with consequent increase of awareness about concerned trials and the no need of further communications/coordination. In summary, achieved results allow inferring that workload still remained at proper level without negative effects on ground performances of both ECs and PCs. Situational Awareness Situational awareness is the continuous extraction of environmental information, the integration of this information with previous knowledge to form a coherent mental picture and the use of that picture in directing further perception and anticipating future events. Within a scale from 0 to 6, ATCOs had to evaluate their level of situational awareness, where 0 means poor level, 3 means neutral and 6 means high level. Mean Situational awareness Overall ATCOs Figure 69: Groundside HP objective HP038 - Situational Awareness Overall Figure 69 clearly shows that on average ATCOs often achieved a good level of situational awareness. The rationale behind this trend tended to be assessed through a set of specific questions: The ability to anticipate traffic behaviour; The way they are not focused on problem/area of the screen; 124 of 213

125 Their evaluation that there is no potential risk to forget important information; Their ability to plan and organise activities according to their own preference; The unwanted surprise due to unexpected event; The no loss of time spent to search for info on the screen Figure 70: Groundside HP objective HP038 - Situational Awareness Considering that the range of value can vary from zero (poor level of situational awareness) to six (high level of awareness), the achieved scores are in general positive. This allows inferring that situational awareness did not change so much with respect to baseline conditions or, in case of some dimensions, achieved positive values (anticipate flights request, plan and organisation, no surprise due to unexpected event). This trend is aligned and coherent with the one of workload discussed in previous paragraph. Task Time Execution 0 Anticipation Not focused on problem No risk to forget Planification Another marker that could influence performance is the time/effort devoted to the execution of tasks. The figure hereafter shows achieved scores with respect to four topics deemed able to influence the execution of activities, i.e., - Traffic complexity (if changed and with which trend) - Sector capacity (if changed and with which trend) Not surprised - Conflict detection (if changed and with which trend) - Conflict resolution (if changed and with which trend) No search for information Traffic complexity Sector capacity 27 Figure 71: Groundside HP Objective HP038 - Task Time Execution Figure 71 provides positive results meaning that for the majority of ATCOs the Direct Routings did not affect the time devoted to the execution of tasks Conflict detection Conflict resolution Increase Decrease No N/A 125 of 213

126 Project Number LSD Edition As for complexity, more than half of users (65%) did not notice variations with respect to current situation, 28% experienced an increase. This increase is explained by free comments provided below. As for capacity, perception of no-change is stronger (78% did not notice variations), remaining parts were equally divided into increase of capacity (08%) and deterioration (08%). Some controllers reported that the increase of conflicts possibility could influence sector capacity in a negative way. Finally the two direct tasks were assessed, i.e. detection and resolution of potential conflicts. Basically unchanged conditions were registered for both tasks, i.e. 66% of no-variation in case of detection and 71% for resolution. Results are aligned with the ones of EXE-LDS Free comments provides an explanation quite similar to the one experienced during EXE-LDS : (a) The less time they have to operate due to shortest route; (b) The increase of conflict possibilities on the area (more potential crossing points) and; (c) The occurrence of new types of conflicts with which they are not used to deal with. These free comments explain the experienced increases in terms of traffic complexity and consequently conflicts detection and resolution. Summing up, new Direct Routes did not introduce significant variations in terms of tasks execution. Time devoted to them is quite close to the one necessary in current operations, thus without negative effects on users performances. Communications and Coordination Three ad-hoc questions were posed to controllers for the assessment of this topic, i.e. Changes in the coordination procedures; Changes in terms of information exchanged with the other team member; Changes in the amount of ATC communications (the same already treated for the topic workload ); Coordination Procedures Shared Information ATC Communications Figure 72: Groundside HP objective HP038 - Communications & Coordination The management of Direct Routings did not influence the current level of communications (both for ATC communications and information shared with other team members) and the coordination task. More than 50% of the ATCOs did not experience changes: they used the same coordination procedures as todays Moderate peaks in terms of growth were recorded for coordination procedures (24%) and information shared with other team members (24%). These peaks can be explained through free notes provided in questionnaire: (a) some DRs were closed to sector boundaries and had then requested further coordination among involved controllers to guarantee proper level of safety in managing the flights, (b) the unfamiliarity with routings, (3) some issues with OLDI (on Line Data Interchange) that induced an increase of phone usage and coordination in general. These 3 elements could impact the needs of coordination Increase Decrease No N/A 126 of 213

127 Project Number LSD Edition Summing up, no significant changes in communications and coordination occurred during trials. Trend of this dimension is aligned with the one of workload and situational awareness. Trust in the Concept Another marker able to influence the general controller performance is the trust in the new concept/route introduced through FREE Solutions. Relevant aspects investigated during trials were: Usefulness of new routes; Reliability of new routes; Controllers confidence while working with new procedures. Within a scale from 0 to 6, ATCOs had to evaluate these 3 topics, where 0 means poor level of trust, 3 means neutral and 6 means very high level of trust Usefulness Reliability Confidence Figure 73: Groundside HP Objective HP038 - Trust Based on Figure 73, it is possible to infer that: - ATCOs are often confident with new procedures related to this trial, - they often found direct routings useful, - they also found that direct routings were often reliable compared to baseline operations. This implies that controllers often trusted in the concept and in its rationale that had driven the design. Achieved scores are quite higher (i.e. much positive) than the respective one collected during EXE- LDS Job Satisfaction It addresses the human well-being, the cognitive comfort in executing the prescribed work. During trials this indicator was investigated at high level posing indirect questions, i.e. acceptability of the FREE Solutions agreed route considering the balance between gain on route length and workload. 127 of 213

128 Figure 74: Groundside HP objective HP038 - Operative Acceptability Figure 74 shows positive results of operative acceptability. More than 80% of the interviewees perceived as acceptable the compromise route gain/induced workload because directly experienced the efficiency and the overall benefit of proposed routes, against 10% that assumed a negative position. 86% of them declared being able to clear flights to follow planned Direct Routes and only 10% were not able. New routes revealed to be effective, applicable and then able to influence in a positive way the users acceptability related to operative concept. Nevertheless, some free notes revealed that a proper sector configuration is essential for an effective implementation of DRs and the subsequent acceptability by controllers. Thus, good level of satisfaction is supposed to positively influence users acceptability. Integration of results related to operative acceptability and trust allows inferring that proposed Direct Routes can be introduced easily in real operations, without jeopardizing the users attitude to the change. Potential Errors Purpose of questionnaire was also to assess that new routes did not induce errors in executing operations with dangerous side effects on controller performance and then on safety level of operations. This dimension was investigated considering two different point of sight: leading metrics like error, reliability of new concept (Figure 75) and, lagging properties such as operational aspects (Figure 76). Regarding leading metrics (no risk to forget and reliability) quite positive values were achieved. Considering that scale varies from 0 (negative feedback) to six (very positive), scores are close to four that means controllers often did not bump into the risk to forget important info related to the flight and often deemed the concept design reliable Acceptability Ability 4 Yes No N/A 128 of 213

129 Figure 75: Groundside HP Objective HP038 - Potential errors a Regarding lagging metrics, next figure sums up those operative aspects already investigated in other paragraphs but deemed strongly linked to assessment of errors induced in the users Figure 76: Groundside HP Objective HP038 - Potential errors b Based on results (summarized in Groundside HP Objective HP038 - Potential errors b Figure 76), we can conclude that new concept was designed and applied in a way that still allowed controllers the correct management of trials and baseline flights (82%) and according to a set of information deemed accurate (86%). Communications/coordination did not change for most of controllers (65%) thus operational conditions were quite close to the current ones. Nevertheless, free comments provided by ATCOs tended to show that sectors affected by mixed traffic (i.e. vertical movements and over-flights) did not allow a proper application of DRs, with effects on acceptability of new concept. In general, all these intermediate results allow inferring that new Direct Routes did not induce incorrect actions or errors in controllers, both ECs and PCs. Changes to Tasks and Roles Identifier Objective Supportive Objective 82 Abitlity to manage mixed traffic 0 4 Exercise No risk to forget Changes in ATC communication/coordination Reliability OBJ-LSD0105-HP040 To demonstrate that changes to controller tasks and role (EC, PC) are identified to allow flight trials on direct routings supporting specific traffic flows to take place OBJ-LSD Table 60: Success Criteria - Changes to Tasks and roles OBJ-LSD Accuracy of information related to FREES procedures 7 7 Increase Decrease Yes No N/A 129 of 213

130 Project Number LSD Edition Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP042 To demonstrate that changes to controller tasks and role (EC, PC) are managed during direct routings supporting specific traffic flows flight trials OBJ-LSD Table 61: Success Criteria - Changes to tasks and roles (2) OBJ-LSD These objectives are focussed on the assessment of potential changes to controllers activities and roles/responsibilities due to the execution of FREE Solutions trials, additionally to the detection and mitigation of potential errors. During EXE-LDS , no significant changes were recorded. Tasks and roles were mainly adherent to the baseline ones, due to the fact that the applied concept was a sort of improvement of current tracks. Additionally, the trial was managed according to a flight-oriented approach, i.e. specific flights affected by the experiment were highlighted and handled in accordance with FREE Solutions plan. In this exercise the approach was no more oriented to selected flights but to specific new waypoints making up new FREE Solutions routes. This aspect was highlighted during preparatory brainstorming Error! Reference source not found. in a certain way as potentially able to influence and change ATCO s tasks and then his/her actual role. Above elements were translated into devoted questions of questionnaire, already treated in previous sections. For the different arguments, ATCOs did not perceive any significant changes related to their tasks with respect to baseline situation. However, free notes provided some justifications important to underline in this section: a. The use of direct routings could provide less time to operate due to shortest route, b. The use of direct routings could imply to have more conflict possibilities on the area, reducing then the capacity, c. These conflicts are considered as new ones (controllers are not used to), d. As previously said, sectors affected by mixed traffic did not allow a proper application of DRs, with effects on acceptability of new concept, e. Some variations were observed in conflicts resolution: they used more often tactical vectoring, instead of shortcuts (thought intermediate route s point) Results referred to safety confirmed that during this trial the safety level of operations was still assured. In general, comparing results per different topics, no significant changes occurred at groundside in case of Direct Routes execution. Current roles and linked tasks revealed be stable and adherent to the nowadays situation. Nevertheless, some issues are underlined and need to be taken into account for the implementation of DRs in everyday life AUs Analysis As for the ATCOs, during the execution phase, a customized questionnaire was provided to flight crews involved in the trial. The format of questionnaire was intentionally similar to the one used during EXE-LDS in order to guarantee a coherence of approach in collecting results (i.e. HP objectives and related questions to submit to crews). Very few, minor changes were introduced in terms of terminology update (Direct Routes instead of City Pairs) and in terms of questions deletion because not still applicable considering the new scenario. Airlines outside the Consortium were involved in the trials due to the possibility to plan the new FREE Solutions Direct Routes, too. However they were not involved in the HP data collection. Decision was taken internally the Consortium. Rationale at the basis of this position has been the requirement to guarantee consistency and homogeneity of collected scores over the three bunches of the project. Extend data collection outside the Consortium would have introduced variability and heterogeneity of HP data due to the fact that i) airlines outside Consortium but interested to fly new Direct Routes were 130 of 213

131 Project Number LSD Edition not predictable before the bunch and ii) flight crews of these mentioned airlines were not prepared regarding the project and HP activities before the bunch. Contrary to EXE-LSD , the amount of flights actually involved in EXE-LSD was not predictable. City Pairs bunch was flights-oriented, that had meant to select and keep trace of specific set of flights. Direct Routes bunch was routes-oriented, that meant to identify and keep trace of new optimized segments (potentially flyable by all aircraft, belonging or not to the Consortium). For this Exercise the HP format was made available through ad-hoc on-line survey service to all flights and was asked to the crews actually involved in the bunch to provide feedbacks. A total of 386 fights were reported as measured from consortium airlines. Theoretically, it would have been implied the submission of the questionnaire to both PF (Pilot flying) and PM (Pilot monitoring) per each flight, and the subsequent collection of 772 questionnaires. Actually, 124 questionnaires were returned and were included in an aggregated way PF and PM feedback. Questionnaires were filled-in by pilots coming from three AUs of the Consortium, i.e. AZA, RYR and SWISS. AUs Overall Swiss 3 AZA 49 RYR Table 62: AUs Questionnaires - Direct Routings Scores are reported in the following subsections and organized according to the different Human Performance objectives linked to this EXE-LSD More details can be found in the devoted document Human Performance Assessment Plan [10]. Roles and Responsibilities Identifier OBJ-LSD0105-HP033 Objective To demonstrate that pilot role and responsibilities (PF, PM) are clear and consistent during direct routing supporting specific traffic flows flight trials Supportive Exercise OBJ-LSD Objective Table 63: Success Criteria - Roles and Responsibilities OBJ-LSD The coverage of this objective foresees the assessment of working methods and operational information/procedures in terms of i) any eventual change with respect to current situation and ii) trend of change (i.e. improvement or deterioration), if occurred. Eventual changes to these elements could affect roles and responsibilities of pilot due to their mutual dependence. Clear way of work/procedure influences the role of the actor and his/her boundary of action. Vice versa, unclear or confusing procedures may negative influence the working action of the user and compromise his/her understanding about limit of intervention. Relevant aspects asked to pilots were: routine working methods (if changed or not); Exhaustiveness of operational information preparatory for the trial. 131 of 213

132 Yes No N/A Figure 77: Airside HP Objective HP033 Roles and Responsibilities Results are quite aligned with the ones achieved through City Pairs bunch. Routine working methods remained unchanged for most of the pilots (i.e. 88%). Current way of work still remains effective and applicable for the crew operations also in case of Direct Routes. Only 05% of pilots declared a modification of working methods. Unfortunately this figure was not supported by free comments or explanations. Also the second topic confirmed the trend already experienced during City Pairs bunch. For most of the pilots the question was not applicable (i.e. 83%). Switching from EXE-LSD to EXE- LSD , positive feedbacks regarding suitability of information pre-flight increased from 07% to 15%. Collected scores confirm that role and responsibility for both PF and PM are still clear and consistent due to the non-change of current routine working methods and the operational information. Operating Methods and Procedures Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP037 To demonstrate that pilot operating methods and procedures (PF, PM) are correctly defined for flight trials to take place supporting direct routings on specific traffic flows OBJ-LSD Table 64: Success Criteria - Operating Methods and Procedures OBJ-LSD Suitability of crew operating methods and procedures represents another important marker for the HP assessment. These elements are complementary to roles and responsibilities topics. Assessment of PF/PM operating methods and procedures implied to check: 5 Modification of routine working methods adherence to planned FREE Solutions route; operational ability to perform fuel check during the flight as requested by current procedures; eventual change to routine working methods (if changed or not); availability of proper information to safely execute the trial. Collected scores are proposed in the following figure Exhaustiveness of operational information 132 of 213

133 Yes No N/A Figure 78: Airside HP Objective HP037 Operating Methods and procedures Positive results of first bunch were confirmed also for Direct Routes one. Crews did not experience particular difficulties in terms of flyability of new FREE Solutions routes, i.e. 92% of them declared actual execution of planned route. This score is slightly higher with respect the one achieved during City Pairs (where 87% were able to fly new routes). Only 06% of interviewees encountered some problems and were not able to perform the new route. The most frequent reasons were i) route deviation (due to traffic or adverse weather conditions) and ii) misunderstanding at first contact with ATCOs regarding trial route to be followed. Regarding the routine fuel check task, 92% confirmed be still able to comply with this activity most of the time. In situation like bad weather that implied deviation from planned route, crews were still able to check fuel consumption and argued that fuel consumption at the time of deviation was still lower than the one expected by standard/baseline route. Routine working methods were very rarely modified (i.e. only 05%). In most of the cases (88%) the current methods were still valid. Declared modifications were mainly referred to the need of manually upload the route on the Flight Management System (FMS). Nevertheless crews were aware that this disadvantage is a sort of bias related to the experiment and the fact that proposed Direct Route was still new for the planning system. Once FREE solutions route become operative, they can be easily found and select on FMS because already stored. Furthermore, information provided to the crews was deemed sufficient for the safe conduct of flight. For Direct Routes achieved score was 90% with a slight decrease with respect the one collected in the first exercise (that was 95%). Matching positive feedbacks from crews and the stability of operations during trial (i.e. quite adherent to the current ones), can be assumed that operating methods and procedures (for PF and PM) still remain clear and consistent. Pilot Performance Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP039 To demonstrate that pilot (PF, PM) performance is appropriate during direct routings supporting specific traffic flows flight trials OBJ-LSD Table 65: Success Criteria - Pilot Performance OBJ-LSD Evaluate crew performance means the assessment of users perception with respect to different markers/dimensions that basically are: o o Ability to fly planned FS Route workload (mental, physical); situational awareness; Ability to perform routine fuel checks 5 7 Modification of routine working methods 0 10 Sufficient information 133 of 213

134 Project Number LSD Edition o o o o tasks time execution; job satisfaction in adopting the new operational concept; load of ATC communications; potential errors induced by the introduction of the proposed concept. Some of these markers are interconnected among them, e.g. tolerable level of workload and ATC communications make the acceptability of new concept smoother and then allow the achievement of positive results in terms of task execution, job satisfaction and then trust in the new concept. Workload Figures hereafter reports pilots feedback regarding workload experienced during the trial. In details, topics proposed to crew were: any eventual changes of workload level, variation of performance due to workload variation, typology of workload mainly experienced Yes Increase 39 Decrease No N/A Differences in workload levels Performance alteration by increased workload Figure 79: Airside HP Objective HP039 Workload Workload level did not change for half of the users (52%). This result confirmed initial hypothesis collected during brainstorming preparatory to the trial Error! Reference source not found. where participants declared that cognitive workload would have been quite similar to the current one because of availability of published routes (i.e. stable and known by all involved actors). For those who have experienced a reduction (39%), free comments gathered at the end of trials confirmed this trend due to (i) a better organisation of work (i.e. less input on FMS because direct clearance already planned and uploaded) and (ii) a less frequency changes (i.e. silent cockpit). Mental load decreased thanks to the reduction of communication with ATCOs, too. All these aspects influenced in a positive way performance of pilots because they could better focus on the aircraft management with subsequent reduction of operational fatigue. Only 03% suffered an increase of workload. Some crews reported that this increase occurred during ground operations due to the manual upload of FREE Solutions route on the FMS. They were aware that this was a bias related to the novelty of proposed routes. The experienced level of workload did not dramatically affect pilots performance. Half of users deemed the question not applicable, 37% declared to changes and only a 13% suffered alteration of their performance due to increase of load. 134 of 213

135 Mental Physical Physiological Other N/A Figure 80: Airside HP Objective HP039 Typology of workload Regarding the typology of workload, most of the pilots deemed the question not applicable. Of the remaining scores, the mental effort was confirmed the most affected one (i.e. 22%) as happened for EXE-LSD Matching results collected through the different questions, workload appeared unchanged or slight decreased (compared to current operations). Proper levels of crew performance were still assured during the trial. Situational Awareness Unfortunately, due to the questions provided to crews, it was not possible to provide significant information about Situational Awareness. Tasks time execution & Job satisfaction Investigate this dimension implied the assessment of those elements deemed by AUs sensitive to the new concept, i.e. execution of proposed new route, check of intermediate fuel consumption, comparison between final remaining and final reserve fuel, perceived efficiency of operations Yes No N/A Ability to fly planned FS Route Ability to perform routine fuel checks Final remaining fuel>final reserve fuel Figure 81: Airside HP Objective HP039 - Task time execution Very positive results were achieved for the first three topics. Most of the pilots were able to actually execute FREE Solutions route (92%) and to perform on-board routine activities, i.e. intermediate fuel check (92%) and comparison remaining/final reserve (94%). 0 9 Perception of flight efficency improvement 135 of 213

136 Project Number LSD Edition Regarding efficiency of operations, scores were spread out between positive perception of improvement (58%) and negative one (32%). Finally, due to the questions administrated to crews, it was not possible to provide significant information about Job Satisfaction. Results allow declaring that tasks/operations were successfully executed as planned before the trial and respecting routine nowadays on-board activities. Load of ATC communications Communications with ATCOs could influence overall pilots performance. Specific question was devoted to the investigation of this aspect. Figure hereafter reports users perception regarding potential changes of communications load Increase Decrease No N/A Perception of change in number of ATC communications Figure 82: Airside HP Objective HP039 - ATC communications Half of interviewees experienced a decrease of communications (56%), whereas the reaming part declared no changes with respect to current operations (40%). This trend is exactly opposite to the one registered during EXE-01. During EXE-01 pilots mainly perceived no change of communication load (53%) and a bit reduction (43%). In case of Direct Routes, mainly a decrease that a no-change. This means that Direct Routes scenario introduced marked positive effects on communication load that were clearly perceived and declared by airside users. Free comments provided by some pilots confirmed this statement. They declared less frequency changes (i.e. silent cockpit) and reduction of communication with ATCOs thanks to the provision of planned better route, the direct one. Potential Errors Scope of questionnaire was also the assessment that Direct Routes scenario did not induce errors and misunderstanding in the operations execution. Questions administered to pilots were mainly focused on: potential safety effects due to changes of workload level, potential safety effects due to modification of working methods, acquisition of any information deemed proper for the safe execution of trial. Figure hereafter shows subjective scores. 136 of 213

137 Figure 83: Airside HP Objective HP039 - Potential errors For the first topic, half pilots declared no effects (50%) and the other most ones deemed the question not applicable (48%). Only 2% revealed that the safety was compromised by workload variations but it was mainly due to the manual upload of FREE Solutions route on the FMS. They also indicated that they were aware of this bias related to the novelty of proposed routes. For the second topic the users were quite aligned in declaring the question out of scope (77%). Furthermore, the information provided to crews was considered sufficient and appropriate for the safe execution of trial (90%). New Direct Routes did not induce crews in performing wrong operations or errors, also because operating methods and procedures were still quite similar to the current ones (as occurred for EXE-LSD ). Changes to Tasks and Roles Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP043 To demonstrate that changes to pilot tasks and role (PF, PM) are managed during direct routings supporting specific traffic flows flight trials OBJ-LSD Table 66: Success Criteria - Changes to Tasks and Roles (2) OBJ-LSD This objective is the final one related to the assessment of airside HP aspects. It is devoted to the identification and of any potential changes to crew activities and roles/responsibilities due to the execution of FREE Solutions routes. This objective is very close to the ones already discussed in previous sections and therefore it represents the closure, the wrap-up of overall airside assessment. It implies the verification of those elements declared by pilots potentially affected by a change, i.e.: Safety compromised by workload variations procedure of routine fuel check during the trial; routine working methods; operational information related to roles & responsibilities to be adopted during the trial; safety level of operations. Figure hereafter is mainly focussed on the first three aspects reported above. 23 Safety compromised by working methods alteration 0 10 Provision of sufficient information Yes No N/A 137 of 213

138 Yes No N/A Figure 84: Airside HP Objective HP043 Tasks and Roles No significant changes occurred to airborne operations. Current way to perform routine fuel check was successfully applied in most of the cases (92%). Routine working methods did not change for 88% of pilots. For the 5% who experienced a modification of routine working methods, it was due to the novelty of proposed routes and the manual upload of FREE solutions route on the FMS. Operative information was deemed still exhaustive and not impacting crew roles and responsibilities. No significant changes were then experienced and reported by crews of different airlines. Furthermore, potential for pilot errors were not registered. During trials the level of safety was still assured as occurs nowadays (See results related to potential errors in the subsection above). Cases of safety degradations did not occur during trials. Pilots did not experienced errors or misunderstanding induced by the new Direct Routes General Results 4 Ability to perform routine fuel checks 4 The collected recommendations and the performed formal and informal peer reviews contributes to the satisfaction of the following demonstration objective OBJ-LSD OBJ-LSD To give input to airspace community interested in DR/FR operations Success Criterion Results Production of a list of recommendations for the Fully achieved, a list of FAB community recommendations is provided in the dedicated sections of this document Formal feeding of OFA and OFA with FREE Solutions results Production of a list of recommendations to update/improve existing regulations/standards in support of Free Routing operations Table 67: Success Criteria - OBJ-LSD as results of live trials activities Fully achieved due to provision of a list of recommendations, participation to joint meetings and active OFAs involvement as external document reviewers No specific need to change/improve existing regulations/standards highlighted Results impacting regulation and standardisation initiatives Results impacting regulatory and standardisation activities should be analysed to present the needs for Regulation and/or Standardisation. Analysis of Exercise Results Unexpected Behaviours/Results 5 Modification of routine working methods Exhaustiveness of operational information 138 of 213

139 Project Number LSD Edition Different issues were reported by several members from the consortium (AUs, ANSPs, CSFP) and are detailed hereafter. For Airlines, it appeared that: o o o An issue related to the rolling UUP every 1 hour, a procedure had been implemented asking the dispatch to retry the DR routing one hour before EOBT to take the last opportunity. But, LIDO, instead of working as foreseen, was considering the DR routing always open. So REJ were received and when recalculated, another REJ appears. Decision was taken to suppress the new procedure and not recalculate for opportunity, just check REJ as usual; UUP checking had to be performed manually and it was required to be performed every hour. As there were different routings for every destination flight dispatcher tasks were not easy as it was hard to filter all the UUP entries for their appropriateness for several flights and associated routings; An issue related to cost effectiveness: Sometimes, there was no benefit, because it was impossible to climb or descent during the DR portion. As a matter of fact, it is enough to make the DR more costly. For example, the ARKIP-LMG DCT. For ANSPs, it appeared that: o o o Some DRs, close to ACCs boundaries, were not integrated in the neighbour system, when it is not a COP (e.g. KUMIN-OTKOP). These DRs should be integrated in the neighbour s system. Some inconsistencies between ATC plan and FMS plan were underlined. For example, for three flights using the DR TINOT-NIVDA-TABOT, an inconsistency was underlined between the SID ATC (SOSUR) and the SID pilot (TINOT). Some problems related to Flight plans with odd FL: the first day of the trials (21st of November), ¾ of the aircrafts with the DR VANAS GODRA DITON had odd FL over KINES or VANAS instead of even FL. In the RAD it was well specified "even" FL. Eurocontrol has been advised immediately and then the problem was solved without having any further information about it Quality of Demonstration Results As it may be verified in the other sections of this chapter, the approach adopted was a combination of FPL information and actual track data, which gave as an output high quality results compared to the conventional approach used when performing RTS simulation activities The savings in terms of distance, duration and fuel burn are consistent when measured from the NM or the AUs' perspectives. Moreover, the assessment of real trajectories confirms the actual improvement that was expected on the flight plans basis. The results are lower but this is in line with SES established flight efficiency indicators. As trials took place during a low traffic period and the number of trial flights was too low to create a significant effect on ATC operations and controller workload, no impact on capacity was expected. However, the fact that sector occupancy was not affected brings confidence in the feasibility of the designed solution at higher traffic rates. During screening of questionnaires and subsequent delivery of statistics an anomaly was observed. For some questions of filled-in templates, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve the full score (i.e. 100%) through the sum up of afferent intermediate quotes. As a final effect, a margin of uncertainty is caused and induced subsequent effects, first on the analysis of data and secondly, on the reporting. However, this uncertainty can be classified as not severe Significance of Demonstration Results The number of flight trials is quite significant, and the operational environment unchanged. It guarantees the confidence in the results published. 139 of 213

140 Project Number LSD Edition Conclusions and recommendations Conclusions This section summarizes results achieved with the execution of the FREE Solutions live trials concerning Direct Routings. Main conclusions are drawn from the results per KPA reported in section of this document and are listed below: CONC 1. Compared to bunch 1, the possibility to open available DRs to flights out of the consortium enhanced the expected benefits. CONC 2. Results tended to prove that the following DR [ARKIP-DCT-LMG] could be considered as a quick win. Its implementation is still under assessment and should be published in RAD, Appendix 7 (FUA Restrictions). CONC 3. For ENAV, DRs live trials represented an intermediate step toward Free Routing implementation. It allowed pre-viewing traffic pattern distribution within existing airspace sectorization and with current ground system, giving confidence to keep working on this thread. Operational impact of DRs operations was positively assessed and as a consequence expected implementation gained operational staff support. Delivered benefits can be summarized with the following measured values, which provide readers with the delta between the live trials operational scenario and the baseline scenario. Trial Flights Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) Recommendations Table 68: DRs Weekend delivered benefits This section is devoted to the list of proposed recommendations for planned FREE Solutions live trials exercises. REC 1. DR live trials confirmed the need to plan intermediate points to allow fuel check operations as previously stated. REC 2. To facilitate the CSFP calculation and according to the length of the proposed DRs, it is necessary to insert a new point in the middle of the DR to allow flights to climb or descent. REC 3. ANSP ground systems would benefit from dynamic COPs to allow for cross-border DRs segments not constrained by existing COPs REC 4. It is necessary to integrate in ground systems the proposed DRs by neighbours when these DRs are too close to the boundaries and especially when the entry/exit point is not a COP. REC 5. Future design of DRs close to sector boundaries should be avoided in order to not have high amount of not concerned traffic close that boundaries which may trigger the need for additonal tasks/coordination by controllers 6.4 Demonstration EXE-LSD Report Exercise Scope In this specific exercise, Direct Routings were identified in support of specific traffic flows. The selected DRs intercepted restricted/reserved airspace structures. The negotiation among FMPS, AMCs, military Approved Agencies and NM had defined the allocation of restricted/reserved airspace structures. Adequate notification to airspace users were ensured through the AUP/UUP process. 140 of 213

141 Project Number LSD Edition DR possibilities were available for trial flights operated by AUs participating in the FREE Solutions project and organised for the purpose of the project activities. FPL routings of trial flights had to contain at least one FREE Solutions DR as designed in the context of WP3 activities Exercise Scenario Figure 85: DRs weekdays - SCN-LSD Exercise scenario SCN-LSD refers to Normal day of Operations during week days, which means any period of time starting from Monday morning at 07:00 UTC up to Friday night at 23:00 UTC. Scenario is applicable to En-Route phase and to the consortium airspace. It is related to the execution of cross-accs Direct routings. Participating ACCs in the DR weekday trial included Reims, Bordeaux, Marseille, Geneva, Zurich, Milan, and Padua ACCs. Aircraft flew in accordance with existing operational procedures in controlled airspace. In the same way, for involved ACCs operations were ensured through existing working procedures. This scenario is applicable to DRs interfering with reserved/restricted airspace structures. Their availability was the result of coordination with national AMCs, military Approve Agencies and NM, properly notified to airspace users through the AUP and UUP process for any update of the airspace status. The coordination and notification process focused on airspace activations and associated restrictions to inhibit flight planning through military training areas (as opposed to today s common practice to work with route availability rather than airspace availability). When using airspace activations in AUP/UUP, ANSPs and AMCs have to ensure that the publication of those airspaces used for dynamic activation via AUP/UUP includes the complete airspace volume relevant for flight planning (including flight planning buffer zones, FBZ, if any). The Network Manager ensures a timely collection and distribution of the airspace status updates sent by participating ANSPs. Based on this information AUs participating in the trial updated their flight plans in order to take advantage of effective airspace availability (or to avoid airspace recently activated) Exercise Objectives This exercise addresses the demonstration objectives and related KPAs listed in the table below. 141 of 213

142 OBJ ID OBJ Description KPAs and TAs OBJ-LSD OBJ-LSD OBJ-LSD OBJ-LSD OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized medium-long haul traffic in terms of efficiency/environment sustainability To give input to airspace community interested in DR/FR operations To demonstrate the possibility to correctly calculate, file & process flight plans based on airspace availability rather than route availability To demonstrate that an airspace management tailored on AU needs along with an improved cooperation with AMC can contribute to optimize the cost for airlines as well for air navigation services To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably safe Efficiency; Human Performance; Airspace Capacity - En- Route. Predictability and ATC Planning General N/A General N/A Efficiency Safety Human Performance Table 69: EXE-LSD Demo Objectives Conduct of Demonstration Exercise EXE-LSD Exercise Preparation Pre-trials Analysis To assess the potential impact of the proposed DRs usage the following approach was undertaken: Identification of the military areas crossed by proposed DRs; Traffic crossing areas before DRs implementation was measured; Traffic crossing areas on new trajectories containing proposed DRs was measured; The two traffic samples were compared in terms of number of flights crossing the areas, time of the day and potential gains. These criteria led to the selection of the DRs summarized in the table below (i.e. SCN-LSD ). 1 FJR BALEN FL245 FL660 2 GTQ BEGAR FL245 FL660 3 LASAT RESIA FL245 FL660 4 RESIA BABAG FL245 FL660 5 VANAS GODRA FL250 FL660 6 GODRA DITON FL250 FL660 7 TOBSO BZO FL290 FL460 8 BZO ETPUB FL290 FL460 9 ETPUB TALEP FL290 FL of 213

143 ARKIP LMG FL250 FL ERIXU VALKU FL250 FL660 Table 70: Selected DRs weekdays The evaluation of potential performance gains (based on DDR and NEST tools) showed that AUs could save 2397 NM (average 7.7 NM/flight) using proposed DRs (See table below). Number of flights Total distance (NM) Reference ,539 Using proposed DCTs ,142 Table 71: DRs weekdays potential benefits evaluations Flight Planning The Weekday DRs tested for the ASM scenario were planned and flown according to the effective availability of the route segment affected by the segregated area. The updating of the segregated area availability status was executed by rolling EUUP, issued every 1 hour; as a consequence, a specified Direct Route segment became available if the correspondent segregated area was not active. The Weekday DRs tested for the ASM scenario were planned and flown according to the effective availability of the route segment affected by the segregated area. The updating of the segregated area availability status was executed by rolling EUUP, issued every 1 hour; as a consequence, a specified Direct Route segment became available if the correspondent segregated area was not active The EUUP and restricted area allocation EUUP was published every hour, exactly on the hour, and its publication could be verified in the Network Operation Plan (NOP). For DRs whose availability was subject to one or more segregated areas AUs had the responsibility to verify which segregated area(s) affected a specific DR Figure 86: Examples of segregated airspace being released by EUUP during trial period The size of the balls in the figure above is directly dependent on the amount of EUUP updates published after receiving national UUP updates. The ones with the label highlighted are the ones referring to the segregated areas part of the EXE-LSD Exercise execution 143 of 213

144 Project Number LSD Edition The exercise EXE-LSD was executed in the week from Monday 23 rd to Friday 27 th of November The Direct Routes listed in the section were part of the ASM scenario and their actual availability is described below: GTQ BEGAR was available for ASM scenario from 23rd to 27th of November, from 18:00 Z to 23:00 Z updated with rolling UUP every 1 hour until 20:00 UTC. ARKIP LMG was available for ASM scenario only for City Pair LFPG-LFBD, from Monday to Friday during all the AIRAC 1512 cycle, updated with rolling UUP every 1 hour. ERIXU VALKU was available for ASM scenario only for City Pair departing LFPG and arriving DIAP, DNAA, DNMM, FCPP, FKKD, FOOL, FZAA or GABS, from Monday to Friday during all the AIRAC 1512 cycle, with rolling UUP every 1 hour. BEGAR ODINA, VANAS GODRA and GODRA DITON were available for ASM scenario from Monday 23rd to Friday 27th updated with rolling UUP every 1 hour. RESIA BABAG, TOBSO BZO, BZO ETPUB, and ETPUB TALEP were available for ASM scenario on Friday 27th, updated with AUP on Friday morning. The detailed time activation and arrival/departure restrictions were published in the EUROCONTROL NM Route Adhere Document (RAD) Special Events FREE Solutions Flight Trials Deviation from the planned activities EXE-LSD suffered from Reims ACC strike which directly and indirectly affected live trials activities. This represented the major deviation for this exercise limiting the possibility to plan and then execute some of the available DRs segments. Air Traffic Controllers industrial action in Reims ACC between 23 rd and 27 th November generated 65,000 minutes of ATFM delay and required protective measures at Maastricht and Karlsruhe neighbouring ACCs (5,500 minutes) Figure 87: Top 20 Delay Locations for en-route Delays in November 2015 Additionally, not sufficient data were collected for the EXE-LSD to allow the execution of a separate SAF and HP analysis, ending in a common analysis, for both weekends and weekdays, reported in the related sections of the EXE-LSD Finally, EXE-LSD was open to all flights instead of consortium airlines only as initially planned and reported in [1] Exercise Results Summary of Exercise Results Results per KPA Environment Fuel Efficiency OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized medium- 144 of 213

145 long haul traffic in terms of efficiency/environment sustainability Success Criterion Results Efficiency: Positive impact on fuel saving and Achieved 12 CO2 emissions Table 72: Success Criteria Environment-Fuel Efficiency OBJ-LSD KPA KPI Metric Results Environmental Sustainability Time saved Delta Minutes FPL(ref) v FPL (scn) and Fuel -233mins Efficiency CPF(ref) v CPF(scn) -251mins Distance saved Delta Nautical Miles FPL(ref) v FPL (scn) nm CPF(ref) CPF(scn) -1596nm v Flights planned flight planned to conduct the trial solution Number flights of 208 FPL(ref) v FPL (scn) Flights that have actually flown the trial solution Number flights of 328 CPF(ref) v CPF(scn) Fuel burn saved enroute Delta Kg FPL(ref) v FPL (scn) kgs CPF(ref) CPF(scn) -5683kgs v CO2 saved en-route Delta Kg FPL(ref) v FPL (scn) kgs CPF(ref) CPF(scn) kgs v 12 Benefits achieved are to an extent which is less than expected due to Reims industrial action during live trials weekdays 145 of 213

146 NOx saved en-route Delta Kg FPL(ref) v FPL (scn) -965kgs CPF(ref) CPF(scn) -362kgs v Table 73: Environment-Fuel Efficiency Results - Metrics and Measures Weekdays Direct Routings The analysis of the CPF(ref) and CPF(scn) data shows 328 flights which planned benefit from DRs availability during EXE-LSD with an average distance saving of4.9nm/flight. The average distance saving per flight is not only due to DRs usage but it also depends on other variables that may positively or negatively affect results like, for example weather (wind, temperature) and ATC interventions due to existing traffic situation. However, it must be pointed out that the average distance saving result is a relevant one spreading benefit at network level. Those benefits may be further augmented if geographical scope of such operations is further enlarged and the time window expanded Airspace Capacity En-route OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized mediumlong haul traffic in terms of efficiency/environment sustainability Success Criterion Results Capacity: no negative local and network impact Promising results. No Negative effect Table 74: Success Criteria - Capacity OBJ-LSD KPA KPI Metric Results Airspace Capacity (enroute) ATFCM delay Minutes None attributed to DRs Sector occupancy ACC throughput Number of aircraft per hour Number of aircraft per hour Change not considered significant enough to measure quantitatively Change not considered significant enough to measure quantitatively Sector workload Delta % points Change not considered significant enough to measure quantitatively Conflicts Number of potential conflicts Change not considered significant 146 of 213

147 per sector enough to measure quantitatively Table 75: Capacity Results - Metrics and Measures Weekdays Direct Routings Predictability and ATC planning OBJ-LSD To demonstrate the advantages of using new DR catching the most penalized mediumlong haul traffic in terms of efficiency/environment sustainability Success Criterion Predictability: Positive impact on flight punctuality and on ATC planning Results Not Achieved Table 76: Success Criteria Predictability and ATC Planning OBJ-LSD KPA KPI Metric Results Predictability Actual en-route time duration vs. planned en-route time duration Delta Minutes An expected reduction of the delta time between reference and scenario examples, indicating a more predictable scenario, was not demonstrated in the data analysed. This may be due to wind or other influencing factors Table 77: Predictability and ATC Planning Results - Metrics and Measures Weekend Direct Routings Civil-Military Cooperation and Coordination OBJ-LSD To demonstrate the possibility to correctly calculate, file & process flight plans based on airspace availability rather than route availability Success Criterion Results Airspace volumes to be Fully achieved avoided, together with their associated activation periods, and level bands, are correctly communicated and processed by all stakeholders (AMCs, ANSPs, military authorities, aircraft operators, network manager). The Majority of all flight plans pass IFPS checking automatically or the majority of selected flights files the FPL in accordance to the availability of identified DRs Partially Achieved, some unexpected behaviours were reported in the section Table 78: Success Criteria Civil-Military Cooperation and Coordination OBJ-LSD of 213

148 OBJ-LSD To demonstrate that an airspace management tailored on AU needs along with an improved cooperation with AMC can contribute to optimize the cost for airlines as well for air navigation services Success Criterion Results Reduction in cost per single flight (Fuel/Tons and Time/Minutes) Optimized airspace allocation from AMCs Achieved, but to a less extent than expected due to Reims industrial action which forced re-routing of some flights and delays Fully Achieved Table 79: Success Criteria OBJ-LSD KPA KPI Metric Results Civil-Military Updates of EAUP by Number of - 5 days Cooperation and Coordination EUUP updates reference period: 7 UUP updates in 4 days (1 to 3updates / day) Availability of TSA/TRA to GAT per day Hours minutes and - 5 days trial period: 2 UUP updates in 2 days - 5 days reference period: booking time 140h 45min released 22h (16% from booked) TSAs availability: 82%+3% release - 5 days trial period: booking time 127h 15min released 16h30 (13% from booked) TSAs availability: 84%+2% release 148 of 213

149 Average AO reaction time interval to re-file FPL following UUP before EOBT Hours Minutes and No results partly due to Reims ACC industrial and mainly to consortium AOs unable to analyze EUUP data and react to airspace availability changes by changing flight plans. To this end a specific recommendation was inserted in the section Table 80: Civil-Military Cooperation and Coordination - Metrics and Measures Weekdays Direct Routings Safety Questionnaires collected during Direct Routes trials that included both EXE-LSD and EXE- LSD , did not allow the discrimination of information with respect to the two different exercises. Both airside and groundside collected scores were essentially referred to EXE-LSD and then discussed in the dedicated section. No dedicated Safety results are then available for EXE-LSD Human Performance As for the Safety section, there are no HP results available for EXE-LSD General Results As per SAF and HP, these aspects are addressed in the section of EXE-LSD Results impacting regulation and standardisation initiatives Nothing to be reported in this paragraph Unexpected Behaviours/Results During this exercise the design of GTQ-BEGAR as both CDR and as FREE Solutions DR led airspace users to receive flight plans rejections when containing that DR. As a result some users were forced to re-file their routes until the situation was resolved leading to a loss of traffic on this DR. In addition, the usage of VANAS-GODRA-DITON DR was affected by system limitations. The trial showed that it is not possible to check the status (active / inactive) of more than one military area modules when creating certain restrictions in the RAD (e.g.: "Whenever military area modules A & B are both inactive, DR X-Y shall be compulsory for certain flights"). This system limitation prevents from moving from route availability to airspace availability. Actually, it was not possible to define a compulsory restriction on a DR crossing several TRA modules. During the trial, an instruction to IFPS operators to manually ignore errors in certain cases was given and a manual workaround was performed (i.e. operator had to manually ignore RAD errors). Obviously, this contingent situation is achievable during a limited period such as EXE-LSD , but it is definitively not applicable during daily operations as it is prone to errors. This led to a request for system update to Eurocontrol Network Manager. Impossibility to use the ERIXU-VALKU DR, as not permitted by the system. In fact, LIDO system considered that route segment closed during weekdays even if was always available. 149 of 213

150 Project Number LSD Edition Finally, AUs reported the difficulty to plan properly in the ASM scenario. In fact, during week days, ASM had some difficulties when assessing flight in the plan preparation and definition some days before trial. Therefore the number of flights previously considered to be part of this scenario was lower than expected Quality of Demonstration Results Nothing to be reported in this paragraph Significance of Demonstration Results Nothing to be reported in this paragraph Conclusions and recommendations Conclusions CONC 1. Increased availability to GAT of the concerned TSAs/TRAs was greater in the trial week than in the reference period, resulting in additional benefits delivered. This triggers the conclusion that increased coordination and exchange of mutual information among involved partners is crucial for the setting up of efficient and stable flexible use of airspace operations (see section for further details); CONC 2. Within Italian airspace a new approach was adopted concerning improved use of existing TSAs/TRAs. Those areas are now available for civil use unless specified that are used for military purposes. The previous approach adopted the other way around: airspace first inaccessible and then released for civil usage; CONC 3. Usage of available DRs show that the system of FBZ, area activations and corresponding FUA restrictions for flight planning worked well during live trials. To summarize, measurement from the entire week of trials with the average distance savings and the related benefits achieved are reported in the table below. Any additional detail for each single DR is provided in the detailed assessment as reported in the table hereafter. Scenario Economy for week days (23-27 Nov 2015) all DRs traffic Total flights on DCT Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) 23 November November November November November TOTAL week 150 of 213

151 Table 81: DR Weekdays and Final Total Overall average distance savings: 8.92 NM. This number is a significant number that provide hints for the potential benefits that may be obtained by increasingly making available TSA/TRA as a results of continuous civil-military cooperation, considering that such result is obtained per one week of trial only Recommendations REC 1. The NM system should allow to check the status (active / inactive) of more than one TSA/TRA module when creating restrictions in the RAD; REC 2. Flight Planning systems supporting flight dispatcher should be capable of associating a DR segment to a TSA/TRA area when released. With current systems DRs are assigned from point to point. REC 3. Taking into account current flight planning systems limitation and to avoid manual modification of flight plans, any rejection should be provided to AUs an hour and a half prior EOBT, instead of an hour. This would allow to respect the threshold of an prior EOBT to communicate to ANSP any flight plan cancellation REC 4. ANSP ground systems would benefit from dynamic COPs to allow for cross-border DRs segments not constrained by existing COPs REC 5. ASM process would benefit from additional indicators to assess civil-military coordination and cooperation. Analysis of EAUP and EUUP in the trial did not provide complete and comprehensive results. 6.5 Demonstration EXE-LSD Report Exercise Scope The aim of the FREE Solutions demonstration activities was to allow airspace users to file and execute optimized FPL routings. In this specific exercise as Free Routings, optimized routings were made available for flight trials in a defined Free Route Airspace. In phase with ERNIP Part 1 European Design Methodology, Free Route Airspace (FRA) is defined as A specified airspace within which users may freely plan a route between a defined entry and a defined exit point, with the possibility to route via intermediate way points, without reference to the ATS route network, subject to airspace availability. Within this airspace, flights remain subject to air traffic control [6]. The airspace subject to the FREE Solutions project formed an integral part of the overall European ATM network. Flights entering and exiting the FREE Solutions airspace across its lateral or vertical boundaries had to file at least one Entry and Exit Points. FR possibilities were made available for all flights and not limited to the consortium flights only 13. In line with the airspace design solutions and publication, during the FRA trials in MAR 2016, AUs were able to optimize route selection in the applicable area, and thereby obtained benefits from choosing and planning routes according to their needs (i.e. flying their optimised D-day route, which could be the shortest route, cheapest route, the fastest route (tailwind) or any other route). The Airspace Users were free to define UPRs taking into account the network constraints already defined and shared. For detailed information, please refer to the FREE Solutions FRA Trial CONOPS V FREE routing task force, established at project level, defined a set of criteria to be applied to determine eligibility of flights entering FREE routing area 151 of 213

152 Project Number LSD Edition Exercise Scenario Exercise scenario SCN-LSD referred to Free Routing Area availability during weekends. The FREE Solutions FRA trial was conducted during one single weekend (during the 5 th and 6 th March 2016) starting from Saturday at 01:00 (UTC) up to Monday at 01:00 (UTC). To allow late flights and provide more flexibility to AUs, a Free Routing transition period was set-up in which Free Routing flights may be executed, but may not be planned. The length of this transition period was 30 minutes. The operational environment was composed by a mix of FRA and DRA airspace. The airspace considered for the trial encompasses the following areas: - Geneva and Zurich ACC Area of Responsibility (AoR) combined above FL365 - Roma/Milano/Padova/Brindisi ACC AoR combined above FL365 - All Malta ACC AoR combined above FL365 - D/Z sectors of Marseille ACC combined above FL195 In this FRA (Italian and Malta airspace more information are provided in the section for Aix and Skyguide Airspace) users were free to plan optimized routes starting from a specified entry point up to a specified exit point (i.e. the current Coordination Points COPs published at the boundaries of the trial airspace), via intermediate waypoints (published inside the FRA). These Intermediate Points and the COPs had to be used by AUs for flight planning in order to: - climb or descent through the vertical limit of the FRA in order to join or depart airports below (cf vertical transition to and from the Free Routing area) - optimize their profile - circumnavigate a particular area - indicate a change in flight level or speed - remain compliant with FRA areas boundaries rules (DCT shall remain inside the FRA area) Flight planning were performed using existing waypoints and current working methods were applied to pilots and controllers. Only waypoints or Nav Aids were used, Lat/Long points were not plannable. There was no requirement to define Arrival and Departure Points. AUs chose the most appropriate Intermediate Point to join or leave the ATS network (or the FRA). Eligible flights that make a change in cruising flight level resulting in a transition to/from FRA had to file the portion of the flight outside FRA along the standard ATS route network. Definition of ad hoc procedures agreed between single ANSP and AUs were possible to ensure safe transition. Concerning connectivity with fixed ATS route network, Entry/Exit Points into/out Free Routing operations airspace had to take into account adjacent airspace where Free Routing operations airspace was not implemented. Entry/exit Points were defined to allow for a structured transition between the two operational environments, and not necessarily at the FIR or ATC unit boundary. The vertical transition with the fixed ATS route network used published Intermediate Points. Coordination rules were not modified but if needed, specific coordination procedures had to be applied. If needed, planner controller could coordinate via phone with interested ACCs every Free Routing flight reporting the following information: clearances issues and OLDI activation point. It was the responsibility of each FREE Solutions ANSP to ensure connectivity with neighbouring ANSPs airspace not involved in the project. To minimize the impact of trials operations on existing operational procedures it was agreed to keep the already existing RAD restrictions and FLAs applied with neighbouring ACCs. All flights entering the defined airspace were eligible for FRA trial operations within the horizontal and vertical limits of the applicable airspace. 152 of 213

153 Exercise Objectives Figure 88: EXE-LSD Involved ACCS This exercise addresses the demonstration objectives and related KPAs listed in the table below OBJ ID OBJ Description KPAs and TAs OBJ-LSD OBJ-LSD OBJ-LSD To demonstrate the feasibility of FR operations on Cross Border Areas To give input to airspace community interested in DR/FR operations To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably safe Efficiency Capacity Predictability and ATC Planning Human Performance General N/A Safety Human Performance Table 82: EXE-LSD Demo Objectives Conduct of Demonstration Exercise EXE-LSD Exercise Preparation For assessment of the FREE Solutions - FRA Airspace Design, the System for Assignment and Analysis at a Macroscopic level (SAAM) modelling tool has been used. It permits to assess quantitative information in support of the development of the airspace structure, route network and sectorisation. Results obtained from SAAM can refine the requirement for fast-time or real-time simulations. 153 of 213

154 Project Number LSD Edition Airspace Design Swiss Airspace: SKYGUIDE In the defined airspace involved (LSAZ Zurich ACC and LSAG Geneva ACC), skyguide proposed a limited number of new DCTs starting from different low levels to accommodate also some ARR/DEP flows within AoI No. FROM TO Lower Vertical Limit (FL) Upper Vertical Limit (FL) Figure 89: Skyguide - Overview - new DCTs Utilization Time Availability Operational Goal Direction of Cruising Levels 1 VANAS GODRA H24 Free Solutions Even 2 OMASI GODRA H24 Free Solutions Even 3 RONOP BALSI H24 Free Solutions Even 4 PUNSA UNKIR H24 Free Solutions Even 5 GODRA KOGAS H24 Free Solutions Odd 6 KOGAS BALSI H24 Free Solutions Odd 7 GODRA BANKO H24 Free Solutions Odd 8 BANKO MEDAM H24 Free Solutions Odd 9 RAVED GODRA H24 Free Solutions Odd 10 RAVED LUTIX Only available for traffic ARR H24 Free Odd 154 of 213

155 11 BEGAR ODINA NATOR LUTIX LSGG/LFLL Not available for traffic ARR LSZA/LSZL/LSZS. Only available for traffic ARR LSGG/LFLL 13 ABESI LOKTA H24 14 ABESI VEDOK H24 15 RESIA HOC H24 16 SUXAN HOC H24 17 GODRA DITON H24 18 GODRA VEDOK H24 19 MOPAN LUTIX Only available for traffic ARR LSGG/LFLL H24 H24 H24 Solutions Free Solutions Free Solutions Free Solutions Free Solutions Free Solutions Free Solutions Free Solutions Free Solutions Free Solutions Odd Odd Even Even Even Even Even Even Od Table 83: List of skyguide - New DCTs Within skyguide AoR, during FREE Solutions FRA Trial, the ATS network remained available for flight planning purposes French airspace: DSNA In the defined airspace involved, (D/Z sectors of Marseille ACC combined above FL195) DSNA participated in the FRA trial with application of the concept of Direct Routing from Entry and Exit point. Comprehensive list of new 82 DCTs was published and plannable for all AUs Figure 90: Marseille ACC D and Z sectors - Overview DRA DCTs No FROM TO Lower Vertical Limit (FL) Upper Vertical Limit (FL) Utilization 155 of 213

156 1 BATIV MUREN BATIV RIXOT BATIV SORAS BATIV TABOT BAVON ELVOP Only available for traffic ARR LEBL/GE 6 CIRTA ELSAG CIRTA NEGAT DOLIS ELSAG DOLIS GARPU DOLIS NEGAT DOLIS OLNUL ERPIN CIRTA ERPIN DOLIS ERPIN KAMER ERPIN MOUET ERPIN MUREN ERPIN OTARO ERPIN REQIN ERPIN SALMA ERPIN TABOT ISTER OKSER KAMER OKSER KAMER OLNUL MORSS TABOT MORSS XATOS MOUET BAVON Only available for traffic ARR LEBL/GE 27 MOUET GARPU MOUET NEGAT MOUET OKSER MOUET OLNUL NEGAT BALEN Only available for traffic DEP Ajaccio Group, Bastia Group With ARR LEBL/GE 32 NEGAT CIRTA Only available for traffic DEP Ajaccio Group, Bastia Group 33 NEGAT DOLIS Only available for traffic DEP Ajaccio Group, Bastia Group 34 NEGAT MUREN Only available for traffic DEP Ajaccio Group, Bastia Group 35 NEGAT OTARO Only available for traffic DEP Ajaccio Group, Bastia Group 36 NEGAT RIXOT Only available for traffic DEP Ajaccio Group, Bastia Group 37 NEGAT SORAS Only available for traffic DEP Ajaccio Group, Bastia Group 38 NENAK DOLIS NENAK KAMER NENAK MOUET NENAK MUREN NENAK OTARO NENAK REQIN NENAK SALMA NENAK TABOT OLNUL DOLIS OLNUL KAMER OLNUL SALMA OLNUL TABOT OLNUL XATOS OSPOK ELSAG OSPOK SALMA OSPOK TABOT OSPOK XATOS of 213

157 55 OTARO ELSAG OTARO GARPU OTARO NEGAT OTARO OKSER OTARO XATOS REQIN GARPU REQIN NEGAT SALMA MUREN SALMA OLNUL SOTAX ELSAG SOTAX GARPU SOTAX NEGAT SOTAX OKSER SOTAX TABOT SUPUX OTARO SUPUX SOTAX TABOT BAVON Only available for traffic ARR LEBL/GE 72 TABOT MORSS TABOT MUREN TABOT OKSER TABOT OLNUL TABOT SOTAX XATOS BALEN Only available for traffic ARR LEBL/GE 78 XATOS MORSS XATOS MUREN XATOS OKSER XATOS OLNUL XATOS OTARO Table 84: List of Marseille ACC D and Z sectors - DRA DCTs Within Marseille ACC, D and Z Sectors, during the FREE Solutions FRA Trial, the ATS network remained available for flight planning purposes Italian airspace: ENAV Within Free Route Airspace Italy (FRAIT), AUs may plan user-preferred trajectories using significant points, five letter name codes and/or published En-route radio navigation aids. 157 of 213

158 Figure 91: ENAV Italy: Overview E(Entry) and X(Exit) FRA points FRAIT entry and exit points were the actual CoPs published at the boundaries of the Italian airspace; FRA FRA relevance Point Parity FRA FRA relevance FRA_ITALY E ASKOT EVEN FRA_ITALY EX PAN EVEN ODD FRA_ITALY E BABLO EVEN FRA_ITALY EX PAPIZ EVEN ODD FRA_ITALY E BAXON ODD FRA_ITALY EX PEVAL EVEN ODD FRA_ITALY E DETSA ODD FRA_ITALY EX PINUK EVEN ODD FRA_ITALY E DILIN EVEN FRA_ITALY EX RESIA ODD EVEN FRA_ITALY E DIRAB ODD FRA_ITALY EX RIFEN ODD EVEN FRA_ITALY E DOKAR ODD FRA_ITALY EX RUTOM EVEN ODD FRA_ITALY E ELSAG EVEN FRA_ITALY EX SENTI EVEN ODD FRA_ITALY E GIRIS ODD FRA_ITALY EX SUBOK EVEN ODD FRA_ITALY E GOGEM FRA_ITALY EX SUSIP EVEN ODD FRA_ITALY E LAPRI ODD FRA_ITALY EX TIBRO ODD EVEN FRA_ITALY E LIKDA ODD FRA_ITALY EX TIGRA EVEN ODD FRA_ITALY E LURAG ODD FRA_ITALY EX TISAL EVEN ODD FRA_ITALY E MEGER EVEN FRA_ITALY EX TORPO EVEN ODD FRA_ITALY E MOULE ODD FRA_ITALY EX TORTU EVEN ODD FRA_ITALY E NATAG ODD FRA_ITALY EX TUPAL EVEN ODD FRA_ITALY E NIBEL EVEN FRA_ITALY EX XATOS FRA_ITALY E NITAM FRA_ITALY EX XOLTA ODD EVEN FRA_ITALY E NOSTA EVEN FRA_ITALY X ABDAB ODD Point Parity 158 of 213

159 FRA_ITALY E ODINA ODD FRA_ITALY X ABESI EVEN FRA_ITALY E OLGAT EVEN FRA_ITALY X AOSTA EVEN FRA_ITALY E OLPIX ODD FRA_ITALY X BARPI EVEN FRA_ITALY E PELOS ODD FRA_ITALY X BELIX ODD FRA_ITALY E ROTAR EVEN FRA_ITALY X BORDI ODD FRA_ITALY E TABOT ODD FRA_ITALY X CERVI EVEN FRA_ITALY E TOBSO ODD FRA_ITALY X DOBIM EVEN FRA_ITALY E UNIMI ODD FRA_ITALY X GINOX FRA_ITALY E VELUG EVEN FRA_ITALY X GOPAT FRA_ITALY EX ABREG ODD EVEN FRA_ITALY X IBENI EVEN FRA_ITALY EX ADEXI EVEN ODD FRA_ITALY X IVKAL FRA_ITALY EX AIOSA ODD EVEN FRA_ITALY X KUBUD ODD FRA_ITALY EX BALEM EVEN ODD FRA_ITALY X KUKEV EVEN FRA_ITALY EX BEVIS ODD/EVEN FRA_ITALY X LABIN ODD FRA_ITALY EX BIBAN ODD/EVEN FRA_ITALY X MADKA EVEN FRA_ITALY EX BRENO ODD EVEN FRA_ITALY X MALUG EVEN FRA_ITALY EX CANNE ODD EVEN FRA_ITALY X MARON ODD FRA_ITALY EX CORSI ODD EVEN FRA_ITALY X MIRSA EVEN FRA_ITALY EX DINOB EVEN ODD FRA_ITALY X NIKOL EVEN FRA_ITALY EX DOPEL EVEN ODD FRA_ITALY X NOLSI ODD FRA_ITALY EX GISAM ODD EVEN FRA_ITALY X PUNSA EVEN FRA_ITALY EX GOKEL FRA_ITALY X RONOP EVEN FRA_ITALY EX KATTI EVEN ODD FRA_ITALY X SAFFA EVEN FRA_ITALY EX LATAN EVEN ODD FRA_ITALY X SOTOV EVEN FRA_ITALY EX LEVDI EVEN ODD FRA_ITALY X SOVOX EVEN FRA_ITALY EX LONDI ODD EVEN FRA_ITALY X SUPUX ODD FRA_ITALY EX LORED ODD EVEN FRA_ITALY X SUXAN EVEN FRA_ITALY EX LORNO ODD EVEN FRA_ITALY X TALEP ODD FRA_ITALY EX NELDA EVEN ODD FRA_ITALY X TUNEX ODD FRA_ITALY EX NOSTO EVEN ODD FRA_ITALY X VAMTU ODD FRA_ITALY EX ORTAP EVEN ODD FRA_ITALY X VAPUP ODD FRA_ITALY EX OSMAR EVEN ODD FRA_ITALY X VENIM ODD Table 85: List of ENAV - Italy E(Entry) and X(Exit) FRA points The list of five letter name codes 5LNC in the table above was published in ENR 4.4 and available as E(entry) and/or X(exit) points into FRAIT airspace above FL365. AUs were also able to use intermediate points between entry and exit points within FRA area in order to wind optimize their profile, circumnavigate a particular area, indicate a change in flight level or speed, or remain compliant with FRA areas boundaries rules (DCT shall remain inside the FRA area). 159 of 213

160 Figure 92: ENAV Italy: Overview I(intermediate) FRA points The ATS network was kept available for flight planning purposes in the so called Free Route Airspace above 365. Overflight traffic could be planned either directly between FRAIT entry point and FRAIT exit point or via intermediate points. The list of five letter name codes 5LNC and NAV points provided above as published in ENR 4.1. and ENR 4.4 as I(intermediate) points into FRAIT airspace above FL365. Overflying traffic with a cruising level below FL365, but requesting a change in cruising level above FL365: could plan the portion of their flight below FL365 along the published ATS route network; could indicate a change in cruising level in relation to a valid FRA Intermediate point and continuing flying in accordance with FRA operations or using the fixed route network published in AIP-Italia. Overflying traffic with a cruising level above FL365, but requesting a change in cruising level below FL365: could plan the portion of their flight above FL365 according to FRA procedure or using fixed route network published in AIP-Italia; 160 of 213

161 could plan the remaining portion of their flight below FL365 using the published ATS route network. Transition from FRA to the fixed route network had be done using FRA intermediate point Figure 93: ENAV Italy: Overview IA(I-Intermediate and A-arrival) FRA points Malta Airspace: MATS Within Malta Free Route Airspace, AUs may plan user-preferred trajectories using significant points, five letter name codes and/or published En-route radio navigation aids. Malta Free Route Airspace entry and exit points were the actual CoPs published at the boundaries of the Italian airspace; All the significant points could be planned as intermediate points. 161 of 213

162 Figure 94: Malta Overview E(entry),X(exit) and I(intermediate) - FRA points Area FRA Point Area FRA Point Area FRA Point FRA_LMMM E EKLIS FRA_LMMM X ABRAM FRA_LMMM I INBIN FRA_LMMM E INDOT FRA_LMMM X ASKOT FRA_LMMM I KOLEX FRA_LMMM E LUMED FRA_LMMM X BONAR FRA_LMMM I KUTOS FRA_LMMM E MARON FRA_LMMM X DILIN FRA_LMMM I LPD FRA_LMMM E OLMAX FRA_LMMM X ELIMO FRA_LMMM I MABOX FRA_LMMM E SARKI FRA_LMMM X LOTIN FRA_LMMM I MADIR FRA_LMMM E VENIM FRA_LMMM X VARIG FRA_LMMM I MINVA FRA_LMMM EX ADEXI FRA_LMMM I AGARI FRA_LMMM I NEVNA FRA_LMMM EX ARLOS FRA_LMMM I ALIXA FRA_LMMM I NIBLO FRA_LMMM EX BASMO FRA_LMMM I AMIBO FRA_LMMM I NUPMA FRA_LMMM EX BIRSA FRA_LMMM I ARTIB FRA_LMMM I OBITA FRA_LMMM EX DEMAG FRA_LMMM I ASDAX FRA_LMMM I OMARA FRA_LMMM EX LEVDI FRA_LMMM I BINKO FRA_LMMM I PEGIT FRA_LMMM EX LONDI FRA_LMMM I DEXOL FRA_LMMM I RATOK FRA_LMMM EX LORED FRA_LMMM I DINUX FRA_LMMM I REPTA FRA_LMMM EX NELDA FRA_LMMM I DIRKA FRA_LMMM I ROBIM FRA_LMMM EX NEVIK FRA_LMMM I DOBIX FRA_LMMM I ROLEV FRA_LMMM EX OMENI FRA_LMMM I DOKIK FRA_LMMM I RUBRI FRA_LMMM EX ORTAP FRA_LMMM I EKOLA FRA_LMMM I RUDOG FRA_LMMM EX PAN FRA_LMMM I ELKAN FRA_LMMM I SOPIR FRA_LMMM EX RALAK FRA_LMMM I EMLAR FRA_LMMM I SUSOM FRA_LMMM EX RASNO FRA_LMMM I EVIRA FRA_LMMM I TIMBI FRA_LMMM EX SENTI FRA_LMMM I EVLIB FRA_LMMM I UPLIT FRA_LMMM EX SONAK FRA_LMMM I GODAK FRA_LMMM I VANIX FRA_LMMM EX SUBOK FRA_LMMM I GZO FRA_LMMM I VERAK FRA_LMMM EX TISAL FRA_LMMM I IDANI Table 86: List of Malta E(entry), X(exit) and I(intermediate) FRA points The list of five letter name codes 5LNC and NAV points provided above was published in ENR 4.1. and ENR 4.4 as E(entry), X(exit) and I (intermediate) FRA points above FL365. AUs were also able to use intermediate points between entry and exit points within FRA area in order to wind optimize their profile, circumnavigate a particular area, indicate a change in flight level or speed, or remain compliant with FRA areas boundaries rules (DCT shall remain inside the FRA area). The ATS network was kept available for flight planning purposes in the so called Free Route Airspace above FL365. For the Inbound traffic, an appropriate list of intermediate points was notified to permit the transition between the FRA and the current ATS Network. 162 of 213

163 Project Number LSD Edition Flight planning FPL Item 15: Route - Segments between entry, exit and intermediate points had to be indicated by DCT in item 15 of the flight plan. For example: LONLA DCT RIMOS DCT TAKET. There were no restrictions within each FRA area on the DCT segments length ICAO FLOS (ODD / EVEN) was applied in Free Route Airspace. For flight planning purpose AUs have been requested to take into account flight level orientation for each significant point and en-route radio navigation aid in accordance with published information in each AIP, as prescribed by the ERNIP Part 1, Chapter 6. En-Route Design Methodology, paragraph 6.5. Free Route Airspace Design [6]. Existing points close to the operational borders of each FRA area (defined as COPs in existing LoAs) were defined as Entry/Exit Points to/from each FRA area. Existing COPs which were entry-only or exit-only for fixed ATS route network remained entry-only or exit-only in FRA. DRs segments between Entry, Intermediate and Exit points had to remain contained within the defined boundaries of each FRA area. Flight plans with segments that partially crossed the lateral limits of each FRA area were rejected by IFPS. Eligible flights that make a change in cruising flight level resulting in a transition to/from FRA had to file the portion of the flight outside FRA along the standard ATS route network. Definition of ad hoc procedures agreed between single ANSP and AUs were possible to ensure safe transition Figure 95: FRA Generic Examples (Exit - Entry) All flights within FRA had to leave FRA area before the end of the Free Routing opening hours. Any flight plans received that did not comply with this rule was rejected by IFPS. However, according to ICAO flight planning rules, a change of EOBT of up to 15 mins was not communicated to IFPS. In addition, delay en-route could be incurred due to circumstances that were not known before departure. A Free Routing transition period was set-up in which Free Route flights could be executed (i.e. 30 minutes). The transition period was only used on a tactical level and not for flight planning purposes Operational Staff Briefings Based on the already mentioned guidelines document, a set of briefings with operational personnel (ground and airborne sides) was conducted to provide advices for a correct execution of the trials. The objectives of theses briefings were to provide a general description of the FREE Solutions Project, and a detailed description of the third set of flight trials planned for March 2016, providing also guidelines and advices for a correct execution of the trials. As for the previous EXE (001, 002, 003, 004), the following activities were undertaken: Leaflet preparation for ATCOs involved in trials to clarify scope, ambition, needs of the project, and describing any further operational procedure required; 163 of 213

164 Leaflet preparation for Pilots involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required Leaflet preparation for Flow Managers Positions involved in trials to clarify scope, ambitions, needs of the project, and describing any further operational procedure required Advanced sharing of ATCOs feedback questionnaires Advanced sharing of Pilots feedback questionnaires Liaison with the identified Point of contact for each ACC involved in the flight trials for setting up the relevant operational scenario. Liaison with the identified Point of contact for each airline involved in the flight trials for setting up the relevant operational scenario Questionnaires Questionnaire Target Question ID Question description Hazard ID reference ANSP 4.1 ANSP 4.2 ANSP 4.3 ANSP 4.4 ANSP 4.5 ANSP 4.6 ANSP 4.7 ANSP 4.9 ANSP 4.10 ANSP 4.11 AU 1.1 AU 1.2 AU 1.3 Did you encounter any problems with flight plans? The amount of tactical rerouting was within acceptable limits Did you encounter any technical issues (i.e. HMI/FDP/OLDI) I was able to manage interaction between traffic flying DRs and other flights using the current network I was able to manage deviations to the FPL route due to active military areas/adverse meteorological conditions Have contingency procedures (if any) been adequate to cope with an unexpected event? Did you notice any change in the amount of ATC communications/coordination required compared to routine operations? Has the sector configuration been adequate to manage the traffic in control during the trials? Concerning the new DR scenario, did you manage conflicts within your area of responsibility or at the boundary? (entry/exit points) Did you manage unexpected interactions with active military areas? Have you been able to fly your planned FREE Solutions Route? Have you been able to perform routine fuel checks without any problem? Did you notice any change in the amount of ATC communications compared to routine HZ-01 HZ-04 HZ-05 HZ-09 HZ-07 HZ-02 HZ-04 HZ-05 HZ-09 HZ-02 HZ-04 HZ-04 HZ-03 HZ-03 HZ-04 HZ-06 HZ-08 HZ-07 HZ-01 HZ-02 All AUs hazards 164 of 213

165 Questionnaire Target Question ID Question description Hazard ID reference operations? AU 1.4 AU 2.1 AU 3.1 AU 3.2 AU 4.1 AU 4.2 Final remaining fuel has been final reserve fuel? Did you feel that safety was ever compromised due to deviations for adverse weather conditions (if any)? In case you noticed variations in workload level, what kind of difference did you experience? In case you noticed variations in workload, did you feel that these variations compromised safety? Was Free Solutions operational information, provided before the flight, exhaustive with regards to roles and responsibilities, working methods and operational requirements? Did you perceive any improvement with regards to flight efficiency? HZ-02 HZ-06 All AUs hazards All AUs hazards All AUs hazards N/A Exercise execution Table 87: Safety related questions - FRA The FREE Solutions FRA trial was conducted during one single weekend (during the 5 th and 6 th March 2016) starting from Saturday at 01:00 (UTC) up to Monday at 01:00 (UTC) Deviation from the planned activities Originally when defining the trials operational conditions, a single FRA was set up with crossborder activity between the 4 ANSPs. However, 2 FRA and 2 DRA were used due to the need to use existing COPs. In accordance with the outcomes of the operational solution design and live trials preparatory activities (WP3 and WP5), DSNA and Skyguide decided to ensure Direct Routing operations instead of Free Routing Operations for the EXE-LSD The ATS route network had to be kept due to difficulties using the CFSP and to avoid the exclusion of some AUs. Finally, the HP analysis was limited to the groundside, due to the few replies to the distributed questionnaire from the airside Exercise Results Summary of Exercise Results Results per KPA This section details the results obtained by applying the methodologies described in the section of this document for each addressed KPA Environment Fuel Efficiency 165 of 213

166 Project Number LSD Edition The results collected and analysed for this KPA contributes to the satisfaction of the following demonstration objective OBJ-LSD OBJ-LSD To demonstrate the feasibility of FR operations on Cross Border Areas Success Criterion Results Efficiency: Positive impact on fuel saving and Achieved apart from Malta ACC CO2 emissions where no traffic planned within FRA Table 88: Success Criteria Environment Fuel Efficiency OBJ-LSD The table hereafter summarizes results 14 observed and KPI used. KPA KPI Metric Results Environment- Fuel Efficiency Time saved Delta Minutes R(ref) v R(scn) 179mins CPF(ref) v CPF(scn) - 46mins Distance saved Delta Nautical Miles R(ref) v R(scn) -1463nm CPF(ref) v CPF(scn)-324nm Flights following filtering of the sample that have flight planned to conduct the trial solution FPL(ref) v FPL (scn) Flights that have actually flown the trial solutions CPF(ref) v CPF(scn) Fuel burn saved en-route Number of flights Number of flights Delta Kg R(ref) v R(scn) -9520kg CPF(ref) v CPF(scn) -3701kg 14 Some flights were removed from the final economy scenario calculations as follows: (1) long range flights (North Atlantic oceanic, to/from Asia, South America) due to significant difference between reference and solution scenarios; (2) flights with a distance between reference track and solution track of more than 40NM; (3) some flights with different trajectories and with different routings. Therefore from the 271 flights initially identified as trial participants 163 were used for the direct comparison with the reference scenario following the application of the above criterion. 166 of 213

167 CO2 saved enroute NOx saved enroute Delta Kg Delta Kg R(ref) v R(scn) kg CPF(ref) v CPF(scn) kg R(ref) v R(scn) -127kg CPF(ref) v CPF(scn) - 63kg Table 89: Environment - Fuel efficiency results - Metrics and Measures Free Routing For any further details, refer to the detailed analysis reported in Error! Reference source not found Airspace Capacity En-Route The results collected for this KPA contributes to the satisfaction of the demonstration objective OBJ- LSD , which has dedicated success criterion for airspace capacity. OBJ-LSD To demonstrate the feasibility of FR operations on Cross Border Areas Success Criterion Results Capacity: no negative local and network impact Achieved. No negative effects Table 90: Success Criteria - Capacity OBJ-LSD KPA KPI Metric Results Airspace Capacity (enroute) ATFCM delay Minutes None attributed to flights of the trial No negative effects Sector occupancy ACC throughput Number of aircraft per hour Number of aircraft per hour Change not significant enough to measure quantitatively Change not significant enough to measure quantitatively Sector workload Delta % points Change not significant enough to measure quantitatively Conflicts Number of potential conflicts per sector Table 91: Capacity Results - metrics and measures Free Routing Change not significant enough to measure quantitatively Qualitative results (received from Skyguide and ENAV) regarding controllers workload are reported in the devoted section Predictability and ATC planning The results collected for this KPA contributes to the satisfaction of the demonstration objective OBJ- LSD , which has dedicated success criterion for predictability. OBJ-LSD To demonstrate the feasibility of FR operations on Cross Border Areas Success Criterion Results Predictability: Reduction in the variation of the Not achieved. An expected reduction distribution of actual en-route time duration vs. of the delta time between reference planned en-route time duration. and scenario examples, indicating a more predictable scenario, was not 167 of 213

168 demonstrated in the data analysed. This may be due to wind or other influencing factors Table 92: Success Criteria - Predictability OBJ-LSD KPA KPI Metric Results Predictability Actual en-route time duration vs. planned enroute time duration Delta Minutes Ref v Scn An expected reduction of the delta time between reference and scenario examples, indicating a more predictable scenario, was not demonstrated in the data analysed. This may be due to wind or other influencing factors Table 93: Predictability Results - Metrics and Measures Free Routing For any further details on the assessment, data measurement and results, refer to the detailed analysis reported in Error! Reference source not found Safety This section reports the main results about safety for the FREE Solutions Free Routing operational scenario. Full analyses are reported in [4]. Safety was evaluated through: - safety assessment and safety brainstorming, - calculation of residual fuel at arrival and - answers to post-trial questionnaires. In particular, post-trial questionnaires, focused on ground-side (answered by 173 ENAV ATCOs of all 4 ACCs) and air-side (answered by 41 pilots from Alitalia) were analysed aiming to validate identified hazards and to conclude if they had an impact on the Safety of the operations or not ANSPs analysis This section shows results according to the GSN Arguments structure 15 in order to demonstrate that every aspect of the Safety assessment was considered pointing out the connections between the arguments, the hazards and the dedicated questions of the questionnaire. ARG Equipment affected by the change The purpose of this argument is to demonstrate that the potential failures or issues regarding the Equipment affected in organizing and executing the flight trials have been considered. The Equipment elements that were taken into account refer to flight plan management systems and HMI, OLDI, FDP. The results presented below show that few ATCOs in all Italian ACCs did experience some problems with flight plans or technical issues (i.e. HMI/FDP/OLDI). In the context of hazard assessment, this situation is typically considered as a possible cause for an increase of workload for planner air traffic controllers. 15 Full GSN Argument structure is available in the Safety Study [4] 168 of 213

169 Project Number LSD Edition It is also necessary to underline that the N/A (Not Applicable) answers are not analysed unless they represent a significant percentage of the total answers (i.e. >50%). Despite the fact that, they could be reported as not happening of a specific event reported in the question, in most of the cases. For completeness, the results of the questionnaires and the most significant rationales/explanations for some answers are provided below. Figure 96: ARG3.2.2 ANSP Equipment The above results show that ATCOs did not experience problems with flight plans or technical issues (i.e. HMI/FDP/OLDI). Free comments permit to underline some problems and issues encountered during flight trial scenario: - The need of data search, that could have caused also an increase of workload for air traffic controllers, - Absence of planning new points, - Difficulty to identify trial flight, - Unavailability of some flight plans in FDP, - Inability to send traffic via OLDI, - The necessity to transmits/receive some estimates, - Some ABI error. 100% 80% 60% 40% 20% 0% ARG Airspace configuration ARG Equipment affected by the change 4.1 Did you encounter any problems with flight plans? 4.3 Did you encounter any technical issues (i.e. HMI/FDP/OLDI)? This argument aims to demonstrate that the potential issues related to the airspace configuration were considered. ATCOs were asked to evaluate the adequateness of the airspace configuration during the trials. This topic is mostly linked to sector overload issue with a cascade effect on the ATCOs workload. Several causes of sector overload were identified: unexpected inbound flights entering the sector, inaccurate monitoring of flights and insufficient sectorization. The collected answers demonstrate the airspace configuration was adequate enough to allow a safe management of the traffic during the trials. For completeness, the results of the questionnaires and the most significant rationales/explanations for some answers are provided below. n/a No Yes 169 of 213

170 100% ARG Airspace configuration 80% 60% 40% 20% n/a No Yes Figure 97: ARG3.2.3 ANSP Airspace Configuration According to free comments, airspace configuration was evaluated as adequate because (i) few flights had to be managed during the trial and (ii) extra personnel was present to assist ATCOs on duty, while this configuration was evaluated as inadequate due to (i) some sectors size (in Roma ACC for example) and because lot of traffic flows were on sector boundaries. ARG Operational and flight procedures This argument was structured to demonstrate that the flight and operational procedures to be used during the flight trials are adequate and not demanding by a safety perspective. Six issues have been evaluated: Resuming the assigned route after a deviation, Interactions between flights flying DRs and other flights, Adverse meteorological conditions, Unusual entry/exit points, Navigation issues, 0% Crossing of active military areas. 4.9 Has the sector configuration been adequate to manage the traffic in control during the trials? The controllers on duty were asked to answer to the questions and to provide justifications if they wanted to. 170 of 213

171 Figure 98: ARG ANSP - Operational and flight procedures According to the Italian ATCOs answers and related justifications the following considerations were done: - N/A answers mean that the ATCOs did not experienced what they were asked to evaluate, - they did not encounter any problems in re-routing traffic. In some cases there were no conflicts. - there was extra personnel available for helping controllers on duty (one of the mitigations identified and applied to reduce risk) - in case of conflicts, some of them occurred near the boundary (especially between Milan ACC and Padua ACC) while others in typical well known areas. - ATCOs easily managed the interactions between DR flights and other flights, - during the trial, it was not needed to apply contingency procedures, - military areas were not active at the moment of the flight trials, - bad weather conditions were not experienced. Thus, it is reasonable to affirm that the ATCOs were always in control and confident with the implementation of DR concept as they had been informed about the potential issues before the trial execution. However, some ATCOs declared that they contingency procedures were not adequate to cope with unexpected events (less than 8%). Some justifications for these few answers were provided: 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 4.2 The amount of tactitcal rerouting was within acceptable limits 4.4 I was able to manage interaction between traffic flying DRs and other flights using the current network one ATCO declared that no training was provided and that he/she encountered some OLDI problems, another ATCO declared that no procedures were available but at the same time the traffic lload was very low. A deeper analysis of free text comments provided by the same ATCOs answering "NO" the only issues encountered were related to increase of coordination and rerouting and problems with OLDI. Also, they registered low traffic load. Thus most of "NO" answers should be reasonably considered as mistakes". ARG People related issues ARG Operational and flight procedures 4.5 I was able to manage deviations to the FPL route due to acxtive military areas/adverse meteorological conditions 4.6 Have contigency procedures (if any) been adequate to cope with an unexpected event? 4.10 Concerning the new FR scenario, did you manage conflicts within your area of responsibility or at the boundary? 4.11 Did you manage unexpected interactions with active military areas? n/a No Yes 171 of 213

172 Project Number LSD Edition This argument was pointed out to determine the impact on the human perspective of the operational concept introduced by the Free Routings flight trials execution. The questionnaire sections referred to situational awareness and to the level of ATCO s workload during the trials execution. According to the answers, ATC communications increased especially for PLN controllers (48%). For completeness, the results of the questionnaires and the most significant rationales/explanations for some answers are provided below: - Increase of phone coordination for PLN - Increased coordination to manage rerouting - Creation of new conflicts due to new DRs - Increase of coordination with neighbouring ACCs for approval request and FPL data exchange 4.7 Did you notice any change in the amount of ATC communications/coordination required compared to routine operations? N/A No Increase Decrease Figure 99: ARG ANSP - People related issues ARG Environmental and meteorological conditions The argument was identified to consider the impact of the environmental and meteorological conditions on the operations. This topic was tackled by the question 4.5 ( I was able to manage deviations to the FPL route due to active military areas/adverse meteorological conditions ). In fact, adverse meteorological conditions is a hazard whose effect is trial flight needs to deviate from the assigned route to avoid adverse weather conditions As already mentioned during the analysis of the ARG 3.2.4, adverse meteorological conditions were not experienced and/or did not affect the trials AUs analysis This sub-section shows the results according to the GSN Arguments structure in order to demonstrate that every aspects was considered pointing out the connections between the arguments, the hazards and the dedicated questions of the questionnaire. ARG Equipment affected by the change 172 of 213

173 % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1.1 Have you been able to fly your planned FREE Solutions Route Figure 100: ARG AU - Equipment The results clearly show that no issue was experienced by the pilots due to Equipment-related issues (87.8% of the answers were YES ). It should be noted that the question 1.1 has been proposed to the pilots mainly to understand if no issues were recorded during the trials but focusing on the operations rather than on the equipment. It is possible to assume that since the most of the pilots have been able to fly their planned route then it is clear that no issues linked to the flight planning occurred. Free comments provided by pilots can explain the percentage of no answer : ATCOs provided a direct route that was better than the planned one. ARG Operational and Flight Procedures This argument was structured to demonstrate that the flight and operational procedures to be used during the flight trials are adequate and not demanding by a safety perspective. N/A NO Yes Did you notice any change in the amount of ATC communications compared to routine operations? No Decrease Figure 101: ARG AU Operational and flight procedures - Communications Communications with the ground were not affected by the trials execution for most of the interviewed pilots (73.2%). For the remaining 26.8%, they indicated that the amount of ATC communications was even lower compared to daily operations. It is then possible to confirm that there was no degradation of the Safety level of the operations. 173 of 213

174 100% 90% % 70% 60% 50% 40% 30% 20% 10% N/A No Yes % Have you been able to perform Final Remaining Fuel has been Final routine fuel checks without any problem? Reserve Fuel Figure 102: ARG AU Operational and flight procedures - Fuel Very positive results were obtained concerning routine checks and final fuel. Few pilots revealed, in free comments, an extra fuel than usual due to bad weather (4.9%). ARG People related issues This argument was pointed out to determine the impact on the human perspective of the operational concept introduced by the Free Routing flight trials execution. Results related to this argument are completed by the HP analysis related to the Workload and Working Methods sections. A set of questions was selected to perform the analysis by a Safety perspective and the results are shown below: 174 of 213

175 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 95.1 N/A Other Physiological Physical Mental % Figure 103: ARG AU - People related issues - Workload 95% of the pilots interviewed reported that there was no effect on workload and only 4.8% declared a variation about the mental workload (2.4%) and the physical one (2.4%). Most of them (41.5%) affirmed that the Safety of the operations was not affected at all with respect to the baseline operations while 58.5% affirmed that this question was not applicable because they did not feel any variations in workload level. As a matter of fact, it can be said that, pilots workload was not affected by trials execution. Hereafter are presented the results related to the Working methods section of the questionnaires: 100% 80% In case of variations in workload level, what kind of difference did you experience? % 40% 20% 68.3 N/A No Yes % Was Free Solutions operational information, provided before the flight, exhaustive with regards to roles and responsibilities, working methods and operational requirements? Figure 104: ARG AU Working Methods ARG Environmental and meteorological conditions This argument was identified to consider the impact of the environmental and meteorological conditions on the operations. Analysis focused on determination of how adverse meteorological conditions could have affected the trials by inducing involved aircraft in deviating from the assigned route. 175 of 213

176 Project Number LSD Edition One question was dedicated to this particular issue Did you feel that safety was ever compromised due to deviations for adverse weather conditions (if any)?. 100% 90% 80% 39 70% 60% 50% 40% 30% 20% 10% 61 N/A NO Yes % Did you feel that safety was ever compromised due to deviations for adverse weather conditions (if any)? Figure 105: ARG AU Environmental and meteorological conditions The answers show that the majority of pilots did not feel that safety was compromised during the flight. It is also necessary to underline that the N/A (Not Applicable) answers are not analysed unless they represent a significant percentage of the total answers (i.e. >50%). Despite the fact that, they refer to the not happening of a specific event reported in the question, in most of the cases. This result matches with what was declared by air traffic controllers (refer to ARG 3.2.4) Safety Summary Results OBJ-LSD To demonstrate that the changes introduced by each scenario foreseen by FREE Solutions Project are acceptably/tolerably Safe Success Criterion Every negative effect on the operations shall be reduced as far as reasonably practicable The Safety of the operations shall remain at or better than its current level of Safety All the critical issues related to the operations management by all the stakeholders involved will be analysed All the evidences requested by NSA will be provided Results Fully achieved no negative effects on operations, according to the argument analysis showed in the previous sections Fully achieved - according to the argument analysis showed in the previous sections Fully achieved - according to the argument analysis showed in the previous sections Fully achieved evidence provided to NSA and no issue reported Table 94: Success Criteria - Safety OBJ-LSD Free Routing Weekends KPA KPI Metric Results 176 of 213

177 Safety Residual Fuel at arrival Tons It was decided to not measure and replace this measurement with ad-hoc questions for pilots Table 95: Safety Results - Metrics and Measures Free Routing Weekends During demonstration planning stage a quantitative measurement for residual fuel was envisaged. The aim was to measure the overall amount (tons) of residual fuel at arrival. However, during live trials preparatory activities it was discussed and agreed with involved AUs to replace this measurement with ad-hoc questions to be inserted in the post-trial questionnaire for pilots (i.e Have you been able to perform routine fuel checks without any problem? Final remaining fuel has been final reserve fuel?). This decision better matched with existing AUs policies about external communication regarding fuel consumption, and it represented a more rapid and practical way to get the same piece of information HP Results Results reported in this sub-section are derived from the brainstorming session executed during the preparatory phase Error! Reference source not found. and the actual execution of Direct Routes trials. They are organised and proposed with respect to the two strands of analysis: groundside and airside ANSPs Analysis As already done in previous exercises, a customized questionnaire was administrated within the different sectors of ACC units directly affected by the trials. Questionnaires analysed here were provided by ENAV and Skyguide. No particular comments or remarks were provided by DSNA. On the ground side the amount of trial flights allowed the collection of 224 questionnaires spread out among the involved ANSPs. The table hereafter provides details on the questionnaires, i.e. amount of formats collected per each ANSP. In some cases information on role performed during live trial was not provided by controllers. For this reason, it was not possible to make a difference between planner and executive roles answers. The following sections summarize results in an aggregated way. ANSPs Overall EC Formats PC Formats Other ENAV Skyguide Table 96: ANSPs Questionnaires - Free Routings Data collected through questionnaires were manipulated and then converted into statistics. Manipulation consisted into the aggregation of ENAV and Skyguide scores and the subsequent organization. Conversion implied that answers to the different choices were expressed in percentage values or in mean values, i.e. partial quotes were reported in reference to the overall amount. Scores (percentage or mean) are reported in the following sections. Results are organized and then proposed according to the different Human Performance objectives linked to EXE-LSD Nevertheless, for some questions, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve the full score (i.e. 100%) through the sum up of afferent intermediate quotes. A margin of uncertainty is caused and induced subsequent effects, first on the analysis of data and secondly, on the reporting. This uncertainty can be classified as not severe because achieved tolerable levels. 177 of 213

178 Project Number LSD Edition Details regarding declared HP objectives can be found on the devoted document HP Assessment Plan [10]. Scores collected through questionnaires are hereafter provided in an aggregated way, i.e. mean value without specifying the role (EC and PC) or the ANSP of origin Roles and Responsibilities Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP061 To demonstrate that controller role and responsibilities (EC, PC) are clear and consistent during free routings in a defined free route airspace flight trials OBJ-LSD Table 97: Success Criteria Role and responsibilities - OBJ-LSD The purpose of this objective foresees the assessment of working methods and operational information/procedures in terms of i) any eventual change with respect to current situation and ii) trend of change (i.e. improvement or deterioration), if occurred. Relevant aspects asked to controllers are presented in the next figure and concerned: Changing of coordination procedures: 42% of the ATCOs did not experience variation. Whereas, 54% declared an increase of coordination procedures. - Variation of shared information with other team members: mixed results are also observed for this topic: 49% of the ATCOs estimated that it did not change, while for 47% a rise in shared information was stated. Results to these two first topics are induced by two different bias : o o Application of Free Routing to flights out of the experimental campaign that required managing re-routing. Some technical issues (i.e. OLDI limitations) requiring extra coordination procedures (e.g. phone coordination, manual rerouting, ). - Accuracy of information related to procedures and working methods: The majority (83%) of the ATCOs estimated that the information related to procedures and working methods was accurate and complete. (For information, only 2% of the ATCOs did not answer to this question) Changing of coordination procedures 49 Changing of shared information Figure 106: Groundside HP objective HP061 In summary, collected scores underline that it is necessary to pay attention to problems experienced during trials inducing a need of extra communications and coordination Accuracy of information related to FREES procedures Increase Decrease Yes No No answer 178 of 213

179 Project Number LSD Edition procedures. Nevertheless, roles and responsibilities from an ATCO s perspective are still clear and consistent. Operating Methods and Procedures Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP065 To demonstrate that controller operating methods and procedures (EC, PC) are clear and consistent during free routings in a defined free route airspace OBJ-LSD Table 98: Success Criteria Operating Methods and Procedures - OBJ-LSD Purpose of this objective is to complement evidence derived through a devoted brainstorming before the trials Error! Reference source not found. with additional more tangible one collected after the execution of the exercise. This objective mainly implied to check the ability to allow flight trials to fly the planned and agreed route. Positive scores were obtained for this topic: 96% of ATCOs were able to provide planned agreed route (Only 1% answered no and 3% did not answered to this question). Figure 107: Groundside HP objective HP065 As for the previous topic, the mixed results obtained for i) the amount of coordination procedures and ii) the amount of shared information among team member do not permit to infer that HP objective was completely assessed and that operations methods and procedures were correctly defined, even if controllers were able to allow flights to fly the agreed route. Controller Performance Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP067 To demonstrate that controller (EC, PC) performance is appropriate during free routings in a defined free route airspace flight trials OBJ-LSD Table 99: Success Criteria Controller Performance - OBJ-LSD Evaluate controller performance means the assessment of users perception with respect to different markers/dimensions that basically are: o Workload, Ability to allow flight trials to fly the FREE Solutions agreed route Yes No N/A o o o o o Situational awareness, Tasks time execution, Communications and coordination, Trust in the concept, Job satisfaction in adopting the new operational concept, 179 of 213

180 Project Number LSD Edition o Potential errors induced by the introduction of the proposed concept. As previously explained, some of these markers are interconnected among them. Scores collected through questionnaires are hereafter reported according to the markers previously introduced. Workload Figures hereafter show controllers feedback regarding level of workload experienced during the trials. Within a scale from 1 to 10, ATCOs had to evaluate the elements making up the workload with respect to baseline operations, where 1 means less demanding, 5 means no change and 10 means more demanding. In order to collect information about workload as objective as possible, attention was paid to the investigation of 3 dimensions: Mental demand, i.e. supplementary cognition effort devoted to the execution of FREE Solutions trial; Physical demand, i.e. extra actions/tasks like more speeches or more interactions through HMI; Frustration, i.e. potential sense of irritation/stress/annoyance Additional mental workload Extra actions undertaken Feelings Figure 108: Groundside HP Objective HP067 Three main dimensions of workload Figure above shows that for the 3 main dimensions assessed, a slight significant change with respect to the baseline situation was observed concerning mental workload and extra actions, as previously underlined. Nevertheless, ATCOs feelings were still positive during the trial (no insecurity, no stress). In order to collect information about workload as objective as possible, the results were enriched through the assessment of ATC communication load Figure 109: Groundside HP objective HP067 - Communications Changes 2 Increase Decrease No change No answer of 213

181 Project Number LSD Edition Figure above provides collected scores (i.e. coordination procedures) in percentage in terms of changes in ATC communications. Contrary to the previous results, most ATCOs declared that communications numbers were unchanged (75%). In summary, achieved results allow inferring that workload still remained at proper level without negative effects on ground performances of both ECs and PCs. Situational Awareness Situational awareness is the continuous extraction of environmental information, the integration of this information with previous knowledge to form a coherent mental picture and the use of that picture in directing further perception and anticipating future events. Within a scale from 0 to 6, ATCOs had to evaluate their level of situational awareness, where 0 means poor level, 3 means neutral and 6 means high level. To assess the situational awareness, a set of devoted questions was proposed: The ability to anticipate traffic behaviour; The way they are not focused on problem/area of the screen; Their evaluation that there is no potential risk to forget important information; Their ability to plan and organise activities according to their own preference; The unwanted surprise due to unexpected event; The no loss of time spent to search for info on the screen Figure 110: Groundside HP objective HP067 - Situational Awareness Considering that the range of value can vary from zero (poor level of situational awareness) to six (high level of awareness), the achieved scores are in general positive. This allows inferring that situational awareness did not change so much respect to baseline conditions or, in case of some dimensions, achieved positive values (anticipate flights request, plan and organisation, not surprise due to unexpected event). This trend is aligned and coherent with the one of workload discussed in previous paragraph. Task Time Execution 0 Anticipation Not focused on problem No risk to forget Planification Another marker that could influence performance is the time/effort devoted to the execution of tasks. Figure hereafter shows achieved scores with respect to four topics deemed able to influence the execution of activities, i.e., - Traffic complexity (if changed and with which trend) - Sector capacity (if changed and with which trend) Not surprised - Conflict detection (if changed and with which trend) - Conflict resolution (if changed and with which trend) No search for information 181 of 213

182 Increase Figure 111: Groundside HP Objective HP067 - Task Time Execution Figure 111 provides positive results meaning that for the majority of ATCOs the Free Routings did not affect the time devoted to the execution of tasks. Histograms are first analysed and explanations are given right after. Regarding complexity, more than half of users (68%) did not notice variations with respect to current situation, 27% experienced an increase. Regarding capacity, perception of no-change is stronger (79% did not notice variations) remaining parts were divided between increase of capacity (08%) and deterioration (05%). Finally the two tasks were assessed, i.e. detection and resolution of potential conflicts. Basically unchanged conditions were registered for both tasks, i.e. 62% of no-variation in case of detection and 71% for resolution. Results are aligned with the ones of EXE-LDS and Free comments provided causes of these increases concerns: (a) The importance of sectors design (some ATCOs underlined that Free Routings were a suitable concept for upstream/scroll sectors); (b) Some difficulties are related to even/odd compatibilities at the sector boundary (c) The increase of conflict possibilities on the area (more potential crossing points) and; (d) The occurrence of new types of conflicts with which they are not used to deal with; (e) A need to adapt the habits of action to a new way of work. As a matter of fact, these elements permit to explain the slight increase in terms of traffic complexity. Summing up, Free routings did not introduce significant variations in terms of tasks execution. Time devoted to them is quite close to the one necessary in current operations, thus without negative effects on users performance. Communications and Coordination Three ad-hoc questions were posed to controllers for the assessment of this topic, i.e Changes in coordination procedures; Changes in terms of information exchanged with the other team member; Changes in the amount of ATC communications (the same already treated for the topic workload ); These questions were already analysed above in the topic dedicated to roles and responsibilities. Trust in the Concept Traffic complexity 8 30 Another marker able to influence the general controller performance is the trust in the new concept/route introduced through FREE Solutions. Relevant aspects investigated during trials were: Sector capacity Conflict detection Conflict resolution Decrease No No answer 182 of 213

183 Usefulness of new routes; Reliability of new routes; Controllers confidence while working with new procedures. Within a scale from 0 to 6, ATCOs had to evaluate these 3 topics, where 0 means poor level of trust, 3 means neutral and 6 means very high level of trust It is possible to infer that: Figure 112: Groundside HP Objective HP067 - Trust - ATCOs are often confident with new procedures related to this trial, - They often found it useful, - They also found that free routings were often reliable compared to baseline operations. This implies that controllers often trusted in the concept and in its rationale that had driven the design. Job Satisfaction (Operative Acceptability) 0 It addresses the human well-being, the cognitive comfort in executing the prescribed work. During trials this indicator was investigated at high level posing indirect questions, i.e. acceptability of the FREE Solutions agreed route considering the balance between gain on route length and workload Usefulness Reliability Confidence 9 Acceptability Figure 113: Groundside HP objective HP067 - Operative Acceptability Figure 113 shows positive results of operative acceptability. More than 80% of the interviewees perceived as acceptable the compromise route gain/induced workload because directly experienced the efficiency and the overall benefit of proposed routes, against a 9% that assumed a negative Ability Yes No N/A 183 of 213

184 Project Number LSD Edition position. 96% of them declared being able to clear flights to follow agreed routes and only a 1% was not able. New routes revealed to be effective, applicable and then able to influence in a positive way the users acceptability related to operative concept. Thus, good level of satisfaction is supposed to positively influence users acceptability. Integration of results related to operative acceptability and trust allows inferring that free routings can be introduced in real operations, without jeopardizing the users attitude to the change. Potential Errors Purpose of questionnaire was also to assess that new routes did not induce errors in executing operations with dangerous side effects on controller performance and then on safety level of operations. This dimension was investigated considering two different point of sight, i.e. leading metrics like error, reliability of the new concept. Regarding leading metrics, quite positive values were achieved. Considering that scale varies from 0 (negative feedbacks) to six (very positive), scores are close to four that means controllers often did not bump into the risk to forget important info related to the flight and often deemed the concept design reliable No risk to forget Reliability Figure 114: Groundside HP Objective HP067 - Potential errors a Regarding lagging metrics, next figure sums up the operative aspects linked to assessment of errors induced in the users Increase Abitlity to manage mixed traffic Assurance of safety level in case of Accuracy of information related to increased FREES procedures coordination/communication Figure 115: Groundside HP Objective HP067 - Potential errors b Decrease Yes No N/A 184 of 213

185 Project Number LSD Edition Looking atthe figure, scores confirmed that new concept was designed and applied in a way that still allowed controllers the correct management of trials and baseline flights (96%) and according to a set of information deemed accurate (83%).. Most of controllers did not respond or found it not appropriate (43%) to evaluate the assurance of safety level in case of increased coordination/communications. The very few negative results (4% of safety level decrease and 3% of not safety level assurance) might have been induced by two validation bias occurred during the trials in the Italian airspace: application of Free Route also to flights not belonging to experimental campaign. This induced extra coordination to manage re-routing. technical issues (i.e. not all FPLs available on FDP, OLDI limits). This induced extra actions normally not requested (e.g. phone coordination, manual rerouting, transmission of estimated times/planned COP). In general, all these intermediate results allow inferring that new Direct Routes did not induce incorrect actions or errors in controllers, both ECs and PCs. Changes to Tasks and Roles Identifier Objective Supportive Objective Exercise OBJ-LSD0105-HP071 To demonstrate that changes to controller tasks and role (EC, PC) are managed during free routings in a defined free route airspace flight trials OBJ-LSD Table 100: Success Criteria - Changes to Tasks and roles OBJ-LSD These objectives are focussed on the assessment of potential changes to controllers activities and roles/responsibilities due the execution of FREE Solutions trials, additionally to the detection and mitigation of potential errors. Above elements were translated into devoted questions of questionnaire, already treated in previous sections. For the different arguments, ATCOs did not perceive any significant changes related to their tasks with respect to baseline situation, except for the coordination procedures that were evaluated as being increased during trials due to some bias. Results referred to safety confirmed that during this trial the safety level of operations was still assured. In general, comparing results per different topics, no significant changes occurred at groundside in case of free routings execution. Current roles and linked tasks revealed to be stable and adherent to the nowadays situation. Nevertheless, some issues are underlined and need to be taken into account for the implementation of Free Routings in everyday life. During the brainstorming session, which involved all the Consortium partners SMEs by direct participation or indirect contribution to the document, the FHA (Functional Hazard Assessment) of the SAM was performed and the severities of the abovementioned hazards have been identified. The details of the hazard identification, together with its related causes and effect analysis, are reported in Free Solution Safety Case 1.0 ([4]). The final version of the Safety Case which has been validated and approved by all the Consortium partners and its results are presented in [4] AUs Analysis As for the ATCOs, during the execution phase, a customized question was administrated to flight crews involved in the trial. A total of 87 fights were reported as measured from consortium airlines. Unfortunately, only 41 questionnaires were returned from Alitalia and were included in an aggregated way PF and PM feedback. 185 of 213

186 AUs Overall Alitalia Table 101: AUs Questionnaires - Free Routings As explain in the section , the HP analysis for Free Routings is purely limited to the ANSPs General Results The collected recommendations and the performed formal and informal peer reviews contribute to the satisfaction of the following demonstration objective OBJ-LSD OBJ-LSD To give input to airspace community interested in FR operations Success Criterion Results Production of a list of recommendations for the Fully achieved, a list of FAB community recommendations is provided in the dedicated sections of this document Formal feeding of OFA and OFA with FREE Solutions results Production of a list of recommendations to update/improve existing regulations/standards in support of Free Routing operations as results of live trials activities Fully achieved due to provision of a list of recommendations, participation to joint meetings and active OFAs involvement as external document reviewers Not achieved Table 102: Success Criteria - OBJ-LSD Results impacting regulation and standardisation initiatives Nothing to be reported in this paragraph Unexpected Behaviours/Results During EXE-LSD some flights not planned as free solution flights were actually rerouted as if they were Free Routing flights and managed in the same manner within Italian airspace. That led to the delivery of additional benefits reported below. The total number of flights is related to the actual tracks that could be compared with baseline traffic in accordance with the specified eligibility criteria reported in Error! Reference source not found.. 5 th March Data Total impacted flights Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) th March Data 186 of 213

187 Total impacted flights Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) Quality of Demonstration Results As it may be verified in the other sections of this chapter, the approach adopted was a combination of FPL information and actual track data, which gave as an output high quality results compared to the conventional approach used when performing RTS simulation activities The savings in terms of distance, duration and fuel burn are consistent when measured from the NM or the AUs' perspectives. Moreover, the assessment of real trajectories confirms the actual improvement that was expected on the flight plans basis. The results are lower but this is in line with SES established flight efficiency indicators. As trials took place during a low traffic period and the number of trial flights was too low to create a significant effect on ATC operations and controller workload, no impact on capacity was expected. However, the fact that sector occupancy was not affected brings confidence in the feasibility of the designed solution at higher traffic rates. During screening of questionnaires and subsequent delivery of statistics an anomaly was observed. For some questions of filled-in templates, feedbacks were not provided by all controllers thus leaving these cells empty. This fact caused the impossibility to achieve the full score (i.e. 100%) through the sum up of afferent intermediate quotes. As a final effect a margin of uncertainty is provoked and induced subsequent effects, first on the analysis of data and secondly, on the reporting. However, this uncertainty can be classified as not severe Significance of Demonstration Results The number of flight trials is quite significant, and the operational environment unchanged. It guarantees the confidence in the results published Conclusions and recommendations Conclusions This exercise demonstrated the feasibility of Free Routing operations with current systems and existing operational procedures, taking into account all the assumptions made and the fact that live trials were, by design, not arranged during peak of traffic. However, the conclusions and recommendations listed below illustrate the lessons learnt collected during the exercise and are shared with the community with the purpose of facilitating future Free Routing initiatives. Disclaimer: Traffic samples used as reference are not exactly the same than the ones presented below (aircraft types, routes, weather). Results are thus to be taken with care. For the explanation of the calculation below, please refer to the deliverable [3]. Date Number of flights Length (NM) Time (min) Fuel (kg) CO2 (kg) NOx (kg) of 213

188 Table 103: EXE-LSD Results per Day Table 104: EXE-LSD Overall results The above numbers depict the positive outcomes of the trials with an average gain of 1,96 NM during 5th March and 2,33 NM during 6th March. In addition 3701 Kg of Fuel were saved leading to reduced CO2 emissions by Kg and 46 minutes of flight time saved. Then, taking also into account collected and analysed feedback, the following conclusions were also drawn: CONC 1. Free Routing live trials demonstrated the expected performance improvement in terms of flight efficiency and environment sustainability (emissions reduction) across the European Network; CONC 2. Lower limit of FREE Route airspace set at FL365 was a constraining factor which limited access to that airspace, due to difficulties in joining that airspace for short haul flights and when performing coordination and transfer with neighbouring ANSPs with LoA that foresaw a lower FL at the entry/exit point. Moreover, due to performance reasons (e.g. heavy A321), some trials could not be flown because they were not able to reach the FL365. FL335 would have been sufficient for all flights to reach the trial airspace CONC 3. Setting of Free Routing area lower limit at FL365 by design led to management of low amount of traffic, which resulted in few Free Routing flights handled in some sectors; CONC 4. The contemporaneity of ATS route network and FREE Route airspace let users to choose the most used and easy to plan solution, resulting in a number of flights not planning as Free Solutions flights; CONC 5. Free Routing live trials demonstrated that there was no negative impact on capacity at both local and Network level, but taking into account that demonstration were conducted during quiet periods of the year by design, with the amount of traffic not at higher peaks. However, controller feedback from questionnaires and interviews indicate no significant issues relating to the changes being trialled. Although the feedback makes clear that such changes would have to be fully validated before full implementation and operational use; CONC 6. Taking also into account the outcomes of FREE Solutions trials, ENAV decided to set FL335 as lower limit of FREE Route airspace when implementing such kind of operations at the end of 2016; CONC 7. Current systems (i.e. Ground FDP and Airborne FMS) are constrained by limiting factors that required implementation of ad-hoc procedures adopted during trials, like for example: additional supervisory staff, telephone coordination for transmission of estimates. Those peculiarities were related to: 188 of 213

189 Project Number LSD Edition a. Impossibility to plan waypoints (entry, exit and intermediate points) for the main CFSPs (i.e. LIDO, SABRE, Jeppesen, Rock Route). The procedure adopted was to plan segments designing more a DRA than a FRA; b. availability of Free routing segments loaded in the FMS database. FMS have a set of pre-stored segments which allow companies to choose the most favourable route in accordance with identified selection criteria (e.g. the cheapest route). Therefore, availability of additional route segments shall trigger updates of on board databases; c. ground ANSPs manage cross-border coordination through OLDI and, in some cases, the systems need to know the next point on the route even if this point is beyond Area of Responsibility. CONC 8. For DSNA, the planned and published DCTs of 200Nm were rejected from the STIP/STPV: It was impossible to process flight plans automatically. Moreover, using DCTs of 200 Nm did not allow AO to make a change of FL that was not planned in the FPL. CONC 9. Due to RAD constrains and differences between CFSP and IFPS processes led to the rejection of some Flight plans. CONC 10. From DSNA point of view, the present ATCO interfaces do not allow them to have sufficient information about adjacent FRA: A mandatory point is needed before each Entry point and after each Exit one to allow the flight plan processing. CONC 11. As to ATCO perceived workload, feedbacks collected during trials can be summarized as follows in comparison with today s operations: a. Reduced ATC communications, with positive effects on operational fatigue, while an increase in ATC coordination was underlined due to some OLDI limitations leading to additional tasks; b. initial part of flight less subjected to intermediate level off and deviation from standard route; c. manual intervention required with no possibility to use direct routings to solve conflicts, only headings were applicable; d. concerns were raised by controllers related to «not concerned» traffic close to sectors boundaries which may interfere with «concerned» traffic, due to the fact. CONC 12. Great potential of OPR, but during trial no great benefit for SWISS because: a. many flights with high potential of benefit were operated by RJ1H (max. ALT FL310) b. many flights with high potential of benefit were operated by OAW wetlease c. savings from UPR did not compensate for lower ATC charges in LA/LD/LQ/LY" CONC 13. For Lufthansa, a more flexible and expanded timeframe for the trials would have led to more results concerning their long-haul fleet, because the flights only take place once a day. This led to the fact, that out of 9 possible flights only 2 flights were able to fly the trial routing due to unfavorable winds which would have led to higher costs of several USD. CONC 14. Taking also into account the outcomes of FREE Solutions trials, skyguide decided to implement, during Q1 2017, the following DCTs: a. VANAS GODRA from FL305 H24 b. BEGAR ODINA from FL195 night c. SUXAN HOC from FL245 night d. GODRA DITON from FL245 H of 213

190 Project Number LSD Edition Recommendations REC 1. Technical system (NM, AUs and ANSPs) requires improvements to support the implementation of Free routings. REC 2. Training sessions are deemed necessary to support the transition induced by changes. REC 3. Future trials should be organized for a longer period than a week-end as the organization of those events is complex and demanding due to the amount of work for the airspace design modifications (and update of flight planning systems), then a longer period will give users and operators the possibility to get accustomed with new/modified working methods and operations. Ideally it is suggested to envisage live trials operations which last for an entire AIRAC cycle; REC 4. Future trials should avoid mixing of Free Routing models which increases difficulties and need for ad hoc solutions resulting in not exploiting of all potential free routing benefits. There is a need to harmonize the approach of implementing FRA (transition between FRA and not FRA at European level, the number of needed points and segments, the length of segments, the number of transition points ) and to make sure this is taken into account by future flight planning capabilities and ATC operations; REC 5. Existing national and international plans for ground FDP and CFSP evolutions should be seen as an enabler for full implementation of Free Routing concept and associated benefits; REC 6. On board Flight Management Systems route segments databases should be capable of selecting the additional route segments possibilities allowed by Free Routing operations. REC 7. For the future, it is mandatory to aim for a defragmented FRA which works cross border and which is homogenous with regards to the settings or limitations of Minimum Flight- Level and points to be used (Waypoints VS. Lat Long Points). REC 8. For ANSP need and for fuel check routines, there is a need to split the flight plan into segments of 50/60 Nm. 190 of 213

191 Project Number LSD Edition Summary of the Communication Activities Communication activities and exploitation of results are an importan task for any SESAR project. FREE Solutions has not represented an exception with several dissemination actions aimed at communicating project activities results to a wide aviation audience. FREE Solutions communication activities included classical initiatives, (e.g., inter-project coordination, scientific publications, workshops and training activities), but also went a step further by using new ways of spreading information (e.g., social networks and information upload to websites). In parallel to communication, exploitation of results also played a major role, namely for some ANSPs partners as the VLSD project has achieved a strong impact towards the FRA solutions deployment. Exploitation activities of the FREE Solutions consortium were structured along several dimensions, including an overall ad-hoc end-of-project communication/dissemination event with more than 50 participants representing main stakeholders such as AUs, NSAs, ANSPs, CFPS, Professional Associations and so forth. FREE Solutions teams informal communication activities performed at some national meetings/seminars, with the involvment of media too, also represented an additional means to convey project results to decision and policy makers as well as to national political level. More in detail, FREE Solutions communication activities were performed in the period October 2014 through July Partners have generally followed the agreed communication approach included in the Demonstration Plan while executing their communication roadmaps alongside the life of the project; to this regard, Task 6.2 has implemented a number of communication actions in accordance with the actual execution of the demonstration exercises. In fact, being the core of the project the Flight Trial campaign, a dedicated team was tasked for the proper promotion of the project in this phase through a tailored marketing campaign. All data gathered during the first set of flight trials (city pairs) were elaborated in order to quantify the confidence perceived by the involved actors for each scenario, recommendations for improvement, elements to support and/or speed-up the regulatory processes both at European level and at national ones, issues, risks and blocking points. FREE Solutions project activities are mainly focusing on two ATM Functionalities already identified in the Pilot Common Project: Flexible Airspace Management and Free Route; Network Collaborative Management. The scope of the project is the definition and demonstration of Free Route operational solutions over a wide multi-fab continental area to improve ATM performances, flexibility and efficiency. This can be attained following the criteria below: Demonstrate the benefits of the proposed FREE Solutions in a cross-border/multi-fab scenario to the wider aviation community; Accelerate the cross-border Free Route operational acceptance and the subsequent deployment of the proposed operational solutions; Develop a positive business case for the deployment of cross-border Free Routing operations; Deliver recommendations and more advanced ATM system requirements in support of the deployment/certification of Free Routing operations; Contribute to SESAR work by delivering a comprehensive assessment on the operational and technical feasibility of Free Routing operations. The dissemination process was presented to the air transport community through the following means: Dedicated web pages to showcase FREE Solutions progresses and results; 191 of 213

192 News via distribution to provide high level information on the Project, including articles on specific topics and interviews to project participants; Publications of articles and press releases or editorial initiatives targeted to promote project results; Cascading key messages via social media (Twitter); Workshops, Exhibitions: Project promotion/presentation to the aeronautical community via major conferences and appropriate fora (e.g. World ATM Congress in Madrid, SESAR Demo Days); Short "About" project description was released in simple language consistently with the SJU guidance material accessible to non-specialist readers Partners internal channels of communication among which intranet, corporate alerts or company newsletter magazine; Final Dissemination Workshop. Each Partner identified an officially nominated communication focal point to execute this task throughout the Project lifecycle. addresses of Communication Points of Contact are given below: Name Organisation Morten Grandt DFS François Richard Bole DSNA Jerome Dufossez DSNA Giancarlo Ferrara ENAV Cristiana Abbate ENAV Naomi Galea MATS RaimundFridrich SKYGUIDE Pascal Latron SKYGUIDE Christofer Brain EUROCONTROL/NM Borce Dvojakovski EUROCONTROL/NM Franck Montoya EUROCONTROL/NM Jacqueline Coquel AIR FRANCE Daniele Di Febo ALITALIA Andreas Herzog-Havel LUFTHANSA Choorah Singh RYANAIR 192 of 213

193 Paolo Iacobelli RYANAIR Andreas Triska SWISS Table 105: Communication Points of Contact As already mentioned, the initial communication timing schedule is being refined through the project lifespan and particularly with the actual development of the flight trials campaign. Most of communication activities are centred on the identified set of demonstration flights. The table below is the list of planned communication activities as reported in Demonstration Plan: the column reference number has been added to show a direct link with the articles, leafleats, magazines actually released so far. Media / Activity When Who responsibl e Where Expected Cost Directly Targets Referen ce number TV News (internal media) Launch of project activity DSNA DSNA: Headquarters + all main operational units Cost of HR Staff Visitors DSNA Leaflet TV News Information and briefings in involved ACCs Spring 2015 (City Pairs live trials) DSNA Intranet DSNA : Headquarters, operational units including overseas territories Cost of HR Staff Visitors DGAC magazine Customers news TV News Information and briefings in involved ACCs Autumn 2015 (Direct Routing live trials) DSNA Internet Intranet DSNA : Headquarters + all main operational units Cost of HR DSNA customer s External stakehold ers Staff 24 DSNA Leaflet Customers News TV News Information and briefings in involved ACCs Winter- Spring 2016 (Free Routing live trials) DSNA Intranet DSNA : Headquarters, operational units including overseas territories Cost of HR DSNA customer s External stakehold ers Staff Annual report DSNA n.a. Cost of HR External Stakehold 193 of 213

194 Media / Activity When Who responsibl e Where Expected Cost Directly Targets Referen ce number ers Article Project Announ cement Issue: July/Aug ust 2014 ENAV Cleared /ENAV Magazine n/a ENAV Employee s and external stakehold ers 1 Press release Project Launch Issue: Septem ber 2014 ENAV Cleared /ENAV Magazine n/a ENAV Employee s and external stakehold ers 2 Press release Project Launch Issue: Septem ber 2014 ENAV Newsletter Nr.9 v.it/enavwebpo rtalstatic/eclea red/ita/articoli /art092014/arti colo5.html n/a ENAV Employee s and external stakehold ers 3 Presentation and/or participation in any SJU led activity at the CANSO World ATM Congress Q ENAV/FRE E Solutions Project Manager Madrid n.a ATM Stakehold ers at large 9 Leaflet at Eurocontrol Network Manager User Forum 28-29/01/2 015 ENAV/FRE E Solutions Project Manager Brussels n.a ATM Stakehold ers at large 4 Press releases Articles Flight Trials Winter- Spring 2015 ENAV&ALL n.a. n.a. ATM Stakehold ers at large (City Pairs live trials) 194 of 213

195 Media / Activity When Who responsibl e Where Expected Cost Directly Targets Referen ce number Press releases Articles Press releases Articles Flight Trials Autumn 2015 (Direct Routing live trials) Flight Trials Winter- Spring 2016 (Free Routing live trials) ENAV&ALL n.a. n.a. ATM Stakehold ers at large ENAV&ALL n.a. n.a. ATM Stakehold ers at large Open Days Flight Trials End of March 2015 ENAV Roma ACC Milano ACC tbd ATM Stakehold ers at large Communicat ion sessions on SESAR trials / results during NM users forums General Info on forthcoming trials, Available results of demonstratio ns January 2015, 2016, 2017 EUROCON TROL NM NM users forums Labour costs AOs, ANSPs 4 EUROCONT ROL Extranet and Usual Publications Synthesised Results of demonstratio Last quarter of 2016/ First quarter of 2017 EUROCON TROL NM Extranet Labour costs AOs, ANSPs, Other users of EUROCO NTROL extranet of 213

196 Media / Activity When Who responsibl e Where Expected Cost Directly Targets Referen ce number ns / Potential for implementati on Articles in internal media Articles in internal media Articles in internal media Articles in internal media Update Internet skyguide Annual report skyguide Autunm 2014 Spring 2015 (City Pairs live trials) Autumn 2015 (Direct Routing live trials) Winter- Spring 2016 (Free Routing live trials) Regularl y from autumn 2014 until end of project Skyguide n.a. Cost of HR Staff 21 Skyguide n.a. Cost of HR Staff skyguide n.a. Cost of HR Staff Skyguide n.a. Cost of HR Staff Skyguide n.a. Cost of HR External Stakehold ers Skyguide n.a. Cost of HR External Stakehold ers Internal distribution of FABEC Com material (newsletter, newsflash, ) tbd Skyguide n.a. Cost of HR FABEC staff SWISS On- 2 SWISS SWISS On Labour cost SWISS of 213

197 Media / Activity When Who responsibl e Where Expected Cost Directly Targets Referen ce number Board Magazine Launch of Project Activity and general information for passengers Waves: 1 st quarter 2015 and 4 th quarter 2016 with results Board Magazine passenge rs SWISS Employee Journal AIRMAIL (monthly) 2Waves : 1 st quarter 2015 and 4 th quarter 2016 with results SWISS Airmail Labour cost Various range of SWISS employee Table 106: Planned communications activities The following table is the list of the activities actually performed: each one with its reference number, to have a direct comparison between forecasted and actual communication activities. Who Media Link Target group(s) Date Subject Languag e Refer ence numb er ENAV 1 Corporate Magazine Cleared t/portal/page/port al/portaleenav/ Home/Comunica zione?currentpa th=/enav/it/comu nicazione/pubblic azioni/cleared All Employees ATM Community (Italian) July/August 2014 General Info on Demonstrati ons and Free Solutions Insight Italian ENAV Corporate t/enavwebportals tatic/ecleared/it A/articoli/art /articolo5.html All Employees ATM Community (Italian) September 2014 General Info on Project Launch Italian 2 ENAV Corporate Magazine Cleared t/enavwebportal Static/cleared/CL EARED_8_2014/ CLEARED_8.ht ml#p=1 All Employees ATM Community (Italian) September 2014 General Info on Project Launch Italian of 213

198 ENAV All Partners ATM Event at EUROCONTR OL Stakeholders at Large October 2014 Project Leaflet "Demonstrati ng SESAR: From Innovation to Free Route Operations English 4 ENAV SESAR Annual Demo Worskhop SESAR JU/ ATM Community Toulouse, October 2014 Project Presentation English 5 ENAV Twitter FREESolutions1/ Stakeholders at Large November 2014 Project Launch English 6 ENAV Twitter FREESolutions1/ Stakeholders at Large Several messages sent out March 2015 about flight trials execution English 7 ENAV Bluemed Newsletter ed.aero/public/re port/blue_med _Newsletter_issu e_6.pdf.pdf ATM Community July 2015 Article MATS and the Free Solutions Project: from Innovation to Free Route Operations English 8 ENAV World Congress ATM PAVING THE WAY FOR THE ATM Community 11 March IMLEMENT ATION OF 2015 THE SESAR FREE ROUTE SOLUTION English 9 ENAV Corporate Magazine Cleared t/ec5/enav/it/pdf/ cleared/cleare D_3_2015_mini_ 1.pdf All Employees ATM Community (Italian) Article on Flight Trials March 2015 Execution Campaign Italian 10 ENAV Corporate t/enavwebportals tatic/ecleared/it A/articoli/art /articolo3.html All Employees ATM Community (Italian) News on Flight Trials March 2015 Execution Campaign Italian 11 ENAV Corporate Magazine Cleared t/ec5/enav/it/pdf/ cleared/cleared_ genn_2016.pdf All Employees ATM Community (Italian) January 2016 Article on Free Route Flight Trails Campaign Italian 12 ENAV Corporate Annual Book Photo t/ec5/enav/it/pdf/ cleared/enav_im All Employees ATM Community (Italian) March 2016 News on City Pairs Flight Trials Italian of 213

199 2015 un mag_2016.pdf anno di Immagini ENAV Free Solutions Leaflet WAC 2016 ATM Community 8-10 March 2016 Madrid English 14 ENAV Free Solutions Section on International Strategy Brochure WAC 2016 ATM Community 8-10 March 2016 Madrid English 15 EUROCON ROL NM Airline Operations Group (AOG) 15 meeting (EUROCONTR OL HQ) control.int/sites/d efault/files/public ation/calendar/ev ent-meeting- extclndr v62.pdf ATM Community with more than 40 airlines October 2015 Free Trials Route English 16 ALITALIA 24/09/ Internal Newsletters ALITALIA Staff 11/03/ /03/2015 Italian 24/03/2015 ALITALIA Technical communication with flyers and dedicated e- mail ALITALIA dispatchers and pilots n.a. Italian 18 DLH Internal Newsletter 19 SWISS Inside Swiss Article on the Project SWISS Employees SWISS investigates direct flight routes English 20 SWISS SWISS Magazine Article on the Project SWISS Employees Free Solutions English 21 SWISS Internal Newsletter My SWISS Article /Interview SWISS Employees 04 June 2015 Interview to Pilot Andreas Triska German 22 SWISS Onboard Magazine World of Swiss Article SWISS Passengers Interview to Pilot Andreas Triska German 23 SWISS POLITIKEN Newsletter (DK) Article on Danish Press Publich at Large 04 May 2015 Interview to Pilot Andreas Triska Dutch 24 SKYGUID E Corporate Communication ide.ch/webpublic ations/annual_r SKYGUIDE Employees 10 October 2014 Annual Report English of 213

200 eport_2014/en/in dex.htm DLH Article on Intranet DLH Employees March 2015 Flight Trials Presentation German 26 DLH Intranet DLH Employees Article on Flight Trials Execution German 27 DSNA Newsletter oppementdurable.gouv.fr/2 DSNA & Vous Issue SESAR- Website Free-Route.html DSNA Employees Externally & 06 October 2015 Article on Free Route English 28 ALITALIA Internal Newsletter Alitalia Employees 24/09/ /03/ /03/ /03/2015 Content: Poject and Trials Italian 29 ALITALIA Internal Newsletter Alitalia Employees November 2015 Autumn Trials Italian 30 ALITALIA Internal Technical Communication (flyers dedicated s) and Dispatchers and Pilots 2015 Italian 31 ALITALIA (in coordinatio n with ENAV) External External Communication Italian Media Audience 2016 Final Results Italian 32 Lufthansa Politikbrief Ministry transport of June 2016 Final results Germany 33 ENAV/ALL Final Communication Workshop ATM Community 14 July (more than participants) Rome English Table 107: Actual communications activities - city pairs, DRs & FRA The Picture below provide a one-shot view of performed communications activities which are not all reported for the matter of document readability. 200 of 213

201 Figure 116: Examples of Communications Activities - City Pairs 201 of 213

202 Project Number LSD Edition In addition to the promotion through traditional media channels, FREE Solutions also explored and widely utilized Twitter as a social on-line communication channel. All in all, these activities supported and added value to the overall findability and visibility of FREE Solutions online. The Twitter channel was actively used during the lifetime of the project, and in general these types of channels benefit from a higher number of individuals than those engaged actively on more traditional channels. In this framework FREE Solutions Twitter ( was widely used as an internal/external conversational/communication channel on the news and events related to the project activities. Twitter was mainly used during the Flight Trials sessions and all linked dissemination events and, with the support of a number of other tweeters and linked twitter-accounts, the hash tag #FREESolutions was used. Twitter was also widely used for the promotion of events where FREE Solutions presented and/or participated. Some interactions with individuals, press channels and other projects were achieved too. The combination of individuals and projects re-tweetting, visualizing, liking, interacting and commenting on the news posted and the ability to share images and links to individual project members have provided a good vehicle to broaden the visibility of FREE Solutions outside the core project community (e.g. ECA, AEA, IATA etc.) and with respect to the aviation sector press (e.g. Air Traffic Management, Avionics Magazine, Aviation Week etc.). The recorded statistics shows the viral reach for FREE Solutions content on its Twitter account. Even with almost 160 followers, the content reached in many cases more than 2500 individuals/company/associations/press reps which have direct relationships with project members and/or the ATM/Aviation domain Figure 117: Examples of Communications Activities RTS 202 of 213

203 Figure 118: Examples of Communications Activities Free Routing Figure 119: Examples of Communications Activities Free Routing (2) 203 of 213

204 Figure 120: Examples of Communications Activities Leaflet Figure 121: Examples of Communications Activities Brochure Figure 122: Examples of Communications Activities Final Event 204 of 213

205 Next Steps 8.1 Conclusions In the framework of Free Solutions project a thousand of trial flights, roughly, were executed, demonstrating the feasibility of Free Route operations with the current systems and operational procedures. This was achieved through the use of both direct routing and free routing operations, leading to the delivery of a set of measured benefits in terms of reduced fuel burnt and related CO2 and NOx emissions, reduced flight time and routes length. Those benefits were compared against current daily operations (i.e. reference scenario) and were measured for the trials periods only. The overall results of the project are then summarized in the table below, showing that the project demonstrated the expected benefits, in terms of fuel efficiency-environment sustainability and with no negative impact on the safety of operations and both local and network operational capacity Table 108: Overall results of the FREE Solutions demonstrations In addition to the demonstrated benefits it is worth mentioning that FREE Solutions trials directly contributed to various operational implementation programmes, by providing outputs and lessons learnt. In particular, it contributed to: The Implementation of FRA in Italy above FL335 at the end of the year 2016; Based on trials results and feedback, the following DR segments are going to be published, as part of the future plans for implementation: o VANAS GODRA from FL305 - H24; o BEGAR ODINA from FL195 night; o SUXAN HOC from FL245 night; o GODRA DITON from FL245 - H24; o ARKIP LMG from FL245 - H24 (included FUA restrictions). For further details and analysis, please refer to the following sections (Error! Reference source not found., Error! Reference source not found.error! Reference source not found., Error! Reference source not found., Error! Reference source not found., Error! Reference source not found.). Besides, the following table intends to describe in a single snapshot how the different FREE Solutions exercises contributed to the reference SESAR Solutions. In particular, it synthetises the results obtained for the different KPAs and TAs together with the considerations made for the supporting technical enablers. In addition, some notes concerning areas of improvement are also identified. SESAR SOLUTION #32 Free Route through the use of direct routing for flights both in EXERCISES KPA/TA/ENB Live Trials Results Limitations and Opportunities for Improvements EXE- Safety No negative effects LSD on operations 001; EXE- during live trials LSD of 213