ICAO EUR Region Performance Framework (ICAO EUR Doc 030) World ATM Congress Madrid, 5 th March 2014 ICAO European and North Atlantic Office 6 March 2014 Page 1
Objectives - Present the context: the ICAO global framework and regional initiatives - Describe the ICAO EUR Region performance framework - Raise awareness for non-regional stakeholders and provide information to support implementation for regional stakeholders - Collect inputs and views from participants 6 March 2014 Page 2 2
Introduction States in the ICAO EUR Region, responding to ICAO requirements, decided to elaborate a Regional Performance Framework, to improve the performance of the aviation system and indentify areas where improvements are possible To prepare such framework a dedicated Task Force was created The TF analysed what already in place in/outside the Region to understand the context, to make maximum use of what available and avoid duplications The TF prepared a comprehensive framework with KPAs/KPIs and guidance material to support implementation We are at the beginning of the implementation phase 6 March 2014 Page 3 3
ICAO Performance Based Approach ICAO European and North Atlantic Office 6 March 2014 Page 4
Air Navigation Conference 11 in 2003 How did we start? Endorsement of the global ATM operational concept already at 11 th Air Navigation Conference in 2003 and ICAO is requested to:» develop ATM system requirements» address interoperability and seamlessness» define requirements for global AIM» publish the operational concept» amend the Global Plan» develop a performance framework States and PIRGs consider the Global Air Navigation Plan for CNS/ATM Systems as a catalyst for change, providing a global safety and interoperability framework while allowing regional or local adaptation to efficiently meet regional and local needs States and PIRGs agree on a set of metrics related to KPAs, incorporate these metrics into the performance monitoring process and review their results on a regular basis (EANPG,RASG) 6 March 2014 Page 5
Air Navigation Conference 12 in 2012 How do we continue The Conference unanimously endorsed the Aviation System Block Upgrades (ASBUs) framework introduced by ICAO to set goals in terms of operational improvements on a consensus-driven basis. The Conference also endorsed the draft 4th edition of the Global Air Navigation Plan (Doc 9750), as unified planning mechanism, which was approved by the 38 th ICAO Assembly in September 2013 ICAO is tasked to include the key policy principles in the GANP, develop financial policies, SARPS development plan, define a stable and efficient process to update the GANP ICAO (through Regional Offices) will provide guidance and assistance on ASBUs implementation to States, establish a group and mechanism for interregional cooperation 6 March 2014 Page 6
GANP and GASP Advantages A globally harmonized and time-wise synchronized set of ATM improvements to direct the work of States, ANSPS, AOs, airports, ICAO and other international and standard making organizations A globally agreed ATM baseline through commitment to implement Block 0 A systematic approach to ATM improvements, a packaged solution that includes standards/guidance, technology, procedures, regulatory framework, business case, performance measurements 6 March 2014 Page 7
AN-Conf/12 Recommendations Recommendation 1/15 Performance monitoring and measurement of air navigation systems The Committee noted that the implementation of One Sky would require the use of a common language regarding performance monitoring and measurement. This implies a coordinated approach across States and stakeholders for data provision, collection, storage, protection, dissemination, indicator calculation and use of the results to support the various ATM improvement processes. It was agreed that there was a need to develop a global methodology, to identify metrics and indicators which could be used to allow States and regions to measure and evaluate the effectiveness of their ATM performance initiatives. In doing so, particular attention should be given to avoiding duplication of efforts and to use, to the maximum extent possible, existing arrangements and solutions That ICAO: a) establish a set of common air navigation service performance metrics supported by guidance material, building on existing ICAO documentation (e.g. Manual on Global Performance of the Air Navigation System (Doc 9883) and the Manual on Air Navigation Services Economics (Doc 9161)); b) promote the development and use of leading safety indicators to complement existing lagging safety indicators as an integral and key component to drive improvement in performance and in the achieved management of risk; and; c) encourage the early and close involvement of the regulator and oversight bodies in the development, proving of concepts and implementation of the aviation system block upgrades and regional programmes. 6 March 2014 Page 8
The wider ATM perspective A wider (more regional, if not even global vision) perspective is needed to make even greater gains in capacity and efficiency A far-reaching cooperation is necessary in order to support the implementation of facilities and services over larger geographical areas Enhanced planning perspectives and a global framework for performance measurement is required to support the implementation activities States and all aviation stakeholders should work together towards a common vision (global ATM Operational Concept) use a common planning framework Global Air Navigation Plan (GANP), the regional air navigation plans and several other documents and tools (e.g. European ATM Masterplan) Global Aviation Safety Plan (GASP) utilize harmonised performance objectives (targets, metrics, indicators) Interoperability and harmonization are key factors in order to ensure the required improvements for the future 6 March 2014 Page 9
ICAO EANPG Ensure that air navigation system development plans and actions within the EUR Region remain coherent and compatible with the ICAO Strategic Objectives, with the ICAO global plans (especially GANP and ASBUs, parts of GASP), with those of the adjacent ICAO Regions and with other world-wide provisions. Manage the EUR Air Navigation Plan (Performance directives and associated requirements for facilities and services established through regional air navigation agreements, in support of the global air navigation infrastructure) and facilitate the implementation of the international operational requirements contained therein. Promote and facilitate the harmonisation and co-ordination of the air navigation related programmes (e.g. SESAR, State ATM Corporation modernisation program, NextGen) of other international organisations such as the European Commission, the European Civil Aviation Conference (ECAC), Eurocontrol and the Interstate Aviation Committee of the Commonwealth of Independent States (IAC/CIS). Ensure the interoperability of the EUR CNS/ATM systems with those of adjacent regions and in line with the ICAO GANP. Assist States in their planning and implementation efforts, if and when required. 6 March 2014 Page 10
Performance Based Approach ICAO Performance Based Approach is based on the following three principles: Strong focus on desired/required results Informed decision making, driven by the desired/required results Reliance on facts and data for decision making 6 March 2014 Page 11
Focus on desired/required results Instead of prescribing solutions, desired/required performance is specified. Management attention is shifted from a resource and solution centric view (how will we do it) towards a primary focus on desired/required (performance) results (what is the outcome we are expected to achieve). This implies finding out what the current performance situation is, what the most appropriate results should be, clarifying who is accountable for achieving those results. 6 March 2014 Page 12
6 March 2014 Page 13 ICAO EUR/NAT Office COG PERF TF Workshop 13 Performance terms Performance objectives: Specific, Measurable, Achievable, Relevant & Timely (SMART) objectives defined to satisfy ATM community expectations. Performance indicators: to measure achievement of performance objective Performance targets: value of performance indicators that need to be reached/exceeded to fully achieve performance objective. Performance gaps: Performance indicators compared against performance targets identifying gaps Performance assessment metrics: determine which data needs to be collected to calculate values of performance indicators
6 March 2014 Page 14 ICAO EUR/NAT Office COG PERF TF Workshop 14 Expectations KPA Objectives Indicators Quantitative Targets... Objective is met when indicators meet or exceed targets
6 March 2014 Page 15 ICAO EUR/NAT Office COG PERF TF Workshop 15 Performance assessment Assess current performance: establish baseline & track progress towards performance objectives Identification/diagnosis performance gaps: Performance gaps denote current/anticipated mismatch (unfavorable) between current/planned performance and performance target. Addressing performance gaps: Operational improvements in terms of changes to ATM system that are on transition path towards Global ATM Operational Concept and result in a direct performance enhancement. Developing list of options for operational improvements: Operational improvements made possible by technical systems, human factors, procedure and industrial enablers in terms of feasibility, timing, cost and impact on operational change. Global Air Navigation Plan (Doc 9750) is one of the sources, which can be used. Building/updating transition roadmaps: Performance gaps identify the affected performance objectives. Categorization of operational improvements according to performance objectives leads to development of appropriate shortlist of candidate solutions. If the solution is not included in the transition roadmap, then it should be selected on the basis of maturity or research, if no mature solution is available.
In summary Understanding Performance is key for successful transition from past and current to the future Working under one common umbrella Performance based transition Continued operational improvements Toward a common vision Using the ICAO framework Using the 3-step transition approach Set objectives/metrics Identify and understand performance gaps Select and plan Operational Improvements Supporting the goal of a more seamless and performance based Global Air Navigation System 6 March 2014 Page 16
Existing processes in the Region ICAO European and North Atlantic Office 6 March 2014 Page 17
EU Performance scheme SES II - Performance in RP2 (2015-2019) Binding performance targets in 4 key performance areas (Safety, Environment, Capacity and Cost-Efficiency) Mandatory implementation of a financial incentive scheme for the target achievement on the KPA Capacity Performance planning on FAB level 'gate-to-gate' approach covering the entire chain of air navigation services Target setting on terminal charges as of 2015 (Union-wide target setting foreseen as of 2017) Network Manager integral part of the scheme Clear link of new investments and major overhaul of systems to the EU ATM Master Plan 6 March 2014 Page 18
Performance reports Performance Review Report analysis of the performance of the ATM systems in the KPA of Safety, Capacity, Environment and Cost-efficiency in Europe ATM Cost-Effectiveness Benchmarking Report analysis of cost-effectiveness and productivity in 37 European ANSPs US-Europe comparison reports analysis of the operational and economic ATM performance - Comparison of Air Traffic Management-related Operational Performance 2008-2012 - Comparison of ANS cost-efficiency trends 2002-2011 - Joint publications influenced ICAO and CANSO work on global performance benchmarking 6 March 2014 Page 19
ICAO EUR Region Performance Framework ICAO European and North Atlantic Office 6 March 2014 Page 20
ICAO EUR Performance Framework WHO? WHY To elaborate a Performance Framework for the EUR Region, to improve the performance of the aviation system and indentify areas where improvements are possible WHEN Decision at COG 47 (Rostov on Don, June 2010) HOW Based on clear principles: Produce improvements to the aviation system Avoid duplication of efforts and make maximum use of existing arrangements As far as possible be applicable also in the NAT Region Consider impact on resources Follow a pragmatic and stepwise approach Should also represent Regional input to ICAO HQ activities in developing metrics for global use Support the future developments of EU Performance Scheme 6 March 2014 Page 21
ICAO EUR Performance Framework WHAT? Input ICAO framework (ANC 12, GA38, etc.) EU-ECTL (Single European Sky and Performance Scheme) FAA Russian Federation ICAO EUR Workshops (Rome 2011, Bishkek 2013) Pragmatic approach in developing the proposal Output Development of a comprehensive framework Identification of 6 KPAs out of 11 ICAO KPAs Definition of focus areas, objectives and KPIs Definition of Processes, Roles and Responsibilities Guidance material To assess the Regional performance and to identify areas where improvements are possible 6 March 2014 Page 22 Safety Access and equity Capacity Efficiency Environment Flexibility Global Interoperability Cost Effectiveness Predictability Security Participation by ATM community
Performance Framework Document ICAO EUR Region Performance Framework Document (ICAO EUR Doc 030) was approved by EANPG and COG in 2013 and published in English and Russian language. Performance Framework document describes: INTRODUCTION BACKGROUND INFORMATION RELATIONSHIPS WITH ICAO GLOBAL DEVELOPMENTS AND REGIONAL INITIATIVES PART A SCOPE ROLES AND RESPONSIBILITIES KPA/KPI/METRICS MONITORING, REPORTING AND REVISION PROCESS AT REGIONAL/NATIONAL LEVEL PART B KPAs TEMPLATES FOR THE REGIONAL PERFORMANCE REVIEW REPORT ANNEX 1: TERMINOLOGY ANNEX 2: REFERENCE MATERIAL 6 March 2014 Page 23
KPAs-KPIs summary table KPA OBJECTIVES FOCUS AREAS INDICATORS SAFETY Ensure safety continuous improvement through reduction of ATM related safety occurrences and implementation of uniform safety standards Effectiveness of Safety Management (Safety Maturity Questionnaire) Level of State Safety/Just culture (Safety Culture Questionnaire) Adoption of an harmonized occurrences severity classification methodology CAPACITY Capacity meets demand for enroute and at airports En-route ATFM Delay Airport ATFM Delay Average en-route ATFM delay generated by airspace volume Average ATFM delay per flight in the main airports (to be identified by States) EFFICIENCY Ensure users may use most efficient routes Horizontal Flight Efficiency Average horizontal en route flight efficiency (length of the en route part of the actual trajectory/last flight planned route vs great circle) ENVIRONMENT Contribute to the protection of environment (fuel/co2 emissions reduction) CO2 emissions related to inefficiencies in route extension COST EFFECTIVENESS Contribute to optimization of costs for ANS ATCO Productivity IFR Flights (en-route) per ATCO hour duty IFR flight hours per ATCO hour on duty IFR movements per ATCO hour on duty PARTICIPATION BY ATM COMMUNITY Ensure States' participation to Regional planning and implementation activities Level of participation to meetings Level of responses to planning activities Level of provision of performance results 6 March 2014 Page 24
Safety KPA Overview KPA Objective Indicators Safety Ensure the continuous improvement of safety through the reduction of ATM related safety occurrences and the implementation of uniform safety standards. - Effectiveness of Safety Management (EoSM) - Level of State Just Culture (JC) - Application of a common methodology for classification of occurrences in terms of risk severity 6 March 2014 Page 25
Safety KPA Nature of indicators Effectiveness of Safety Management (EoSM) Based on the annual completion of two questionnaires (State level and ANSP level) Level of State Just Culture (JC) Based on the annual completion of two questionnaires (State level and ANSP level) Application of RAT methodology for Severity Classification Objective is to motivate States to use a common methodology for the severity assessment of safety occurrences Benefit: harmonisation of occurrence statistics 6 March 2014 Page 26
Safety KPA Conclusions Three Indicators These are leading indicators measuring precursors to improved safety And not lagging indicators which are directly related to safety outcome Application Have already been used by many States since 2012 during SES Reference Period 1 (RP1) and now reinforced for RP2 Under ICAO umbrella geographical extension to remaining States of the EUR Region Complex subject, but detailed guidance material is available Fully described in EASA documentation ICAO guidance material summarises and explains the EASA documentation 6 March 2014 Page 27
Capacity KPA Overview KPA Objective Indicators Capacity Ensure that Air Navigation Service capacity meets demand in en-route airspace and at airports - Average ATFM delay per flight generated by the airspace volume (en-route) - Average ATFM delay per flight in the main airports (to be identified by States in advance and based on the regional relevance) 6 March 2014 Page 28
Capacity KPA Application Re-use of ATFM delay indicators as defined in SES Performance Scheme Regulation Measures the delay generating entities (airspace and airports which are bottlenecks), not the penalised entities (flights) Covers majority of States where capacity is an issue ATFCM Area Relevance in remainder of EUR Region to be determined Capacity often not (yet) an issue or only a local issue Aircraft held at dep. airport Departure Airport NM ACC anticipating capacity shortfall ATFM regulation requested by ACC Arrival Airport 6 March 2014 Page 29
Capacity KPA Conclusions Indicators measure The location where the problem is, not where the delay is taken (departure airport) Performance of airspace volumes and airports, not flights Despite the expression as a value per flight Within the Capacity KPA Demand/capacity imbalance Not capacity itself Limitations Not designed to measure excess capacity No data if airspace or airport does not participate in a centralised ATFCM process 6 March 2014 Page 30
Efficiency KPA Overview KPA Objective Indicator Efficiency Ensure that users [can?] use the most efficient routes focussing on the horizontal flight-efficiency Average horizontal en route flight efficiency, defined as the difference between the length of the en route part of the actual trajectory (where available) or last flight planned route and the great circle. Note: specificities shall be considered for flights longer than 1000 nm where the optimum could differ from the great circle (wind optimal routes, etc.). 6 March 2014 Page 31
Efficiency KPA Origin of indicator Re-use of indicator as defined in SES Performance Scheme Regulation The average horizontal en route flight efficiency of [(a) the actual trajectory, or (b) the last filed flight plan trajectory], defined as follows: (i) the indicator is the comparison between the length of the en route part of [(a) the actual trajectory derived from surveillance data, or (b) the last filed flight plan trajectory] and the corresponding portion of the great circle distance, summed over all IFR flights within or traversing the European airspace; (ii) en route refers to the distance flown outside a circle of 40NM around the airports; (iii) where a flight departs from or arrives at a place outside the European airspace, only the part inside European airspace is considered. Note: the SES indicator is expressed as a percentage: Sum of extra distance divided by the sum of corresponding portions of great circle distances 6 March 2014 Page 32
Efficiency KPA Nature of indicator Indicator option A Last Filed FPL Indicator option B Length of Trajectory Reference distance Extra distance Shortest Available Route Shortest Route Awareness and Choice Route Network Design City pair distance Planning (Great circle distance) Based on information known in advance Route & Airspace Availability Business need: Get from A to B = direct route Actual Trajectory Operations Difference due to planning limitations ATC Separation Fragmentation Wind-optimum Cost-optimum Tactical decisions based on updated information requires surveillance data Extra distance Reference distance 6 March 2014 Page 33
Efficiency KPA Conclusions Indicator Measures actual (flight plan data or radar data), achieved, and additional distance flown within an airspace volume (eg within a State) Can cope with overflights Additional distance can be computed separately for flights shorter and longer than 1000 nm Results are geographically aggregatable Represents the overall outcome of several influencing factors, for which different Stakeholders are accountable 6 March 2014 Page 34
Environment KPA Overview KPA Objective Indicator Environment Contribute to the protection of the environment focussing on fuel savings and CO2 emission reductions CO2 emissions deriving from inefficiencies in flight efficiency (conversion of additional distance into CO2 emissions based on standard values formula). 6 March 2014 Page 35
Environment KPA Definitions Reference to excess fuel consumption: Total additional distance flown in the airspace volume (i.e. State) multiplied by a standard fuel consumption factor (value State-specific) Reference to excess CO2 emission: Reference to excess fuel consumption multiplied by 3.15 (net carbon content of kerosene) Trajectory Actual distance Achieved distance Additional distance Standard fuel consumption per km Reference excess fuel consumption Net carbon content of kerosene (3.15) Reference excess CO2 emission 6 March 2014 Page 36
Environment KPA Conclusions Indicator Has low data requirements and is therefore easy to implement Because mostly based on additional distance already computed under the Efficiency KPA Publishes an approximation of excess CO2 emission, resulting from horizontal flight inefficiency The optimum indicator value is not equal to zero Hence the absolute value of the indicator should not be interpreted as representing the CO2 emissions caused by ANS. Indicator can be calculated with IFSET or other tools and should be used for trend analysis only 6 March 2014 Page 37
Cost Effectiveness KPA Overview KPA Objective Indicators Cost effectiveness Contribute to optimize the cost for air navigation services - IFR flights (en-route) per ATCO hour on duty - IFR flight hours (en-route) per ATCO hour on duty - IFR movements (airport) per ATCO hour on duty 6 March 2014 Page 38
Cost Effectiveness KPA Origin of indicator SES indicators not re-usable because dependent on States participation in EUROCONTROL Route Charges System Asking for extensive disclosure of economic and financial information not deemed realistic for the initial EUR Region performance framework Focus is on controller (ATCO) productivity: Staff cost represents the main cost for service provision ATCO staff cost represents the most important share in staff cost ATCO productivity can be expressed in non-financial terms Existing data flow covers many EUR Region States already 6 March 2014 Page 39
Cost Effectiveness KPA Conclusions Indicators Focus on an important component of ANSP costs However with the limitation that working hours per ATCO, employment cost per ATCO and support cost are not covered Re-use data reporting already in place for many States Also reuse existing data definitions and terminology Have relatively simple reporting requirements Only traffic volume and ATCO hours on duty Avoid entering into financial information disclosure issues for the initial implementation of the framework 6 March 2014 Page 40
ATM Community Participation KPA Overview KPA Objective Indicators PARTICIPATION BY ATM COMMUNITY Ensure States participation to regional planning and implementation activities so that air navigation system development plans and action within the EUR/NAT Region remain coherent and compatible with those of the adjacent ICAO Regions and with the ICAO global plan and world-wide provisions. - Level of active participation by States and international organisations - Level of responses to State Letters - Level of provision of requested information or performance results from aviation stakeholders 6 March 2014 Page 41
ATM Community Participation KPA Origin ICAO Manual on Global Performance of the Air Navigation System (Doc 9883) KPA - Participation by the ATM community There are currently no common indicators obtained and some stakeholders are attempting to use the following indicators: number of yearly meetings covering planning, implementation and operation, and covering a significant estimated proportion (e.g. 90 per cent) of the whole of the regional aviation activity number of yearly meetings for planning number of yearly meetings for implementation number of yearly meetings for operations The ATM community is defined in Appendix A of the Global Air Traffic Management Operational Concept (Doc 9854) : It comprises (in alphabetical order) the aerodrome community, the airspace providers, the airspace users, the ATM service providers, the ATM support industry, the International Civil Aviation Organization (ICAO), the regulatory authorities and States 6 March 2014 Page 42
ATM Community Participation KPA Conclusions Three Indicators will be monitored and evaluated by the ICAO EUR/NAT Office Level of participation of States and international organisations to planning and implementation meetings (e.g. EANPG or its contributory groups with a focus on regional coordination/implementation aspects like Aeronautical Fixed Services Group, Air Traffic Management Group, Frequency Management Group, Meteorology Group, Route Development Group, Regional Performance Framework implementation activities) Level of responses to State Letters asking for information on planning and implementation aspects (e.g. Completed Air Navigation Report Forms ARNF, State Letters which assess the status of planning and implementation of key ASBU modules, State Letters concerning the amendment to Regional Supplementary Procedures, Reports on the environmental benefits that were derived from the implementation of operational improvements) Level of provision of performance results from States for the Regional Performance Review Report (RPRR) Crucial support mechanism for PIRGs (EANPG) to monitor planning and implementation activities in the ANP Important for Air Navigation Report Forms (ANRFs) and ASBU monitoring 6 March 2014 Page 43
Implementation and next steps ICAO European and North Atlantic Office 6 March 2014 Page 44
Roles and Responsibilities for implementation Roles and Responsibilities in the functioning of the ICAO EUR Performance Framework EANPG COG and contributory Bodies ICAO EUR Secretariat States Processes at Regional and National level 6 March 2014 Page 45
Process (Regional National) 6 March 2014 Page 46
Implementation of the Framework EANPG 55 (2013) concluded to launch the implementation phase Collect data (2014) Monitor performance Fill the gaps in the system to improve Report performance results in 2015 TF is working to support States and Stakeholders in this implementation Circulation of a questionnaire Identification of focal points Awareness campaign - Meetings and workshops (especially in the Eastern part of the EUR Region) 6 March 2014 Page 47
Implementation of the Framework All material is on the ICAO EUR/NAT Office, Paris website http://www.paris.icao.int/documents_open_meetings/subcategory.php?id=169 http://www.paris.icao.int/documents_open/files.php?subcategory_id=170 Next workshop in Baku (Azerbaijan) 10-11 April 2014!!! ICAO Regional Performance Framework Workshop (State Letter invitation EUR/NAT 14-0109.TEC, from 10 February 2014) 6 March 2014 Page 48
Conclusions ICAO EUR Region has defined a performance framework aimed at monitoring/reporting performance Objective is to identify areas where improvements are possible Built on existing ICAO documents and regional initiatives Proposed as a good practice towards ICAO HQ and other Regions This presentation is available at: http://www.paris.icao.int/documents_open_meetings/subcategory.php?id=169 6 March 2014 Page 49
Thank You 6 March 2014 Page 50