Summary of the Mid-term Review Results

Transcription

1 Summary of the Mid-term Review Results 1. Outline of the Project Country: Cambodia, Lao PDR, Project title: Project for the Capacity Development for Transition Vietnam to the New CNS/ATM Systems in Cambodia, Lao PDR and Vietnam Issue/Sector: Transportation/Traffic Cooperation scheme: Technical Cooperation - Air Transport/Airport Department in charge: Economic Infrastructure Dept. Period of Cooperation: (R/D) January 2011 January 2016 Total cost (as of Apr 2013): 280 million yen Partner Country s Implementing Organizations: SSCA (Cambodia), DCA (Lao PDR), ANSP in Lao PDR (LATM), CAAV (Vietnam), ANSP in Vietnam (VATM, ACV) Japanese Cooperating Organization: Japan Civil Aviation Bureau (JCAB) Related Cooperation: <East Mekong Region> - Master Plan Study on the Development of the New CNS/ATM Systems ( ) - Project for Support on Establishment of the Programs for Operation and Maintenance in Noi Bai International Airport ( ) - Terminal 2 Construction Project in Noi Bai International Airport ( ) <Cambodia> - Advisory Team for Airport Maintenance (Experts) (2002) <Lao PDR> - Project for the Modernization of Equipment for Transition to New CNS/ATM Systems ( ) - Project for Expansion of Vientiane International Airport in Lao PDR ( ) - Air Traffic Safety Improvement Project ( ) 1-1 Background of the Project International Civil Aviation Organization (ICAO) decided the global introduction of New Communications, Navigation and Surveillance/Air Traffic Management (CNS/ATM) Systems, which utilize satellite technology in air navigation, in order to improve safety and efficiency of aircraft operations and to cope with increasing air traffic volumes. Accordingly the ICAO contracting states are enhancing the introduction of new CNS/ATM systems. Responding to this issue, the Government of Japan conducted The Master Plan Study on the Development of the New CNS/ATM Systems for three Eastern Mekong countries of Cambodia, Lao PDR and Vietnam in The study

2 assisted those countries in preparing master plans for system development, human resource development, and development of technical standards, etc. Accordingly, the governments of the three countries are modernizing their air navigation systems recognizing that the transition to the new systems in coordination with neighboring countries is an international obligation. However, they are in need for acquiring knowledge and developing capacity on the new CNS/ATM systems for their civil aviation staff in parallel with modernization of equipment, and requested technical assistance to the Government of Japan. In 2011, Japan International Cooperation Agency (JICA) started The Project for the Capacity Development for Transition to the New CNS/ATM Systems in Cambodia, Lao PDR and Vietnam. The Project has been implementing technical transfer on (1) establishment of Performance Based Navigation (PBN) flight procedures, (2) development of training system for air traffic controllers and air navigation technical staff on the new CNS/ATM systems, and (3) strengthening of safety oversight capability by the introduction of Safety Management System (SMS) in air navigation services. This is a regional technical cooperation project covering the three countries on the basis of bi-lateral agreements between Japan and each of the three countries. The Project term is five years from January 2011 to January Currently, four Long-term Experts, namely Leader/ATM Advisor, CNS Advisor, Flight Procedure Office (FPO) Advisor, and Project Coordinator, are stationed in Hanoi, Vietnam, and provide technical assistance for the three countries from its main Project office located in CAAV in Hanoi, and two Project offices in Phnom Penh and Vientiane arranged by Cambodian and Lao counterpart respectively. 1-2 Project Overview (1) Overall Goal To improve efficiency and safety of flight operations and to enlarge the capacity of airspace through the transition to the new CNS/ATM systems in Eastern Mekong Region. (2) Project Purpose To introduce the new CNS/ATM systems in Vietnam, Cambodia and Lao PDR (3) Outputs: Output 1: To develop capacity for establishing Performance Based Navigation (PBN) flight procedures. Output 2: To develop training system for air traffic controllers and air navigation technical staff on the new CNS/ATM systems. Output 3: To strengthen safety oversight capacity by the introduction of Safety Management System (SMS) in air navigation services (Vietnam and Lao PDR) To establish safety oversight system for air navigation services (Cambodia)

3 (4) Inputs (as of Apr 2013) Japanese side: - JICA Experts Long-term: 4 persons Short-term: 70 persons(1,080days) - Training in Japan: 10 from Vietnam, 7 each from Cambodia and Lao PDR (Total: 24 persons) - Equipment: USD335 thousand (including USD322 thousand for PANADES installed in Vietnam) Vietnamese, Cambodian and Lao sides: - Counterpart: 17 from CAAV, 7 from VATM, 6 from ACV and 1 from VAA, Vietnam (Total 32), 18 from SSCA, Cambodia, 9 from DCA and 6 from LATM, Lao PDR (Total 15) - Office spaces: One office provided both in CAAV, DCA and CATS - Office facilities: provided when necessary for seminar/workshops in three countries 2. Outline of the Mid-term Review Team Review Team Japanese Side Mr. Hiroyuki Ueda Leader Senior Advisor for Transportation Sector, JICA Mr. Kyotaro Harano Air Traffic Management Vice-Chief Air Traffic Controller, Chubu Centrair Airport, Ministry of Land, Infrastructure, Transport and Tourism, Japan Mr. Hideyuki Moriyama Communications, Navigation and Surveillance Air Traffic Services Engineering Division, Air Navigation Services Department, Civil Aviation Bureau, Ministry of Land, Infrastructure, Transport and Tourism, Japan Mr. Kenichi Morikawa Communications, Navigation and Surveillance Air Traffic Services Engineering Division, Air Navigation Services Department, Civil Aviation Bureau, Ministry of Land, Infrastructure, Transport and Tourism, Japan Mr. Kaoru Okada Cooperation Planning Deputy Director, Transportation and ICT Div. 3, Transportation and ICT Group, Economic Infrastructure Dept., JICA Ms. Yuko Tanaka Evaluation Analysis Consultant, Tekizaitekisho LLC Period April 14, 2013 May 18, 2013 Type of Study: Mid-term Review

4 3. Summary of Mid-term Review Results 3-1 Achievements (1) Likelihood of Achieving the Project Purpose Project Purpose: To introduce the new CNS/ATM systems in Vietnam, Cambodia and Lao PDR. At the time of Mid-term Review, there was limited information available regarding the level of achievements of indicators for the Project Purpose as the Project is still in the interim stage. Therefore, it is premature to make any conclusive statements to determine the prospective level of achievements of the Project. Furthermore, the Review Team made some discussions on the wording of the Project Purpose with the Project members, and proposed to modify it as "To enhance the transition to the new CNS/ATM systems in Vietnam, Cambodia and lao PDR for more accurate description. In addition, the Review Team considered that indicators for the Project Purpose are not clearly identified in current PDM (version 1.2 for Vietnam and Lao PDR, version 1.3 for Cambodia). Therefore, the Review Team recommends revising Objectively Verifiable Indicators and Means of Verification for Project Purpose. The details of the proposed modification are explained in Section of country reports. (2) Level of Achievements: Outputs Output 1: To develop capacity for establishing Performance Based Navigation (PBN) flight procedures. Output 1 is composed of the following several sub-outputs: 1) WGS-84 Survey 2) PBN Flight Procedure Design including Ground Validation 3) Flight Validation of Designed PBN Flight Procedures 4) Training of Air Traffic Controllers for PBN 5) Approval of Airlines PBN Operations 6) Provision of RAIM Prediction Information Services (for Vietnam) 7) Publishing of PBN Flight Procedures 1) WGS-84 Survey Theoretical training at CAAV and practical training at Cat Bi Airport for WGS-84 survey was conducted, and critical coordinates of Hue, Phu Bai Airport were surveyed. As a result, Vietnamese authorities learned the procedures for WGS-84 survey including long-baseline analysis for aviation purpose. It was also confirmed by Japanese Short-term Experts that some Vietnamese companies demonstrated capability to conduct WGS-84 surveys with long-baseline analysis and using GPS receivers. Thereafter, ACV conducted supplemental surveys of Hue Airport and completed full survey of Phu Quoc Airport in November 2012.

5 <Cambodia> JICA conducted the Seminar in Hanoi and WGS-84 surveys at Phnom Penh and Siem Reap Airports in 2012 as planned in the PDM. JICA also conducted theoretical training and OJT on the WGS-84 surveys including long baseline analysis. SSCA now understand the verification method of the survey data, the standard survey methods/procedures, as well as a mechanism to oversee the survey process so that required survey data quality for PBN flight procedure designs is ensured. As for the implementation of WGS-84 survey, MOU was signed among SSCA, SCA and MLUC to define roles and responsibilities of relevant organizations/agencies and secure future cooperation. By utilizing the knowledge obtained from the JICA experts and based on the above MOU, SSCA plans to conduct WGS-84 survey of Sihanouk Airport in the second or third quarter of <Lao PDR> JICA conducted WGS-84 surveys at Vientiane Airport in 2011 and Pakse Airport in 2013 as planned in the PDM. JICA also conducted theoretical training and OJT on the WGS-84 surveys including long baseline analysis. DCA understands the verification method of the survey data, the standard survey methods/procedures, as well as a mechanism to oversee the survey process so that required survey data quality for PBN flight procedure design is ensured. According to DCA, National Geographic Department (NGD) is capable of conducting WGS-84 surveys besides long baseline analysis. DCA signed MOU with NGD to secure future cooperation with regard to WGS-84 surveys. NGD will conduct re-survey of Luang Phabang Airport in 2013 to which DCA will participate to supervise the survey work. 2) PBN Flight Procedure Design including Ground Validation The Project implemented the first round of training program that was composed of basic training and OJT (Modules 1, 2 and 3) of PBN flight procedure designs (SID, STAR and APCH). Training provided by the Project included training for PBN design automation tool, PANADES, installed in FPO in CAAV in July At the beginning of the Project, Vietnam had capability to design conventional procedures, and some designers had completed basic training of PBN flight procedure designs. As of April 2013, 8 PBN flight procedure designers are trained a result of the training course conducted under the Project. As for actual design of PBN flight procedures, Vietnamese Task Force conducted the designs of SID, STAR and IAP for Hue Airport, and a part of designed procedures were made effective for temporary operation in March <Cambodia> SSCA has updated the PBN Roadmap, which plans to introduce PBN flight procedures at 3 international airports and 4 domestic airports (Battambang, Stung Treng, Koh Kong, and PreahVihear if required) by At the beginning of the Project, Cambodia had 1 qualified conventional flight procedure designer and 1 official completed basic PANS-OPS training. In

6 total of 4 staff from SSCA are appointed to be the 34 PBN flight procedure designers as a result of the training course conducted under the Project. As for actual design of PBN flight procedures, Cambodian Task Force completed the designs of SID, STAR and IAP for Phnom Penh Airport and conducted a stakeholder meeting with airlines and ANSP. <Lao PDR> The current Lao PBN Roadmap indicates that PBN flight procedures will be introduced at all 4 international airports and 3 domestic airports by 2015 and other 4 domestic airports in At the beginning of the Project, Lao PDR had no qualified conventional flight procedure designer. In total of 4 staff (2 from DCA and 2 from LATM) are appointed to be the PBN flight procedure designers as a result of the training course conducted under the Project. As for actual design of PBN flight procedures, Lao Task Force completed the designs of 13 SID, STAR and IAP for Vientiane Airport in January 2013 and drafted additional 4 procedures. The PBN design work will continue to Pakse and Luang Phabang Airports in ) Flight Validation of Designed PBN Flight Procedures Designed flight procedures must to be validated for flyability, obstacle clearance and GPS signal availability with check flights by flight validation officers before implementation. As for flight validation officers, JICA conducted training by Short-term Experts in January 2013 and plans a training program in Japan in August Vietnamese counterparts conducted the validation of the designed flight procedures for Hue Airport by use of flight simulator of Vietnam Airlines, which is not sufficient as flight validation; but is the most practical method with currently available resources to Vietnam. The Project wait for that ATTECH of VATM procures equipment for flight validation in <Cambodia> SSCA plans to conduct the flight validation of the PBN designs for Phnom Penh Airport in May 2013 with procurement of the services from abroad. As for flight validation officers, SSCA will appoint three officers to be flight validation officers in due course. <Lao PDR> DCA/LATM plan to conduct the flight validation of the PBN designs for Vientiane Airport in June 2013 with procurement of the services from Thailand. As for flight validation officers, DCA appointed 2 officers, however DCA plans to review the current appointment and qualification of flight validation officers after the training in Japan. 4) Training of Air Traffic Controllers for PBN The Project conducted training courses for ATC in September 2012 in Vietnam. Followed by the training, counterparts of each country conducted additional training courses on their own.

7 Counterparts conducted training for Danang Approach and Phu Bai Tower by instructors trained under the Project. In addition, CAAV is currently developing training materials and training programs for related air traffic controllers. <Cambodia> SSCA conducted training in Phnom Penh by instructors trained under the Project. In addition, SSCA modified training materials provided by the Project and utilize them for their training courses. <Lao PDR> LATM will conduct training in Vientiane in late May or early June The syllabus of this training is under approval process of DCA. 5) Approval of Airlines PBN Operations Regarding approval of airline's PBN operations, some progress were made before the initiation of the Project. The Project will proceed to training of more complicated PBN flight procedures such as APV/Baro-VNAV and RNP AR APCH from CAAV has already established the standards for operational approval before the Project, and granted approval to Vietnam Airlines for RNP10 and RNAV5/1 for B777 and Airbus aircraft. At present, 2 flight standard officers who can conduct PBN operational approval are available with CAAV. <Cambodia> SSCA started developing standards for PBN operational approval in September 2012, and will adopt the PBN Operational Approval Handbook before planned implementation of PBN flight procedures at Phnom Penh Airport in July SSCA plans to adopt Advisory Circular for the appointment of one flight standard officer for PBN operational approval. <Lao PDR> DCA has published PBN Operational Approval Handbook in 2010 using COSCAP PBN materials. DCA approved PBN operation for 2 A320s of Lao Airlines and 1 RRJ-95B of Lao Central Airlines by March DCA has 2 flight standard officers capable of developing and updating standards for PBN operational approval. 6) Provision of RAIM Prediction Information Services (for Vietnam) In the PDM, it was planned that Vietnam will procure RAIM prediction equipment to facilitate airspace users of Vietnam FIR. However, due to budgetary constraint of VATM its procurement is planned for <Cambodia and Lao PDR> As this activity is not included in the PDM for Cambodia and Lao PDR, no action has been taken so far. However, the RAIM prediction information may be necessary in the future. The

8 Review Team reports that SSCA plans to procure RAIM prediction information from Aeronautical Radio of Thailand (AEROTHAI). In this regard, DCA mentioned that it will start the study on RAIM prediction information. JICA Experts will assist SSCA and DCA/LATM in this area. 7) Publishing of PBN Flight Procedures Vietnam introduced the first PBN flight procedures for Hue Airport on 15 March 2013 although Hue Airport was closed for runway overlay works till November 2013 shortly after PBN procedures were made available. <Cambodia and Lao PDR> Cambodia and Lao PDR will introduce the first PBN flight procedures (July 2013 for Phnom Penh Airport, and September 2013 for Vientiane Airport) if flight validation is conducted in time. In summary, the overall progress of Output 1 at present is good. The Review Team recognizes that each target country shows good progress in terms of PBN flight procedure designs and a set of approval process before its implementation. On the other hand, WGS-84 surveys and flight validation of designed PBN flight procedures are the most critical risks involved in the implementation of activities under Output 1, which needs to be addressed adequately. Output 2: To develop training system for air traffic controllers and air navigation technical staff on the new CNS/ATM system for Vietnamese trainees. The Output 2 is composed of the following two sub-outputs: 1) Basic CNS/ATM Training 2) Advanced CNS/ATM Training The Project has completed the training of instructors and the production of training material for the New CNS/ATM Systems Training Basic Course. In 2013, while implementing Basic CNS/ATM training courses, the Project will start the training of instructors and the production of training materials for Advanced CNS/ATM courses. 1) Basic CNS/ATM Training The Project trained 16 instructors, who can operate the training courses. VATM and ACV plans to increase this number to 42 so that the training can be implemented more efficiently. Comparing to Cambodia and Lao PDR, Vietnam has larger number of air traffic controllers and air navigation technical staff, therefore Vietnamese counterparts aim to increase large number of trainers in order to effectively cover the whole target group.

9 VATM and ACV established the training plan for year The training material was translated into Vietnamese, and awaits CAAV s approval. Vietnam Aviation Academy (VAA), which conducts training of new air traffic controllers, etc. has adopted the training material developed by the Project into their training courses. Therefore, the sustainability of the basic CNS/ATM training has been ensured in Vietnam. <Cambodia> The Project trained 4 instructors, and they were appointed by SSCA as the course instructors. Actual training started with the first course for air traffic controllers in Phnom Penh in February SSCA has developed the training plan for the basic course, which plans to increase the ratios of trainees who will have completed this course 39% for air traffic controllers and 45% for air navigation technical staff by the end of 2013, and 100% for the both by the end of the Project. <Lao PDR> The Project trained 4 LATM instructors, and they were appointed by DCA as the course instructors. LATM plans to increase one or two instructors each for air traffic controllers and air navigation technical staff. Actual training started with the two courses for air navigation technical staff in March Two of the 4 trained instructors conducted the first two courses in March DCA has developed the training plan for the basic training, which plans to increase the ratios of trainees who will have completed this course to 46% for air traffic controllers and 68% for air navigation technical staff by the end of 2013, and more than 95% for the both by the end of the Project. 2) Advanced CNS/ATM Training The Project has not started activities related to the Advanced CNS/ATM Training, as it plans to implement activities from In 2013, the Project will start the training of instructors and the production of training materials for Advanced CNS/ATM courses, while implementing latter half of the Basic CNS/ATM training courses. In summary, the achievement level of Output 2 is good. The Project trained certain number of trainers for CNS/ATM "Basic Course". The provability of achieving Output 2 is assured if the Project successfully completes training of trainers for advanced level courses. Output 3: To strengthen safety oversight capacity by the introduction of Safety Management System (SMS) in air navigation services (Vietnam and Lao PDR) To establish safety oversight system for air navigation services (Cambodia) Output 3 includes components that are responsible by the regulator and the ANSPs as follows: For Regulators (CAAV, SSCA, DCA): 1) Development of SSP and regulations on SMS for ANS 2) Certify ANSP s SMS

10 3) Conduct safety audit of ANSP For ANSPs (VATM, ACV, CATS, LATM): 1) Develop SMS Manual/Procedures 2) Operate SMS 3) Receive CAAV s audit and respond to recommendations Most of those activities were implemented by Vietnamese authorities without the assistance of JICA Experts. Input from JICA has been limited in this Output partly because JICA Experts needed to assist activities related to WGS-84 surveys which were subsequently added in the Project activities and to commit to establishing FPO as early as possible, which is the most critical element of the Project, with the limited resources from the Japanese side. CAAV developed regulation and guidelines on SMS for ANS in VATM developed SMS manual, and CAAV granted an approval in November Accordingly, VATM started SMS operations in January ACV is updating ANS component of its SMS to meet CAAV s requirements. SSP was approved by MOT in May <Cambodia> SSCA is in the process of developing Civil Aviation Regulations (CARs). The first French expert has assisted SSCA in the developing its CAR Part 12 Aerodromes. And the second French expert assistance followed with the drafting of Air Navigation Regulations (ANR), which will be a part of CAR. The draft ANR prepared by the Air Navigation Standards and Safety Department and the draft of SSP and SMS prepared by Security and Safety Oversight Department of SSCA are in the process of approval by Secretary of State of the SSCA in the 3rd quarter of 2013.In terms of the development of Cambodian SSP, SSCA is in the process of reviewing the final draft and it will be sent for approval by the Secretary of State of the SSCA for approval in the 3 rd quarter of On the operator side, CATS developed its own SMS manual and started operating SMS before the initiation of the Project. <Lao PDR> DCA established SSP in October 2010 and SMS Regulations in December The JICA Experts suggested that SMS procedures and guidelines are to be developed to facilitate ANSP s compliance to the SMS Regulations. As for audit of ANS facilities, DCA conducted safety audits at 11 LATM facilities in Audit teams provided recommendations to LATM based on findings of the audits. On the operator side, LATM has a SMS manual with limited scope, i.e., SMS for ATC at Vientiane Airport. To comply with DCA s SMS Regulations in December 2012, LATM plans to modify its SMS manual by adding CNS components. JICA experts will follow-up and support LATM to upgrade the SMS. In summary, the progress in Output 3 has been slow in comparison with those in Outputs 1 and 2,

11 the Review Team appreciates that counterparts of three countries have been making the best efforts. At the same time, the Review Team expects that Japanese side will allocate more resources timely to the need of the Output 3 activities, as also desired by the Task Force 3 members. 3-2 Results as per the Five Evaluation Criteria (1) Relevance The relevance of the Project remains high for the following reasons: - The Project design is in line with national policies of Vietnam, Cambodia, and Lao PDR; namely "Aviation Transport Development Master Plan of Vietnam(2009)" and "CNS/ATM Implementation Plan up to 2020 and Vision to 2030" in Vietnam, "Aviation Transport Policy and Strategies " in Cambodia and "Strategic Development Plan for and Action Plan for " in Lao PDR, all of which identify transition to the new CNS/ATM systems as one of the priority areas within the government. - As a contracting state of ICAO, three countries are committed to implementing the transition to the new CNS/ATM systems. Three countries reported the PBN Roadmap to ICAO APANPIRG. The three countries PBN Roadmaps up to year 2015 accord with ICAO PBN Global Goals and Asia/Pacific Regional Plan. - The Project is consistent with priority areas for Japan s Assistance Strategy to three countries. The Project is located under the development issue of Urban Development, Transportation and Communications Network Development in Vietnam, Improvement of Social and Economic Infrastructure and Conditions for Economic Progress in Cambodia and Improvement of transport network in Lao PDR. (2) Effectiveness The effectiveness of the Project is assured for the following reasons: - The possibility of achieving the Project Purpose To introduce the new CNS/ATM system in Vietnam, Cambodia and Lao PDR by 2015 is considered to be moderately high as a result of the Project. All three Outputs contribute to the achievement of the Project Purpose; therefore there are clear logical relations that link the Project Outputs with the Project Purpose. - Target number of airports for PBN implementation (indicator for Output 1 as PBN Roadmap planned to be modified) is yet to be determined among the Project members. During the Mid-term Review study, the Review Team intended to determine target numbers for the mentioned indicators, however more sufficient time was required to set up longer-term target (for the year 2015) discussed and agreed at the institutional level. The effectiveness of the Project will increase if targets of the mentioned indicators are identified in due course. - For better understanding and more effective project management, the Review Team suggests the modification of PDM in terms of Objectively Verifiable Indicators and their means of verification, as well as some minor changes in wording of the Project Purpose.

12 (3) Efficiency Overall, the level of efficiency of the Project is considered to be appropriate since most inputs of the Project were appropriately utilized to produce outputs. For example, the equipment for designing PBN flight procedure is procured and installed in FPO in Hanoi to be shared among the three countries. Some of the training courses were implemented jointly by inviting participants from all of the target countries. However, there are some risks or inhibiting factors for efficient implementation of activities as follows: - Regarding the adequacy of input and level of activities, limited availability of resources of JICA short-term experts in a timely manner resulted in some delays in the progress of Output 3 during the first half of the Project. Efficiency of the Project would increase if current assignment of JICA Experts is allocated appropriately for Output 3, as well as for follow-up of risk factors such as WGS-84 procedures and guidelines under Output 1. - Distribution of tasks among long-term experts can be reviewed to accommodate the needs under Output 1, 2 and 3 identified during the Mid-term review. The revision of mission plans by experts as well as simplification of structure of project meetings may be considered to enhance efficiency of the Project. (4) Impact Impact is a viewpoint that asks whether expected or unexpected long-term effects are brought about as a result of the Project. Overall Goal, which is expected to be achieved within three to five years after the Project completion, is one of the expected impacts of the Project. As for the level of achievement of overall goal To improve efficiency and safety of flight operations and to enlarge the capacity of airspace through the transition to the new CNS/ATM systems in Eastern Mekong Region, it is verified that there are some positive factors that can contribute to its achievement as follows: - The Project is enhancing capacity to establish PBN flight procedures in Cambodia, Lao PDR and Vietnam as a whole; hence the outcome of the Project would contribute to enlarge capacity of airspace in the region. - A new route was established in 2013 for the direct flight between Hanoi and Phu Quoc Airports, as a result of ATS cooperation meeting among Cambodia, Lao PDR and Vietnam. The meeting considered this achievement as a demonstration of close cooperation and collaboration in the Eastern Mekong region, with assistance of the Project 1. - No negative impacts have been reported so far. (5) Sustainability The sustainability of the effects of the Project is considered from three aspects, i.e. 1) policy, 2) organizational and financial, and 3) technical aspects. 1 Minutes of LJPM/17 and CJPM/17

13 1) Policy Aspects From the policy perspective, transition to the new CNS/ATM systems will remain one of the priority areas in Lao PDR. DCA developed Strategic Development Plan for and Action Plan for , emphasizing the transition to the new CNS/ATM system as one of the priorities for air navigation sector. 2) Organizational and Financial Aspects In terms of organizational and financial aspects, it is premature to determine level of sustainability of the effect of the Project. - Financial as well as institutional arrangements for implementing WGS-84 survey and flight validation are the most critical elements under Output 1. The prospect for sustainability would increase if coordinated efforts as well as budget are secured to implement WGS-84 survey in collaboration with related agencies such as NGD. Similarly, the prospects for sustainability would also increase if flight validation services are provided at reasonable costs by ATTECH of Vietnam as it plans to procure necessary equipment for flight validation in In terms of training for air traffic controllers and air navigation technical staff, the organizational as well as financial sustainability would be enhanced if new CNS/ATM training courses are to be incorporated into regular training program implemented by DCA and LATM. 3) Technical Aspects As for technical aspects, it is also premature to determine the technical sustainability of the Project. However, there are some encouraging factors to contribute to sustainability such as the followings: - Regarding PBN flight procedures, 4 flight procedure designers are appointed by DCA with capability for designing PBN flight procedures as a result of the training under the Project. In addition, 13 PBN procedures 2 were designed for Vientiane Airport by C/P as a result of OJT on PBN flight procedures. - The Review Team also notes some encouraging factors under Output 2, that LATM in coordination with DCA and CATC show good progress for the implementation of the basic level training program. Although the advanced level component is yet to be started, the instructors from LATM reported that they are confident to carry out new CNS/ATM Systems Training course with support of JICA Experts. 3-3 Factors that promoted/ inhibited realization of effects (1) Promoting factors JICA Master Plan Study Project ( ) served as effective preparatory phase of the Project implementation; Baseline Survey enhanced the Project to set up targets for Output s level 2 6 PBN SID, 6 PBN STAR, 1 IAP

14 indicators; Project's goals are in line with ICAO s requirement on PBN and SSP/SMS, which promotes strong initiatives from counterparts; (in case of Lao PDR) an on-going grant project will be a promoting factor for PBN flight procedure design; and DCA conducted annual safety audit even before the project undertakes activities on this subject. (2) Inhibiting factors Limited availability of resources by Japanese side caused certain delays in implementation of Output 3; (in case of Cambodia and Lao PDR) some indicators whose target are yet to be determined (for example, number of airports for WGS-84 surveys and/or PBN implementation); WGS-84 surveys and PBN flight validation turn out to be risk factors for all three countries. 3-4 Conclusion The Project has been making good progress so far. Regarding the level of achievements, both the Output 1 and the Output 2 are considered to be good. Regarding the Output 3, the progress is slow compared to previous two Outputs; however the Review Team appreciates the three countries ownership and initiative to bring forward the activities under the Output 3. Regarding the Project Purpose, it is premature to determine the prospective level of achievements. In terms of the Five Evaluation Criteria, the relevance is considered remain high, the effectiveness is assured and the efficiency is appropriate. Some positive impacts have been observed in terms of enhanced collaboration among the Eastern Mekong countries. Regarding the sustainability of the Project, it is premature to determine organizational, financial and/or technical aspects of sustainability, though it is considered to be relatively high from the policy aspect. 3-5 Recommendations On the ground of the results of the study summarised above, the Review Team has made the following recommendations to the Project. (1) Modification of PDM (for three countries): Details are shown in table 6.1 of country reports. (2) Recommendations regarding Output 1 1) Actions to Critical Activities to Achieve Output 1 (for Vietnam): WGS-84 survey, flight validation of designed PBN flight procedures and provision of RAIM prediction information services are most critical activities to achieve the Output 1 target. The Review Team recommends to both Vietnamese and Japanese side to undertake adequate actions including increased allocation of MOT budget to the relevant organizations. 2) Review of PBN Roadmap (for Cambodia and Lao PDR): From the viewpoints of cost-effectiveness and aircraft capability, it would be necessary that the PBN Roadmap should be reviewed. 3) PBN Implementation in En-route Phase (for three countries): The Review Team recognized the

15 benefit of the implementation in en-route phase had not been realized yet in three countries. Three countries would need to coordinate with adjacent states to establish routes for PBN flight such as RNAV5 routes. 4) WGS-84 survey (for three countries and JICA experts): necessity of developing technical guidelines for WGS-84 surveys is well recognized in three countries. The Review Team recommends that JICA Experts should provide timely support to CAAV, SSCA and DCA as they commonly have difficulty in this area. 5) Preparation for Transfer of PANADES Flight Procedure Design System (for Vietnam and JICA Experts): JICA Experts should conduct transfer of knowledge so that CAAV can take adequate actions through communication with the supplier of the system when troubles and irregularities occur. In addition, it is necessary to develop operation manual of PANADES. Roles and responsibilities of CAAV, the supplier and users shall also be documented. It is also necessary that the Vietnamese side will secure budget for the annual maintenance fee of the system after the Project period. (3) Recommendations regarding Output 2 Assurance of organizational and financial sustainability of new CNS/ATM training program (for Cambodia and Lao PDR): SSCA, DCA and LATM are suggested to incorporate new CNS/ATM subjects into their regular training programs targeting especially new recruits of ATC and air navigation technical staff. (4) Recommendations regarding Output 3 1) Enhancement of Output 3 Activities (for JICA Experts): the Review Team recommends the Japanese side in particular that assistance to three countries should be enhanced timely to the Project activities. The adequate amount of resources should be shifted to the Output 3 by direct involvement of the JICA Experts to the Output 3 trainings to the possible extent. The Review Team also recommends the Japanese side, if it is necessary and adequate as well as if the budget allows, to consider additional training in Japan or the third country to enhance the progress of the Output 3. 2) Specific fields for auditors (for Vietnam): Japanese side confirmed that auditors are expected to provide oversight to regulations of each specific field, i.e. ATM, AIS, PANS-OPS, CNS, MET and SAR, and the Review Team suggested that the target in the PDM be field specific. (5) Recommendations regarding Project's Implementation Process 1) Transfer of knowledge at the Project Meetings (JICA Experts): The Review Team observes that the Project Meetings are more oriented for monitoring of the activities by the counterparts. It is suggested that the regular Project Meetings be utilized more for technical transfer, for example on WGS-84 survey guidelines, SMS procedures and guidelines, and safety audit of ANSPs, for increased effectiveness and efficiency of the Project activities. 2) Communications among JICA Experts, Japan Civil Aviation Bureau (JCAB) and JICA (for

16 Japanese side): In order to ensure necessary assistance from JCAB and timely arrangement by JICA, parties concerned are requested to enhance close coordination and communications. 3) Improvement in administrative process (for Vietnam): The Team observed the tendency within Vietnamese counterparts to take a lengthy process to make decisions with regard to various Project activities, which have consequence on JICA Experts activities as well as the progress under the regional cooperation framework. Thus, the Team recommends that Vietnamese counterparts consider streamlining the administrative process to more effectively and efficiently support the Project. 3-6 Lessons learned Followings are common lessons learned from Project s experiences for similar types of cooperations in the future: 1) For a technical cooperation on recently developed new concepts and/or technologies, such as advanced level CNS/ATM and/or SSP/SMS, existing resources from the Third Countries (such as training institutions and/or instructors) may be strategically combined to increase efficiency as well as effectiveness of the Project. 2) For a regional cooperation targeting various countries with different starting points like this Project, it is important in the planning stage of the project to identify common issues as main components of the project for effective implementation. 3) For Japanese experts to manage a project for the first time, advice from an experienced expert at an early stage would be very beneficial. In many cases, those experts expressed that the mid-term review turned out to be a useful exercise to learn how to improve the management of the project, but they had not been able to obtain such advice before. Providing a short-term assistance by an experienced expert at the beginning of the project would be an effective measure for smooth implementation of the project.