NETLIPSE Infrastructure Project Assessment Tool Stuart Baker, Deputy Director of National Rail Projects Department for Transport, UK Vienna, September 28, 2010
Contents 1. Refinement of IPAT 2. IPAT Assessment Process 3. Pilots and Progress 4. IPAT Deliverables 5. Future of IPAT 6. Use and Value of IPAT
Objective of the IPAT Objective: The IPAT assesses the quality of the management of the totality of the project including the plan to reach the objectives and to implement the outputs (not just the physical construction) Predictive value: The IPAT highlights weaknesses and strenghts in management and the realism of the project delivery plan for the whole project and specifically for the next project phase
Purpose of the IPAT The core purpose of the IPAT is to assess the fitness of the project organisation: this will define the ability and timing of funding by the EU/national government The IPAT does not help to decide on the feasibility of a project, neither on the need for funding IPAT can indicate early warning signals of a project running out of schedule and/or budget
IPAT Conceptual model and themes Political context (Usually a Ministerial client) T1 Client / official sponsor T 1 Political Context T 2 Objectives, Purpose and Business Case T 3 Functional Specifications Related projects T4 T6 T3 T2 Project Delivery Organisation T12 T5 Stakeholders T 4 Interfaces T 5 Stakeholder Management T 6 Finance T 7 Legal procedures T7 T8 T9 T10 T 8 Technology T 9 Knowledge T11 T10 Organisation & Management Private companies (contractors, advisors, operators, etc.) T11 Contracting T12 Risks
IPAT Questionnaire Level of scoring RECENT REFINEMENT: We will score on the level of subthemes (27) instead of questions old style (168) Implication: Individual assessors must score the aspects for consideration to reach a score on the level of subtheme -> stronger reliance on the expert judgment of the assessors
Example Questionnaire new style with open questions on a subtheme 5A. To what extent does the manner stakeholder identification is carried out by the PDO provide sufficient conditions for entering the next project phase? Consider the following aspects: stakeholder (including media) identification stakeholder categorisation regular updating of stakeholder database understanding of prioritisation of stakeholders identification of stakeholder relationships and networks monitoring of changes in the stakeholder network(s)
Score labels 4 Point forces thinking REFINEMENT: Description of score labels have been adjusted Option Label This reflects Score = 1 Score = 2 Score = 3 Score = 4 Very negative contribution to a successful project Negative contribution to a successful project Positive contribution to a successful project Very positive contribution to a successful project An immediate need to review and improve An urgent request to improve (weakness) Generally good with areas for improvement Very good and incorporating best practice
Levels of importance These are different at different times within a Project s lifecycle RECENT REFINEMENT: The methodology used to determine the levels of importance is adjusted. New methodology: 1. Determine the levels of importance by dividing 100 points over the themes per phase 2. Divide the points attributed to a theme over the subthemes within that theme
Graphic: Total Score Gotthard Base Tunnel 100,0% 90,0% Total weighted percentage score 80,0% 70,0% 60,0% 50,0% 40,0% 30,0% 20,0% 10,0% 0,0% M1 M2 M3 M4 Project phase
Options for Presenting Results: 1. Column Discussion: what graph is most suitable? What do those present think is the most suitable / understandable / useful graph? Score per theme Score 4 3,5 3 2,5 2 1,5 1 0,5 0 1 2 3 4 5 6 7 8 9 10 11 12 Theme
Options for Presenting Results: 2. Score matrix Project assessment FB 4,0 3,5 11 9 10 8 5 7 6 1 3 2 12 3,0 4 2,5 Score 2,0 1,5 1,0 0,5 0,0 0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5 5,0 Weight
Options for Presenting Results: 3. Spider diagram plus circle Political context Risks 4 Objectives, purpose and BC 2,4 3 Contracting 3,7 8% 11% 8% 2 Functional specifications 10% 10% n and Management 3 6% 7% 3 Interfaces 9% 9% Knowledge 4 5% 10% 7% 3,5 Stakeholders 2,5 2,8 Technology 3,8 Legal procedures Finances Weight Score
The assessment process Management organisation Organise Training Program Assessors team Client / PDO Training Assessors Training Assessors Nominate project for IPAT Select Assessors team Prepare site visit Deliver background information Interviews (app. 6) Score and analyse (sub)themes Write assessment report Deliver assessment report Interviews (app. 6)
Qualification criteria assessors Qualifications of lead assessors: At least 10 years of practical experience and responsibility within major infrastructure projects Good working knowledge of English. Independent from the project to be assessed. Successfully passed the IPAT Assessors Training. Other assessors: Significant practical experience and responsibility within infrastructure projects Good working knowledge of English. Independent from the project to be assessed. Successfully passed the IPAT Assessors Training
Composition of an IPAT assessment team Criteria: A lead assessor and two or three other qualified assessors; At least one of the assessors should have local knowledge and experience; At least one of the assessors needs good working knowledge of the native language; Maximum one of the assessors is involved in an IPAT assessment for the first time. Additional: One of the team members is secretary and takes care of organising the site visit and is the contact person for the PDO / C/S and the NETLIPSE organisation. The assessors team is composed by the management organisation (NETLIPSE) by selecting people from a shortlist of assessors and a pool of lead assessors.
IPAT Assessors Course 2-day course (morning, afternoon, evening and next morning) Outline for the training programme: Theory IPAT Assessors IPAT The model TEN-TEA Project managers Skills IPAT The Assessment IPAT The Interpretation Applied skills IPAT The case
Pilot projects Pilot pilots (ex post assessment) West Coast Main Line Gotthard Base Tunnel Pilot project (ex ante assessment) Fehmarnbelt Pilots still to undertake: Lock Waaslandhaven (Flanders, Belgium): Oct Railway project Koper Ljubljana (Slovenia): Oct Options: Vienna main Station? Figueras Perpignan (France)? ODA London on Transport (UK)? Others?
4. IPAT Deliverables by December IPAT Brochure: available IPAT Manual: completed, final improvements are made on Questionnaire and scoring model IPAT Assessment Reports: 1 completed (FB), 2 in progress (WCML, GBT), 2 still to do ideally IPAT Assessors Training: course drafted available in November
5. Future of IPAT as a Tool Within TEN-T Annual Call 2010-2012: 1. IPAT Assessments: 5 IPAT Assessments Reports on TEN-T projects 2. IPAT Upgrading: IPAT Questionnaire version 2.0 IPAT Manual version 3.0 3. Developing a Training Programme: LIPs Management Course for project managers IPAT Course for Assessors IPAT Course for TEN-T managers IPAT Course for applicants (PDOs/Clients)
6. Use and Value of IPAT Use by a Client/Sponsor as check on health of delivery or projected delivery against plan Use by a Project Team to self assess Possible/Likely use by EC as one of their criteria for decision making on the award of grants and the timing of likely spend Possible use by the EC as a project monitoring tool Use by funding bodies (e.g. EIB)