Development of a Cost-Effective Poverty Assessment Tool. Contractor s Final Report

Development of a Cost-Effective Poverty Assessment Tool Contractor s Final Report Lima, Peru. October 2004

Acknowledgements We wish to thank Ms. Julia Johannsen and Mr. Manfred Zeller for their help and valuable guidance in every stage of the survey, especially in relation to quality assurance procedures for data gathering and processing. We are also very thankful to all the staff who was directly involved in the project, including interviewers who had to work long hours under harsh weather and even unsafe public conditions. This survey would not have been possible without their valuable efforts.

Contents Key Staff in Peru...4 Definitions...4 Territorial / administrative divisions...4 Geographical Regions...4 Other Terms...5 Survey Organization and Execution...5 Functions...5 Selection and Training of Survey Personnel...6 Fieldwork...6 Fieldwork Planning and Programming...6 Fieldwork Execution...7 Fieldwork Control and Supervision...7 Data Entry and Validation...8 Sampling...8 Sample of MFI Clients...8 Sample of Non-Clients...8 Routes...10 Description of routes and assigned personnel...10 Supervision... 14 Routes selected for supervision....14 Supervision Procedure...14 Supervisors Reports...15 Major Findings... 15 Learning Curve...15 Improved Wording Needed...15 Estimated workload based on the actual sample...15 Errors in the lists of entering clients in areas of recent expansion...16 Most common errors when filling questionnaires out...16 Difficulties found in MFI selection...16 Curious happenings during fieldwork...18 The Case of the CRAC Cruz de Chalpón...18 3

Final Report Key Staff in Peru The Peruvian staff involved in the project included the following professionals: M.Sc. Luis Castillo Quintana. Project Manager and Director Econ. Pedro Llontop Ledesma. Fieldwork Manager, Est. Mario Reyna Farje Espinoza. Statistician. Definitions Many terms are peculiar to a given country, i.e. the meaning they are given there may not be the same or as common as elsewhere. Territorial / administrative divisions The Peruvian territory is administratively divided first into departments (departamentos) and therefore a department is the largest territorial division. Departments are in turn broken down into provinces (provincias), and finally provinces are subdivided into districts (distritos.) As the decentralizing regionalization process consolidates, a further governance tier is currently in progress that of regions. For the time being, however, regions and departments encompass the same territories and consequently we will no longer consider regions in the discussion that follows. Geographical Regions Traditionally, the Peruvian territory has been divided into three geographical regions as a result of the influence of the Andes, the large mountain range that runs North- South throughout the territory. Accordingly, such regions take the shape of three North-South longitudinal strips. The strip between the Sea and the Western slopes of the Andes is called the Coast. The highlands crowned by the Andes, with its fertile and habitable valleys are called the Sierra. The land East of the Andes where tropical forests and rainforests thrive is called the Jungle, and is sometimes referred to as the Amazonian region/area. Climatic and cultural differences are believably assumed to influence and/or determine their inhabitants distinct patterns of living. 4

While such names refer directly to dominant geographical features, it should not be understood as restricted to them, but only as labels of such extensive tracts of land. It is within such a context that we should understand such apparent oxymora as Urban Jungle (neither should it be thought of as equivalent to asphalt jungle, but rather as the urban areas within the geographical region called the Jungle ) and the like. Other Terms Route. An established or selected course of travel and the assigned territory to be systematically covered CRAC. Initials of Caja Rural de Ahorro y Crédito, Spanish for Rural Savings and Credit Union CMAC. Initials of Caja Municipal de Ahorro y Crédito, Spanish for Municipal Savings and Credit Union CAC. Initials of Cooperativa de Ahorro y Crédito, Spanish for Savings and Credit Union EDPYME. Initials of Entidad de Desarrollo de la Pequeña y Micro Empresa, Spanish for Micro and Small Business Development Institution. Survey Organization and Execution Functions Instituto Cuánto was responsible for the overall implementation of the survey in compliance with the instructions by, and in coordination with, the IRIS Center. The survey direction and management was structured as follows: 100 Project Manager. Responsible for both technical and managerial direction of the survey in all stages. He also comprehensively assesses, controls, and supervises the progress of the survey. 200 Translator. Responsible for translating and back-translating documents in Spanish into English and vice versa, as well as facilitating communication between IRIS researches visiting Lima and local staff. 300 Statistician. Responsible for preparing the sampling frame, as part of the survey design, as well as for determining the procedures leading to the selection of sampling units. 400 Fieldwork Manager. Responsible for implementing the survey fieldwork, from staff training and planning to actual fieldwork to the submission of completed questionnaires for data entry. 5

500 Regional Supervisors: Responsible for coordinating the fieldwork in each area surveyed. 600 Logistics Manager: Responsible for delivering any materials on a timely manner, and managing current expenses. 700 Data Processing Officer. Responsible for processing data, designing the data entry shell, and validating the data entered. Selection and Training of Survey Personnel This involved two stages: First, personnel recruiting and technical training for fieldwork supervisors and interviewers, and second, a course for critics and codification operators. Instituto Cuánto has available a large directory of experienced interviewers, which take part of social and economic studies on a regular basis. Pretest Survey This survey will allow us to administer the questionnaires proposed for the study so as to make it possible to correct any errors, measure the time required to complete a questionnaire per household, as well as to improve the instructions given on the manner both conglomerates and households were to be located. Fieldwork This part of the survey refers to the execution of a number of tasks, subtasks, and operations intended to gather data from selected households to meet survey objectives. Fieldwork involved the following tasks: Fieldwork Planning and Programming This task involved providing information on fieldwork routes, time required to complete the fieldwork, detailed determination of goods and services required for the successful completion of each task, fieldwork budgeting and allocation of funds to each route. Planning and programming required taking into account sample size and distribution. One of the basic elements of this process is determining the time required to get to the selected sampling unit. In this regard, the following movements should be taken into account: Within the route Between routes Between districts Between provinces Between departments 6

To that end, district-, province-, and department-level maps, roadmaps and street maps of the districts selected, as well as auxiliary information indicating distances, types of roads, and the usual means of transportation were provided. It was very important the broad expertise and knowledge of the staff involved in this task. Fieldwork Execution This task involves gathering information from selected households by administering the questionnaires designed for this purpose. The main guidelines to execute the survey are listed below: The information from each selected household is gathered through direct interview. Fieldwork schedule is flexible, depending on circumstances and the appointment arranged by the interviewer or Team Leader with the household head. The Team Leader allocates the workload to each interviewer Work is done Monday to Saturday. Only previously appointed interviews are carried out on Sundays, as required, to complete the weekly workload or to retrieve information. As the information from individual households is completed, each interviewer submits all completed questionnaires to his/her Team Leader for data cleaning and validation The Team Leader supervises interviews to correct interviewers initial mistakes and check for compliance with methods and procedures. As Team Leaders are responsible for the technical and administrative performance of survey fieldwork, they had to be continually in contact with the Project Manager, who consequently controls and supervises the overall progress of the survey. Team Leaders are required to submit fieldwork progress reports every other day, either via email o telephone. Fieldwork Control and Supervision It involves checking the survey fieldwork for adequate technical performance and compliance with methods, budget, and scheduled deadlines as planned for the fieldwork. Control and supervision is performed by the Team Leader. 7

Data Entry and Validation This stage involves gathering, coding, and entering the data from completed questionnaires. Data entry is carried out under the Data Processing Officer in a roomy and adequately equipped computer room by experienced data entry operators. Double entry of data is performed to prevent potential data entry errors. Sampling Factors to be taken into consideration for sample selection Sample of MFI Clients Step 1. Once we had available the lists provided by the selected MFI, entering clients were sorted by district. Then two districts were selected by applying PPS sampling. Step 2. Where selected districts had over 100 clients each, 100 were chosen from each to get a sample size of 200. Where districts had fewer than 100 clients, two districts were randomly selected and the sampling units were allocated to both districts according to their sizes. There was only one exception to this rule. The CRAC Cruz de Chalpón had too few new clients. As a result, we had to select 5 districts to cover the total sample of 200 clients. Step 3. Sample units were allocated as shown in Table 1. Sample of Non-Clients Step 1. (Using PPS method) At the beginning we decided to select 6 departments from which the sample will be taken. The following departments were selected: - Arequipa - Cusco - La Libertad - Lima (twice) - Piura 8

Table 1. Allocation of the MFI-Client sample among selected MFIs. MFI Department Province District Total Sample Backup CAC San Isidro Lima Huaral Chancay 57 32 3 Lima Huaral Huaral 299 168 17 Total 356 200 20 CMAC de Chincha Ica Chincha Chincha Alta 349 100 10 Ica Chincha Pueblo Nuevo 244 100 10 Total 593 200 20 NGO Caritas del Huánuco Huánuco Amarilis 412 111 11 Perú Junín Chanchamayo Chanchamayo 331 89 9 Total 743 200 20 Edpyme Edyficar Lima Lima Chorrillos 252 100 10 Lima Lima San Juan de Lurigancho 783 100 10 Total 1035 200 20 CRAC Cruz de Lambayeque Chiclayo Chiclayo 136 119 12 Chalpón Cajamarca Jaén Jaén 52 46 5 Lambayeque Ferreñafe Ferreñafe 24 21 2 Lambayeque Lambayeque Lambayeque 16 14 1 Total 228 200 20 CAC San Pedro Apurímac Chincheros Ranracancha 98 25 3 Apurímac Chincheros Anco Huallo 700 175 17 Total 798 200 20 Step 2 Since Lima was taken twice, we decided to draw one more department out of the remaining ones. The selected department was Cajamarca Step 3. When we analyzed the sample, we observed that no department in the Amazonian region had been selected. To solve this, we applied PPS sampling to the departments in the Amazonian region alone. The selected department was Loreto. Step 4 With a sample of 7 departments, we allocated the 800 non-client sample among the selected domains as shown in Table 2. Table 2. Non-Client sample allocation among selected domains. Domain Total % Population % Sample Coast and Lima 400 0.518 0.500 Metropolitan Lima 200 0.289 0.250 Urban Coast 132 0.178 0.165 Rural Coast 68 0.052 0.085 Sierra 266 0.350 0.333 Urban Sierra 99 0.126 0.124 Rural Sierra 167 0.224 0.209 Jungle 134 0.131 0.168 Urban Jungle 66 0.060 0.083 Rural Jungle 68 0.071 0.085 TOTAL 800 1.000 1.000 9

Table 3. Districts selected Arequipa La Libertad Lima Cerro Colorado Chao Ate Mariano Melgar La Esperanza El Agustino Tiabaya Trujillo Lima Rímac San Juan de Miraflores Cajamarca Loreto Santiago de Surco Cajamarca Iquitos Encañada Punchana Querocoto Yurimaguas Cusco Echarate Quiquijana Wanchaq Piura Chulucanas Pariñas Sullana Step 5. (Using PPS Sampling) We selected 6 districts from Lima and 3 from all other departments. Table 3 shows the selected districts. Step 6 Since there is no recent information about which districts have rural areas, we selected randomly, according to our needs, which districts would be rural and which urban. The districts that were chosen as rural were verified is they indeed have rural areas from were to take the rural sample. Step 7 The sample was finally allocated as shown in Table 4: Routes Fourteen routes were required to cover all selected areas from which the sample was to be taken. Table 5 shows the routes covering the MFI-Client sample areas and Table 6 shows the routes covering the Non-Client sample areas.. Description of routes and assigned personnel Only two out of the fourteen routes (routes 13 and 14) required the use of Quechua-speaking interviewers. Most interviews were therefore held in Spanish. Except for the team who covered route 6 only, all teams covered two routes, one after the other. Table 7 details the sequences followed. Route 1 with 2. Route 3 with 4 and 5. Route 6 with 7. Route 8 with 9 and 10. Route 11 alone. Route 12 alone. Route 13 with 14. 10

Table 4. Detailed allocation of Non-Client Sample among selected districts Total Coast Sierra Jungle Urban 497 332 99 66 Rural 303 68 167 68 TOTAL 800 400 266 134 Arequipa 100-100 - Urban 66-66 - Mariano Melgar 33-33 - Tiabaya 33-33 - Rural 34-34 - Cerro Colorado 34-34 - Cajamarca 100-100 - Rural 100-100 - Cajamarca 34-34 - Encañada 33-33 - Querocoto 33-33 - Cusco 100-66 34 Urban 33-33 - Wanchaq 33-33 - Rural 67-33 34 Echarate 34 - - 34 Quiquijana 33-33 - La Libertad 100 100 - - Urban 66 66 - - La Esperanza 33 33 - - Trujillo 33 33 - - Rural 34 34 - - Chao 34 34 - - Lima 200 200 - - Urban 200 200 - - Ate 33 33 - - El Agustino 33 33 - - Lima 33 33 - - Rímac 34 34 - - San Juan de Miraflores 34 34 - - Santiago de Surco 33 33 - - Loreto 100 - - 100 Urban 66 - - 66 Iquitos 33 - - 33 Punchana 33 - - 33 Rural 34 - - 34 Yurimaguas 34 - - 34 Piura 100 100 - - Urban 66 66 - - Pariñas 33 33 - - Sullana 33 33 - - Rural 34 34 - - Chulucanas 34 34 - - 11

Table 5. Routes for MFI-Client areas Route 1. Lima: 200 households Huaral Huaral Lima Urban Coast Chancay Huaral Lima Urban Coast Route 2. Ica: 200 households Chincha Alta Chincha Ica Urban Coast Pueblo Nuevo Chincha Ica Urban Coast Route 3. Lima: 200 households Chorrillos Lima Lima Metropolitan Lima San Juan de Lurigancho Lima Lima Metropolitan Lima Route 4. Junín: 89 households Chanchamayo Chanchamayo Junín Rural Jungle Route 5. Huánuco: 111 households Amarilis Huánuco Huánuco Rural Sierra Route 6. Lambayeque: 154 households Chiclayo Chiclayo Lambayeque Coast rural Ferreñafe Ferreñafe Lambayeque Coast rural Lambayeque Lambayeque Lambayeque Coast rural Route 7. Jaén: 46 households Jaén Jaén Cajamarca Rural Jungle Route 8. Lima: 200 households Ate Lima Lima Metropolitan Lima El Agustino Lima Lima Metropolitan Lima Lima Lima Lima Metropolitan Lima Rímac Lima Lima Metropolitan Lima San Juan de Miraflores Lima Lima Metropolitan Lima Santiago de Surco Lima Lima Metropolitan Lima Route 14. Apurímac: 200 households Anco Huallo Chincheros Apurímac Sierra rural Ranracancha Chincheros Apurímac Sierra rural 12

Table 6. Routes for Non-Client areas Route 9. Arequipa: 100 households Mariano Melgar Arequipa Arequipa Urban Sierra Tiabaya Arequipa Arequipa Urban Sierra Cerro Colorado Arequipa Arequipa Sierra rural Route 10. Piura: 100 households Chulucanas Morropón Piura Urban Coast Pariñas Talara Piura Urban Coast Sullana Sullana Piura Coast rural Route 11. La Libertad - Cajamarca: 200 households La Esperanza Trujillo La Libertad Urban Coast Trujillo Trujillo La Libertad Urban Coast Chao Virú La Libertad Urban Coast Cajamarca Cajamarca Cajamarca Sierra rural Encañada Cajamarca Cajamarca Sierra rural Querocoto Chota Cajamarca Sierra rural Route 12. Loreto: 100 households Iquitos Maynas Loreto Urban Jungle Punchana Maynas Loreto Urban Jungle Yurimaguas Alto Amazonas Loreto Rural Jungle Route 13. Cusco: 100 households Wanchaq Cusco Cusco Urban Sierra Echarate La Convención Cusco Rural Jungle Quiquijana Quispicanchi Cusco Rural Sierra Table 7 Route Supervisor Interviews Working Days 1 1 5 30 2 1 5 30 3 1 6 26 4 1 2 29 5 1 3 27 6 1 5 27 7 1 0 30 8 2 6 27 9 1 3 27 10 1 3 29 11 1 3 46 12 1 2 30 13 1 5 21 14 1 5 30 13

Supervision A number of supervising visits to most fieldwork teams were carried out intended mainly to observe the their performance as they administered the Composite Questionnaire to respondents, as well as to verify the quality of the information provided by MFIs that took part in the survey. Routes selected for supervision. The following routes were visited by Mr. Luis Castillo, Project Manager and Director. Department of Lambayeque: Route of MFI CRAC Cruz de Chalpón s clients (cities of Chiclayo and Lambayeque.) Department of La Libertad: Non-Client Route. (Cities of Trujillo and Moche) Department of Loreto: Non-Client Route. (City of Iquitos y Town of Punchana.) Department of Ica: Route of MFI CMAC de Chincha s clients. (City of Chincha, Town of Pueblo Nuevo.) Department of Arequipa: Non-Client Route. (City of Arequipa and Cerro Colorado) Department of Lima: Route of MFI CAC San Pedro s clients. (Cities of Huaral y Chancay) All fieldwork teams in Metropolitan Lima were supervised by Mr. Pedro Llontop, Head Survey Officer. Mr. Llontop had also been commissioned with supervising fieldwork teams in Cusco and Apurímac; however, these two cities could not be supervised as flights to Cusco and Apurímac were cancelled due to harsh weather conditions. Supervision Procedure The following steps were followed in every province visited: Initial Briefing with the Supervisor An overall briefing on fieldwork progress and Q&A on matters directly related with the Composite Tool. Initial review of all completed questionnaires: Those which required further clarification were set aside. Determining the average time required for interviewers to complete a questionnaire. First Overall Meeting with Fieldwork Teams Exchange of opinions and Q&A related to the Composite Questionnaire. On-Site Fieldwork Visit A given area was chosen only when work was in progress at that time. The distances to be covered by interviewers were calculated, as well as the time required to 14

complete them. The daily workload was checked for completion, and a sample questionnaires were partially filled for illustrative purposes. Completion of Review of All Questionnaires Intended to assess the individual performance of each interviewer. Final Briefing with the Fieldwork Team. Discussion of main findings of questionnaire review. Individual instructions intended to improve team performance were given. In this last meeting administrative issues were discussed, largely related to increased operational costs. Supervisors Reports After the first inspection visits we determined the need that each supervisor prepared a report detailing the main occurrences in their work. This would be used as input for a planned meeting with Ms. Julia Johannsen, Such feedback would be useful to pinpoint the main concerns and challenges found in the application of the Composite Tool. These reports were timely submitted to Ms. Julia Johannsen and were personally elaborated in the meeting held in Lima with the attendance of all supervisors and interviewers. Such reports were extended with the conversation with interviewers, who described the challenges and facilities found in each fieldwork area. Fieldwork supervision was deliberately carried out when the Composite Questionnaire was being applied. We did not intend to supervise interviewers performance in administering the LSMS Expenditure Module (Benchmark Questionnaire.) Major Findings Learning Curve It was surprising how readily interviewers eventually were able to administer the Composite Questionnaire, averaging less than 90 minutes to complete it. Improved Wording Needed On the other hand, it was quite clear to us that some effort should be done to improve the wording of some questions; especially those who were intended to elicit information on housing rent or sale value from respondents Estimated workload based on the actual sample Peru is a very diverse country, and a range of distinct reactions to survey interviews is normally found. 15

Table 8. Domain Urban Coast Urban Sierra Urban Jungle Rural Coast Rural Sierra Rural Jungle Clients 2-3 2-3 N/A 2-3 2-3 N/A Non-Clients 2-3 3-4 3-4 2-3 3-4 3-4 Table 8 summarizes the modules of the Composite Questionnaires that may be applied per workday: Errors in the lists of entering clients in areas of recent expansion All MFIs were requested the same information: a list with the names and addresses of entering clients belonging to branches where a recent expansion had been taken place, with entering clients meaning people who had been their clients for at most 6 months. However it seems that the concept of being a client for at most 6 months was not clear enough, as there were cases in which: The list included underage people with savings accounts. The list included clients of the MFI who had repaid a first loan and had been granted a new loan after a few months, which diverted from what had been requested. It was not rare the existence of clients who. Applied for, and were granted, a loan using a relative s address, but the borrower himself lived elsewhere, even abroad! Declared the house they were renting as their domicile but had moved elsewhere without reporting their new addresses. Most common errors when filling questionnaires out Supervision visits revealed that errors were limited to a few questions. Table 9 shows the most common errors found. Some errors found in the supervision could be attributed to interviewer s confusion. This in turn may have been the result of insufficient number of examples during training sessions, and the fact that supervisors did not hold ongoing updating briefings with their teams. Difficulties found in MFI selection We contacted friends and colleges in USAID by means of Mr. Jaime Gieseke, who referred us to Mr. Armando Pillado from COPEME. COPEME provided us with a list containing: Some of the main Municipal Savings and Credit Unions (CMAC), Micro and Small Business Development Institutions (EDPYMEs) and Rural Savings and Credit Unions (Cajas Rurales) Contact people. Some indications on how to locate them. 16

Table 9. Most common errors when completing questionnaires. Page Question Page Question B12 Terms were not clearly understood 2 5 B15 Unlikely high amounts for poor households estimates. E15 The cultural peculiarities of some 11 populations made it difficult to 11 correctly ask this question G1.20 Respondents felt bothered as they considered the question to be 15 18 repetitive and often did not answer it. 20 H7, 9 Respondents provided confusing answers. They confused their economic situation with their political discomfort 21 D1a-b Either there was no real estate market or people refused to provide property value E17 Both respondent and interviewer misreport the social program providing aid. H6 Answers were often not realistic considering the area surveyed. Questions I10 onwards: Confusion with amounts and names of NGOs providing free services was a constant error. We contacted some of the main domestic NGOs providing microcredit. We visited directly to Chief Officers responsible for microcredit programs. We started conversations with senior officers of Banco del Trabajo, Banco de Crédito and tried to meet with a senior officer of Mi Banco. However, and despite all our efforts, we could not manage to get any support from these institutions. At the same time, IRIS provided us with information and contacts from the Federation of Savings and Credit Unions. We established the first contacts with them and prepared a short list of financial institutions willing to cooperate with us. Generally speaking, financial institutions showed a strong opposition to provide us with the requested information. They claimed it was not considered to be public information. We were reminded that people with moderate or high economic or financial position had been victims of theft and kidnappings. In a few cases, the information on who is who or how much they hold was provided to criminals by personnel from financial institutions themselves. Despite those difficulties, we managed to get support from: NGO Caritas del Perú, EDPYME Edyficar, Credit Unions San Isidro de Huaral and San Pedro de Andahuaylas. CRAC Cruz de Chalpón de Chiclayo. CMAC de Chincha. In all cases we had to make use of our professional and social relationships to contact people who may be willing to support our survey, as we requested. But we were not able to check the quality of the lists we were given which make it even more difficult to locate MFI clients. 17

Curious happenings during fieldwork Although this issue may have been extensively reported by Ms. Julia Johannsen, we would like to highlight only two facts: That one of our fieldwork teams was interviewed by a Peruvian TV station because they had been bearing freezing temperatures for over 48 hours inside a bus that was prevented from moving forward by the snow. That a campaign of enforced collection of debts took place in Chincha at the same time we were carrying out the interviews, which caused lots of clients to disappear temporarily or even to move to other address. Remarkably enough, respondents continually expressed their discomfort towards some questions that tried to depict their living conditions. Most people felt offended because they felt their privacy was being invaded. Some respondents even decided to discontinue the interview and refused roundly to resume it. The Case of the CRAC Cruz de Chalpón We planned to complete 200 interviews to clients of this MFI. The fieldwork was carried out at the same time as the Copa América Perú 2004 Football Tournament was played in several cities of Peru. Chiclayo was one of the venues of this continental football tournament. As a result of this, the number of crimes increased because criminals, especially thieves, from other places came to the city. In the city of Chiclayo, during the development of fieldwork a number both minor and serious crimes were perpetrated and extensively covered by the local media. This alarmed local residents against strangers visits. The CRAC Cruz de Chalpón provided us with a relatively small list of clients in the city of Chiclayo. The number was small due to the fact that its clients are persons somehow linked to agriculture and the trade of agricultural products. This year there was a severe drought in Northern Peru and as a result there were only a small number of new credits approved between January and May 2004. These factors affected the performance of interviewers assigned to this route, to the extent that it was the first place the Project Manager had to visit for supervision. 18

The interviewers made up to 4 attempts to have MFI clients participate. Nevertheless, in spite of such multiple attempts, we were not able to complete the number of interviews assigned. All possible backup respondents were used so as to complete all the assigned questionnaires. However, out of all 200 interviews to clients of the MFI assigned for the entire route, we were able to complete only 175. This was the only route we were not able to complete the assigned workload. In many cases respondents refused to answer the second questionnaire, in spite of having promised to do so. The combination of these factors affected severely the performance of our work. And undoubtedly small, incomplete lists were the principal cause of our failure to meet objectives as planned in this area. 19