Japan. Administered Account for Selected IMF Activities (JSA) Independent External Evaluation. February 2010 PARTNERSHIPS

Transcription

1 GL BAL PARTNERSHIPS Shared Objectives Joint Action Real Impact Independent External Evaluation February 2010 Japan Administered Account for Selected IMF Activities (JSA) I n t e r n a t i o n a l M o n e t a r y F u n d

2 International Monetary Fund (IMF) Office of Technical Assistance Management (OTM) Independent Evaluation of the Japan Administered Account for Selected IMF Activities (JSA) February, 2010 (MURC)

3

4 Table of Contents List of Abbreviations...iii Executive Summary Evaluation Results and Recommendations... v I. Evaluation Results....v II. Recommendations xiii Chapter 1 Outline of the Independent Evaluation Background and Objectives Implementation and Study Activities Documents, Data and References Evaluation Criteria and Verification Method Review of Evaluation Reports... 7 Chapter 2 Outline of JSA and Japanese ODA Outline of JSA ODA Policy of Japan Consistency of IMF/JSA TA and Japanese ODA Chapter 3 Evaluation Results through Data Analysis and Questionnaires Review of Evaluation Reporting System of JSA Efficiency and Cost Considerations Evaluation Results of the Questionnaire Surveys Chapter 4 Evaluation Results through Field Survey Evaluation Results of Field Interviews Cases of Good Practices Chapter 5 Overall Evaluation Results and Proposed Rating System Overall Evaluation Results Proposed Rating System Chapter 6 Recommendations to IMF and MOF Recommendation to IMF Recommendations to MOF i

5 Appendix A. Questionnaire Survey Sheet: Beneficiaries B. Questionnaire Survey Sheet: Experts C. Results of Questionnaire Survey: Beneficiaries D. Results of Questionnaire Survey: Experts E. JSA Evaluation Questionnaire (Attachment IV of JSA Guidelines) F. Results of Collected JSA Evaluation Questionnaire G. Schedule of IMF Headquarter meetings H. Schedule of the Field Survey conducted I. List of JSA-funded TAs in the evaluation period ii

6 List of Abbreviations ADB -- Asian Development Bank AfDB -- African Development Bank AFRITAC -- Africa Regional Technical Assistance Center AML -- Anti-Money Laundering APD -- Asia Pacific Department BI -- Bank Indonesia BOT -- Bank of Tanzania CFT -- Combating the Financing of Terrorism DFID -- Department for International Development (UK) EBRD -- European Bank for Reconstruction and Development FAD -- Fiscal Affairs Department FDI -- Foreign Direct Investment FY -- Financial Year GBS -- General Budget Support HQ -- Headquarters IDB -- Inter-American Development Bank IEO -- Independent Evaluation Office IFC -- International Finance Corporation INS -- IMF Institute IT -- Information Technology JICA -- Japan International Cooperation Agency JSA -- Japan Administered Account for Selected IMF Activities LEG -- Legal Department MCM -- Monetary and Capital Market Department MOF -- Ministry of Finance MOU -- Memorandum of Understanding MURC -- Mitsubishi UFJ Research & Consulting NBC -- National Bank of Cambodia NIS -- National Institute of Statistics OAP -- Regional Office for Asia and Pacific OED -- Office of Executive Directors OTM -- Office of Technical Assistance Management PFM -- Public Financial Management PHRD -- Policy and Human Resources Development PRGF -- Poverty Reduction and Growth Facility RA -- Resident Advisor RAP -- Resource Allocation Plan ROSC -- Report on the Observance of Standards and Codes RSN -- Regional Strategy Note SBV -- State Bank of Vietnam iii

7 SIDA -- Swedish International Development Cooperation Agency STA -- Statistics Department SWAP -- Sector Wide Approach TA -- Technical Assistance TAIMS -- Technical Assistance Information Management System TCAP -- Technical Cooperation Assistance Program TOR -- Terms of Reference UN -- United Nations UNDP -- United Nations Development Programme UNFPA -- United Nations Population Fund WB -- World Bank WTO -- World Trade Organization iv

8 Executive Summary Evaluation Results and Recommendations The evaluation results and recommendations in this study are based on the following sources of information and the analyses conducted by the study team. The project-level evaluation results from the current IMF/JSA reporting system, i.e. Project Assessment Report and JSA Evaluation Questionnaire 1. The Questionnaire Survey conducted under this evaluation, addressed to the beneficiaries (recipient governments) of JSA and experts employed by IMF for JSA TA projects. The results of a field survey conducted by the study team for four countries, Cambodia, Indonesia, Vietnam and Tanzania (East AFRITAC), in August The interviews conducted at IMF headquarters mainly with OTM, TA departments and regional departments and OED Japan, in July The interviews in Japan with Japanese MOF, OAP and ex-advisors to central banks in Asia under IMF and JICA TA. The desk analyses of the relevant information collected during this study. I. Evaluation Results 1. For this evaluation study, the evaluation of IMF/JSA TA for effectiveness and benefit to the recipient countries and accountability to the Japanese general public are categorized by the terms of reference (TOR) with six criteria: 1) Quality of Activities; 2) Effectiveness and Sustainability; 3) Efficiency; 4) Consistency with Japanese ODA Policy; 5) Visibility of Japan as Donor; and 6) Advantage of IMF Scheme. On the other hand there are six criteria by OECD DAC Criteria for Evaluating Development Assistance (relevance, effectiveness, efficiency, impact and sustainability), widely applied to development aid evaluation, have been modified for this evaluation, in seven criteria, namely 1) Relevance, 2) Effectiveness (Impact), 3) Efficiency, 4) Sustainability, 5) Transparency, 6) Consistency with Japanese ODA and 7) Visibility of Japan as a donor. These sets of criteria were applied to conduct this evaluation work. 1. Evaluation Results based on OECD Evaluation Criteria 1) Relevance 2. The Questionnaire Survey conducted by the study team, for both the recipient countries and experts, clearly indicated that they recognize IMF/JSA TA projects as being relevant to recipient government needs and priorities. The answers for consistency with 1 One-page questionnaires collected by OTM within five weeks of the completion of each JSA project, as defined by Guidelines for the Use of Resources for Technical Assistance Activities from the Administered Account for Selected Fund Activities-Japan (JSA Guidelines). v

9 the recipient government policy were either Excellent (64%) or Good (36%), and there was no answer for Modest or Poor, as shown below. Answers by beneficiaries for consistency with government policy Answer Response Share Excellent % Good % Modest 0 0 Poor 0 0 No knowledge The field survey results indicated that IMF/JSA TA projects are relevant to the recipient countries developing priorities, based on their country contexts, and also relevant to IMF priorities, whose regional strategy is indicated in RSNs. 4. According to the interviews with TA departments at IMF headquarters, IMF TA projects are highly specialized and quite focused on the relevant development issues in monetary and finance policies, legislation process and procedures, which make the TA highly relevant compared to TA by other donors. 2) Effectiveness (Impacts) 5. Overall results of the evaluation surveys indicate that IMF/JSA TA projects are basically effective and achieved the desired goals. The results of the Project Assessment by TA departments indicate that TA projects have been regarded as effective in light of the targeted objectives. The results of the JSA Evaluation Questionnaire (by the beneficiaries) indicated an 80% yes response regarding the usefulness of the TA, as shown below. Answers by beneficiaries for usefulness Answer Response Share Yes % Partially Yes % 6. Based on our questionnaire surveys for the beneficiary countries and experts, the IMF/JSA TA projects have mostly achieved their expected results, with most answers as excellent (39%) or good (62%), and with no answers for Modest or Poor. TA outputs have been used by authorities and contributed to improvement of knowledge, awareness and motivation for reforms. In addition, TA outputs, to some extent, have directly led to improvement of policies and their instruments in the recipient countries. Major reasons for their effectiveness are: 1) expertise of advisors, 2) experience of experts in other countries, 3) communication skills of experts in understanding country context, and 4) backstopping by headquarters. 7. The field survey results show that IMF/JSA TAs are quite effective in generating (or contributing towards generation) of tangible results. Main reasons for the effectiveness vi

10 are: 1) expertise of experts, 2) focus on competitive sectors, 3) selection method (roster system) of experts, 4) communication skills of experts, and 5) backstopping by IMF headquarters. 3) Efficiency 8. Based on our field survey results, IMF/JSA TAs seem to be efficient due to their well-focused approach and selection of competent experts. Overall evaluation results of the study are basically positive in regard to the efficiency of IMF/JSA TA projects. 9. Regarding the efficiency in terms of cost, JSA is well managed through cost-effective means such as the roster-based selection of experts, smooth implementation supported by strong backstopping from headquarters, and repeated assignments by the same expert whose ability is proven. The budget per project and standard cost of long-term experts indicate that these are well within the reasonable range, compared with TA provided by Japanese ODA agencies and other international organizations. 10. The questionnaire survey results for JSA experts indicate that in comparison with other international organizations, IMF/JSA TA is higher in quality by 92%, while the cost is higher by 46%, indicating comparative higher efficiency in terms of quality and cost consideration, as shown below. Answers by Experts for Comparison with Other International Organizations Points of Difference Answered as yes Quality is higher for IMF/JSA 92% Monthly remuneration is higher for IMF/JSA 46% 4) Sustainability 11. According to our questionnaire surveys for both recipient countries and JSA experts, major factors for sustainability of TA projects such as institutional memory, staff capabilities, or budget allocation are achieved in most of the counterpart authorities, as shown below. The majority of the experts have expressed their shared views that sustainability of IMF TA depends on a willingness to reform by authorities. Answers by Experts for Levels of Sustainability Answer Response Share Excellent % Good % Modest % Poor 1 1.2% No knowledge 7 8.5% 12. The field survey results indicate that the sense of ownership by recipients varies by country, e.g. Vietnam with strong ownership. The concept of increasing the sense of vii

11 ownership for countries with weak ownership needs to be addressed to support the sustainability of the TA contribution. 13. As for financial sustainability, it is forecasted that IMF needs to rely more on externally financed sources, due to decreasing internally funded sources. IMF has introduced a charging system for internally-funded TA, which might not be significant in terms of volume, but it will have an impact in enhancing ownership of the recipient countries. Therefore, an application of the charging system to JSA will be needed to be considered. 5) Transparency 14. Although the IMF/JSA TA projects deal with policy-level and highly specialized areas of finance and monetary systems, the field survey results indicate that there is room for improvement in sharing information with other donors, including Japan, regarding the activities of IMF/JSA TA projects. 15. The recruitment system to select experts for IMF TA is by a roster system, due to the limited number of projects and competent experts to be employed. Although the current system is considered to be reasonable for operational efficiency, enhancement in transparency of the recruitment system is needed. 6) Consistency with Japanese ODA 16. Regarding the consistency of JSA with the Japanese ODA policy, three main documents compiled by the Japanese Government (ODA Charter, Mid-term Policy and Country Assistance Programs) were examined. The result of this examination was an assessment that IMF/JSA TA is highly consistent with the Japanese ODA policy. 7) Visibility of Japan as a donor 17. The overall evaluation results indicate that the visibility of Japan as a donor for IMF/JSA TAs is relatively low compared with the high scores gained for other aspects of evaluation and for effectiveness and benefit to the recipients. 18. The Questionnaire Survey results show that Japanese sponsorship of JSA TAs is known by 71% of the high ranking officials in the recipient governments, recognizing JSA is financed by Japan. On the other hand, 96% of experts answered that they are aware of the Japanese sponsorship. The narrative comments by experts indicated their support and understanding on the need for enhancing visibility of Japan as a donor. 19. The results of the field survey revealed that in particular countries, e.g. Cambodia, visibility is very high on the recipient side as well, partly because of the volume of JSA TA delivered and the IMF-Japan (MOF) joint mission s visit in early However, in other countries visited, e.g. Indonesia, Tanzania and Vietnam, the awareness on the recipient authorities was low, suggesting some room for improvement. viii

12 Answers by beneficiaries for recognition of Japan as a donor Answer Response Share Yes, very much so % Yes, relatively so % Not so much % Not at all 1 2.6% No knowledge % 2. 1) Evaluation Results Based on TOR criteria Overarching Objectives 20. The overarching objectives are defined by the three points below. (1) To assess whether the JSA has been an effective instrument for capacity building in countries, covering issues of (a) quality of TA, (b) sustainability, and (c) efficiency; (2) To determine whether the administration of and information on the JSA is sufficient to provide accountability and whether the JSA provides good value for money, covering issues of (a) consistency with Japanese ODA, (b) visibility of Japan as a donor, (c) supporting reasons for IMF TA; and (3) To make recommendations to improve the JSA in furtherance of the objectives in (1) and (2) above. 2) The JSA as an Effective Instrument for Capacity Building in Countries 21. The followings are findings by the evaluation team for each of the subjects being questioned. (a) Quality of JSA TA 22. Is the JSA TA of high technical quality consistent with international best practice and member country priorities? Most of the recipient country counterparts recognize that the quality of IMF/JSA TAs is at the best international level. Most of the recipient countries also agree that IMF/JSA TA's are quite consistent with their developing priorities. 23. Is the involvement of area departments useful in determining member country TA priorities, integrating them with Fund surveillance and lending activities, and providing quality control? The involvement of area departments is useful and critical because initial identification of needs is done by the area departments. They monitor surveillance results and consider what kind of TA activities and areas are necessary. Area departments compile Regional Strategy Notes (RSN) which outlines regional issues to be addressed and assistance priorities and plans in each country. ix

13 24. What measures are in place in TA departments to ensure that IMF/JSA TA is timely and of high quality? Have these measures been adequate/effective? TA departments provide a strong back stopper service, which is key for quality control of JSA and other TA activities. The headquarters staff in TA departments are specialists themselves, who have deep knowledge and experience in the subject areas. This backstopping function differentiates IMF TA from other donors, where usually, the generalist staff manages TA activities. As for timeliness of TA, JSA is a quick funding source compared with other externally funded TAs. 25. Are the current evaluations/assessments of JSA activities adequate/effective? The current evaluation/assessment system of the IMF/JSA TA seems to have much room for improvement from the viewpoints of both accountability and management improvement. (b) Sustainability of Results Achieved Through IMF/JSA TA 26. Have JSA activities as stand-alone, or in conjunction with IMF financed activities, achieved tangible and sustainable results in beneficiary countries? Most recipient country counterparts recognize that the implemented IMF/JSA TAs have been able to generate tangible outcomes, in some cases impacts, in their countries. The coverage of IMF/JSA TA, however, varies with countries quite widely. (c) Efficiency and Cost Considerations 27. Is IMF/JSA TA a cost-effective way to deliver capacity building assistance to countries, especially in relation to other comparable TA delivery modes (as determined by the evaluators), and bearing in mind the difficulties inherent in measuring the benefit of capacity building activities? Since many (most) recipient countries regard the contents of IMF/JSA TA as unique, the comparison with other developing partners seem not to be relevant. However, if we dare make some comparison, there seem to be several factors which contribute to the cost-efficiency of JSA. First, the roster-based selection process of experts is more cost-efficient than public bid or open application procedures. Second, the repeated assignment of the same qualified expert whose ability is proven, is efficient in many respects due to the established relationship and experience in the field. Third, strong support and quality control of IMF headquarters functional departments contribute to the smooth and efficient implementation of IMF/JSA TA. The standard cost of dispatching the expert, as far as the remuneration rate is concerned, seems to be reasonable compared with similar Japanese TA schemes. 28. Is the cost of IMF/JSA TA reasonable in relation to other comparable TA providers? The Budget per project (around US$150,000) and the standard monthly remuneration rate for dispatching experts as determined in the JSA Guidelines seems to be reasonable compared with similar cases in Japanese bilateral technical assistance such as JICA. Furthermore, the disbursement rate is quite high, 96% on average from 1993 to 2009, which is another aspect of sound management and administration by the IMF department in charge. x

14 3) IMF/JSA TA Delivery is Providing Good Value for Money for Japanese Taxpayers (a) Consistency of JSA Activities with Japan s ODA policy 29. Have JSA activities been consistent with Japan s ODA policy? There are three major documents compiled by the Japanese Government which represent the Japanese ODA policy, i.e. ODA Charter, Mid-term Policy and Country Assistance Programs. Examination of these documents show that IMF/JSA TA activities are highly consistent with the Japanese ODA policy. 30. Have the JSA activities overlapped negatively with other Japanese ODA initiatives, e.g., bilateral ODA and/or activities of Japan funds at other international institutions? If so, how can these overlaps be best removed/addressed? None of our findings indicate the existence of duplication or overlap between the IMF/JSA TA and other Japanese ODA technical assistance implemented. This is partly due to the difference in areas of specialty focused on by the IMF and Japanese ODA. Nevertheless, there is an important and necessary complementary role for IMF/JSA TA in light of future development of Japanese bilateral ODA in the financial and fiscal sectors. (b) Visibility of Japan as a Donor 31. Has the JSA provided adequate visibility to Japan as an ODA donor country? Partly yes and partly no. There is room for improvement, in particular on the beneficiary side, with more institutionalized arrangements by the IMF and the Japanese authority in charge. 32. Has the JSA enhanced the visibility of Japan as an ODA donor country? Partly yes and partly no. Since Japan is recognized as a large donor, in particular in Asia, the effectiveness of enhancing visibility through JSA is, if properly implemented, is marginal. However, in countries where the Japanese presence is not strong, e.g. Africa, utilization of JSA, as a part of IMF which has strong influence in the financial and fiscal sector, might have more effectiveness. 33. Have the publicity guidelines in the JSA been effective? If not, how can they be changed to improve the visibility of Japan as an ODA donor country? There seem to be various possibilities for improving Japan s visibility including publicity activities by building into administrative process. This is discussed under recommendations in the next chapter. (c) Support for IMF TA Versus other Avenues for Providing Development Assistance 34. What are the advantages of supporting IMF TA, as compared to using Japanese funds for other development assistance? For Japan, the areas involving financial systems, fiscal management and macroeconomic analysis are areas with limited human resources that need to be supplied in TA projects to developing countries. The advantage of sponsoring IMF TA through JSA is that it expands the opportunity for Japanese bilateral ODA coverage, xi

15 with possible collaboration and complementary exchange among experts and ODA agencies; typically JICA. IMF TA in African countries. e.g. Tanzania, is regarded in Japan as a pioneering effort, compared to some countries in Asia, e.g. Vietnam, where the country is already utilizing JICA TA in the central banking and fiscal areas. A strategy can be implemented to make use of JSA for pioneering efforts in a less experienced country. The coverage of bilateral donors will increase when Japanese ODA catches up in providing its own TA directly to the recipient countries. (see the illustration of this concept below) Complementary Role of IMF TA and Bi-lateral TA xii

16 II. Recommendations Recommendations to IMF 1. Improvement in Transparency 35. It is recommended that the results of project-level evaluation by recipients, collected through JSA evaluation questionnaires, should be summarized and reported with more detailed analysis of the evaluation results in the JSA Annual Report in order to present this information to the general public both in Japan and in other countries. 36. In order to maintain the current recruitment system which is considered to be operating efficiently, it is recommended that there be disclosure of the recruitment process for experts to be employed by the IMF for JSA TA. Particular emphasis should be placed on the specific qualifications required, relevant background and experience along with communication skills with recipient agencies and IMF headquarters. This will help to increase transparency in the recruitment procedure since the recruitment process does not utilize an open-bid system similar to other donor agencies. 37. It is recommended that information sharing with other donors regarding the contents of TA activities be institutionalized. This will provide enhanced transparency and possible complementary effect with the other donors input, in particular with Japan, i.e. the Japan International Cooperation Agency (JICA). Since there might be a limitation in information disclosure with the authorities in the recipient countries, the level and content of information sharing must be considered in light of country and sector context as well as potential interest by other donors in the field. The IMF Resident Representative should assume a coordinating function in the field. 2. Improvement of Management 38. It is recommended that management of JSA Project Proposals and Project Assessments be improved regarding the points listed below: Improvement in consistency in the information to be filled-in; Improvement in filling out verifiable indicators for the projects with guidance regarding the intention and the concept; Improvement in describing the assumptions for the project; Improvement in filling in the matrix for the mid-term objectives; Improvement in describing the achievement of the projects and their intended targets; Improvement in procedures regarding the extension of projects with conditions to be clarified. 39. In order to enhance the effectiveness, impact, and efficiency of projects, it is xiii

17 recommended that a strategy should be established for programming the TA project in combination with long-term and short-term (peripatetic) experts, depending on the level of focus for a particular issue in a particular country, with attention toward: More focus regarding both the IMF Context and the Country Context in selecting the country and areas of TA. An exit policy and targets that need to be achieved within a limited period. This will enhance ownership of the recipient, which is also indicated in the 2005 IEO Report as a need for graduation of long-term experts. 40. It is recommended that more focus on sustainability and ownership of the recipient organizations occur by incorporating them in the JSA Proposals as well as in the Project Assessment, through applying methodologies in objective evaluations. 41. It is recommended that OTM improve the managing and administering JSA project proposals, project assessments, and JSA evaluation questionnaires, through a more efficient system to store, file, collect, and sort data on the IMF intranet. While we discovered that OTM started its improved system for collecting the questionnaires, more attention is necessary to sustain the effort given the limited resources in OTM. 3. Improvement of Visibility 1) Symbolic Improvement 42. It is recommended that a logo and/or concept be printed in support of the JSA and this logo should appear on IMF/JSA TA relevant materials, documents, reports and name cards of all the experts, to indicate clearly that JSA is being financed by Japan. 2) Enhance Visibility of the Recipients 43. As the results of this evaluation study indicate, the recipient authorities need to be more informed of the Japanese contribution to IMF/JSA TA projects. In this regard, it is recommended that the IMF enhances its procedures, so that the recipients of JSA TA, at various levels, are well-informed of Japan as the donor. 3) Enhance Japan s Involvement in JSA projects 44. It is recommended that the recruitment of Japanese experts for short-term assignments for seminars and workshops be strengthened, especially for assignments of less than one week. While the current number of Japanese experts in the roster is 30 out of a possible 1,350 in total as of June 2009, cooperation with Japanese authorities is needed to up-date and improve the roster. 45. It is recommended that regional forums are organized and conducted in order to introduce activities covered by IMF/JSA TA in Asia and/or Africa. These forums are to be xiv

18 held in Japan, with the participation of IMF experts, recipient governments, The Japanese Government, ODA agencies, Japanese experts, scholars, the private sector and the mass-media. This will facilitate the: (i) dissemination of information focused on relevant experience, (ii) publicizing JSA s financing support, and (iii) strengthening communication with the Japanese ODA circle. Recommendations to MOF 1. Improvement of Transparency and Information Disclosure 46. It is recommended that public relations should be strengthened in order to relay information on JSA projects and their achievements to the general public, i.e. taxpayers, through seminars, forums, websites, the writing of articles on ODA publications and other media. 47. It is recommended that Information sharing should be enhanced regarding Japanese ODA agencies and relevant groups, academic circles, experts and consultants, with a focus on project-level information for JSA projects. This will lead to the identification of interesting issues to be covered by the proposed regional forum. 48. It is recommended that a strategy is created to make use of IMF TA and the interests of Japanese ODA in the field of financial and fiscal system development in developing countries with attention to: Policy of MOF and MOFA for ODA policy, utilization of Japanese resources and utilizing IMF capacity in a complementary arrangement. Strategy to use the JSA as a tool to supplement Japanese ODA. Develop documents to explain the Japanese historical experience in monetary and fiscal systems, and policy implementation experience as reference material for developing countries. Establish a strategy to mobilize JSA and JICA experts, depending on the country context in development needs of the target countries financial and fiscal system, and possible involvement of Japanese bilateral assistance. 2. Improvement of Visibility 1) Symbolic Improvement 49. It is recommended that cooperation with IMF occur in the symbolic improvement of JSA related documents to be used by IMF staff as well as experts. In order to have the recipient government be aware of the Japanese assistance, the use of bilateral channels to inform the recipient government can be effective. xv

19 2) Enhance Japan s Involvement in JSA Projects 50. It is recommended that the number of Japanese experts on the IMF roster of experts be increased with screening by capacity and appropriate tasks to be covered. Despite the efforts by IMF and MOF, in the past to increase Japanese participation, use of Japanese nationals for IMF/JSA TA is quite limited. In order to improve the participation of Japanese experts, more participation of officials from MOF, Bank of Japan and Financial Services Agencies should be encouraged with cooperative effort by the respective departments among Japanese agencies in seeking capable staff as IMF/JSA TA experts. 51. In order to increase awareness by Japanese domestic interested parties, it is recommended regional forums be organized to disseminate information on the activities of JSA experts, in particular, with countries that have limited information about the Japanese ODA circle. 52. It is recommended that a system to have JSA experts establish communication with JICA at headquarters as well as field offices should be set up. This could include possible collaboration in the field. xvi

20 1. Outline of the Independent Evaluation Chapter 1 Outline of the Independent Evaluation 1.1 Background and Objectives Background 53. Since 1990, the Japanese Government has provided financial support for IMF technical assistance to its member countries to strengthen their capacity to formulate, implement, and maintain macroeconomic and structural adjustment programs. Japan s contributions are provided through the Japan Administered Account for Selected IMF Activities (JSA), and Japan has been the largest contributor to the IMF s technical assistance (TA) activities 54. As of the end of 2008, Japan has made contributions totaling approximately US$279 million to finance IMF TA activities. These funds have been used to cover costs of selected IMF TA projects, including the salaries and travel costs of short- and long-term TA experts and costs associated with organizing seminars and workshops. 50 percent of each annual JSA contribution is targeted at TA activities for countries in the IMF s Asia and Pacific departments and selected countries in Central Asia. The use of JSA funds has been flexible, with the main focus of the IMF/JSA TA program being support for low- and lower-middle-income countries in building institutions and capacity to implement growth-enhancing economic policies. Objectives 55. This evaluation is being requested and sponsored by Japan s Ministry of Finance, in its capacity as the Japanese agency responsible for the JSA, and is the first evaluation of its kind. The evaluation s overarching objectives are: (1) To assess whether the JSA has been an effective instrument for capacity building in countries, covering issues of 1) quality of TA, 2) sustainability, and 3) efficiency; (2) To determine whether the administration of and information on the JSA is sufficient to provide accountability and whether the JSA provides good value for money, covering issues of 1) consistency with Japanese ODA, 2) visibility of Japan as a donor, 3) supporting reasons for IMF TA; and (3) To make recommendations to improve the JSA in furtherance of the objectives in (1) and (2) above. 1.2 Implementation and Study Activities 56. Mitsubishi UFJ Research & Consulting Co., Ltd. (MURC) was appointed by IMF as a consultant to conduct this independent evaluation through a contractual arrangement between IMF and MURC by Agreement Number 1507, dated June 8, 2009, and MURC, as a contractor, is responsible for setting up study plans, conducting data/ information collection 1

21 1. Outline of the Independent Evaluation and their analyses, implementing field surveys and questionnaire surveys, and reporting. MURC formed a team of experts listed below to conduct this task augmented by one outside advisor from the Japan Economic Research Institute (JERI). The flow of the work conducted is depicted in Chart 1.1, and brief contents of the work are described below. Evaluation Team Members Mr. Hidekazu Tanaka, Team Leader (MURC) Mr. Jun Arakawa, Program Evaluation Expert (MURC) Mr. Yasuhiro Sakon, Program Evaluation Expert (MURC) Mr. Yoichi Hara, Financial Management Specialist (MURC) Ms. Masumi Shimamura, Aid Policy Specialist (MURC) Mr. Hisatsugu Furukawa, Advisor (Japan Economic Research Institute) Chart 1.1 The Evaluation Study Flow Chart Preparation and Interviews in Japan 57. The work started with listing and sorting of JSA projects during the evaluation period with data available from the IMF web-site as well as information provided by OTM. At the same time, review of annual reports, project proposals and assessment reports for each JSA project were conducted, and review of the past evaluation reports for IMF TA was also covered. Interviews with relevant organizations and experts in Tokyo took place with the Ministry of Finance, Regional Office for Asia and the Pacific (OAP), former IMF TA advisor to the National Bank of Kyrgyz Republic and the former JICA TA advisor to the State Bank of Vietnam. These interviews provided the team members with basic knowledge and 2

22 1. Outline of the Independent Evaluation ideas of JSA and TA in the areas concerned by IMF, in developing countries and transition economies. Discussion at IMF HQ 58. The MURC Evaluation Team, with all five members, visited IMF Headquarters from July 13 to 17, 2009, to have discussions with the OTM, and conducted interviews with key IMF staff at Regional departments, MCM, FAD, STAT, the Legal Department, IEO, IMF Institute and the Executive Director s Office. The arrangement of interviews and coordination of the team was managed and assisted by the Office of Technical Assistance Management (OTM), by the following contacts. Mr. Alfred Kammer, Director, Office of Technical Assistance Management (OTM) Mr. Harish Mendis, Senior Technical Assistance Officer, OTM Mr. Barrie Williams, Senior Budget Assistant, OTM Field Visit Survey 59. In order to verify the achievement of results, effectiveness, and efficiency of IMF/JSA TA projects, visit interviews were conducted in Indonesia, Cambodia, Vietnam and Tanzania during August 17 to 26, The evaluation in the field was focused on the benefit and effectiveness to the recipient countries and visibility of Japan as a donor. Through the field visits, good practice cases were selected for each country as the result of the interview in the field. Questionnaire Survey 60. The questionnaire surveys were conducted through and web-based collection of answers from recipients as well as experts involved in TA activities financed by JSA. The questionnaire distribution, collection and compilation of the results were conducted by Rapid Access International (RAI), a Washington DC based consultant, through a sub-contracting arrangement by MURC. 1.3 Documents, Data and References 61. The Team reviewed the documents related to activities of JSA and evaluation work done by IMF and the World Bank, as listed below. 1) JSA Annual reports ( ): available through the IMF website English version: Japanese version: Annex 1 of each Annual Report 2004 to 2008 was reproduced in a form using Excel, by the team, to be used for further analysis. 3

23 1. Outline of the Independent Evaluation 2) JSA Project Proposals and Assessments: files were provided by OTM FY 2004 (May 2003-April 2004) FY 2005 (May 2004-April 2005) FY 2006 (May 2005-April 2006) FY 2007 (May 2006-April 2007) FY 2008 (May 2007-April 2008) Note: The assessment reports are not available for the projects being extended to the present or for projects cancelled. 3) Beneficiary Survey Results 62. For each JSA project, at the time of project completion, a Beneficiary Survey sheet is distributed for the beneficiaries (counterpart recipient organization) to respond with checks and narratives or notes. The team obtained the returned sheet copies through OTM. 4) Reference Project Documents 63. The JSA project documents, back-to office reports, mission report, presentation material and related documents for selected JSA projects were made available from TA departments through OTM to the evaluation team. 5) Evaluation Reports 64. Among several evaluation reports for IMF TA, the following were considered as highly relevant to the work, and carefully reviewed by the team. The review results for some of these reports are provided in Chapter 1.5. Enhancing the Impact of Fund Technical Assistance, April 3, 2008, Office of Technical Assistance Management (In consultation with other Departments), IMF Evaluation of the Technical Assistance Provided by the International Monetary Fund, January 2005, Independent Evaluation Office, IMF Evaluation of the Technical Assistance Subaccount for Iraq, March 2008, Bruce Murray (Team Leader) Independent Evaluation: African Regional Technical Assistance Center (AFRITAC), March 2009, Bruce Murray (Team Leader) 65. The report below is an evaluation for the Japanese trust fund for TA activities under PHRD, a reference for a similar contribution to the World Bank. Evaluation of the Policy and Human Resources Development Trust Fund, January 2008, UNIVERSALIA 4

24 1. Outline of the Independent Evaluation 1.4 Evaluation Criteria and Verification Method 66. The evaluation work was conducted through: 1) Review of IMF/JSA TA activities; 2) Evaluation of JSA TA projects; and 3) Field visit interviews to recipient countries to determine the overall scope of IMF/JSA TA activities. The following methodology was applied for each of these tasks. 1) Review of IMF/JSA TA Activities 67. The holistic review of the IMF/JSA TA activities was conducted from a multi-dimensional viewpoint. Based on the contents reported in the Annual Report, the general characteristics of the past JSA TA activities were analyzed by areas, regions/countries, and year. In particular, among the three main areas listed below, the subjects covered, as well as the activities of countries with repeated TA recipients. Chart 1.2 Core Areas of IMF Technical Assistance Areas Subjects Fiscal Policy and Management Tax policy, Tax and customs administration, Expenditure policy, Budgeting and public expenditure management, Fiscal management, Fiscal federalism Macroeconomic and Financial Multi-sector statistical issues, Balance of payments and external debt Statistics statistics, government finance statistics, Monetary and financial statistics, National accounts and price statistics, Data dissemination standards Monetary Policy and Financial System Other Areas Central banking and currency arrangements, Monetary and exchange policy operations and public debt management, Financial market development, focusing particularly on money, government debt, and foreign exchange markets, Exchange systems and currency convertibility, Payment systems, Bank supervision and regulation, Bank restructuring and banking safety nets Legal, Training and Anti-Money Laundering/ Combating the Financing of Terrorism 2) Evaluation Criteria of IMF/JSA TA Projects 68. For this evaluation study, the evaluation of IMF/JSA TA for effectiveness and benefit to the recipient countries and accountability to the Japanese general public are categorized by the terms of reference (TOR) with six criteria: 1) Quality of Activities; 2) Effectiveness and Sustainability; 3) Efficiency; 4) Consistency with Japanese ODA Policy; 5) Visibility of Japan as Donor; and 6) Advantage of IMF Scheme. On the other hand there are six criteria by OECD DAC Criteria for Evaluating Development Assistance (relevance, effectiveness, efficiency, impact and sustainability), widely applied to development aid evaluation, have been modified for this evaluation, in seven criteria, namely 1) Relevance, 2) Effectiveness (Impact), 3) Efficiency, 4) Sustainability, 5) Transparency, 6) Consistency with Japanese ODA and 7) Visibility of Japan as a donor. These sets of criteria were applied to conduct this evaluation work. 5

25 1. Outline of the Independent Evaluation Category Effectiveness and benefit to the recipient countries Accountability to the Japanese general public Chart 1.3 Evaluation Criteria of IMF/JSA TA Projects Viewpoint Quality of Activities (1) Quality and consistency with the priority of recipients (2) Effectiveness of regional departments (3) Quality and timelines of Technical Assistance Departments (4) Evaluation to JSA TA projects for relevancy and effectiveness Effectiveness and sustainability (1) Achievement of issues through TA (benefit to the recipients) Efficiency (1) Efficiency (value for money) of JSA TA projects (2) Appropriateness of cost for JSA TA projects Consistency with ODA Policy (1) Consistency with Japanese ODA Policy (2) Duplication with other technical assistance by Japan, including measures to be taken Visibility (1) Level of visibility of JSA TA projects (2) Improvement of visibility through JSA TA projects (3) Effectiveness of guidelines for improving visibility Advantage of IMF scheme (1) Advantage of IMF scheme compared with other schemes 69. In regard of the effectiveness, the following logic model is taken into account as a base: Chart 1.4 Basic Logic Model for IMF/JSA TA Projects External factors Inputs Activities Outputs Outcomes Impacts Immediate Intermediate Final JSA Funds: TA Activities & Outputs: TA Outcomes: TA Impacts: IMF Experts: HQ Resources: - Capacity building (Human, Institutional) - Materials, seminars, WSs, daily guidance - Policy-making & -reforming capacities - Human & Institutional - Socio-economic changes and/or - Policy designing - Laws, regulations, manuals - Including management & system -related capacities - Policies and/or policy reforms in place improvements - Management & organization reforms - System architectures - Laws, regulations, manuals - Management & organization reforms - System architectures Major targets of this evaluation 6

26 1. Outline of the Independent Evaluation 70. The questionnaire survey was conducted and addressed to counterparts of recipient governments as well as experts sent through JSA projects. The questionnaires, covering the above points, were sent via instructing respondents to submit responses to the web-site created for this evaluation work. The contents of the questionnaires were coordinated through consultation with OTM and addresses were provided by relevant TA departments of IMF. 3) Field Visit Interviews to Recipient Countries 71. Taking into account the above evaluation coverage, the selection of the countries to be visited was based on the usage of IMF/JSA TA, intensity of Japanese bilateral ODA, and availability of the counterparts during the designated visit period. For each field visit, conducted by two team members to each country, interviews were made based on criteria to the recipient ministries and agencies, IMF resident representatives, technical assistance center, experts under JSA and other donors including Japan and the World Bank. 1.5 Review of Evaluation Reports 72. As a reference to the evaluation work, some of the past evaluation reports closely related to the evaluation of JSA were reviewed and their brief contents are introduced in this section. The reports reviewed were 1) Evaluation of IMF TA by IEO, 2) Enhancing the Impact of Fund TA by OTM, and 3) Evaluation of the PHRD Fund of the World Bank. 1) Evaluation of The Technical Assistance provided by The International Monetary Fund prepared by the Independent Evaluation Office of International Monetary Fund, January 31, In this evaluation work, the Independent Evaluation Office (IEO) of IMF assessed its own technical assistance (TA) to its member countries. The methodologies used were desk reviews of relevant data as well as in-depth case studies for six selected countries; Cambodia, Honduras, Niger, Ukraine, Yemen, and Zambia. The case studies for six countries were compiled based on the field visits and interviews in each country with public officials and donors, reviews of past evaluations, and interviews with IMF staff and other stakeholders. In the evaluation analysis, the TA process was examined by dividing it into three stages; (1) Prioritization: How are countries' TA needs identified? What can be done to make the process more strategic so as to increase the relevance of IMF TA activities? (2) The delivery process: What factors influence the effectiveness of the various modalities for TA delivery? (3) Monitoring progress and evaluating impact: How is progress tracked and what factors contribute to the impact of TA? 74. Major findings and recommendations are summarized as follows: Prioritization: Seventy percent of IMF TA is provided to countries with per capita income 7

27 1. Outline of the Independent Evaluation below $1,000, and therefore, IMF TA is well targeted to low-income countries. Furthermore, the report analyzed the level of linkage between TA priorities and key policy issues identified in Article IV consultations. Regarding the linkage with Article IV consultations, the evaluation report presents interesting analysis in its Chapter 4 (Table 4.1. on page 45), where the strength of the link between Article IV consultations and IMF TA is examined for 18 countries. Chart 1.5 provides a summary of the scores. Chart 1.5 Strength of the Link between Article IV Consultations and IMF TA PEM and Budget Revenue (Tax and Customs) Monetary and Exchange Rate Policy Banking Supervision and Restructuring Statistics Average Score Bangladesh Belarus Bulgaria Burkina Faso 3 1 n.a. n.a Cambodia Cameroon 3 3 n.a. n.a Ecuador Ethiopia Honduras Jamaica Morocco Niger 3 3 n.a. n.a Tonga Turkey Ukraine Vietnam Yemen Zambia Average score Source: IMF documents. 1 = weak or no apparent link; 2 = partial link; 3 = close link between surveillance and TA from IMF; n.a. = not applicable (for these countries, responsibility for subject area rests with the regional central bank (Banque Centrale des Etats d Afrique de l Oeust). (Source: Evaluation of The Technical Assistance provided by The International Monetary Fund, IEO-IMF, 2005, p.45) 75. On average, the strength of the links vary from country to country with a range of 1.7 at the lowest end to 3.0 at the highest and total average of 2.3. (The designation of this 3-point scale is as follows: 1 = weak or no apparent link; 2 = partial link; and 3 = close link between surveillance and TA from IMF.) It should be noted that Cambodia, where JSA plays a significant role and is the subject of our field survey this time, obtained an average of 2.8, which is the second highest after Ukraine. Cambodia reached the full score of 3.0 for the four TA areas (Revenue, Monetary and Exchange Rate Policy, Banking Supervision and Statistics) and 2.0 for PEM and Budget. This result implies that, given the magnitude of JSA in the country, IMF TA is very appropriately designed and provided with the right priority-setting in Cambodia. On the contrary, it is noted that the score for Vietnam is rather low with a 1.8 average. However, the strength of the link differs sharply according to TA areas, i.e. there is a close link in banking supervision, but partial or weak links in other 8

28 1. Outline of the Independent Evaluation areas. 76. The analysis of short-term versus long-term modalities for providing TA is another interesting issue noted in the report. 2 This raises a question regarding the result of the 1999 Fund-wide evaluation which suggested that shorter term modalities were more cost effective. The report argues that long-term experts (LTE's) can play a critical role and provide significant values to recipient countries. As evidence of this line of argument, the report presents the cases in Niger, Cambodia, Honduras and Ukraine, where the works of LTE's were highly appreciated by the counterpart officials. It concluded that there is an appropriate mix between resident experts and short-term experts depending upon the level of institutional capacity and country context. 77. The delivery process: The recipient countries have generally been satisfied with the experts sent by the IMF. They appreciate the experts hands-on role in training and coaching, accessibility, and emphasis on teamwork. The report recommends enhanced involvement of the authorities in the preparation of terms of reference (TOR), which led to stronger ownership and shared expectations by the authorities and IMF about final objectives. 78. Monitoring progress and evaluating impact: Progress has generally been achieved in enhancing the technical capacity of the agencies that the IMF typically supports. The report recommends that IMF documentation and reporting be improved to track the different stages of progress toward meeting the final objectives of TA, with clear indicators of progress at the outset. Also, the report suggests a need for unbundling between short- and medium-term indicators that capture different stages of the results chain, which provides a track record in implementing TA in making decisions about future TA. 79. In conclusion, key recommendations were made as follows: The IMF should develop a medium-term country policy framework for setting TA priorities, incorporating country-specific strategic directions and linked to more systematic assessments of factors underlying past performance. The IMF needs to engage countries to help them articulate their medium-term capacity-building needs in the IMF's areas of responsibility and in accordance with the PRSP. The IMF should develop more systematic approaches to track progress on major TA activities and identify reasons behind major shortfalls. Greater involvement by the authorities and counterparts in the design of TA activities and arrangements for follow-up should be emphasized as a signal of ownership and commitment. 2 This analysis is presented in detail in B. Short-Term Versus Long-Term Modalities for Providing TA (pp.73-75) and in C. Attributes of a Good Long-Term Expert (pp.76-77) of the IEO Report. 9

29 1. Outline of the Independent Evaluation 80. In addition to the major report, there are case studies on specific countries followed by recommendations. For example in Cambodia, some of the recommendations are as follows: In order to avoid countries becoming addicted to TA it is necessary to plan how to phase out TA advice in order to encourage the authorities to start walking on their own, and to help them graduate. One possibility is to shift from long-term to peripatetic experts as capacity is successfully built and a certain threshold is surpassed. (The lack of funding for implementing and following-up TA recommendations became an obstacle to the large impact of TA.) A closer coordination with international donors to provide a full package where TA recommendations are matched with resources from these donors would be ideal. Pooling donors resources, to make sure ex ante that TA recommendations will be properly funded, is desirable. 2) Enhancing the Impact of Fund Technical Assistance prepared by the Office of Technical Assistance Management (In consultation with other Departments), April 3, This paper was prepared with the view to make Fund TA more effective, ensure that TA resources are allocated efficiently, and that TA is better coordinated with recipient countries, other providers, and development partners. The Medium-term Strategy and Independent Evaluation Office (IEO) recommendations have called for substantial changes in the way the Fund conducts business, including information management and disclosure. There is a supplemental document to this paper titled: Dissemination of Technical Assistance Information (SM/08/97). The main issues discussed in this paper for substantial reforms (in prioritizing, monitoring, evaluating, and costing TA) are summarized as follows: Enhance the integration of TA with surveillance and Fund lending operations. Enhance prioritization by better aligning TA with strategic objectives of the recipient country and coordinating with other TA providers. Make the TA more responsive to changes in priorities. Better integrate TA into the Fund s medium-term budget to make priority-setting easier. Introduce Fund-wide performance indicators (moving to results-focused management of TA projects) to make TA more transparent and accountable. Refine the costing and billing of TA projects to help ensure efficient allocation of resources, better inform management decisions and enhance accountability. Strengthen partnerships with donors with a more proactive external financing strategy. 3) Evaluation of the Policy and Human Resources Development (PHRD) Trust Fund was prepared by UNIVERSALIA, January The PHRD trust fund to the World Bank is another technical assistance contribution 10

30 1. Outline of the Independent Evaluation to an international organization by the Ministry of Finance of Japan. Due to its relevance to JSA, the contents of this report can be of reference for JSA evaluation. PHRD TA Effectiveness 83. Regarding the effectiveness of PHRD TA, the report describes: The PHRD TA is a highly effective facility that has produced quality results in almost all areas. As an investment, it has led to a high level of outputs and good quality results for the resources expended. PHRD technical assistance is rated highly by both the World Bank and government officials, particularly in terms of the quality of project preparation for Bank loan operations. PHRD grants do not appear to duplicate other Official Development Assistance (ODA) resources. 3 Recommendations 84. The report concludes by presenting the following six recommendations 4. (1) A Bank-wide discussion, possibly led by the Operations Policy and Country Service Department (OPCS), should be initiated to develop a position paper on the financing of project preparation in the Bank and to make the experience of PHRD TA more widely known to Bank investors. (2) Given a future in which the increasing demand for project preparation support may exceed the supply, the World Bank and the Government of Japan will need to establish more strategic priorities for PHRD TA to ensure that it manages available resources in the most cost-effective way. The Recommendation 2 includes, among others; revising allocation between IDA/ non-ida countries and targeting climate change grants to encourage broader based mitigation projects (3) The Government of Japan and TFO (Trust Fund Operation) should identify tools and processes that support a greater recognition of Japan s dedicated support to good project preparation through PHRD TA. The Recommendation 3 includes: symbolic changes such as putting the Japanese flag on PHRD TA documents; Process changes to enhance recognition of Japan as the sponsor of TA; Strategic changes for more country-level coordination between WB and the Japanese Embassy at the selection stage. (4) The TFO and the Government of Japan should consider limited additional flexibility in PHRD TA grant conditions in order to increase the effectiveness of the grant program. (5) The World Bank and Government of Japan should consider ways to increase efficiency of the PHRD TA grant approval process, to avoid unnecessary delays in a process that can take six months to a year or more to complete. (6) TFO should ensure greater oversight of the PHRD TA reporting system to check that GRM (Grant Reporting and Monitoring) reports are complete and that they provide adequate data to monitor and assess the efficiency and effectiveness of the PHRD TA grants. 3 Page xiii, Final PHRD Synthesis Report, UNIVERSALIA, January Quoted from page xv to xvii of Final PHRD Synthesis Report, UNIVERSALIA, January

31 2. Outline of JSA and Japanese ODA Chapter 2 Outline of JSA and Japanese ODA 2.1 Outline of JSA Background and Objectives 1) Background 85. The history of JSA dates back to 1990, when Japan started to provide grant contributions to support the IMF s technical assistance to member countries. Among the external financial sources which supported IMF TA, JSA is one of the major financial sources. In fact, since 1990, Japan has been the largest contributor to the IMF s technical assistance (TA) activities. 5 In FY2009, external financing from bilateral and multi-lateral donor partners occupied close to 70 percent of technical assistance delivered in the field, and the rest was sourced from IMF internal financing. Despite the trend of increasing financial sources from bilateral and multilateral donors, the Japanese share among the external financing source remains the largest. In FY2009, JSA financing accounted for about one-third of total external sources, which plays a dominant role in supporting the IMF TA. Chart 2.1 shows the historical trend of JSA s share of financing of the IMF s field-delivered technical assistance from FY2000 to FY2009. Chart 2.1 JSA Share of Financing of Technical Assistance, FY2000 FY Percent IMF JSA Others FY (Source: JSA Annual Report 2009 and others) 5 Contributors to IMF TA other than Japan include Australia, Austria, Belgium, Canada, China, Denmark, Finland, France, Germany, India, Ireland, Italy, Korea, Kuwait, Luxembourg, the Netherlands, New Zealand, Norway, Oman, Portugal, Qatar, Russia, Saudi Arabia, Spain, Sudan, Sweden, Switzerland, the United Arab Emirates, the United Kingdom, and the United States. (In addition, Singapore, Brazil, the Arab Monetary Fund and the African Development Bank provide support for IMF Institute s regional training centers.) 12

32 2. Outline of JSA and Japanese ODA 86. Chart 2.2 shows the historical trend of annual contributions which were used solely for the technical assistance portion of the JSA since its foundation in The figure indicates that since 1990 Japan has made annual contributions for JSA totaling $305 million, out of which $278.9 million was used for TA projects including the activities of the Regional Office for Asia and the Pacific. Chart 2.2 Annual Contributions by Japan for Technical Assistance FY1990 FY US$ 15.0 Million FY (Source: JSA Annual Report 2009) 87. The detailed breakdown of Japanese contributions and JSA is also indicated in Chart 2.3. Chart 2.3 Contributions by Japan FY1990 FY2009 (In millions of U.S. dollars) FY FY 2001 FY 2002 FY 2003 FY 2004 FY 2005 FY 2006 FY 2007 FY 2008 FY 2009 Total FY1990- FY2009 JSA Technical Assistance Asia Scholarship Program Advanced Scholarship Note: Technical Assistance includes activities of the Regional Office for Asia and the Pacific. (Source: JSA Annual Report 2009) 6 JSA supports the activities implemented by the IMF Regional Office for Asia and the Pacific (OAP) in Tokyo and also the Japan-IMF Scholarship Program for Asia (JISPA), which is administered by OAP. This evaluation is limited to TA activities financed by the JSA, and does not cover OAP or the scholarship program. 13

33 2. Outline of JSA and Japanese ODA 2) Objectives 88. The main objective of JSA TA has been to provide assistance in institutional and capacity building for low-income countries and lower-middle-income countries to sustain their economic growth. Regarding the target area of support, the use of JSA generally reflects the priorities of IMF technical assistance as a whole. JSA TA has been targeted mainly in the monetary and financial area, fiscal and budget management, Anti-Money Laundering (AML) and Combating the Financing of Terrorism (CFT). 89. As for the usage of funds, JSA funds can be used to cover the cost of short- and long-term TA experts and other costs associated with conducting seminars and workshops. The selection of TA experts is based on quality in international best practices depending on the requested subject of TA. Regarding geographic coverage, the JSA funds can be used in countries all over the world with no restrictions, but high priority is placed on TA activities in Asia and the Pacific region as well as Central Asia. In this regard, there is a general guideline within the IMF that about half of JSA funds should be used in these regions. 90. The Japanese authorities also place high priority on assistance for countries that have demonstrated strong effort and good track records in the implementation of economic reform policies Institutional Set-up at IMF 91. In providing JSA TA, along with other IMF TA, the functional departments, such as the Fiscal Affairs Department (FAD), Monetary and Capital Markets Department (MCM), and Statistics Department (STA) play a substantial role in project formulation and selection of experts, while area departments play a key role at the initial stage in needs identification. The Office of Technical Assistance Management (OTM) coordinates JSA policy and procedures in consultation with other IMF departments, such as various area departments and the Japan Executive Director s Office. OTM also functions as a window in communicating with Japanese authorities. 1) Preparation and Formulations Process of JSA projects 92. Projects to be funded from the JSA as well as other IMF TA are planned and prepared within an annual cycle. The next Chart 2.4 shows the preparation process within IMF. 7 This is indicated in the paragraph Technical Assistance Funded by the JSA on page 7 of Japan Administered Account for Selected IMF Activities (JSA) ANNUAL REPORT Fiscal Year

34 2. Outline of JSA and Japanese ODA Chart 2.4 Workflow of Preparation and Formulation of JSA Projects Area Departments (area specialists) -Prepares Regional Strategy Notes (RSN), which reflects policies of each mission chief Functional Departments (FAD, MCM, Statistics, etc) (subject specialists) -Review needs of recipients countries and set priorities considering RSN -Prepares Resource Allocation Plan (RAP) -Considers the fund source of each possible project (JSA is considered to be the most reliable and flexible fund source among external financing.) Assistance request from member countries Functional Departments (FAD, MCM, Statistics, etc) Regarding the requested projects; - Conduct Diagnostic Study: identify issues to be addressed and consider countermeasures to be taken by IMF. Donor coordination is done to avoid duplications. Resident Representatives and experts support the mission depending upon needs. - Prepare Project Proposal: Scope of works, target, implementation period, mode of assistance are proposed. Background and necessity, goals are explained. Committee on Capacity Building (CCB) reviews proposal Area Departments -Receives feedback and endorse Office of Technical Assistance Management (OTM): -Check the conformity with JSA guidelines -Prepares indicative list of JSA candidate projects Japan Executive Director s Office: -Receives indicative list of JSA Ministry of Finance, Japan: - Receives indicative list of JSA and gives approval (Source: MURC based on interviews at IMF Headquarters) 93. First, Area Departments assume their role to identify needs in areas and countries, considering the results of surveillance activities and TA needs. The Regional Strategy Note (RSN) compiled by each Area Department is a basic document which outlines regional issues to be addressed, assistance policy and plan in the region and respective countries, and future direction. Then, Functional Departments review needs of recipient countries and set priorities considering RSN, and prepare another key document called the Resource Allocation Plan (RAP), which presents the forecasts of the regional distribution of both 15

35 2. Outline of JSA and Japanese ODA internally- and externally financed TA, including JSA resources. The RAP is agreed upon after extensive discussions among Area Departments, Functional Departments, and OTM. Through these documentation and agreement processes of RSN and RAP, the needs and priorities for TA in each country are reviewed and discussed. 94. At the beginning of each fiscal year, IMF provides Japan s Ministry of Finance with an indicative list of projects which are planned to be submitted during the year for consideration. Thereafter, individual projects are submitted for approval on a monthly basis through the Office of Japan s Executive Director at the IMF. 2) Project Assessment and Monitoring 95. After the project is completed, project managers at IMF TA-providing departments prepare a project assessment report. In addition, project managers collect the recipient s evaluation of JSA projects using simple questionnaire sheets distributed to recipients in order to extract lessons for further improvement of IMF TA for the future. 96. Moreover, a joint Japan-IMF mission for JSA projects is organized, and two or three selected countries are visited in order to review projects every year. Japan s MOF receives first-hand information from these field visits as to whether JSA funds are effectively used in each country. Also, consultation meetings between IMF and MOF are held occasionally (semi-annually, as defined in the guideline 8 ) where they discuss agendas such as effectiveness of JSA, expected use of JSA resources, the plan of joint field visits and other relevant issues Record of Past Activities 1) Transition of JSA amount (commitments and disbursements) 97. FY1993 to FY2009: As Chart 2.5 Annual Commitment and Disbursement of JSA TA, FY1993-FY2009 indicates, between FY1993 and FY2009, a cumulative commitment for technical assistance under the JSA was $262.5 million for 1,852 projects. Out of this committed amount, $251.9 million has been disbursed. 98. FY2004 to FY2008: Between FY2004 to FY2008, as Chart 2.6 Annual Commitment and Disbursement of JSA TA, FY2004-FY2008 shows, a cumulative sum of $88.4 million was committed for 560 projects in total, and cumulative $90.9 million was disbursed. 8 Guidelines for the Use of Resources for Technical Assistance Activities from The Administered Account for Selected Fund Activities-Japan issued in July

36 2. Outline of JSA and Japanese ODA 2) Geographical distribution of JSA funds 99. FY1993 FY2009To date, JSA covered 124 member countries and 13 Regional TA Centers and Training Institutions. During FY1993 FY2008, countries in the Asia and Pacific region received $90.7 million, or 37% of TA projects approved during this period. Africa received the next largest share, totaling $59.5 million, or 24% of total approvals. The detailed geographical distribution including FY2009 is indicated in Chart 2.7. Chart 2.5 Annual Commitments and Disbursement of IMF/JSA TA, FY1993-FY2009 Number of Projects Committed Committed (US$ Million) Disbursed (US$ Million) FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY FY Total 1, (Source: JSA Annual Report 2009) Chart 2.6 Total Commitments and Disbursement of IMF/JSA TA, FY2004-FY2008 FY Number of Projects Total committed Total disbursed US$88.4 million US$90.9 million (Source: JSA Annual Report 2009) Chart 2.7 IMF/JSA Commitments for Technical Assistance by Region, FY1993 FY2009 (Million US dollars) FY FY 2001 FY 2002 FY 2003 FY 2004 FY 2005 FY 2006 FY 2007 FY 2008 FY1993- FY2008 FY2009 Total Share Total Share Africa % % Asia and Pacific % % Europe % 2 13% Eastern Europe and Central Asia % 0 0% Middle East and Central Asia % % Latin America and Caribbean % 0.7 5% Multiple regions % 0 0% Total % % (Source: JSA Annual Report 2009) 17

37 2. Outline of JSA and Japanese ODA 100. FY2004 to FY2008: The cumulative results of regional distribution between FY2004 to FY2008 are indicated in Chart 2.8. The total amount committed during the five years was $88.4 million, which was allocated for 560 projects in total, and $90.9 million was actually disbursed. Chart 2.8 IMF/JSA Commitments for Technical Assistance by Region (FY2004 FY2008) (Millions U.S. dollars) Region FY 2004 FY 2005 FY 2006 FY2007 FY2008 FY FY (%) Africa % Asia and Pacific % Europe % Eastern Europe and % Central Asia Middle East and Central % Asia Latin America and % Caribbean Multiple regions % Total % (Source: JSA Annual Report 2009) 101. In this period, Asia and Pacific received $36.1 million, or 40.6% of the total, which is higher than the long-term trend. Ranked second was Africa, which received a total $25.2 million, or 28.3% share. The share of geographic distribution from FY2004 to FY2008 is shown in Chart In the recent trend over a five year period ( ), the Asia and Pacific region continues to be the largest recipient of JSA. The combined share of Asia and Pacific and Africa was 68.9% from 2004 and Considering the fact that the combined share was 61% from 1993 to 2008, it is noted that these regions are given more priorities in recent years within the total portfolio of JSA. 18

38 2. Outline of JSA and Japanese ODA Chart 2.9 Percentage Share of Cumulative Commitment by Region FY2004-FY2008 Latin America and Caribbean 4.8% Middle East and Central Asia 10.1% Multiple regions 4.7% Africa 28.3% Eastern Europe and Central Asia 5.8% Europe 5.5% Asia and Pacific 40.6% (Source: MURC based on the data of JSA Annual Report 2009) 3) Distribution of JSA Funds by Subject Area 103. FY1993 FY2009: As Chart 2.10 shows, during FY1993 FY2008, JSA funds were spent on the Monetary and Capital Markets area, which received a total of $95.5 million, or 39%, the largest share among six subject areas. The second largest area was Fiscal, with the amount totaling $66.6 million, or a 27% share. Then Macroeconomic Statistics follows with $41.1 million (17%) and Training with $28.8 million (12%). For FY2009 only, the Fiscal area received $5.8 million, the largest share of 40%, and Monetary and Capital Markets came in second with $4.0 million and 27%. Chart 2.10 JSA Annual Commitments for Technical Assistance by Subject Area FY FY 2001 FY 2002 FY1993 FY2009 FY 2003 FY 2004 FY 2005 FY 2006 FY 2007 FY 2008 (Million U.S. dollars) FY1993- FY2008 FY2009 Total Share Total Share Fiscal % % Monetary and capital markets Macroeconomic statistics % % % % Training % % Legal % 0.2 1% Others % 0.1 0% Total % (JSA Annual Report 2009) % 19

39 2. Outline of JSA and Japanese ODA 104. FY2004 to FY2008: Chart 2.11 indicates the trend of distribution of committed amount by subject areas between FY2004 to FY2008. Chart 2.11 JSA Commitments for Technical Assistance by Subject (FY2004 FY2008) (Millions U.S. dollars) Subject FY 2004 FY 2005 FY 2006 FY2007 FY2008 FY FY (%) Fiscal % Monetary and Capital % Markets Macroeconomic % statistics Training % Legal % Others % Total % (Source: JSA Annual Report 2009) 105. When we compare the long-term trend of sixteen years (FY1993-FY2008) and the most recent structure, the notable difference is that Fiscal has now become the priority area rather than the Monetary and Capital Markets. The current share of Fiscal in 2009 is 40%, which is much higher than the 27% recorded by Monetary and Capital markets. With Fiscal and Monetary and Capital Markets combined together, these two subject areas have constantly stayed as the major recipient, which received 64.1% of the total amount committed during FY2004 to 2008, and 67% in FY2009. The detailed percentage share of subject area distribution from FY2004 to FY2008 is shown in Chart Chart 2.12 Percentage Share of Cumulative Commitment by Subject FY2004-FY2008 (Source: MURC based on the data of JSA Annual Report 2009) 20

40 2. Outline of JSA and Japanese ODA 106. As this graph shows, in the recent five years, Monetary and Capital Markets occupies the largest share of 38%, which is almost the same as the long-term trend of 39% between FY1993 and FY2008. Ranking second is Fiscal with 26%, again almost the same level as the long-term trend. After that, Macroeconomic statistics follows with 17%, and then Training and Legal with 11% and 6% respectively. 2.2 ODA Policy of Japan 107. The Japanese official development assistance (ODA) is structured as indicated in Chart The modes of assistance are classified into two major categories; (A) Direct cooperation with recipient countries, and (B) Cooperation through International Organizations such as IMF, the World Bank and United Nations. The support to JSA falls under category (B). In other words, direct cooperation with recipient countries is equivalent to bilateral assistance, and cooperation through international organizations is equivalent to multilateral assistance. Chart 2.13 Classifications of Japanese ODA (Source: Japan s International Cooperation, Ministry of Foreign Affairs) 108. The direct cooperation with recipient countries is further divided into (a) Bilateral Donation, and (b) Government Loan (which means Yen Loan available through JICA) 9. The Bilateral Donation consists of (i) technical cooperation and (ii) grant aid. When the Japanese Government sends experts overseas, this is included in this technical cooperation category The historical trend of Japanese ODA on a net disbursement basis from 2004 to 2008 is indicated in Chart Japanese ODA loan is only by Japanese Yen currency. 21