Vacancy
Background:
The OSCE Permanent Council Decision No. 588 from 2003 specifically mandates the OSCE Presence in Albania (hereinafter “the Presence”) to work in the area of parliamentary capacity-building. Based on previous support the Presence has provided to the Albanian Parliament, Switzerland has entrusted the Presence with an extra-budgetary contribution for a four-year long project “Support to Parliament and Civic Education in Albania”. The Presence has supported the strengthened role of the Albanian Parliament and its capacity building in performing the parliamentary core functions since 2003, using a multi-year and multi-angle approach. The Support to Parliament and Civic Education Project in Albania (“PACEP” or the “Project”) builds on the Presence’s 15-year long work with the Kuvendi, or Assembly of Albania (the “Parliament”). The Project aims to support more effective and efficient work of the Parliament and improved civic education in Albania, in full partnership with the Parliament and relevant stakeholders. It addresses core functions of the Parliament through human resource strengthening, innovative technology solutions and improving transparency and public outreach. The Project targets the parliamentary staff, members of Parliament (“MPs”), civil society and the public at large. The project is structured so as to deliver results that will contribute to the long-term Outcomes as follows:
Main Objective
The OSCE Presence in Albania seeks for an international Evaluation Consultant to develop an overall evaluation strategy for the four-year PACEP project and related work-plan for regular evaluations. The aim is to design a sustainable evaluation process that starts with a baseline to be used for regular, at least yearly, follow up evaluations to assess the project results and provide recommendations for further project implementation. The evaluation strategy and work-plan shall include relevant operational indicators for all activities planned under PACEP, including also the sub-activity level, plans for the content of datasets to be compiled as basis for results based management, as well as methodology for compiling the datasets. The evaluation reports conducted on a yearly basis, using the baseline and recommendations developed by the international consultant, will serve to assess the progress made and will guide the mid-term and long-term project planning in order to ensure successful achievements well as sustainability of project results.
Tasks and Responsibilities
The Evaluation Consultant will work under the supervision and overall guidance of the Programme Manager and Project Manager, and in close consultations with the Programme Coordination Unit of the Presence. S/he will be expected to develop two concrete products: A. A comprehensive evaluation process, including a baseline. The consultant will develop and elaborate upon the project evaluation process for the purposes of PACEP, including a baseline guided by the project document, its Theory of Change and Logical Framework. Essential part of this product will be to detail datasets feasible to be received and compiled as well as designing relevant milestones or indicators that will be effective in informing about the progress towards the project objective and set results. B. A work-plan with an evaluation matrix. The consultant will develop a work-plan with an evaluation matrix to be used for the yearly evaluation processes. The matrix, once completed, should be the basis for determining the project progress, and will be used as such for progress reporting needs of PACEP.
More specifically, the Consultant shall Within the Task A: A comprehensive evaluation process, including a baseline 1. Develop a Baseline for the yearly project evaluations: Using project documents, desk research, existing OSCE and its Presence in Albania’s tools and processes as well as interviews with project team and relevant stakeholders, the consultant will define a Baseline: set of areas, including indicators, which will need to be evaluated on a yearly basis for the Project Manager. Based on the developed Baseline, the Project Manager shall be able to judge overall achievement of project results as well as project sustainability. This should help with the development of mid-term plans, timely adjustments of project activities, as well as serve for the different needs of project reporting internally and externally. The Baseline will comprise a set of feasible and clear questions and needed data to arrive at the information needed. The availability of the data and accessibility of interviewees needs to be ensured. The addressees for the questions and methodology on collecting the needed data will be part of the Baseline, as well as relevant indicators that take into account also the indicators of the project document. 2. Develop Evaluation questions: A set of up to 10 main evaluation questions will be developed to be used for the yearly evaluations. These questions shall be divided into sub-questions. The evaluation questions shall be related to the OECD-DAC standard set of evaluation criteria: relevance, comparative advantage, efficiency, effectiveness, and sustainability etc., as broadly outlined below. The evaluation shall also assess to what extent gender equality has been mainstreamed in the project, to what extent project activities have been coordinated with other executive structures in the OSCE, and whether project progress has been continuously monitored. Relevance: Did the project activities address the needs of the beneficiaries? Sub-questions: How were those needs identified (through surveys, evaluations, assessments, interviews, etc.)? Comparative advantage: did the OSCE have a comparative advantage in one or several ways as an assistance provider in comparison to other assistance providers? Efficiency: Were activities carried out on time and within the available budget resources? Sub-questions: Were outputs (e.g., number of trainees) generated in the most cost efficient way (i.e., cost per unit of output), and in comparison to similar assistance activities carried out by other actors (e.g. European Union, Council of Europe, National Democratic Institute, Westminster Foundation etc.)? Effectiveness: Have project objectives been achieved? Sub-questions: To what extent have short-term outcomes been achieved (as a consequence of activities and outputs)? To what extent have mid-term outcomes been achieved (as a consequence of short-term outcomes)? Thus, what difference has the assistance made for the beneficiaries in the short-term and mid-term, and what was the OSCE’s key contribution as compared to other assistance providers? Finally, what were the major factors (i.e. obstacles, facilitators) influencing the achievement/non-achievement of project outcomes? Impact: Has the assistance generated long-term outcomes (impact)? What was the OSCE’s contribution when parsing out confounding factors (e.g., activities of other assistance providers)? Finally, what were the major factors (i.e., obstacles, facilitators) influencing the achievement/non-achievement of impact? Gender mainstreaming: Has a gender equality perspective been integrated in the project as defined in the 2004 OSCE Action Plan for the Promotion of Gender Equality? Monitoring: Did the project have a system for tracking outputs (output-based monitoring) and outcomes (outcome-based monitoring)? Vertical and horizontal co-ordination: Has the work been coordinated with other departments / units? Has it been coordinated with other OSCE executive structures, including the Secretariat? Sustainability: To what extent may the project benefits (i.e., the three types of outcomes) be expected to continue, or deteriorate, in the aftermath of OSCE assistance? What are the major factors influencing sustainability? Is some type of continued OSCE engagement warranted to ensure that the project benefits do not deteriorate? Additional evaluation questions, as appropriate. 3. Develop Methodology to be followed: The description of the evaluation approach should include an explanation of the ways that data will be collected and analysed. Some evaluations may be statistical and data-driven (as allowed by data availability), other evaluations may by necessity (lack of data) be more reliant on interview information and triangulation of a series of data sources. The underlying population (all the relevant instances) should also be explained, as well as the sample (some of the relevant instances) to be looked at by the evaluation shall be clarified and justified. Furthermore, the identified unit of analysis that will provide credible answers to the evaluation questions should also be explained. “Unit of analysis” is about the time-intervals at which data on indicators is collected and tracked. It may be monthly (i.e., the unit of analysis is “project month”), quarterly, every six months, annually (i.e., the unit of analysis is “project year”) etc. The smaller the unit of analysis (i.e., the more often that indicator is measured), the more data will be available for observation, and thus the more solid the conclusions. The unit of analysis is often determined by data availability, and is commonly a trade-off between what would be an ideal approach and what is a feasible approach. Data sources such as OSCE or third-party entities (institutes, organisations, online archives, academic research, scientific journals, household surveys, etc.), as well as data collection methods (i.e., surveys, interviews, observation, documents including OSCE reports, third-party studies, etc.), shall also be presented here. Indicators can be simple (e.g., “yes-no”, “0-1”), indexes or proxy/indirect indicators. In this regard, identify and recommend valid indicators by consulting the project theory of change, project Logical Framework, as well as scientific studies and evaluations; and OSCE documents or handbooks. Thereafter, and as appropriate, identify and recommend reliable data sources for the indicators by consulting related scientific studies, data collections and evaluations; project/program documents and reports; and third-party data obtainable from IGOs, NGOs, institutes and academia.
Within Task B: A work-plan with an evaluation matrix 1. Create a work-plan for regular evaluations: For the purpose of creating a detailed evaluation work plan, the Consultant will also define who of the project team and from where will collect which data during the year. These materials will then feed into the annual evaluation. 2. Create a Matrix for operationalising the evaluation work-plan: A separate evaluation matrix will be created in order to operationalise the evaluation work-plan. This will take into account existing recommendations and tools within the OSCE.
Additional Expectations and Activities
The international evaluation expert is expected to: • Conduct desk research and collect and work with best practice examples of evaluations of parliamentary support projects. • Conduct desk research using all material provided by the project manager, related to the project documents and activities in the context of PACEP to date. • Provide a draft Evaluation Strategy for the PACEP project, based on comprehensive consultations in country with all segments of stakeholders of the PACEP project (list of stakeholders to be coordinated with the Project Manager). • Be available for online and long distance consultations and questions with the PACEP project team, the main beneficiary, the Albanian Parliament as well as other key stakeholders in the process. • Prepare and manage in cooperation with the PACEP project office the 5-day in-person consultation process in Tirana with the Albanian Parliament and key stakeholders of the draft version of the overall Evaluation Strategy. • The expert will incorporate into the final product, the final version of the Evaluation Strategy, all feedback received and all findings from the consultation process. The expert will provide the final version of the overall Strategic Plan to the PACEP project team. • Provide a Baseline to be an integral part of the Evaluation Strategy for the PACEP project. • Provide a detailed work-plan for the PACEP project regular evaluations, including clear guidance on the material to be collected by the project team during the year, and who collects what and from where. These materials collected during the year, will feed in to the annual evaluation to be conducted by an international expert. The work-plan will also include details on the methodology to be applied for annual evaluations with feedback from stakeholders in in-person consultations. • For the final products, the expert will take into consideration all the comments from the Presence, and feedback from key stakeholders, which will be provided to the expert during the consultation visit. • Be in close cooperation on a daily basis with the project manager and the Presence staff in charge of the parliamentary support programme during the time of work in Tirana and conduct regular skype/phone conferences with the project team during home based work. Constant contact for information sharing and questions will be conducted via e-mail.
Expected Output/Deliverable • Baseline for annual or mid-term evaluations of PACEP (maximum of 10 pages). Due within two weeks from the start of the contract • Draft Evaluation Strategy for the PACEP project, including outline of the process and necessary tools [such as questionnaires, survey templates] (maximum of 15 pages). Due within four weeks from the start of the contract • Detailed work-plan for PACEP evaluations (maximum 10 pages). Due within six weeks from the start of the contract.
Necessary Qualifications:
• Have an advanced degree in political science, social science, law, project management, administration.
• Documented project management experience of at least 5 years, particularly in monitoring and evaluation of comprehensive and complex multi-year and multi-angle support projects;
• Professional experience with parliamentary support projects is an asset;
• Previous experience in working in an international environmentis desirable;
• Computer proficiency;
• Fluency in English.
Remuneration
The payment for this consultancy will be in accordance with the OSCE established rates and upon submission and approval of the deliverables mentioned above.
How to apply
All applications are due by 20 November 2019. If you wish to apply for this position, please use the OSCE's online application link found under link https://jobs.osce.org/. The OSCE retains the discretion to re-advertise the vacancy, to cancel the recruitment, to offer a contract with modified Terms of Reference or for a different duration. The OSCE is committed to diversity and inclusion within its workforce, and encourages qualified female and male candidates from all national, religious, ethnic and social backgrounds to apply to become a part of the organization. Please be aware that the OSCE does not request payment at any stage of the application and review process