Monitoring and evaluation

OBJECTIVES AND OUTCOMES

The monitoring and evaluation (M&E) component of the project Managing Digital Transformation in the Health Sector (ManagiDiTH) , aims at achieving the following:

The M&E component is led by Mundiserviços – Companhia Portuguesa de Serviços e Gestão, Lda.

The following documents support the M&E component

The M&E is organized into four blocks which have the following specific evaluation objectives:

1. Monitoring & Evaluation Design

To design and M&E plan and its activities;
To validate the M&E plan with protect members;
To support the capacity building of the actors involved in the ManagiDiTH project implementation and monitor and evaluate the project´s results during its implementation, in particular the ones obtained with the implementation of the Master Program by the three partner universities.

2. On-going evaluation

To know the state of development of the project, according to its implementation plan;
To evaluate the project implementation verifying the suitability of strategies implemented by the coordination and the consortium dynamics, compliance with timetable and execution, achievement of the targets foreseen;
To produce recommendations to support the project management and its implementation.

3. Results’ evaluation

To evaluate the fulfillment of the goals foreseen in the Master’s course according to the initial planning;
To assess the satisfaction of the different stakeholders involved in the project;
To contribute to the improvement of the project, through the lessons learnt on teaching activities, on methodologies and on the results that are known throughout the project implementation.

4. Impact evaluation

To analyze the impact of the training programme in terms of the students’ pathways after completing the course;
Analyze the direct / indirect effects in terms of the digital transformation within the health sector ecosystem.

EVALUATION METHODOLOGY

Our Approach

The approach that underlies this intervention has for fundamental characteristics:

✓ It should be an instrument to support change;

✓ It has a specific purpose;

✓ It should achieve tangible and measurable results;

✓ It should use a variety of evaluation techniques and methods.

In this approach we are assuming that the activities planned at the beginning of the project, with the respective actions, deadlines and responsibilities are the starting point (baseline) of the evaluation methodology.

The M&E team will make detailed recommendations on:

the work plan for the remaining period of the project, considering the on-going evaluation process, which will provide an opportunity to assess the project success or failure and prompt necessary adjustments;
evaluation of the progress in project implementation, measured against planned outputs set forth in the Project Grant Agreemetn in accordance with rational budget allocation and the assessment of features related to the process involved in achieving those outputs, as well as the initial and potential impacts of the project;
underline main reasons and issues related to targets not adequately met;
evidence based information that is credible, reliable and useful, through a participatory and consultative approach ensuring close engagement with the key stakeholders.

The evaluation will also identify lessons learnt and best practices from the project which could be applied to future and other on-going projects by the consortium partners and the executive agency (HADEA).

MAIN EVALUATION DIMENSIONS

Globally the process allows, besides the analysis of the results achieved, to analyses the Master program in the dimensions of relevance, effectiveness, efficiency and usefulness/sustainability.

The key concepts are as follows:

1. Relevance

Refers to the appropriateness of the explicit objectives to the change needs initially defined

2. Effectiveness

Aims to determine whether the formulated objectives have been achieved, what successes and difficulties have been observed and the adequacy of the formulation of the expected results and related activities.

3. Efficiency

This is determined by taking into account the products and results achieved in relation to the resources mobilized. It is equivalent to the terms “saving resources” or “minimizing costs”.

4. Utility

Looks at the impacts obtained by development interventions in relation to socio-economic needs at the level of analysis of national strategies and policies in terms of economic and social development.

5. Sustainability

Determined by taking into account the durability of results and impacts in promoting the desired changes.