A five-step method

The ImpresS ex post method is structured in 5 main steps : preparation, dialogue, construction, measurement and validation.

1. Preparation phase

This first phase of the study is conducted within the evaluation team, with the researchers involved in the production of the outputs of the innovation.

  • Define the perimeter of the case study: timeline, stakeholders, territory;
  • Draft the stakeholder map;
  • Define the cluster of projects that contributed to the innovation;
  • Draft hypothesis for potential impacts: expected impacts or impacts already observed, using scientific literature, project documents or researchers’ knowledge;
  • Draft innovation story: first innovation story based on researchers’ knowledge of the field and documentation;
  • Draft hypothesis for the impact pathway: first impact pathway based on researchers’ knowledge of the field and documentation.

2. Dialogue phase

In order to refine it, the evaluation team submits its hypothesis to the various stakeholders who participated in the innovation process through a participatory workshop or individual interviews.

  • Refine the impact pathway hypothesis;
  • Refine the innovation story;
  • Establish a participatory list of impact descriptors: stakeholders involved in the innovation process express what changes they or others have experienced (negative or positive) and how evidence for those changes can be gathered. The evaluation team will later translate them into impact indicators, in order to guide data collection.

3. Construction phase

  • Systematic documentation and characterization of the impact pathway elements;
  • Identification of learning situations;
  • Identification of key moments of interaction with public actors.

4. Measurement phase

  • Complete the characterisation of impacts, including intensity (degree of change) and amplitude (distribution of change);
  • Gather data on the impact pathway : conducting surveys, interviews or focus groups to account for the “boxes” of the impact pathways and the contribution of research to the impacts (arrows);
  • Develop the impact radar based on experts’ knowledge (focus groups, interviews); 
  • Consolidate the innovation story: gathering of evidence through documentation or interviews so as to strengthen the innovation story by triangulating sources of information.

5. Validation phase

The evaluation team presents its findings to the innovation’s stakeholders through a participatory workshop, in which results are ameliorated and validated:

  • Complete innovation story, impact pathway and impact assessment;
  • Participatory validation of impacts and of the impact radar;
  • Formulate stakeholder recommendations on the evaluation method and on the innovation process.