External Evaluations

The Volkswagen Foundation perceives itself as a learning organization in which quality assurance plays a central role. This is particularly the case for the Foundation’s core activity: Research funding. Measures of quality assurance are implemented at all stages of the funding process – starting with the development of new funding offers, through application and review procedures, monitoring activities up to the conclusion of single projects or funding initiatives.

In 2005 the Foundation expanded its concept of quality assurance by the instrument of "external Evaluation". In external evaluations independent experts carry out comprehensive assessments of the Foundations objectives, funding concepts, instruments, and procedures of the Foundation. External evaluations serve to provide assessments of the efficiency and goal attainment of the management and funding activities as well as to identify future perspectives.

The chart illustrates the Foundations different evaluation levels above individual funding projects. They encompass funding initiatives, funding and management areas as well as the institution as a whole.

The evaluation approach developed by the Foundation combines accepted standards of quality assurance with a certain degree of flexibility in respect of the area under evaluation. One of the most important underlying principles of all evaluation activities is that they do not take place as routine but only as required.

Following standards apply to all evaluation procedures:

Terms of Reference: Goals and framework of evaluation are formulated in concrete questions.

Self-evaluation report: The current status, problem issues and the Foundation’s future expectations of the area under evaluation are summarized in a report.

Evaluation panel: For each specific evaluation, an evaluation panel comprising up to seven excellent external experts is recruited. They should be open to new ideas, methods, and concepts that cross the borders of single disciplines.

Timing of evaluation: As a rule, and in order to ensure that recommendations can be implemented, evaluations take place while the respective measures are running (mid-term). Furthermore, in limited cases, the Foundation carries out ex-post evaluation. An overall evaluation of the institution as a whole takes place every eight to ten years.

Methods: All evaluation procedures include different assessment methods such as participation in symposia, questionnaires, and in-depth interviews.

Assessment: The basis of every evaluation is the qualitative assessment of the panel (peer review); in some cases, quantitative indicators  will be used complementary (informed peer review).

Presentation of results: The chairperson of every panel presents the results and recommendations of the assessment at the meeting of the Board of Trustees.

Implementation: At the subsequent meeting of the Board of Trustees, the area under evaluation presents its views on and implementation proposals of the evaluation results and recommendations.

The Foundation has set the following organizational framework:

Duration: Evaluation procedures, including their intensive preparation, are spread over a period of between nine and twelve months.

Costs: Members of the evaluation panel are reimbursed in the form of attendance fees and travel expenses.

Foundation staff involved: The preparation and organizational support of evaluation activities falls to the Foundation’s Evaluation Officer. The self-evaluation report is compiled by the area under evaluation. The Secretary General is substantially involved in the conceptual development of every evaluation.

Transparency: Evaluation measures serve to foster internal learning processes. The evaluation reports are not made public.

Evaluations undertaken by the Volkswagen Foundation:

By mid-2017, the Foundation had completed evaluations of ten funding initiatives as well as five funding and management areas. The first overall evaluation of the whole institution was carried out in 2011/2012; the second overall evaluation will take place in 2019/2020.

An evaluation of funding policy will be started in September 2017. Evaluation activities on asset management and the area of finance & administration are envisaged in 2018/2019.

Publikations

Saß, U. (2017): Externe Evaluationen brauchen internes Vertrauen, in: duz – Deutsche Universitätszeitung, 06/2017, S. 61-63.

Reinhardt, A., Saß, U., Heidler, R. (2016): Evaluation with sound judgment: Conference on "Evaluation of research funding: Review, insight, outlook”, in: fteval-Journal for Research and Technology Policy Evaluation, Issue 42, November 2016, S. 44-47.

Saß, Uta (2014): Wir wollen es wissen: Warum Stiftungen fragen (sollten), wie sie wirken, in: Stiftung  & Sponsoring 04/2014, S. 18-20.

Saß, Uta (2010): Evaluationen – Instrumente der Qualitätssicherung der VolkswagenStiftung, in: Zeitschrift für Evaluation (ZfEv) 1/2010, S. 143-152.