8 theses for a lot element in research funding

Since 2017, the Volkswagen Foundation has been testing a novel procedure for selecting projects for funding in the initiative 'Experiment!': Following initial quality assurance by a jury, it selects a share of the funded projects by lot. What effect does this random element have on the selection of projects?

The following theses are based on practical experience and initial results from the accompanying research.

In the case of highly competitive procedures, the lottery relieves the burden on reviewers faced with the problem of differentiating quality among a large number of equally high-ranking proposals.

Decisions by lot are free of any bias and of any influences caused by group dynamics.

The lot is blind to quality. To work properly, therefore, the process requires an initial quality assurance.

In the event where a reviewer panel does not cover all topics equally well, such a procedure ensures fairness among eligible applications.

Regarding the outcome, diversity is enhanced by drawing lots. Often, procedures based on consensus tend to favor established topics and conventional methods ("mainstreaming").

Accompanying research shows that the partially randomized selection, i.e. including a lottery, encourages the submission of risk-taking research proposals.

The introduction of a randomized element is broadly welcomed by the scientific community, including reviewers, and by an increasing number of funding organizations.

Selection by lot has to be regarded as a useful supplement – but not an alternative – to peer review and cannot replace scientific discourse.

What is your opinion on lottery procedures in research funding? Join the discussion on Twitter: #PeerReviewLottery.

Download: 8 theses for a lottery element

You can download a printable version of the 8 theses here (PDF).

Background: the funding initiative 'Experiment!'

With its funding initiative "Experiment!", the Volkswagen Foundation supports scientists who want to test a radically new and risky research idea. Failure of the concept and unexpected findings are acceptable results.

Facts and figures

The initiative was started in November 2012, funded projects are selected via a partially randomized procedure since 2017.

The initiative addresses researchers in science and engineering as well as in the life sciences. The funding is limited to 18 months and up to 120,000 Euro. The projects selected for funding cover a wide range of subject areas, with participants from research institutions across Germany.

The accompanying research project

The lottery as a new selection element and the entire selection process are being evaluated. In our interview "High level of acceptance for project selection by lot", Dr. Dagmar Simon and Dr. Martina Röbbecke (Evaconsult) report on initial results from their accompanying research project.

Partially randomized selection: infografic

Infographic on the selection of funded projects for the 'Experiment!' initiative.

Download: Publication on the accompanying research project

Free Download (English abstract): "Die Macht des Zufalls - Neue Wege für die Förderung riskanter Forschungsideen?" (PDF) by Martina Röbbecke and Dagmar Simon, from: Fo 1+2, 2020, Universitätsverlag Webler.

Text "Falling Walls Science Breakthroughs of the year"

Teil-randomisiertes Verfahren unter den "Breakthroughs of the year" bei "Falling Walls"

Das teil-randomisierte Verfahren der VolkswagenStiftung war unter den 10 Finalisten des Wettbewerbs "Breakthroughs of the year 2020" der Falling Walls Foundation in der Kategorie "Science and Information Management". Dr. Henrike Hartmann, die Leiterin der Förderabteilung der VolkswagenStiftung, erläuterte das Verfahren im Video.