Assisting behavioral science and evidence-based policy making using online machine tools (POLTOOLS)

Summary

Objective

The objective of the project is to assist behavioral science and evidence-based policy making by creating and validating methodologies and tools that will facilitate three interconnected key workflows in the science-to-science and science-to-policy interactions and outcomes:

1. rapid knowledge creation
2. rapid knowledge curation, integration and aggregation
3. rapid policy making

Contribution

Our contribution to the “preparedness for future crisis at global scale”

• Consolidate and curate information coming from heterogeneous sources (i.e., also considering grey literature, preprints, discussions and peer-review on social media etc.)
• Accelerate and support informed factual-based decision making for relevant stakeholders
• Support and enhance the interaction between different stakeholders (e.g., “policy sprints”/“rapid open think tanks”)

Methodology

We will use insights and technologies from research on AI/NLP, cognitive/behavioral sciences, and collective intelligence to produce the following, interconnected sets of methodologies and tools (i.e., the project’s final deliverables):

• methodology for engineering and enriching a relevant knowledge base (i.e., re-using existing and creating new ontologies and knowledge graphs, word/paragraph embedding spaces, enrichment algorithms etc. seeded by the human curated SciBeh raw knowledge base)
• intelligent search, interaction, and collaboration interfaces (i.e., scientific and policy experts interacting with the knowledge base and each other)
• methodology for semi-automatic summarization of the outputs of collective problem solving “sprints” (i.e., “policy sprints”/“rapid open think tanks”)

Extended project context

Social and behavioral measures remain central to the world’s response to COVID-19. Though the emphasis on different aspects of the behavioural sciences will change as the pandemic unfolds, one unifying theme connects all threads: Providing a suitable and useful evidence base for high-stakes policy decisions under time pressure requires rapidly drawing together research across sub-fields and disciplines that are presently, at best, loosely interconnected; formulating and conducting new research; distilling findings into formats digestible by policymakers, and providing expert guidance to decision-making bodies.

New ways of engaging the whole scientific community are needed to support these urgent needs while avoiding the adverse consequences of group-think in small groups of scientific advisors. And all this has to happen at speed, without sacrificing scientific quality and integrity, and without needlessly reinventing wheels. Furthermore, the facts that scientific evaluation is increasingly found on social media (esp. Twitter), that scientists take on ever more public roles, that there is increased demand for transparency of science-to-policy interactions, and that even alternative science advisory bodies have emerged emphasizes the important question of how the behavioural sciences should adapt to best support evidence-based policy in the rapidly changing, high stakes environment of a crisis.

To help meet the challenges stemming from COVID-19 and other, future global crises, there is thus an urgent need to develop, deploy and empirically evaluate online machine tools (e.g., intelligent search and collaboration interfaces) that improve the scientific process and the interface between behavioral science and evidence-based policy making.

This project (POLTOOLS) builds on already operating systems implemented by the SciBeh initiative and methodologies developed by havos.org. The project will pursue the following three objectives by implementing suitable tools to achieve them in three respective real-world use cases:

1. Facilitating rapid knowledge creation by connecting and enriching existing, ad hoc infrastructure to support efficient and rapid development, analysis, evaluation, and dissemination of emerging and extant research (e.g., consolidating a preprint’s timely, but scattered discussion on Twitter into a short, digestible format to assist the evaluation of preprints).

2. Facilitating rapid knowledge curation, integration and aggregation using natural language processing (NLP), information retrieval technologies and minimal, scalable human curation (e.g., visualizing emerging topics in the SciBeh knowledge base using NLP tools, such as topic models).

3. Facilitating rapid policy making (e.g., by organizing machine-assisted rapid open think tanks by implementing a hybrid human/machine approach to create and synthesize key arguments and insights from scientific and policy discussions).