Coding the future: delegates of the third Evidence Synthesis Hackathon, Canberra, Australia.

By Neal R Haddaway 1,2; Martin J Westgate 3; Carina van Rooyen 2; Witness Mapanga 4; Mandlenkosi Ngwenya 5; Christopher Penkin 6

1 Stockholm Environment Institute, Linnégatan 87D, Stockholm, Sweden; 2 Africa Centre for Evidence, University of Johannesburg, Johannesburg, South Africa; 3 Fenner School of Environment & Society, Australian National University, Canberra, Australia; 4 Centre for Health Policy, University of the Witwatersrand, Johannesburg, South Africa; 5 Department of Planning, Monitoring and Evaluation, Pretoria, South Africa; 6 Digital Solution Foundry, Johannesburg, South Africa. 



Although the term the ‘Fourth Industrial Revolution’ has become a bit of a buzzword in South Africa, two recent commissions/task teams – one appointed by the President of the country and another by the Minister of Higher Education – hint at the potential of artificial intelligence and biotechnologies to disrupt the way we do research. Of these two technologies, it is especially machine learning that has the potential to reduce the time and effort of laborious repetitive activities in any evidence synthesis process.

What is evidence synthesis?

Evidence synthesis is the process of converting large bodies of scientific research (such as articles, reports, and data) into reliable and digestible evidence that can inform management or policy. It is vital to have important decisions about public policy informed by rigorous scientific evidence to ensure public funding is used efficiently, and only on projects that we know stand a chance of working. Reliable evidence enhances transparency, public trust, and accountability in decision-making processes.

But there’s a catch: evidence synthesis projects can be extremely time-consuming and costly. This is because reviewers must read through vast numbers of research papers from a range of different sources.

Some researchers believe that evidence syntheses can be made more resource-efficient, timely, transparent, rigorous, and accessible by developing, and using, freely accessible software tools and workflows. This new wave of technological advancements could change the face of evidence synthesis – letting machines do much of the laborious work so that humans can focus on planning, thinking, and implementing.

Making evidence synthesis human-friendly with technology

To support how technological advancements can improve evidence synthesis, Neal Haddaway and Martin Westgate created the Evidence Synthesis Hackathon (ESH) series of events in 2017 to bring together world-leading researchers, practitioners, and software developers to produce new Open Source tools and frameworks to support evidence synthesis.

Three ESH events have been run so far: two in 2018 in Stockholm, and one in 2019 in Canberra. These highly interactive events have involved 64 participants from 13 countries on six continents. Some 19 projects have been initiated, including tools that help researchers visualise databases of studies and those for extracting information from documents. Participants of these events have also produced discussion papers that introduce new ways to think about evidence synthesis.

Many of these outputs are already being used by researchers, such as EviAtlas: a tool for producing maps of evidence unearthed during systematic literature reviews. This tool converts a database into a set of attractive, interactive figures and tables that show patterns in the ‘evidence base’ of where knowledge gaps and clusters might exist.

EviAtlas also allows users to produce free, interactive websites displaying the nature of the evidence on a geographical map – something that would previously have been expensive to do and highly complex to achieve.

Funding the future

So far, ESH has received generous funding of around 100,000 Euro over three years from six different institutions. This funding has enabled effective meetings to be organised, supported participants from around the world to attend those events, and covered publication costs for research papers coming out of the event. Our funders include universities, research councils, and software organisations from around the world.

In 2019, we were very fortunate to receive R250,000 from the Faculty of Humanities at the University of Johannesburg, facilitated through the Africa Centre for Evidence. With this funding, three African participants were welcomed for the first time to the Hackathon in Canberra, Australia in April 2019. Sadly, there were an additional two fully-funded participants who were denied visas by the Australian government at the last moment, preventing their involvement in the hackathon (this seems to be an increasingly common problem for Africans). The three participants from Africa that were able to attend were integral to the discussions and coding work at our Hackathon, and were instrumental in its success.

Participating in the Evidence Synthesis Hackathon

Two of the participants were experienced software programmers: Christopher Penkin and Mandlenkosi Ngwenya. They produced a tool to keep track of and save web-based searches for research: something that has until now been almost impossible to do in evidence syntheses. The tool is a Chrome browser extension that logs user search information and downloads it into a central database. This tool represents a huge step forward in ensuring the transparency, efficiency, and repeatability of producing evidence syntheses, all central tenets for rigorous evidence synthesis work.

The third attendee from Africa – Witness Mapanga – is an evidence synthesis specialist. Witness worked with other world-leading researchers on projects to build a brighter future for evidence-use in policy. We plan to publish this work by the end of the year.

Future events

Plans are underway for future ESH events in South Africa, Norway, Canada, and the Netherlands in 2020 and beyond. Through holding the ESH in Africa, we hope to foreground the particular needs of and solutions for evidence synthesis in resource-constrained environments, and thereby ensure that such technologies enhance equity in evidence synthesis production. Funding for these and future events will be vital to ensure equitable representation across disciplines and backgrounds. The continued interest in these events demonstrates that they are critical to building communities of practice: networks of researchers, data scientists, and software developers focussed on driving progress towards a sustainable future as part of the Fourth Industrial Revolution. Concurrently, technology is moving forwards rapidly and in ways that may make a perfect marriage with key aspects of evidence synthesis methods, for example with machine learning and transparent web scraping.

As we continue to see more and more research published across the globe (currently around 3 million articles per year), we will need new technology that can help us synthesise and make sense of this evidence. Otherwise, we risk missing opportunities to make effective decisions and design efficient public policies. The ESH represents a novel but increasingly important part of this pathway.