The tool that captured the global response to an unprecedented crisis

We might want to firmly forget the COVID-19 pandemic, but it is almost inevitable that the future will see new disease outbreaks. How successfully we meet these will depend on whether we can learn from the impacts the coronavirus had, at the health, political, economic, and social level. Countries around the world adopted markedly different approaches to tackling the pandemic – but which were the most effective? And how did this depend on each country’s unique combination of demographics, political institutions, social structures, and health systems?

Answering these questions would be impossible without an accurate record of what happened and when. Thanks to researchers from Oxford University’s Blavatnik School of Government, the world has a powerful tool to assess COVID-19 related polices: the COVID-19 Government Response Tracker. This global dataset, launched just days after the World Health Organization (WHO) began describing COVID-19 as a pandemic, captures in real-time all the COVID-19 regulations introduced throughout the pandemic.

The measures are coded according to their theme – for instance, workplace closures or controls on international travel – but they can also be combined into an aggregate score that gives an overall measure of the strictness of each nation’s response to the pandemic. The resource also includes an archive with the original source materials for each policy.

‘Whilst the Tracker itself does not aim to measure the appropriateness or effectiveness of a country’s reaction, it offers a way to compare responses and learn from one another’ said project PI Thomas Hale, Professor of Public Policy at the Blavatnik School of Government. ‘This granular approach, which allows us to look at measures in aggregate, gives a sense of how “open” or “closed” nations became, which is more helpful than generic labels like “lockdown”, whose meaning can vary across different contexts.’

Covering 187 countries and over 200 subnational jurisdictions, the database is astonishingly comprehensive, with just under 9 million data points. But the story behind it is just as remarkable. As project co-lead Anna Petherick, Associate Professor in Public Policy at the Blavatnik School of Government, said, ‘What started as just a small team of us entering data into spreadsheets rapidly snowballed into a global effort that quickly became a “go-to” source of information for world leaders, policy makers, and the media.’ 

Incredibly, the dataset was compiled almost entirely by volunteers; a diverse, international community of over 1500 people, from undergraduate students to software developers. During the height of the pandemic, their combined efforts influenced high-level policy papers, featured in global media coverage, and ultimately informed the decisions that governments, international organisations, and public health agencies took on their COVID-19 strategies.

But as Anna explained, it all began with a Blavatnik School class back in March 2020.

An introduction to the Oxford COVID-19 Government Response Tracker 

The germ of an idea

‘Those weeks leading up to the first lockdown were filled with so much uncertainty, and all of us – staff and students – were finding it difficult to focus on anything. It was clear that COVID-19 was developing into a situation of unprecedented scale, but just how big the pandemic would become and how long it would last were completely unknown’, Anna said. Whilst teaching a class as part of the Blavatnik School’s Master of Public Policy course, the team realized that the cohort of students in the room represented around 50 different countries. ‘Very pragmatically, we already had a diverse community capable of reading a large proportion of the official updates released by governments across the world.’ This gave Thomas the idea for a global COVID-19 policy tracker, and he asked for my help to get it off the ground.’ The proposal was that the tracker would record policy changes continuously to provide the most up-to-date comparative tool possible.

The impact of the COVID-19 Government Response Tracker has been nothing short of extraordinary. It has been used by governments all around the world, the UN, the WHO, and hundreds of news articles in dozens of different languages. That really is quite an extraordinary impact in such a short period of time, and none of that would have been possible without the extraordinary contributions of the volunteers.

Professor Irene Tracey, Vice-Chancellor of the University of Oxford

Thomas and Anna launched the Tracker on 22 March 2020, the day before the UK first went into lockdown. Around 30 students from the Blavatnik School and other parts of the University immediately volunteered to track policy updates, enter and code the data, and act as reviewers. Their contribution was invaluable since these tasks were not possible to automate, due to the data collection relying on a wide variety of publicly-available information sources, from government statements and news reports to social media accounts. The team initially recorded information on 11 different areas, including school closures, public events cancellations, and public information campaigns, besides monetary measures and emergency investment in healthcare. (This ultimately expanded to 23 indicators, including vaccine policies).

‘At the time, there was a lot of anxiety among our students about what would happen to their exams, so the Tracker gave them something to focus their minds on, and a sense of agency that they were contributing something towards softening and eventually overcoming the pandemic’, said Anna.

A simulation of data from the Oxford COVID-19 Government Response Tracker, produced by Our World in Data. This is an example of the variation in governments’ responses to COVID-19 across the world on Friday 30 April 2021. Credit: Our World in Data.

Scaling up

Despite the students’ enthusiasm, the sheer volume of data flowing in meant the project quickly required more people power. ‘The first few months were exhausting, and I was working late into the evening most days and weekends too’, said Anna. ‘We felt an enormous sense of responsibility, and a sense that we had to get this right to meet people’s needs.’

One key challenge was designing a framework that could capture and compare all the different kinds of policies governments were adopting. While many governments did similar things, nuance and context were very important, creating difficult choices and trade-offs for the team. ‘Since day one we have been building the airplane as we fly it,’ Thomas said.

Another major hurdle was developing a database that could handle complex data and adapt over time, whilst remaining easy for volunteers from around the world to use. Fortunately, Thomas’s partner Dr Sam Webster, a financial risk manager with experience in handling large data sets, was able to build the basic system in one marathon weekend.

But the biggest need was getting a critical mass of volunteers from around the world. Reaching out to Oxford University’s alumni community, they received an ‘overwhelming response’, with many signing up to volunteer and help raise awareness of the project.

‘Suddenly word was getting out here, there, and everywhere’, said Anna. ‘As one example, a medical student at The Federal University of Pará in Belém, Brazil, heard about it somehow and recruited, at that stage, a large proportion of our entire Brazil sub-national team.’ Even major corporations lent their support. On hearing that the Tracker was built using their software, Microsoft encouraged their staff to volunteer, which led to a sudden surge in sign-ups.

Andrew Read, a digital strategist for Microsoft based in Switzerland at the time, was one of those who responded.

He said:

‘I saw it as an opportunity to do something practical to help even though restricted to working from home. I also found it fascinating to learn how other countries were reacting to COVID-19. I researched and reported on various countries across Europe and Africa, and it was an interesting challenge to track down suitable sources of information, and one I definitely got better at over time.’

Though launched with a budget of zero, the project came to attract financial backing, including newly-launched grants for COVID-19 response projects. ‘This gave us the resource to scale up massively and invest in more support for our volunteers’, said Anna. ‘We hired several research assistants who helped us to develop training modules, run the weekly online check-in meetings, and monitor the vast number of emails and questions.’

These actions enable the volunteer base to swell even further, stretching across the globe. Many saw it as a way to help fight back against COVID-19 at a time when so many of their usual activities were disrupted.  

 Some of the 1500 volunteers who powered the Blavatnik School’s COVID-19 Response Tracker talk about their experiences creating the most-cited dataset of pandemic policy decisions in the world. 

Volunteer Precious Olajide, an undergraduate student studying psychology at Obafemi Awolowo University in Nigeria, recalled:

‘I volunteered for the COVID-19 Government Response Tracker project because I saw it as an opportunity to be involved with something bigger than myself, and to contribute to a project that had a global influence. The community of volunteers was like another family for me; we were interacting all the time with emails and WhatsApp. It was my first involvement in a research project and because the organising team were so supportive, I enjoyed it a lot.’

A global influence

Despite the enormous scale of the dataset, from the beginning it was designed to be accessible and easy for anyone to understand. The entire dataset was made available as an open-access GitHub repository, so that, as Anna said: ‘anyone could download and use the data, whether they were a journalists, policy makers, or even school pupils and teachers.’ Because of this, the dataset soon gained international attention from governments, researchers, journalists, and NGOs around the world.

‘One day in about late March 2020 I made some very basic bar charts comparing the policies of countries according to their income level’, Anna recalled. ‘Suddenly I learnt that they had been sent to the UN Secretary General. Something I had very quickly created on my laptop at home was now being used by policy makers at the highest level.’

Over the next two years, the dataset became a critical tool for decision-makers and provided real-time data for dashboards maintained by the UN, WHO, World Bank and many individual governments and businesses. Our World in Data, for instance, used the Tracker’s data to publish visualizations across several different policy areas.

‘By tracking and comparing policy responses, the Tracker made it possible to study which measures were effective and which ones were not’, said Max Roser, Founder and Director of Our World in Data. ‘This information was crucial for decision-makers who needed to make informed choices about how to respond to the pandemic. No one else was systematically collecting this type of data, so the hundreds of volunteers and the staff who made this possible were doing truly important work.’

The core team were soon regularly engaged in advising governments and international organisations, including the UN Office for the Coordination of Humanitarian Affairs, the World Health Organization, and the UK Cabinet Office, Department of Health and Social Care, and Prime Minister’s office.

A simulation of data from the Oxford COVID-19 Government Response Tracker, produced by Our World in Data. This shows how a ‘stringency score’ of COVID-19 measures varied over the course of the pandemic for six countries. Credit: Our World In Data.

‘After being approached by the Cabinet Office’s International Comparator Joint Unit, we provided fortnightly updates for the Government and Prime Minister on trends and patterns and ad hoc “deep dive” data reports on specific topics, for example education policies’, said Thomas.

The COVID-19 Government Response Tracker has been an invaluable tool that helped researchers, the public, and political institutions around the world to understand and respond to the global pandemic. At Our World in Data, we are all very grateful that this data was collected and that we were able to make it accessible and understandable in real-time to those who were trying to track government responses around the world.

Max Roser, Founder and Director of Our World in Data 

In research, meanwhile, the team published a series of articles in leading journals including Nature Human Behaviour and The Lancet Public Health. Soon, the Tracker became the most-cited dataset of COVID-19 policy decisions in the world, with over three thousand academic citations recorded on Google Scholar to date. At peak periods, the core dataset was being downloaded several thousand times a week.

The Tracker also attracted enormous public interest, being featured in articles published in The New York Times, the Washington Post, Financial Times, National Geographic, BBC News, Al Jazeera, Le Monde, and The Economist, besides others.

But as Anna said, perhaps the most important impact was that the dataset was used directly by governments, international organisations, and public health agencies to decide how to respond to the ever-changing COVID-19 situation.

‘It changed the conversation at a time when certain leaders were questioning the value of science and of bureaucracies. Instead of endless back-and-forth arguments about what each government was or wasn’t doing (because we gave the world the data on that!), the dataset helped to push the discussion towards what was actually working or not.’

A new chapter

I feel very proud to have been involved with the project. Similar to the 1918 Spanish flu pandemic, the impacts of the COVID-19 pandemic will be studied for many years to come, and this dataset will provide researchers with an immeasurably useful resource.

Andrew Read, COVID-19 Tracker volunteer

Thankfully, the peak of the pandemic passed and by the end of 2022, most countries had settled into their version of “the new normal.” With few new COVID-19 policies being released, the team made the decision to stop publishing real-time updates on the Tracker. But as Anna said, this does not mean that the project has come to an end.

‘Our work is changing in type, rather than ending. We will continue to analyse the dataset to better understand the different ways that governments prepare for and respond to pandemics, as well as the effects of COVID-19 policies on behaviour, health, the economy, and other outcomes. The data from the Tracker also has an important role to play in improving our preparedness for future outbreaks, and we are working closely with colleagues across the recently launched Oxford Pandemic Sciences Institute.’

Whilst a return to near-normal life is to be celebrated, the end of the Tracker’s active phase has prompted mixed emotions. As Shoaib Khan, a technology policy researcher and Tracker volunteer from New Delhi, India, said: ‘Being involved with the Tracker has been a wonderful learning experience for public policy enthusiasts like me, and it was highly fulfilling to interact with people from across the world and the team at Oxford. For me, it is a sad moment that the project is ending, but all the contributors will never forget, and they will feel proud to have been a part of such a global collaborative effort.’

Thomas added, ‘Perhaps the most important lesson for me has been the extraordinary power of a group of volunteers, united by a common purpose to contribute to something bigger than themselves, to do something truly extraordinary. The whole world owes them a deep debt of gratitude.’

The Oxford COVID-19 Government Response Tracker team also wishes to thank the project’s major funders: The Blavatnik Family Foundation, Roche, ESRC, the New Venture Fund, PAX Sapiens, and Microsoft.

Key publications:
‘A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker)’, Nature, March 2021.
‘What have we learned from tracking every government policy on COVID-19 for the past two years?’, Blavatnik School of Government Working Paper Series, March 2022.
‘Variation in government responses to COVID-19’, Blavatnik School of Government Working Paper Series, July 2022.
‘What would a data framework for policy responses to pandemic diseases look like?’, Blavatnik School of Government Working Paper Series, February 2023.