Was the Covid Tracer app a success or a failure?

The $6 million+ New Zealand contact tracing app - did it work or was it just a visible and useful reminder of risk to keep people on board with government restrictions?

I still sometimes see around those once ubiquitous QR posters to scan with the Covid Tracer app. And yes, I try to remove them, but what is with the outdoor ones screwed on? They’re a real devil to get off.

As the CovidCard and wearables trial failures showed - the ability to digitally record your private behavior was a major focus of government during the pandemic. To that end, by early April 2020 New Zealand tech firms had inundated the Ministry of Health with their ideas of how it could be achieved.

What became the Covid Tracer app ended up being pursued with only 2 potential suppliers who had directly contacted the Ministry of Health. To do this required an emergency exemption from standard procurement guidelines (which apply for amounts over $100,000) for the duration of the pandemic. Yes, it said that and it was assumed to be 3 months with the right to extend another 3 months. Ha!

They raced to sign a contract within 48 hours so they could begin development. Rush Digital were chosen as the app developer.

The Covid Tracer app was launched on the 20th of May 2020 as New Zealand was slowly climbing down Alert Levels from the height of a full lockdown in March and April.

The tracer app & privacy

Then Director-General of Health, Ashley Bloomfield pushed the app’s privacy features on its launch. This was a big issue that users and the Privacy Commissioner were concerned over.

Oddly, the initial Cabinet papers included a website that everyone could sign up to for adding their contact details on the off chance they became a case (so the Ministry of Health could contact them). This was seen as a good thing based on a poll that said people would do anything, just anything at all, to stop Covid, even supporting the Police accessing their cellphone to find their movements and who they’d been in contact with. I think this website idea must have died a swift death as I didn’t find it mentioned again.

Privacy also cropped up alot in market research that contained public attitudes on the use of an app from various research companies Horizon, IpSos (20 April) and Colmar Brunton (27 April). I found reference to the IpSos and Colmar Brunton research in the paper proposing the Ministry pursue the app but I haven’t found the actual research.

The news media highlighted privacy fears too, and it came up again in a later Colmar Brunton survey in October 2020 on attitudes to contact tracing technologies.

And rightly so, Singapore’s Covid tracing app TraceTogether which was repeatedly held up as the shining example to copy in these documents - had crashed and burned when it got out that police had accessed app data and used it to find people. Further issues cropped up in other countries too - Hong Kong and the Philippines had privacy issues with their apps too.

Bloomfield took pains to add any personal data you entered was “never for enforcement” in press conferences. In the first release you had to enter a name but other details like gender and address and phone were optional. By November requirements to enter your name were removed in order to strengthen the privacy of using the app.

If you used the app - a June paper noted the data is stored for the duration of the pandemic in Amazon Web Servers in Sydney. While the privacy impact assessment has been removed from the Ministry of Health website - an archived copy from early 2022 has the 100 page document.

Bluetooth was added to the app in late 2020 so it could automatically connect with other app users with Bluetooth turned on. A user could choose to send an alert to them if they tested positive. In a briefing on the Bluetooth release then Covid-19 Minister Chris Hipkins said he wanted to see very sound messages as, “…people need to have confidence they will own their own data and this will not be a "big brother" experience.”

Right, so that would be important as they had investigated how “big brother” they could go. Documents on the Bluetooth work discuss other uses too, for instance suggesting banking apps be updated to include a location tracker for users to turn on - so that a contact tracer could find their movements from non-Covid Tracer app use too.

Encouraging compliance

I got the above image from the app’s official website - which while I wrote this post was updated to announce they are finally removing the app in September 2023.

Did scanning help us return to “doing the things we enjoy as soon as possible”?

Similar to what I discussed in contact tracing, the Covid Tracer app was obsessed with metrics (number of scans) yet not so much with solid outcomes from the scans.

It’s a mystery universally acknowledged that if there was no community transmission people were less likely to scan and use masks - by July 2020 the government started pleading people to scan to encourage uptake.

In the first week of August 2020, Bloomfield was planning to email app users and various influencers in public health to support using it. Alert Level changes within August did prompt more scanning - about 1 million scans per day. And there had been 1.8 million downloads of the app (estimated to be 45% of the adult population) by the end of that month.

But despite incredible amounts of research and nudging campaigns, surveys by The Research Agency (TRA) commissioned by the Department of the Prime Minister and Cabinet (DPMC) showed people just weren’t scanning everywhere they went:

This observed ‘low’ usage led to a trial to tap rather than scan to increase compliance, as well as advertising agency pivots to “reposition” the app from a “…scanning activity to stop the virus“ to “…scanning being something we need in order to protect the things we love.”

However I guess it didn’t work as a Cabinet paper in mid-2021 introduced mandatory record keeping which meant requiring people to scan in to close contact businesses or if they did not have the app - manually record their contact details:

It gave further requirements for how QR posters should be clearly displayed on entry. Although Hipkins appeared to note a requirement to have them at each table at every hospitality location seemed excessive - which Hospitality NZ had itself deemed overzealous. It also allowed businesses to order record keeping books in 27 languages! I wonder what happened to those?

The briefing that led to the Cabinet paper and minute for mandatory record keeping cites more research, this time from PwC, who did research into behavioral science and the app:

However what’s really interesting is that mandatory record keeping was not supported by the Ministry of Health, The Ministry of Pacific Peoples nor the Privacy Commissioner.

Not just that but the updates widening community mask mandates in the same Cabinet briefing were done not by the Ministry of Health - who also disagreed with those - but by DPMC.

The Ministry of Health noted mandatory record keeping may not be a proportional response, nor was there any strong evidence that doing so for contact tracing would be effective and it’s use had equity considerations.

The Ministry of Pacific Peoples agreed with the above issues and also noted Bill of Rights advice from Crown Law (which is always redacted in these documents) didn’t agree with it either:

The Privacy Commissioner also objected:

They couldn’t mandate the use of the Covid Tracer app per se because that may contravene both Apple and Google’s Terms of Service and the app could be booted off the platforms.

Despite this record keeping was made mandatory at all close contact businesses, gyms, health care facilities, courts and social service offices and events.

The usual Otago University lobbyists rejoiced in the media as, of course, they’d been promoting it on their ‘public health expert’ blog. They even managed to include while doing so a plea that international flights be suspended. Unhinged!

The briefing to Hipkins in March 2022 to un-mandate record keeping after Covid had well and truly taken off reads…reluctantly…it suggested record keeping and scanning could be useful in the future so businesses should still be required to display QR posters. It allowed that failure to comply could move down from medium to low risk for infringement purposes.

How much did the app cost?

Mostly the costs are redacted in documents but 1 OIA managed to get a response that from March 2020 to December 2020 about $6.478 million was spent on the development and maintenance of the app and QR posters.

The cost appeared to have been paid out of the $500 million fund that was set up initially to strengthen health services. I haven’t found the further costs post 2020 that take into account more work or maintenance of the app.

How useful was the app? Was it successful?

Scanning lots or not at all is 1 metric as was turning on Bluetooth, but success would have meant, presumably during the elimination phase if someone tested positive they could instantly give a full digital diary to contact tracers, send out a Bluetooth alert to their close contacts - and all those actions would quickly and clearly stop transmission.

But an incredibly small proportion of people who became cases had a diary to share that was useful to contact tracers. Useful is the key metric - 1 of the reasons from the CovidCard trial for its failure was contact tracing is manual and nuanced. If you scanned a QR code in an outside park (QR posters were stupidly anywhere, the side of van doors, outside parks and reserves, at playgrounds etc) - the contact tracer may be unlikely to deem that a high risk contact location to follow up.

Locations of interest that were published on the Ministry of Health website confirm this - they were indoor retail stores, cafes, pharmacies etc. If you click on that link you can see how far gone New Zealand was in Covid zero - the locations of interest in the Delta outbreak from August 2021, when the country was in a lockdown, neared 500!

A large music event in early 2022 with a Covid outbreak had somewhat pointless numbers with no actual information on whether a location or Bluetooth alerts meant any impact on case numbers and transmission rates.

Were there any metrics or assessments that the app actually helped contact tracers to do their job quicker? Can we say it stopped transmission? I haven’t been able to find any assessments on how worthwhile the Covid Tracer app was - although there are a lot of academic articles on number of scans and the user experience.

An independent Australian review into their CovidSafe app found it was a complete waste of time and money. It’s hard to find definitive data from other countries too on these apps, a 2021 review across countries came up inconclusive.

Reflections on it from the Ministry of Health staff involved also don’t answer ‘did it work’ but center around designing the app with the perception of privacy.

In late 2020 Otago University did interviews with contact tracing staff to assess the technologies and processes in use. The same author of that is doing another funded study on how digital contact tracing supports improved pandemic responses with a focus on equity.

I bang on about consultants, but those academics did quite well too, no?

Issues with accessing the QR code (particularly for disabled people) in order to scan and having a cell phone for each family member came up in the equity focussed Community Panel that DPMC set up in July 2021. So far the study appears to consist of focus groups so I suspect it will be on similar equity issues, rather than a broader view.

In 2020 a Ministry of Justice unit called Behavioral Science Aotearoa was tasked with behavioral nudging research. An early report noted, “Visibility of COVID prevention behaviors serves as an important reminder and reinforcer of social norms…this exerts a special pressure to comply”. Another report said if “People no longer see the disease as much of a threat….[it] would undermine the government strategy”. 

Visibility of masks was a reason given in support of the first mask mandates on public transport in 2020. From that perspective - is the success of the Covid Tracer app really the success of the ubiquitous QR posters that had to be displayed everywhere - which kept the risk so visible?