Today the National Audit Office has published a report on the challenges in using data across government. It criticises the UK government for the lack of clear and sustained strategic leadership on data, and says that this has led to departments under-prioritising their own efforts to manage and improve data. This is fair, but the report does not appreciate the real complexities of data. To protect people sometimes data about them needs to be hard to use.
The UK has been going through a slightly strange period recently. While it has long been a world leader in the use of data, in the last couple of years it has been sliding. Brexit has overwhelmed everything.
The report says that this has led to a failure to build strong data infrastructure and improve the use of data. We are big fans of better data infrastructure. But the report does not recognise that sometimes data infrastructure needs friction. Sometimes it should be hard to access data, to connect data together and to use it. It should be hard because the checks and balances that make it hard are how we retain trust and reduce the chance of the data being used to harm us.
This is most stark in the examples of the Windrush deportations and overpayment of Carer’s Allowance, and the large range of identifiers for people that are used by different government departments. The report says that these identifiers should be standardised, to save money, to improve services, and to avoid greater harm in the future.
Data, particularly data about individuals, always has gaps and imperfections. We change our names, our addresses, and our gender. Sometimes the gaps in the data are mistakes. Sometimes there are failures in data collection. Sometimes people deliberately hide in the data and give false information. Sometimes they need to hide because they are at risk of harm. Perhaps they are at risk of domestic violence and need to hide their location, or at risk from an authoritarian government which is oppressing a religious or cultural minority. We need to assume that data will have imperfections and plan for that.
Those imperfections can cause harm to people. From the simple example of a letter going to the wrong address, to more complex cases such as that misplaced letter causing someone to be unaware they’ve been diagnosed with a disease.
So, when addressing data quality we need to consider both the harms caused by bad data, and the harms and benefits that might be caused when we make the data easier to use. That way we can try to design the necessary friction into the data infrastructure.
The UK debated identity for citizens a decade ago. Both identity cards and a national register of citizens were roundly rejected. People did not trust the government to use the data well or keep it safe and secure. It is deliberately hard for the UK government to connect together data about individual citizens. It is not impossible (sometimes it needs to be done, for example, to investigate a crime) but it is hard because that extra effort creates the checks and balances which retain trust and prevent harm.
The data lesson from the Windrush deportations and Carer’s Allowance is not simply to increase data quality. It is to recognise that personal data will always have imperfections, those imperfections may be deliberate, and that we should not design public services – particularly critical services like someone’s right to live in a country – that rely on data being perfect.
There are many other areas of the NAO’s report that we strongly agree with, for example the lack of clear and sustained strategic leadership on data, and under-investment in the UK’s data infrastructure. But while every country does need better data infrastructure, we need to recognise that sometimes it will need to have friction.
Peter Wells is director of public policy at the Open Data Institute