Today, the Oxford Internet Institute is launching a new commission on AI and good governance which will examine artificial intelligence in the public sector. The commission aims to work with policymakers from around the world to advise on the most effective and principled ways of using AI, while analysing implementation and procurement challenges faced by governments.
An inaugural paper outlines the commission’s four principles for government use of AI which include inclusive design, informed procurement, purposeful implementation and persistent accountability, all with the aim of protecting democracy. The commission will complete a series of reports in the coming months and aim to come up with guidelines for best practice for policymakers and government officials. It will look at health first, followed by open cities, and then military and policing applications – two areas where governments around the world are investing significant resources.
“For the last couple of years, there has been nominal interest in the role of big data and making use of data in government, but I think Covid has really turned up the pressure on government agencies to use data in effective ways,” says Philip Howard, director of the Oxford Internet Institute. “I think before we get too much more integration of AI and government, we need to set the rules – figure out how we can use AI to do government the way we’d want it to be done.”
In-depth reports from the Cardiff Data Justice Lab and the Guardian have revealed that AI is used fairly extensively by public bodies in the UK, but this is not widely publicised. In late 2019, the Guardian found that one in three local councils were using algorithms to make welfare decisions despite evidence that they were unreliable. Policing is another area where algorithms have controversially been used for predictive purposes (to identify ‘hot spots’ for crime and even individual risk factors). A Public Harms Committee paper released earlier this year pointed out the lack of transparency that surrounds government use of AI.
“Industry will be pushing AI solutions to public agencies pretty aggressively in the years ahead, so one of our goals is to make sure that policymakers ask the right questions and can interpret the answers,” says Howard.
He says that “procurement practices are often pegged to very simple statements” such as fair distribution of resources. However, the stipulation for a competitive tender process doesn’t mean much when there is only a narrow selection of companies that can provide suitably sophisticated algorithms.
Oftentimes, in the case of local government, officials don’t have a strong understanding of AI, meaning accountability can be tricky. “If we don’t fully understand how an algorithm works, how machine learning works, then how can we demonstrate that it’s done something equitable with public resources?” says Howard.
“It’s about making sure that the equity issues are addressed early on by the engineers of a firm, not later on by the PR staff of a firm,” he says of the ‘baked-in’ algorithmic bias that can result from poor training data that doesn’t account for diversity in a population.
In areas such as national security, procurement is shrouded in so much opacity that it’s difficult to ascertain how AI is being used – something the commission hopes to investigate. “One of the fastest routes to undermining democratic values will be if AI and sophisticated machine learning tools are used by national security services in ways that are unchecked,” says Howard.
The committee is composed of a mixture of figures from academia, government and business, including Dame Wendy Hall, chair of the Ada Lovelace Institute and Dr Yuichiro Anzai, chair of the Council for Artificial Intelligence Strategy and adviser to the Japanese government on strategic policy. Perhaps slightly controversially, Dr Rumman Chowdhury, managing director and responsible AI lead at Accenture, is also due to join the committee. This could raise eyebrows given Accenture is a long-standing recipient of UK government public sector contracts and therefore could stand to gain by helping to shape procurement guidelines on AI.