United Kingdom's Open Public Services Network

The Open Public Services Network undertook two major pieces of work exploring how government data, beyond traditional accountability mechanisms like league tables, can be used to evaluate schools. The resulting outputs—A GCSE Schools Guide produced in association with the Guardian, and a report into lack of access to “hard” subjects like triple science at GCSE in deprived areas—succeeded in reaching both parents and policymakers, and drew significant press attention. This study focuses on the contribution open data can make to improving public services.

• OPSN told both a local and a national story with the data it had. The GCSE Schools Guide tool let users see what the data had to say about schools near them. The Lack of Options report took a broad, national perspective.
• The long-term impact of OPSN’s work may best be viewed in the same light as OpenCorporates’ contribution to the campaign for beneficial ownership transparency: just one piece of a complex puzzle.
• Some of the data OPSN used in this case derived from the National Pupil Database, which is not open data. Much of the data needed to meaningfully evaluate public services will never be released—and indeed should never be released—as open data because it contains personal information about service users. It is in the public interest that an understanding of this data does not reside wholly within government. But vested interests and negative press attention driven by a vocal and effective privacy lobby in the UK serve to make this kind of data-sharing policy “a nasty political space”.
• Neither official government interpretations of public services data nor interpretations provided by the press fully meet the needs of public service users. But business models for organisations that could serve the public’s needs are thin.
• Prime Minister David Cameron predicted open data policy would trigger a volunteer “army of effective armchair auditors1” who would interrogate government data in the public interest. In fact, that army has been slow to advance.

Number of users of the Guardian’s GCSE Schools Guide tool on the first day of its release: 20,000

The Open Public Services Network (OPSN) uses data to encourage debate about the quality of public services in ways that engage and empower service users. Hosted at the Royal Society of the Arts, it is run by Charlotte Alldritt, a former government policy advisor, and chaired by Roger Taylor, once a Financial Times journalist who in 1999 co-founded Dr Foster, a ground-breaking information service that focussed on the performance of hospitals and that later attracted controversy when it was part-sold to the government2.

OPSN published its first major report—Empowering Parents, Improving Accountability, about the performance of schools in England—in September 2013. To time with its publication, the Guardian newspaper released a GCSE Schools Guide portal that allowed parents to search through OPSN’s data to discover which schools performed best on which subjects in England, and how accomplished schools were at improving individual pupils’ outcomes in exams. In 2015, OPSN followed up this work with a report highlighting subject deserts, Local Education Authorities (LEAs) in England where challenging subjects such as triple science and modern languages were sparsely offered or not offered at all.

For its first report, Empowering Parents, Improving Accountability3, OPSN’s aim was to see what existing data sources published by the government could tell parents about schools. It drew on open datasets published by the Department for Education (DfE) on data.gov.uk, including exam performance data, and pupil and school characteristics data.

The report also used inspection reports published by Ofsted, the government’s schools regulator. It called on Ofsted to release the data contained in these reports in an accessible format that enables analysis, noting that “at present it mainly exists by school in .pdf format” 4.

For its second report, Lack of options: how a pupil’s academic choices are affected by where they live5 OPSN used data from the DfE’s National Pupil Database. This dataset is not open data, and the pupil-level data it holds is deemed “personal” under the Data Protection Act. The DfE is authorised to share it at various levels of detail under strict terms and conditions with named bodies (including schools, local authorities, and other government departments). Third parties can apply to have access to the data—again, at various levels of detail—to conduct research or provide information services for the purpose of promoting the education or well-being of children.6 The DfE provides guidance on how analysis of the data may be reported in order to protect the privacy of the individual pupils on the database.7

OPSN accompanied both reports with data releases of its own, releasing the data that drove its analyses for others to re-purpose and re-use. The data is released in Microsoft Excel’s Open XML format (.xlsx), under a CC-BY licence.

The aim of the project was to demonstrate that public service transparency, driven by open data can “support a richer and more multi-dimensional approach to accountability than is offered by the current exam results league tables and Ofsted report-based regime”8. OPSN believes that data on schools and other public services represents “a multitude of truths9” that are not captured by current accountability frameworks. In its report OPSN quotes research that shows that such frameworks have “relatively little influence on the choices people make”10. “What good looks like varies against what you’re trying to measure11,” says Charlotte Alldritt. Different service users want to ask different questions to evaluate public services according to their own contexts.

For the 2013 report, OPSN convened a panel of experts representing qualifications authorities, school governors, pupils, teachers, and others. Together they devised a set of features they thought represented the way parents thought about the quality of education offered by schools: facilities; climate of learning; curriculum and pupil outcomes. OPSN then set about interpreting the data sources available to see how fit they were at addressing variations in these features. The data they came up with included new measures, such as rates of uptake of different subjects at GCSE. This data was republished by the Guardian on a postcode-driven searchable portal that allowed parents to compare local schools, the Guardian GCSE schools guide12.

It was during the research for Empowering Parents, Improving Accountability that Roger Taylor says he began to notice how many schools were not showing GCSE results for challenging subjects like the individual sciences (Physics, Biology, and Chemistry—commonly shortened to “triple science”) and modern languages. This observation led to the second of OPSN’s education research projects, using National Pupil Database data to ascertain which schools were not offering these subjects across England.

The report identified six LEAs where 30% or more of schools had no pupils enrolled in triple science: Medway, Slough, Newcastle upon Tyne, City of Kingston Upon Hull, Knowsley, and North East Lincolnshire. In only 41 of England’s 151 LEAs did all schools have at least one pupil enrolled in triple science (see Figure 7).

Figure 7: Access to triple sciences across English schools, taken from Lack of Options report. Copyright OPSN. Reproduced with permission.

The report then shifted its focus to all science GCSEs (including dual-award science, which covers subjects in biology, physics, and chemistry and is worth two GCSEs). This uncovered one LEA where some pupils were taking no science subjects at all (Knowsley). Mapping this data against deprivation data (available as open data on data.gov.uk) showed—with notable exceptions—that there was some relationship (see Figure 8). The report concluded that “fewer science GCSEs per pupil tend to occur in poorer areas” 13.

Figure 8: Number of Science GCSEs per pupil in LEAs coloured by deprivation quintile, taken from Lack of Options report. Copyright OPSN. Reproduced with permission.

The Cabinet Office, which profiled the Guardian GCSE Schools Guide as an open data case study a month after its launch, reports that the portal attracted 20,000 users on its first day of release14. Given the number of pupils receiving GCSE results across England, Wales, Scotland, and Northern Ireland in 2015 was around 700,000, this is a significant proportion of the project’s target audience. The GCSE Schools Guide has also been picked up by the World Bank15 as an example of open data use cases in the education sector.

The 2015 Lack of Options report received substantial press coverage when it was published in 2015. Its headline, that some LEAs in England were subject deserts, was reported extensively by the BBC16 and picked up by specialist education17 news outlets and local papers18 19 20 in the areas highlighted as performing poorly. The Daily Mail21 also gave the story prominence, highlighting its critique of official league tables.

Chris Skidmore MP, a former member of the Education Select Committee, tabled a bill in the House of Commons in response to the Lack of Options report that would guarantee pupils the opportunity to study triple science, and quoted the report’s findings extensively. In closing his speech, he stated:

Poverty of aspiration, which lowers horizons and dims lights that should be burning brightly, still reaches into areas of our education system, and into places where education is most needed to transform young lives. We cannot continue to allow generation after generation of pupils to be let down simply because of the accident of where they were born or what school they attend.22

The OPSN succeeded in engaging its target audiences: the public, the media, and policymakers. This is impressive in itself. But beyond that, how can we understand its impact? Will OPSN’s intervention affect subject availability in deprived areas?

Like much proposed legislation put forward by backbenchers, Chris Skidmore’s bill did not make it far through the UK’s legislative process. In any case, an expert convening in response to Lack of Options attended by the author in June 2015 concluded that the problems highlighted by the report were far more likely to be addressed by broader education reforms. In the end, the Lack of Options report may prove to be just one piece of evidence put forward on the long journey towards policy reform23.

The success of OPSN’s interventions in the education space mirrors that of an American project undertaken by ProPublica in 2011. The Opportunity Gap app used US Department of Education data to show that where some states, like Florida, offer rich and poor students roughly equal access to high-level courses, other states, like Kansas, Maryland, and Oklahoma, offer less opportunity in deprived districts. In his analysis of this project, ProPublica’s Scott Klein writes:

We … worked really hard at making sure the app told a “far” story and a “near” story. That is, the app needed to present the reader with a broad, abstract national picture—specifically, a way to compare how states did relative to each other on educational access. But given that abstraction sometimes leaves readers confused as to what the data means to them, we also wanted readers to be able to find their own local school and compare it to high and low-poverty schools in their area.24

OPSN’s work also tells two different stories. The appeal to the public (through the GCSE schools guide) and the appeal to the press and policymakers (through the Lack of Options report) are subtly different, and point to an understanding that drives OPSN’s work: Service users want to ask different questions of public service data than service providers. At the heart of this approach lies a challenge to models of thinking about public service improvement through open data.

“People need simple heuristics to guide them in their lives and understand what is going on25,” says Roger Taylor, Chair of the OPSN. He is deeply sceptical about the model of public service improvement through open data vaunted by David Cameron, who, when he announced new measures to publish local government spending in 2010, predicted the birth of “a whole army of effective armchair auditors looking over the books26”.

High-profile cases of public spending waste have been exposed thanks to open data, for example, when hundreds of millions of pounds in potential annual savings on prescriptions for statins were identified in 201227. But since the government’s big push to publish more government data in order to improve public services in 2010, several commentators have noted that Cameron’s army are, for the large part, still yet to advance.

In November 2012 the think tank Policy Exchange blamed the lack of armchair auditors on the fact that data like the Combined Online Information System of government spending (COINS) was “unusable”28. And in April 2015 the Institute for Government pointed out that much of the data armchair auditors would need to hold the government to account was inaccessible, or of poor quality29.

But the data may only be half the problem. Roger Taylor says that while good data analysis doesn’t take a large corporation to do, it does need significant resources:

Analysing complex datasets and trying to get useful signal out of all the noise is expensive and time-consuming. It isn’t something you can do in your armchair …. It really does come down to small numbers of extremely talented and able people.30

Drawing on his experience with Dr Foster, he worries that business models to support such operations “are very thin”. Although he is clear that the Department of Health “never once attempted to use its influence to affect anything we said,” after it paid £12m for a 50% stake in the company in 2006, “basic economics” meant the organisation ended up focussing on how to meet the information needs of NHS organisations and healthcare professionals, not how to meet the information needs of the public and patients:

Professionals always err towards the more cautious side, with the result that what you tend to get from more professional organisations are really complex presentations of data, with massive amounts of caveats about over-interpretation. They tend to prefer presenting the raw data rather than [stating] what it means, with the result that it means nothing to the public and they really can’t act on it.31

This situation disempowers service users:

All this information about hospitals or doctors or whatever: if it can’t be used to answer the question “should I have this treatment?”, “should I let this doctor operate on me?” —if it can only be used inside the system—then all that does is simply allow the public to watch through the glass window and see how the professionals sort it out among themselves.32

Taylor adds that traditional media approaches to public service evaluation often mislead the public because “the way our media dialogue works is through over-simplification”. But he also believes that even when the media take a more data-literate approach, the media business model doesn’t support the sorts of personalised information services people need:

[To] get this information to the point where it works for an individual … and have a business model working you’ll probably need a pretty intimate connection with that individual …. An old fashioned publishing model, where you try and earn revenues from publishing ratings or advice sheets or that kind of stuff—it’s really hard to make that work.33

It’s important to note that the Lack of Options report was produced using non-open data: a version of the National Pupil Database for which researchers need the approval of the Department for Education in order to access. Taylor believes that, through the open data agenda, “we’ve set a standard for data release that is wholly inappropriate to most of the information that is of interest in terms of public services”:

For anything to do with mental health, education, to make sense of what is going on you need to understand about the outcomes for people, what happened to them and data about their personal circumstances. There’s no way [you can] put that data together in usable formats and then simply put it out under an Open Government Licence.34

“It is in the public interest that an understanding of what [the data says] does not reside wholly within government,” says Taylor, but getting this data out of government and into the hands of approved third parties, for example on terms similar to those of the National Pupil Database, is hard. “It’s a nasty political space to try and move forward on,” says Taylor. He believes that vocal and effective privacy campaigners in the UK mean this type of government data sharing attracts negative press attention, while behind the scenes, vested interests such as professional bodies and government departments use their power to block progress. By contrast:

Open data is really straightforward. So you see why all the political focus has gone in that direction. But unfortunately, we aren’t actually going to get many useful outputs from that process.35

For open data advocates

Open data advocates should recognise that releasing personally identifiable, sensitive data as open data is not appropriate.

Open data advocates should partner with privacy advocates to encourage policymakers to engage in this “nasty political space” in ways that are technically literate and respect privacy.

For funders

Funders need to be realistic about the capacity of volunteer “armchair auditors” to improve public services. Targeted support for skilled infomediaries should be part of the open data funding mix.