Today, the Central Digital and Data Office (CDDO) and Centre for Data Ethics and Innovation (CDEI) have published the first reports from our transparency standard pilots. The public sector algorithmic transparency standard is one of the world’s first initiatives of its kind. It provides a framework for public sector organisations to share information on their use of algorithmic tools with the general public and other interested stakeholders.
Developing and piloting the algorithmic transparency standard
The standard was developed in response to commitments the government made to algorithmic transparency in the National Data Strategy and National AI Strategy, as well as calls from civil society organisations for greater transparency around the use of algorithmic tools in the public sector. We published the first version of the standard in November 2021. For more information on how and why we developed the standard, see our previous blog.
Since January this year, we have been piloting the standard with teams across the public sector. The piloting process involves:
- An introductory session with the relevant team
- A follow up workshop to work through the content of the standard template in more detail and consider how it will apply to the team’s chosen algorithmic tool
- A first draft of the template, which the team completes with support from CDDO and CDEI
- An evaluation workshop to discuss the team’s experience in completing the template
- Publication of the final report on the gov.uk landing page
The pilot reports we have published today are:
- The QCovid team - who piloted the standard with a COVID-19 clinical tool used to predict how at risk individuals might be from coronavirus; and
- GOV.UK Data Labs team - who piloted the standard for their Related Links tool, a recommendation engine built to aid navigation of GOV.UK
Alongside the pilots, we have gathered feedback on the standard through further discussion with officials from government departments and public bodies, and an open call for feedback from members of the public. We also held two roundtable discussions with suppliers of algorithmic solutions to incorporate the perspectives of third party suppliers on the standard. The roundtables were facilitated by techUK and the Crown Commercial Service respectively and were attended by nearly 100 representatives from the private sector.
We are currently collating and evaluating all the feedback we have received.
Preliminary findings from the pilots
Preliminary findings from the pilots suggest participating teams are generally supportive of algorithmic transparency and its potential benefits but also mindful of its risks and challenges.
Some key themes have emerged through the pilots and we will incorporate this feedback in an updated version of the standard. Teams told us that:
- The algorithmic transparency standard is a useful way to provide information about algorithmic decision-making processes to interested users. For example, the QCovid team said the process of documenting this information could help explain to patients how and why clinical recommendations are made.
- Completing the template can help public sector organisations ask useful questions internally.
- The template should be as user-friendly as possible, and we should specify the level of detail required in the accompanying guidance.
- There are areas that need further specification in the first version of the standard, which we will seek to amend and refine when revising the template.
- There is still work to be done to practically implement the standard, including clarifying who should be responsible for completing the template and at which stage in the tool’s lifecycle.
Next steps
We are completing the pilot evaluation in order to update the standard and develop the accompanying guidance, which we will publish later this year. Alongside this, we will publish a blog providing more detail on the findings from the evaluation and how these have been incorporated into the revised standard, and we will keep releasing further pilot reports in the coming months.
We are also exploring avenues to promote uptake of the standard across government and the public sector more widely. This includes seeking the Data Standards Authority's endorsement of the standard as an official cross-government standard.
Leave a comment