What is an AI risk assessment?
An AI risk assessment is a written check of one AI tool or use before it goes live. It lists what could go wrong, such as a wrong output being used or confidential data leaking, scores how likely and how serious each risk is, and records what you'll do about each one and who owns it.
How is an AI risk assessment different from a DPIA?
A data protection impact assessment (DPIA) looks at the risks to people from the use of their personal data, and UK GDPR sometimes requires one. An AI risk assessment is wider: it also covers wrong outputs, security, suppliers and running costs. If the tool uses personal data, you'll often need both.
The DPIA template for AI projects covers the data protection side.
What risks should an AI risk assessment cover?
The template starts with nine common risks: a wrong output used without checking, data going in that shouldn't, people seeing documents they couldn't open before, the supplier keeping or training on your data, processing outside the UK, unfair outputs, hidden instructions in documents, the tool changing, and costs growing.
Guidelines from the National Cyber Security Centre (NCSC) cover AI security across design, development, deployment and day-to-day running, and they're written for organisations of every size, including small ones.
The AI risk assessment template
This is the same template as the Excel and Word files. The Excel version has the risk register ready to score.
AI risk assessment
One assessment per AI tool or use. Fill in the parts in square brackets, score each risk, and agree a measure for anything rated 6 or above before the tool goes live. Review it when anything changes. It isn't legal advice.
- AI tool or use
- [What it is, and the supplier]
- Owner
- [Name, role]
- Assessed by
- [Name, role]
- Date
- [DD/MM/YYYY]
1The tool and the job
- What it does
- [The job, in a sentence]
- Who uses it
- [Team, roughly how many people]
- What goes in
- [Documents, emails, recordings, and any personal data]
- What comes out
- [Drafts, summaries, answers, and who sees them]
- The supplier
- [Who runs it, what they keep, where it's processed]
- Who checks the output
- [Name or role]
2How to score
- Likelihood: 1 unlikely, 2 possible, 3 likely.
- Impact: 1 minor, 2 serious, 3 severe.
- Rating: likelihood times impact, from 1 to 9.
- 6 to 9 is high: agree a measure before go-live. 3 or 4 is medium: agree a measure and a date. 1 or 2 is low: note it and review.
3Risk register
Risk A wrong or made-up output is used without being checked Personal or confidential data goes in that shouldn't People can see documents through the tool that they couldn't open before The supplier keeps inputs, or uses them to train its models Data is processed or supported outside the UK without a safeguard Outputs treat some groups of people unfairly Instructions hidden in a document or email trick the tool into doing something it shouldn't The tool stops working or changes, and the job depends on it Running costs grow beyond the budget [Add your own] Likelihood, Impact, Rating, Measure, Owner: filled in for each row in the Word file.
4Sign-off
- Approved by
- [Name, role, DD/MM/YYYY]
- Remaining high risks accepted by
- [Name, role. If personal data is involved, check whether you need a DPIA]
- Next review
- [DD/MM/YYYY, or when the tool, supplier or use changes]
Free to use and change, with no email needed.
How do you carry out an AI risk assessment?
Describe the tool and the job first: what goes in, what comes out, who uses it and who checks it. Then go through the risk list with the people who'll use it and whoever runs your IT, score each risk, agree a measure for anything rated 6 or more, and name an owner.
- Describe the tool and the job. What goes in, what comes out, who uses it and who checks it.
- Get the right people in the room. Someone who'll use it, whoever runs your IT, and your data protection lead if personal data is involved.
- Go through the risk list. Keep the ones that apply, add your own and score each one.
- Agree a measure for the high ones. Anything rated 6 or more gets a measure and an owner before go-live.
- Sign it off. Someone with authority accepts what's left, and you set a review date.
How do you score AI risks?
Score each risk for likelihood and impact from 1 to 3, then multiply them. A rating of 6 to 9 is high, so agree a measure before the tool goes live. A rating of 3 or 4 needs a measure and a date. Rough scores are fine, as long as everyone agrees the measures.
What should you ask an AI supplier?
Ask what they keep and for how long, whether your data trains their models, where it's processed and supported, who at the supplier can see it, and whether they'll sign data processing terms. Get the answers in writing and record them in the template's supplier row.
- What do you keep from our inputs and outputs, and for how long?
- Is our data used to train your models? Can we switch that off?
- Where is it processed and stored, and where do your support staff sit?
- Who at your company can see our data, and when?
- Will you sign data processing terms with us?
The ICO, the UK's data protection regulator, also publishes an optional AI and data protection risk toolkit, a spreadsheet for checking the data protection risks of an AI system in more depth.