Show us our processes that ensure we manage AI risk effectively
- 4-Minute Read
- For senior leaders seeking to put in place an AI governance process.
One of the main concerns of Boards is risk management and compliance. This is particularly relevant in AI where Directors and Officers can, under certain circumstances, face personal, sometimes even criminal, liability, not just the company. This matters because most breaches are dealt with at the company level, but criminal liability is personal, and Directors and Officers (D&O) insurance is designed for civil claims, not criminal penalties.
What triggers AI compliance action?
Modern (generative) AI is a powerful technology. But it is just that, a technology. At its core, it is a prediction engine – it looks at what you entered (the prompt), matches it to what it has seen before (its training) and gives you what it decides is the most likely output (the response). Probability plays a role in how these AIs work and so if you ask the same question twice, you may not get the same answer. For example “What is the capital of France” might give you “Paris” 999 times out of 1000, but one time it might respond “F”, as in the capital letter in the word.
This makes it fundamentally different to conventional computer technology. With conventional software, you test it and it provides an answer. Repeat and you will get the same answer. AI does not work like that. Worse, it can give answers that sound plausible and confident but are simply wrong – often called “hallucination” – and users need to be aware of this. AI can be wrong and if you are making business decisions based on what it says, you need to have checks in place.
Where to start
Whatever the trigger, almost all companies are going to need some AI governance and organisations that operate in regulated sectors, routinely process confidential data or have customer-facing AI will need to pay particular attention. But even companies that only use AI built into off-the-shelf products should still have appropriate, lightweight processes in place.
The three pillars
AI governance can be thought of as three pillars:
- Awareness – understanding the risks, actively monitoring and assessing
- Management – processes, procedures, accountability and ongoing control
- Evidence – not just having a well-functioning process but also the ability to evidence this should the need arise
Governance is not just an IT problem
It is tempting to think of AI governance as something that is handled by the IT department. Keeping a register of approved tools, monitoring activity on the company network and looking for suspicious activity. But AI governance is much wider than this and touches on almost every department. HR will need to worry about bias in the AI being used for resume scanning. The legal team need to concern themselves with contracts that have been drafted by AI and may contain incorrect references. The customer service team need to know what the company chatbot is allowed to promise, because the company, not the chatbot, will be held to it.
AI is a cross-functional problem and requires a cross-functional response. While the Board is the ultimate owner, many companies create cross-functional AI oversight committees that are responsible for AI governance. The committee will typically review all the relevant processes, assess any new AI deployments that are proposed, investigate any AI incidents or near misses, monitor the uptake of AI
training and be aware of the latest state of AI and the regulations surrounding it.
Governance is also an enabler
Many companies have reported implementing AI pilots that either never go live or need substantial redevelopment because the AI governance issues were not considered at the outset. For example, it may be attractive to develop an AI to provide credit scoring for customers, only to find later that fully automated credit decisions are restricted in many jurisdictions – including under Kenya’s Data Protection Act – unless there is a route to human review. Proper AI governance ensures that the AI that is already in place is being properly used, and is a key enabler when looking at new uses for AI, helping avoid last-minute issues that arise from loose controls.
Key elements of your first plan
Where is the business compared to where it needs to be? Is there a register of all the AI in use in the business? Is there a risk register and is it maintained? What happens if there is an incident? Are the staff aware of AI risks and responsible AI management? Questions such as these will lead to a gap analysis, which underpins an improvement plan.
Core policies are put in place, for example an AI
governance and ethics policy, acceptable use policy, clear lines of accountability, regular and well documented oversight, well-defined metrics and regular reporting at
Board level.
One of the advantages of AI is that it is readily accessible to anyone, you just talk to it. But that is also one of its principal risks – confidential
data can leave the business, malware can be brought in and AI generated reports can be used for decision-making, based on false or misleading information. So training needs to reach everyone who uses AI, not just the technical teams.
- Key Takeaways
- Irrespective of the trigger, all companies need AI governance
- It is a whole-company issue, not just limited to the IT department
- Good AI governance is an enabler. It reduces the risk of incidents, makes regulatory reporting simpler and prevents otherwise good projects being delayed or cancelled because of core (generally non-technical) issues.
To explore further
Take a look at our Resources page with many other helpful articles on AI and governance
Try our tools and solutions
Discover Akili’s tools and solutions designed to help you understand,
assess and govern AI with confidence.