Who Actually Owns AI Risk?

Episode transcript

Transcript provided for accessibility and search. Minor transcription errors may remain.

AI Security MultipliersEpisode 2: Who Actually Owns AI Risk?Host: SteveWho Actually Owns AI Risk?Steve: AI is turning up everywhere in the enterprise.In customer service.Software development.Finance.HR.Cybersecurity.Marketing.Operations.And increasingly, inside systems that are actually making decisions or taking actions.Which raises a fairly obvious question.Who owns the risk?Is it the CIO?The CISO?The Chief Data Officer?Legal?Privacy?The business?Or do we create a Chief AI Officer, give them a new organization, a very impressive job title, and hope they sort the whole thing out?I'm Steve.And this is AI Security Multipliers.Today, I want to talk about something I've seen organizations struggle with whenever a new technology crosses traditional organizational boundaries.Ownership.Because AI risk doesn't fit neatly into one department.And I think trying to force it into one may actually be part of the problem.So today we're going to figure out who actually owns AI risk.And perhaps more importantly, who shouldn't.Everybody Owns AI... Until Something Goes WrongThere's a fairly predictable pattern with new technology.When the opportunity looks enormous, everybody wants to be involved.There's a strategy group.An innovation team.Technology gets involved.The business gets involved.Somebody creates an AI Council.Possibly an AI Steering Committee.Then another committee appears to coordinate the first committee.There are workshops.Roadmaps.Proofs of concept.And quite a lot of PowerPoint.Everybody is terribly excited.Then somebody asks:"Who owns the risk?"And suddenly the room becomes noticeably quieter.Because ownership of the opportunity and ownership of the consequences aren't always treated quite the same way.I've noticed organizations are remarkably good at finding the CISO when something contains the words risk, data, cloud, AI...or, increasingly..."We're not entirely sure how this works."And I understand why.AI introduces cybersecurity risks.Absolutely.But it also introduces privacy risk.Legal risk.Regulatory risk.Operational risk.Reputational risk.Data risk.Financial risk.Ethical risk.Third-party risk.And plain old business risk.Cybersecurity is part of that.But cybersecurity isn't all of that.Which leads to the first principle for today's episode.The CISO shouldn't become the person who owns everything vaguely frightening about AI.Tempting though that apparently is.Let's Follow One AI Use CaseLet's make this practical.Imagine a business function comes to you with an idea.They want to use AI to improve customer service.Nothing particularly exotic.The AI will analyze customer information, review previous interactions, suggest responses to customer questions, and help employees resolve issues more quickly.There is a legitimate business case.Customers get faster answers.Employees spend less time searching through information.Potentially lower operating costs.Everybody wins.So who owns the risk?Cybersecurity might look at authentication.Access controls.Encryption.Logging.APIs.Application security.Third-party security.Maybe prompt injection or data leakage.Privacy looks at personal information.Legal looks at contractual obligations and regulatory requirements.The data owner needs to understand what information the system can access.Technology needs to understand the architecture.Procurement needs to understand the vendor.Compliance may have requirements depending on the industry.And the business needs to understand what happens if the AI gives the customer the wrong answer.Now imagine the AI tells a customer they're entitled to a refund when they're not.Is that a cybersecurity incident?Probably not.What if it exposes another customer's personal information?Now cybersecurity and privacy are very interested.What if it discriminates against a particular group of customers?That's another category of risk entirely.What if it works perfectly from a security perspective but produces such consistently terrible customer service that customers leave?Still a risk.Just not one the CISO should own.This is why asking:"Who owns AI risk?"may actually be the wrong question.There isn't one AI risk.There are lots of them.Accountability and Ownership Aren't Quite the Same ThingThis is where I think terminology matters.Organizations need accountability for AI.Somebody needs to make sure there is a governance model.Somebody needs to make sure decisions get made.Somebody needs to make sure risks don't simply disappear into the gaps between departments.But accountability for the governance system doesn't mean that person owns every risk created by AI.Think about cloud computing.The CISO doesn't own every risk created by AWS or Azure.The CIO doesn't own every business decision made using Salesforce.And the Chief Data Officer doesn't suddenly own the consequences of every spreadsheet containing customer information.At least I hope not.We learned, eventually, that technology risk has to be distributed to the people capable of understanding and managing the specific risk.AI should work the same way.You can have centralized governance.You can have centralized standards.You can have common policies.Common risk assessment methods.Common technology platforms.But the actual risk still needs an owner.And that owner should usually be the person with the authority to make decisions about the activity creating the risk.That's the important bit.Because assigning somebody responsibility without giving them authority isn't governance.It's just giving them somebody to blame later.Start With the BusinessSo let's go back to our customer service example.Who owns the business outcome?The business.That sounds painfully obvious.But it's surprisingly important.If a business function wants to deploy AI because it expects to improve customer service, increase revenue, reduce cost, improve productivity, or create some other business outcome, the business should own that outcome.And it should also own the business risks associated with that decision.Security can help explain cybersecurity risk.Privacy can explain privacy risk.Legal can explain legal risk.Technology can explain technical risk.But eventually somebody has to decide whether the overall risk is worth taking.And that decision shouldn't quietly migrate to cybersecurity just because the technology contains AI.I've been in versions of this conversation many times.The business says:"We'd like to do this."Security identifies some risks.And eventually somebody asks:"So, Steve, are you comfortable with it?"That's a perfectly reasonable question.But sometimes what they're actually asking is:"Would you mind accepting this business risk for us?"And those aren't quite the same thing.As a security leader, I can tell you whether I believe the cybersecurity controls are adequate.I can tell you where I see weaknesses.I can explain possible consequences.I can recommend additional controls.I can even say:"I think this represents a significant cybersecurity risk."But if the business benefit is substantial and the residual risk is within the organization's appetite, somebody in the business still has to make that decision.The CISO shouldn't be the person deciding whether a customer service strategy is worth pursuing.Otherwise we're going to need a much bigger office.Risk Acceptance Is a Business DecisionThis leads to something I think gets confused surprisingly often.Risk identification and risk acceptance are different things.Cybersecurity might identify a risk.That doesn't necessarily mean cybersecurity owns it.Let's say an AI application needs access to sensitive customer information.Security assesses the architecture and says:"We can reduce the likelihood of unauthorized access significantly, but we can't eliminate it."That's normal.There is no zero-risk architecture.Despite what some PowerPoint slides appear to suggest.So now we have residual risk.Who accepts it?That depends on the consequence.If a system failure could create a material business loss, expose customers, create regulatory consequences or damage the company's reputation, the person accepting that risk needs enough authority to own those consequences.This is where governance becomes real.Because risk acceptance shouldn't mean:"Security sent an email three months ago and nobody objected."Nor should it mean:"We had a meeting and everybody seemed reasonably comfortable."Someone should know:This is the risk.These are the controls.This is what remains.This is the potential impact.And this is the person accepting it.That doesn't need to become bureaucratic.Actually, it shouldn't.But it does need to be explicit.Because ambiguity about risk ownership is remarkably comfortable right up until the day the risk materializes.Then everybody suddenly becomes very interested in the meeting minutes.The Data Owner Still Owns the DataEpisode 1 talked quite a bit about data.And there's a reason for that.AI hasn't made data ownership less important.It's made it more important.If an AI system needs access to HR data, who decides whether it should have access?HR and the appropriate data owner need to be involved.If it needs customer financial information?Same principle.Sensitive engineering information?Same principle.Cybersecurity shouldn't be deciding whether the business is allowed to use its own data.Our job is to make sure the data is appropriately protected.There's a subtle but important difference there.Security might say:"If you give this AI system access to this repository, here are the risks."The data owner should be saying:"Yes, this system has a legitimate business reason to access that information."Those are complementary decisions.And this becomes particularly important with enterprise AI because access can become incredibly broad.We've spent years building enormous repositories of information.SharePoint.Data lakes.Document management platforms.Collaboration tools.Email.Databases.And now we're putting intelligent interfaces over the top of them.Which is wonderful.Until somebody asks:"Why can the intern ask the AI about the acquisition strategy?"At which point the AI governance meeting becomes considerably more interesting.Again, the AI didn't necessarily create the access problem.It exposed it.So data ownership has to be part of AI governance from the beginning.Not invited in three weeks before launch because somebody remembered there might be data involved.There is always data involved.It's AI.That's rather the point.What Does the CISO Actually Own?So if the CISO doesn't own all AI risk, what should the CISO own?I'd start with cybersecurity risk management around AI.Understanding how AI changes the threat model.Protecting AI systems.Protecting the identities used by AI.Protecting data from unauthorized access.Monitoring for misuse.Assessing third-party AI security.Understanding new attack techniques.Making sure incident response can actually deal with an AI-related event.And helping the organization understand the cybersecurity consequences of the decisions it's making.That's plenty.Trust me.We don't need to volunteer for everything else.The CISO should also have meaningful influence over AI architecture and governance.Because cybersecurity decisions made after the architecture has already been selected tend to be rather more expensive.I would much rather have security involved when somebody says:"We're thinking about building an AI agent that can access these five systems."Than six months later when somebody says:"We've built an AI agent that can access these five systems. Can you approve it by Friday?"Those are very different meetings.So security should be involved early.Security should challenge.Security should advise.Security should establish appropriate controls.And in some circumstances, security absolutely should have the authority to stop something.If a system presents an unacceptable cybersecurity risk, somebody has to be able to pull the handbrake.But that doesn't mean the CISO becomes the owner of the business decision itself.Influence isn't the same as ownership.And governance works better when we're clear about the difference.What About the CIO?Then there's the CIO.AI is technology.So perhaps the CIO owns it.Well...partly.The CIO may own the enterprise AI platform.Technology architecture.Integration.Infrastructure.Enterprise applications.Vendor relationships.Service management.Operational resilience.Maybe data platforms.Depending on the organization, quite a lot of the technical environment supporting AI may sit with the CIO.But again, owning the platform doesn't mean owning every use of the platform.Microsoft doesn't own the business decision you make using Excel.And the CIO shouldn't own every decision somebody makes using Copilot.The technology organization should provide safe, reliable, governed capabilities.It should make the approved path easy.It should understand what technology is entering the environment.And it should make sure AI isn't simply becoming another layer of technical debt that somebody will be trying to untangle five years from now.Although, based on thirty years in IT, I wouldn't completely rule that out.But the CIO's role is enabling the organization through technology.The business still has to own what it does with that technology.That's another reason I like the idea of shared risk ownership.The CIO owns important parts of the environment.The CISO owns important security responsibilities.Data owners own their data.Legal owns legal interpretation.Privacy owns privacy expertise and governance.The business owns the outcome.And somebody at the executive level needs accountability for making sure those pieces actually fit together.Do We Need a Chief AI Officer?Which inevitably brings us to the Chief AI Officer.Do organizations need one?Maybe.There are organizations where that role makes perfect sense.Particularly where AI is strategically significant enough to require dedicated executive leadership.But I'd be careful about assuming that creating a Chief AI Officer solves AI governance.It might.It might also create another organizational silo.A Chief AI Officer can coordinate strategy.Drive adoption.Establish standards.Help prioritize investment.Coordinate governance.Build capabilities.And provide executive accountability.All useful things.But if everybody else responds by saying:"Excellent. AI belongs to them now."We've probably gone backwards.The existence of a Chief AI Officer doesn't remove the CISO's security responsibilities.It doesn't remove the CIO's technology responsibilities.It doesn't remove the data owner's responsibilities.And it certainly doesn't remove the business owner's responsibility for business outcomes.One of the dangers with new executive titles is that we sometimes confuse creating a role with solving a problem.Organizations do this quite well.Problem identified.Executive appointed.Organization chart updated.Problem presumably solved.If only it were that easy.The better question isn't:"Do we have a Chief AI Officer?"It's:"Do we have clear accountability for AI strategy and governance?"Those aren't necessarily the same thing.The Committee ProblemAnd yes, eventually we're going to have to talk about the AI committee.I'm not against committees.I've actually served on quite a few.Possibly too many.Committees can be extremely useful when a decision genuinely requires multiple perspectives.And AI absolutely does.The problem comes when the committee becomes the owner.Because committees don't really own risk.People do.Imagine an AI governance committee with representatives from technology, cybersecurity, legal, privacy, data and the business.Excellent.Now imagine a high-risk use case comes before them.Everyone discusses it.Everyone raises concerns.The minutes say:"The committee reviewed the risks."And the project proceeds.Who accepted the risk?The committee?Which person?Who has the authority?Who owns the consequence?If nobody can answer that, the committee hasn't created accountability.It's diluted it.So I like governance forums.I like cross-functional AI councils.I like bringing the right people together.But every significant decision should eventually resolve to an accountable person.Not a box on an organization chart.Not a committee.Not "IT."A person.Because when accountability is shared by twelve people, it has an unfortunate tendency to be owned by none of them.Don't Build an AI Governance IslandThis also connects to something we touched on in Episode 1.Don't create an entirely separate governance universe for AI unless you genuinely need one.Organizations already have risk processes.Cybersecurity risk.Third-party risk.Privacy assessments.Architecture review.Data governance.Model risk.Change management.Legal review.Procurement.Business continuity.Audit.Compliance.The answer shouldn't automatically be to duplicate all of those because the technology now contains AI.Imagine you're buying an enterprise application.Historically, it goes through procurement, security review, privacy review, architecture and legal.Now the vendor adds an AI assistant.Do we suddenly create five completely different AI processes?Hopefully not.We should ask:What changed?What new risks exist?Which existing controls still apply?Where are the genuine gaps?Then extend the governance model.Don't rebuild the entire thing.I said this in Episode 1, and I think it's worth repeating.Govern the risk, not the buzzword.AI is going to become embedded in almost every technology category.Eventually, having a completely separate AI review for everything may become as strange as having a separate "internet-enabled technology committee."We're probably not there yet.AI is still different enough to deserve additional attention.But the direction should be integration.Not permanent isolation.Risk Should Follow the Use CaseHere's a simple way I think about this.Risk should follow the use case.Not every AI application deserves the same governance.Suppose I use an approved enterprise AI tool to rewrite a paragraph I've already written.Low consequence.Now suppose HR uses AI to rank job applicants.Very different consequence.Suppose marketing generates an image for an internal presentation.Different again.Suppose finance uses AI to recommend whether a customer receives credit.Now we're getting more serious.Suppose an autonomous AI agent can change supplier banking information and initiate a payment.Right.I'm going to want a few more questions answered before we switch that on.Same broad technology.Completely different risk.So governance should scale according to things like:What data is involved?What decisions are being made?Who is affected?Can the AI take action?Is a human involved?Can the action be reversed?What's the financial impact?What's the regulatory impact?What's the consequence if the AI is wrong?That's much more useful than categorizing everything as:"AI equals high risk."Because if everything is high risk, eventually nothing is.People stop taking the classification seriously.And then we've created governance theatre.Which is governance that looks extremely impressive right up until you need it to govern something.The Board's RoleThere's one more group we should talk about.The board.Boards are increasingly asking questions about AI.And they should.But I don't think boards need to understand every technical detail of transformer models, embeddings, vector databases or prompt injection.Unless they particularly want to.In which case, clear the afternoon.What boards need is visibility into material risk.Where is AI strategically important?What are the significant use cases?What could create material harm?What controls exist?Who is accountable?What is the organization's risk appetite?And are we learning from incidents and near misses?The board's role isn't to approve every AI application.It's oversight.Management runs the organization.The board needs enough information to understand whether management is doing that responsibly.And I think one of the most useful things technology and security leaders can do is translate AI risk into business language.Not:"We have a prompt injection vulnerability involving an LLM-connected retrieval architecture."Try:"An attacker may be able to manipulate the AI into accessing information or taking actions we didn't intend."Now we're having a conversation.Technical precision matters.But so does being understood.If your board needs a glossary before they can understand the risk, we may need to work on the presentation.A Simple Ownership ModelSo if I were sitting down with an organization tomorrow and trying to clarify AI risk ownership, here's where I'd start.First:Who owns the business outcome?That's usually the business sponsor.Second:Who owns the data?Someone should be accountable for deciding whether that data can be used.Third:Who owns the technology?The platform, architecture, integration and operational environment need clear ownership.Fourth:Who owns the cybersecurity risk assessment?Usually security.Not necessarily the risk itself, but the assessment, advice and relevant controls.Fifth:Who owns privacy, legal and regulatory interpretation?The appropriate specialists.Sixth:Who can accept the residual risk?This is critical.That person needs sufficient authority for the potential consequence.And seventh:Who can stop it?If something changes.If a vulnerability appears.If the AI behaves unexpectedly.If the vendor changes the model.If we discover data is being used incorrectly.Who has the authority to say:"Turn it off."If those seven things are clear, you're already a long way toward sensible AI governance.Notice I haven't mentioned creating a seventy-page framework.We can always do that later if we're feeling nostalgic for traditional enterprise governance.Start with accountability.The MultiplierAnd now, this week's Multiplier.Pick one real AI use case in your organization.Not a theoretical one.Something somebody is actually using or planning to deploy.And ask five questions.Who owns the business outcome?Who owns the data?Who can accept the risk?Who can stop the use case?And who gets the phone call when it goes wrong?You may get five different names.That's okay.Actually, that's probably healthy.AI risk doesn't need one owner.It needs clear owners for the different risks.The problem is when the answer to one of those questions is:"I'm not sure."Or worse:"We thought security owned that."Don't create a new committee.Don't write a new policy.Not yet.Just take one real use case and make the accountability clear.Then do another one.That's this week's Multiplier.ClosingSo, who actually owns AI risk?The slightly unsatisfying answer is:It depends on the risk.But that's also the useful answer.AI isn't a single risk category.It's a technology that can create or amplify many different kinds of risk.The business owns business outcomes.Data owners remain accountable for their data.Technology owns the platforms and architecture it operates.Cybersecurity owns its security responsibilities.Privacy, legal and compliance bring their respective expertise and authority.Executive leadership establishes direction and risk appetite.And the board provides oversight of material risk.What matters is that those responsibilities connect.Because AI lives in the gaps between traditional organizational structures.And risk has a habit of finding those gaps.The goal isn't to find one executive and hand them everything with the letters A and I attached to it.The goal is clear accountability.People should know what they own.They should have the authority to manage it.And when a risk needs to be accepted, somebody should actually accept it.Preferably before the incident.That tends to make the meeting afterwards considerably shorter.That's it for this episode of AI Security Multipliers.If you enjoyed the episode or found it useful, please follow or subscribe to the show. And if you know somebody wrestling with the same questions, send it their way.I'd also really like to hear from you.If there's something you disagree with, a topic you think we should explore, or an AI or cybersecurity problem you're dealing with in your own organization, send it to me.Some of the best ideas for this podcast will hopefully come from the people listening to it.And if you tell me it's all the CISO's responsibility...I may still read your message.I'm Steve.And I'll see you next time.