Your Data Is the Real AI Security Boundary

Episode transcript

Transcript provided for accessibility and search. Minor transcription errors may remain.

AI Security MultipliersEpisode 3: Your Data Is the Real AI Security BoundaryHost: SteveYour Data Is the Real AI Security BoundarySteve: For years, enterprise security had an accidental control that nobody really talked about.Information was difficult to find.I'm serious.We had enormous file shares.SharePoint sites.Email archives.Collaboration platforms.Data lakes.Document management systems.And folders nested inside folders that appeared to have been designed by someone who genuinely disliked other human beings.There was sensitive information everywhere.But finding the right piece of information often required knowing where it was, what somebody had called it, which system contained it, and occasionally remembering the name of somebody who left the company six years ago.It wasn't exactly sophisticated security.But it did create friction.AI changes that.Suddenly, instead of searching through twenty different places, I can simply ask a question.And the AI can potentially find the information, understand it, summarize it, combine it with other information and present the answer back to me in seconds.That's incredibly useful.It's also a fairly significant change to the security model.Because AI doesn't just change what people can access.It changes what they can discover.I'm Steve.And this is AI Security Multipliers.Today we're going to talk about why, in an AI-enabled enterprise, your data may be becoming the most important security boundary you have.The Old Security BoundaryTraditionally, we've thought about security boundaries in fairly familiar ways.The network perimeter.The endpoint.The application.The cloud environment.The database.And, increasingly, identity.We put controls around systems.We authenticated users.We assigned permissions.And we tried to make sure people could only access the applications and information they needed.That model hasn't disappeared.We still need all of those things.But AI introduces something different.An AI assistant can potentially sit across multiple systems at once.It may search email.Documents.Meetings.Chats.Databases.Customer information.Internal knowledge bases.And external sources.From the user's perspective, those systems begin to disappear.They don't necessarily think:"I'm now querying SharePoint."They ask:"What did we decide about Project Atlas?"Or:"Summarize everything we've discussed with this customer."Or:"What were the concerns raised about this acquisition?"The AI figures out where the information lives.That's part of the magic.And part of the risk.Because application boundaries that previously separated information become much less visible to the person asking the question.The interface becomes the AI.But underneath that interface are all the permissions, identities, data repositories and security decisions we've accumulated over years.AI didn't remove those controls.It made them considerably easier to exercise.AI Changes DiscoveryThis distinction between access and discovery is really important.Imagine an employee technically has access to ten thousand documents.Maybe twenty thousand.Maybe considerably more.That's not particularly unusual in a large organization.They're members of security groups.Those groups have access to SharePoint sites.They worked on previous projects.They changed jobs.They joined teams.Someone shared a folder with them four years ago.And, as tends to happen, nobody removed the access afterwards.Before AI, what could they actually do with all of that?Well, they could access it.But first they had to know it existed.Then they had to find it.Then open the documents.Then read them.Then understand how they related to each other.That's friction.Now give that employee an AI assistant that can search across everything they're permitted to access.They can ask:"What projects are we considering cancelling next year?""Which customers have the lowest margins?""Summarize executive discussions about restructuring.""Show me documents discussing potential acquisitions."They may not know any of those documents exist.The AI does the discovery for them.And that's the fundamental shift.The permission hasn't changed.The value of the permission has.We've taken thousands of individually inconvenient permissions and given somebody an intelligent interface capable of using them at machine speed.For years, one of our more effective data security controls was that nobody could actually find anything.It wasn't in NIST.But it was surprisingly effective.AI is removing that control.Permission Isn't the Same as IntentThis leads to one of the ideas I think organizations need to get comfortable with.Permission isn't the same as intent.If I technically have permission to access a document, that tells us something about the security configuration.It doesn't necessarily tell us whether the organization intended me to use that information today.Perhaps I needed it three years ago.Perhaps somebody added me to a project group.Perhaps access was inherited from another group.Perhaps an entire SharePoint site was opened up because somebody couldn't work out which permission was preventing five people from accessing one document.I'm sure that has never happened.But hypothetically.Now imagine thousands of those decisions accumulating over ten or fifteen years.Each one may have seemed fairly harmless.Then AI arrives.And suddenly those historical access decisions can be exercised simultaneously through natural language.That's why I don't think organizations can simply say:"The AI only shows users information they already have permission to see, therefore we're secure."Technically, that statement may be completely correct.Governance-wise, it may be missing the point.The better question is:Should the person still have access to the information?AI forces us to revisit the difference between what the system technically allows and what the organization actually intends.And that's a much more interesting conversation.The SharePoint ProblemLet's talk about SharePoint.Not because SharePoint is uniquely bad.It's actually an incredibly useful platform.But it provides a wonderful example of what happens when collaboration, permissions and time all meet each other.Organizations have been using SharePoint for years.Projects create sites.Departments create sites.Teams create sites.People share documents.Security groups get created.Links get forwarded.Employees change jobs.Projects finish.Organizations restructure.And the information remains.SharePoint has an extraordinary ability to preserve organizational history.Unfortunately, that sometimes includes documents everybody assumed disappeared sometime around 2017.Now put an AI assistant over the top.The AI doesn't care that the document is old.It doesn't know that everyone forgot the site existed.It sees information the user is authorized to access.So when organizations deploy something like Microsoft 365 Copilot, one of the first uncomfortable discoveries can be that the AI is surfacing information people didn't realize they could access.That's not necessarily an AI failure.In fact, the AI may be working perfectly.The underlying permission model is what's surprising you.I've seen this firsthand at meaningful enterprise scale.And one of the lessons is very simple.The AI didn't create the permission problem. It made the existing problem useful. And potentially dangerous.That's why AI readiness has to include data readiness.Otherwise you're putting a very powerful discovery engine over years of accumulated permission decisions and hoping nothing interesting happens.Hope, as usual, is not much of a control.Identity Becomes the QueryThere's another way to think about this.When somebody asks an enterprise AI assistant a question, their identity effectively becomes part of the query.The question isn't simply:"Find information about Project Atlas."It's really:"Find information about Project Atlas that Steve's identity is permitted to access."That's a security decision.Every time.Which means identity governance becomes deeply connected to AI governance.Who is the user?What groups are they in?What role do they have?What systems can they access?What documents have been shared with them?What permissions did they inherit?What should have been removed when they changed jobs?And soon we have another complication.AI agents.Because now the identity asking the question may not be a person.It may be software acting on behalf of a person.Or an autonomous agent with its own identity and permissions.We'll go much deeper into that in another episode.But the principle is already important.AI makes identity more powerful because identity increasingly determines the universe of information the AI can reason over.So when somebody says:"We're deploying enterprise AI."One of my first questions is:"How confident are we in identity and access governance?"Because if the answer is:"Well..."We've probably just found part of the AI program.Data Classification Matters AgainSecurity teams have been talking about data classification for decades.Public.Internal.Confidential.Restricted.Highly confidential.Choose your preferred labels.Then we build policies.We run awareness training.We deploy tools.And somewhere in the organization there is inevitably a PowerPoint deck explaining the difference between Confidential and Highly Confidential that has itself been classified Internal.AI makes classification much more important.Because if we're going to allow intelligent systems to search, summarize and act on enterprise information, we need some understanding of what that information actually is.Is it customer data?Employee data?Financial information?Intellectual property?Source code?Legal material?Board information?Merger and acquisition information?Not every document needs a label.And I don't think the answer is asking employees to make seventeen classification decisions every time they save a PowerPoint.That tends not to end well.But organizations need enough data intelligence to understand where sensitive information exists and how it's being used.Increasingly, that means automation.Content inspection.Sensitivity labels.Data discovery.Pattern matching.Context.DLP.And, where appropriate, AI helping classify the data that AI itself may later consume.Which is slightly circular.But welcome to technology.The point is that data classification can no longer just be a compliance exercise.It becomes part of how we determine what AI should be allowed to see and what controls should follow the information.Somebody Still Has to Own the DataAnd classification brings us straight back to Episode 2.Ownership.Someone has to own the data.Technology may operate the platform.Security may define controls.Privacy may establish requirements.Legal may interpret obligations.But somebody in the business should understand what the data means and who should legitimately use it.That's the data owner.At least in theory.In practice, ask an organization who owns a particular SharePoint site containing eight years of business information and you may get a rather more philosophical discussion.The person who created it left.Their manager retired.The department was reorganized twice.And apparently Dave knows something about it.Dave is on holiday.This is precisely the sort of governance debt AI exposes.If nobody knows who owns the information, who decides whether an AI system should use it?Who decides how long we keep it?Who decides whether access is appropriate?Who understands the consequence if it's disclosed?AI governance can't fix weak data ownership by itself.But it can provide a very good reason to finally address it.And I think that's a recurring theme with AI.It creates new risks.Absolutely.But it also shines an extremely bright light on old ones.Oversharing Is Different When a Machine Can Read EverythingWe should also talk about oversharing.People hear that word and sometimes imagine somebody deliberately putting confidential information in a public folder.That happens.But enterprise oversharing is usually much less dramatic.It's a folder shared with a broad group.A site accessible to everyone internally.A permission inherited from a parent location.A link configured more broadly than intended.A team that accumulated members over several years.Individually, these things may not have caused obvious incidents.Again, because finding the information required effort.AI changes the economics.A person has limited time.They can read only so many documents.An AI system can process enormous quantities of information and identify relationships humans would probably miss.It can combine one document with another.A salary spreadsheet with an organizational chart.A project plan with meeting notes.A customer list with pricing information.Each individual piece may not look particularly sensitive.Combined, they may tell a very different story.That's another reason the data boundary matters.We can't think only about individual files.We also need to think about what AI can infer by combining information.Because the security question is no longer simply:"Can this person open this document?"It may be:"What could an intelligent system learn from everything this person can access?"That's a much bigger question.RAG Doesn't Magically Make Things SafeNow let's introduce another acronym.Because clearly cybersecurity and AI didn't have enough already.RAG.Retrieval-Augmented Generation.At a simple level, RAG allows an AI model to retrieve relevant information from trusted sources and use that information when generating an answer.It's extremely useful.Instead of relying entirely on what the model learned during training, you can connect it to your own enterprise knowledge.Policies.Product documentation.Customer information.Technical documents.Knowledge bases.Whatever is appropriate.This can make AI dramatically more useful inside a company.But I've occasionally heard RAG discussed almost as if connecting the model to an approved data source makes the security problem disappear.It doesn't.You still need to ask:What information is in the source?Who can retrieve it?How are permissions enforced?Can users retrieve information they shouldn't?Can the model expose information through an unexpected prompt?What gets logged?What happens to the retrieved data?Can the system take actions based on it?RAG can help control where information comes from.That's valuable.But the security of the system still depends heavily on the security of the data underneath it.If you connect an AI system to a badly governed repository, you've built a very sophisticated interface to a badly governed repository.The acronym hasn't fixed it.What About Public AI?Of course, not all the data risk comes from enterprise AI searching internal systems.Sometimes the data is going the other direction.Employees put it into external AI services.This was one of the earliest enterprise AI concerns.Someone copies customer information into a chatbot.Or source code.Or financial information.Or a contract.Or a confidential strategy document.Usually not maliciously.They're trying to get their job done.They want the AI to summarize something.Rewrite something.Analyze something.Find an error.And the AI is useful.That's why simply telling employees:"Don't use AI."was never going to be a particularly durable strategy.If a technology saves somebody two hours of work, they're going to be quite motivated to find a way to use it.So organizations need approved tools.Clear rules.Technical controls.And education that explains why certain information shouldn't be placed into certain systems.This is where enterprise agreements, data handling terms, retention controls and vendor architecture matter.Not every AI service handles data in the same way.And the answer shouldn't be based on whether the logo on the website looks reassuring.Understand the service.Understand the contract.Understand how your data is used.Then make a risk decision.DLP Is Useful, But It Isn't MagicThis is where technologies like Data Loss Prevention become important.DLP can identify sensitive information.It can monitor movement.Block certain actions.Warn users.Apply policy based on data type.And increasingly, security platforms can apply those controls across AI interactions as well.That's valuable.But DLP isn't magic.Anyone who's worked with it knows there is a delicate balance between:"Protect the sensitive information."and:"Please stop preventing everyone from doing their jobs."Too loose, and it doesn't protect enough.Too aggressive, and employees discover entirely new and creative ways to move information around your controls.So I see DLP as one layer.Not the strategy.The strategy starts with knowing what data matters.Who owns it.Who should access it.Where it can go.How long it should exist.And what happens when those rules are violated.Then technology helps enforce that.If we start with the tool and skip the governance, we tend to end up with thousands of alerts and a security team slowly losing the will to live.Technology is a multiplier.But it multiplies the quality of the process underneath it.Sometimes that's good.Sometimes it's impressively efficient chaos.Don't Respond by Locking Everything DownNow, after hearing all of this, there's an understandable security reaction.Lock everything down.Remove access.Restrict AI.Block connectors.Disable features.Make everyone request permission.Preferably through a form.We do like forms.But that's not the goal.The purpose of enterprise AI is to make information more useful.If we secure it so aggressively that nobody can actually use it, we've rather missed the point.Security needs to preserve the business value while reducing unreasonable risk.That means appropriate access.Not minimum access to the point of absurdity.If someone legitimately needs information to do their job, AI should help them use it.That's the opportunity.The challenge is distinguishing legitimate access from accumulated access.And those are not the same thing.This is where concepts like least privilege, role-based access, attribute-based controls and just-in-time access become increasingly useful.But we also need pragmatism.You won't clean up twenty years of enterprise permissions before deploying AI.If that's the entry requirement, we'll all be retired before the pilot begins.Prioritize.Find the most sensitive repositories.Find the broadest access.Find the highest-risk AI use cases.Start there.Governance should make AI adoption safer.Not impossible.What Should the CISO Do?So what does this mean for the CISO?First, don't treat enterprise AI purely as an application security problem.Look underneath the application.Look at identity.Permissions.Data repositories.Classification.Data flows.Third parties.Logging.Retention.And the actions the AI can perform.Second, work closely with data governance.Historically, cybersecurity and data governance have sometimes lived in slightly different worlds.AI is rapidly removing that luxury.Third, understand the actual user experience.Don't evaluate AI only through architecture diagrams.Sit down with the tool.Ask questions.See what it retrieves.See what it summarizes.Understand what a normal employee can actually do.I've found that remarkably educational.Architecture diagrams are wonderful.But they rarely say:"By the way, Steve in Finance can apparently see the confidential strategy document."The interface sometimes tells you rather more.And finally, don't make security the department of AI prohibition.Help create the safe path.Because if the approved AI experience is useful, accessible and reasonably frictionless, employees have much less incentive to find an unapproved one.That's good security.And, conveniently, good business.What Do You Do on Monday?So what should you actually do?Start with the AI tools already in your environment.Not the ones on the three-year strategy slide.The ones people are using now.Understand what data each tool can access.Then identify the most sensitive repositories connected to those tools.HR.Finance.Legal.Executive information.Customer data.Intellectual property.Whatever matters most in your organization.Look for broad access.Old groups.Orphaned sites.Stale permissions.Externally shared information.And data nobody appears to own.Don't try to fix everything.Rank it.What would create the greatest consequence if AI suddenly made it very easy to discover?Start there.Then test.Use ordinary accounts.Ordinary prompts.Normal business questions.You're not trying to jailbreak the model.You're trying to understand the experience you've intentionally given your employees.And if the AI finds something surprising, trace the answer backwards.What data did it use?Where was it stored?Why could the user access it?Who owns it?Was the access intentional?That's an AI security assessment that produces something useful.Not another generic list of AI risks.An actual problem you can fix.The MultiplierAnd now, this week's Multiplier.Pick one AI-enabled enterprise application.Then pick one ordinary employee account.Not the CISO.Not an administrator.Not somebody with deliberately broad access.A normal employee.Now ask the AI a few ordinary questions about information that employee probably doesn't need for their day-to-day job.Nothing malicious.Don't try to bypass controls.Don't jailbreak anything.Just ask sensible business questions.Maybe:"What confidential projects are being discussed?"Or:"What strategic changes are being considered?"Or choose something appropriate to your organization.If the AI finds something surprising, don't immediately blame the AI.Ask a much more useful question.Why could this employee access the data in the first place?Then follow the permission.Find the repository.Find the group.Find the owner.And decide whether that access is actually intentional.Do that with one use case this week.Because the quickest way to understand your AI data risk may simply be to use the AI like an employee would.That's this week's Multiplier.ClosingSo, is data really the new AI security boundary?I think increasingly, yes.Not the only boundary.We still need secure applications.Secure endpoints.Secure networks.Strong identities.Good architecture.All the usual things.But AI changes how people interact with information.It removes friction.It improves discovery.It connects information across systems.It summarizes enormous amounts of content.And increasingly, it can act on what it finds.That means old permission decisions suddenly matter more.Old documents matter more.Data ownership matters more.Identity governance matters more.Classification matters more.And understanding where sensitive information actually lives matters a great deal more.The important thing to remember is this:AI isn't just changing the threat landscape. It's changing the value of permissions you already granted.And that's why the answer isn't simply another AI security tool.It's better identity.Better data governance.Better ownership.Better access decisions.And security teams that understand how employees are actually using the technology.The organizations that do this well won't necessarily be the ones that lock AI down the hardest.They'll be the ones that make their data trustworthy enough, governed enough and secure enough that AI can actually make it useful.Because that's ultimately the goal.Not stopping AI.Using it where the opportunity is worth the risk.That's it for this episode of AI Security Multipliers.If you enjoyed the episode or found it useful, please follow or subscribe to the show. And if you know somebody wrestling with AI, data governance or access problems in their own organization, send it their way.I'd also really like to hear from you.If there's something you disagree with, a topic you think we should explore, or an AI or cybersecurity problem you're dealing with, send it to me.And if your organization has perfectly classified data, pristine permissions and nobody has access to anything they shouldn't...I'd particularly like to hear from you.Mostly because I'd quite like to visit.I'm Steve.And I'll see you next time.