Your AI knows too much!

Episode transcript

Transcript provided for accessibility and search. Minor transcription errors may remain.

— Introduction: Your AI Knows Too Much

SPEAKER_00 0:02

This is AI Security Multipliers. AI Cybersecurity Governance with Steve. This week, your AI Knows Too Much. Now that A, I can find information faster than ever before, including information you probably forgot was there. So does that create a new security problem? Or is it exposing the one we've been ignoring for years? To discuss and help with this one, here's Steve.

SPEAKER_01 0:28

There

— Your AI Knows Too Much

SPEAKER_01 0:29

is a slightly uncomfortable moment happening in a lot of organizations right now. Someone deploys an enterprise AI tool. Maybe it's Microsoft 365 co-pilot. Maybe it's another AI assistant connected to corporate information. People start using it. And then somebody asks it a question. And it finds something. Something they weren't expecting it to find. Maybe it's an old strategy document. Maybe it's financial information. Maybe it's an acquisition plan. Maybe it's a spreadsheet containing salaries. Maybe it's an executive presentation from four years ago that everybody assumed had disappeared into the corporate equivalent of a sock drawer. And suddenly there's panic. The AI found confidential information. Right. But here's my question. Could the person who asked the question already access that information? Because if the answer is yes, we've got to be very careful about how we describe the problem. The AI didn't necessarily leak anything. It found something. And those aren't the same thing. I've seen this problem in large organizations for years. Long before Generative A I appeared. Organizations accumulate information, SharePoint sites, Teams channels, file shares, OneDrive folders, old project sites, email attachments, PDFs, spreadsheets, copies of spreadsheets, copies of copies of spreadsheets, and inevitably, final.xlsx, final dash vi two dot xlsx, final dash v2 dash really dash final.x lsx. And my personal favorite? Final dash v2 dash really dash final dash use dash this dash one dot x lsx. Somewhere in there is probably your corporate strategy. For years we got away with fairly poor information governance because finding information was difficult. And that created a strange kind of accidental security control. Nobody could find anything. Then AI arrived. And AI is really, really good at finding things. Which leads me to one of the main ideas I want to explore today. AI often doesn't create your data security problem. It reveals the one you already had. And I actually think that could turn out to be one of the best things that's happened to information governance in years. Let's use

— AI Didn't Give You Access

SPEAKER_01 2:47

Microsoft 365 Copilot as an example. Not because this is an episode about Microsoft, it isn't. The same fundamental issue applies to any enterprise AI system that can retrieve information from corporate data. But Copilot makes the problem very easy to understand. At a simplified level, Copilot works within the permissions of the person using it. If you don't have access to a document, Copilot doesn't magically grant you access to it. That's important. Because sometimes you'll hear a story like, We deployed Copilot, and an employee found this confidential document. And my first question would be, why could the employee access the document? That's the security problem. Now, there's an important nuance here. AI absolutely changes the risk. Because accessibility and discoverability are not the same thing. Imagine there's a document buried seven folders deep in an old SharePoint site. Technically you've had permission to read it for six years. But you didn't know it existed. You didn't know what it was called, you didn't know where it lived. And unless you were prepared to spend Thursday afternoon rummaging around old SharePoint sites, you probably weren't going to find it. AI changes that. You can ask, what are the company's plans for Project Phoenix? And suddenly the system can potentially find relevant information across multiple sources and summarize it for you. The permission hasn't necessarily changed, but the friction has, and that matters a lot. Security professionals need to understand this distinction, because saying, well, the user already had permission, so there's no additional risk, isn't quite right either. AI changes the economics of information discovery. It takes something that was technically accessible and makes it practically accessible at scale. In seconds. And sometimes summarizes the interesting bits for you. That's a meaningful change in risk. But I still wouldn't describe that primarily as an AI security failure. I describe it as AI exposing an information governance failure. Because the fundamental question remains, why did that person have access in the first place? This is

— RAG Without the Computer Science Lecture

SPEAKER_01 4:57

also where we need to talk briefly about RAG, retrieval augmented generation. And I promise we're not going to spend the next 10 minutes drawing architecture diagrams. RAG is actually a fairly simple concept. A normal language model has learned from information during training, but it doesn't automatically know what's in your organization's latest documents. So you give the AI a way of retrieving relevant information. The user asks a question. It finds relevant material, it provides that material to the model, and the model uses it to construct an answer. That's the simplified version. Obviously, there is considerably more engineering underneath it, because apparently nothing in technology is allowed to remain simple once consultants discover it. But conceptually, that's what we're doing. And it's incredibly powerful. Think about what this means inside a large organization. Instead of remembering which SharePoint site contains the document, you ask. Instead of finding the policy, you ask. Instead of searching through fifty pages of documentation, you ask, instead of trying to remember what somebody decided in a project three years ago, you ask. This is exactly the kind of thing I think AI should be doing. This is a multiplier. We're reducing the time people spend searching for information and increasing the time they spend actually using it. Fantastic. But there's a catch. The quality of the security around that system depends enormously on the quality of the information environment underneath it. If your permissions are a mess, I now has a very efficient interface to your mess. If your information classification is poor, I doesn't magically fix that. If you've got abandoned sites containing sensitive information, I may make them useful again, unfortunately. And if everybody has access to everything, because someone clicked everyone six years ago because it was easier. Congratulations, you have democratized information. Possibly rather more than you intended.

— Please Don't Wait Until Your Data Is Perfect

SPEAKER_01 6:58

Now this leads to advice I've heard quite a lot. Before you deploy AI, you need to clean up all your data. I understand the thinking, I really do. But in a large enterprise? Best of luck with that one. You could start cleaning today and finish sometime around the point we're deploying Copilot on Mars. Large organizations have decades of information, millions of documents, thousands of sites, permissions inherited from permissions inherited from permissions. People who left the company seven years ago. Project groups nobody remembers creating. Old business units, acquisitions, divestitures, shared folders, personal folders. And that one SharePoint site apparently owned by someone called Dave. Nobody knows who Dave is, but Dave owns everything. Waiting until all of that is perfect before deploying. AI isn't a strategy, it's procrastination with a governance framework. And I think this is where security teams can sometimes get themselves into trouble. We see a new risk. We decide the safest answer is to stop. No AI until the data is clean. No co-pilot until permissions are perfect. No enterprise search until everything is classified. That sounds responsible, but the business isn't going to wait. People are going to use AI. If the organization doesn't provide a safe, governed way of doing it, employees will find another way. Personal accounts, browser extensions, consumer AI services, copy and paste, upload, enter. And now instead of dealing with Enterprise A, I operating inside your security controls, you're dealing with corporate information leaving them. That's not an improvement. Security cannot become the department of no. Our job isn't to stop people using useful technology. Our job is to help the organization use it safely. So I would take a different approach. Don't try to clean everything, find what matters most. Where is your most sensitive information? Who can access it? Where do you have excessive permissions? Which repositories contain financial data? Employee information, customer information, intellectual property, board material, acquisition information, credentials, security documentation. Start there. Reduce the biggest exposures. Use classification where it helps. Use data loss prevention, fix obviously excessive permissions, remove stale access, review high-risk repositories, then deploy progressively. Measure what happens, learn, improve, repeat. That is considerably less satisfying than announcing a three-year enterprise information governance transformation program. But it has the advantage of possibly working.

— AI Might Actually Fix Information Governance

SPEAKER_01 9:44

Here's the part where my opinion may be slightly controversial. I think AI might be one of the best things to happen to information governance. Not because AI magically fixes information governance, it doesn't. But because AI makes bad information governance impossible to ignore. For years information governance has been one of those things organizations know they should improve. We need better classification. We need better retention. We need to clean up old sites, we need to review permissions, we need better data ownership. Absolutely, put it on the roadmap, and then something more urgent happens, and something else, and something else. And five years later, the SharePoint environment looks like an archaeological dig. AI changes the incentive. Because suddenly the quality of your data environment directly affects one of the technologies your organization most wants to use. Now executives care. Now business leaders care, now employees care. Because the payoff isn't just better compliance, the payoff is better AI, and that's powerful. Good information. Governance improves the quality of what AI can retrieve. Good identity governance reduces inappropriate access. Good classification helps us understand sensitivity. Good retention reduces unnecessary information. Good data ownership creates accountability. Good security architecture allows us to connect AI to useful information without simply opening the doors and hoping for the best. So rather than viewing enterprise AI as the enemy of information governance, maybe we should use it as the catalyst. And I've seen enough large technology environments to know that sometimes you need a catalyst. Policies alone don't change organizations. Neither do PowerPoint slides. Although we do remain extremely committed to trying. So here's

— The Multiplier: Ask Your AI a Question You Don't Want It to Answer

SPEAKER_01 11:39

this week's multiplier. Ask your AI question you don't want it to answer, not recklessly, and preferably in a controlled environment. But think about the information inside your organization that would concern you if the wrong employee could easily discover it. And then test for it. Can someone outside finance find sensitive financial information? Can someone outside HR discover employee information? Can an ordinary employee find executive documents? Can people discover old project information that should have been restricted or deleted? Don't just ask, can copilot access our data? That's not particularly useful. Ask what can copilot make dramatically easier to discover? That's a much better security question. Then take the results and fix the underlying problem. The permissions, the classification, the retention, the ownership, the data. Because if AI finds something a user shouldn't be able to see, don't just blame the AI. Ask why the door was open. There's

— The Problem Was Already There

SPEAKER_01 12:44

a broader lesson here. Every major technology shift exposes weaknesses in what came before it. Cloud exposed weaknesses in perimeter security. Remote work exposed weaknesses in identity. Ransomware exposed weaknesses in recovery. And AI is exposing weaknesses in information governance. That's uncomfortable. But it's also useful. Because now we can see them. The organizations that get the most value from A, I won't necessarily be the ones with perfect data. I'm not convinced those organizations exist. They'll be the ones that understand their risks well enough to move forward anyway. Protect the information that matters, control who can access it, watch what your AI can retrieve, give employees safe tools, improve continuously, and don't spend three years waiting for perfection while everybody quietly uses ChatGPT on their phone. AI didn't create decades of poor permissions. It didn't create abandoned SharePoint sites. It didn't create duplicate files. It didn't decide that everyone needed access to everything. We did that ourselves. And perhaps that's exactly the push we needed. I'm Steve. This is AI Security Multipliers. And remember, AI should multiply your capability, not your mistakes. I'll see you next time.