Home
Tech Grid
News Room
Interviews
CISO POV
Think Stack
Articles
Home | CISO POV | When Security Meets Reality: Making Risk Decisions with Imperfect Information

When Security Meets Reality: Making Risk Decisions with Imperfect Information

You’ve worked across security engineering, consulting, technology leadership, advisory, and now the CISO seat. How has moving between these perspectives changed what you consider a genuinely difficult security problem?

I started out managing global IT for a publicly traded company based in Cambridge, with nine offices around the world. Then we were attacked. That experience shaped how I’ve viewed security ever since: the first question isn’t simply, “How serious is the vulnerability?” It’s, “What needs to happen to keep the business running?”

For example, during an attack, the most technically conservative response might be to shut down systems immediately. But if those systems support critical business operations, taking them offline can create its own serious harm. You have to quickly understand what’s affected, contain the attacker, preserve evidence, and keep the business operating safely enough while you make decisions with incomplete information.

Moving between engineering, consulting, vendor leadership, advisory roles, and the CISO seat has reinforced that the genuinely difficult security problems sit at the intersection of technology and business. What is the organization trying to accomplish? How could its use of technology go wrong? What would the business impact be? Which risks can we accept, and which must we mitigate?

Whatever role I’m in, my job is to understand the business, help its leaders see how technical risks could affect their plans, and give them practical options for addressing those risks.

When building or inheriting a security program, which assumptions about infrastructure, applications, identity, or threats do you challenge before introducing new controls?

All of them. An inherited control reflects a decision made at some point in the past. Its existence doesn’t prove that it’s still the right control, that it addresses the real risk, or even that it works.

Before assessing existing controls or introducing new ones, I want to understand what drives the business, what it needs to protect, how its systems operate, how identities are managed, its risk appetite, and what could materially interfere with its success. I’ll challenge assumptions such as whether the asset inventory is complete, whether access still matches people’s current roles, whether backups can actually be restored, and whether the threats in the risk register reflect how the company operates today.

For example, as a vCISO, I inherited a program that required security approval before every production change. In some environments, the potential impact of a failed or malicious change may justify that level of oversight. But at a technology startup deploying software many times a day, the same control was delaying patches and encouraging the team to batch changes. It was creating more risk than it reduced. The business still needed to prevent unauthorized or unsafe changes, but automated testing, peer review, branch protections, deployment monitoring, and rapid rollback could achieve that objective without obstructing delivery.

So I challenge not only whether a control works, but whether the business needs it, which risk it addresses, and whether it’s the right control for how the organization operates.

Compliance can create non-negotiable requirements. In a SOC 2 examination, you need evidence that the controls operated during the examination period. There may be flexibility in how a requirement is met, including through compensating controls, but you can’t explain away missing evidence. If a standard control meets the requirement without creating disproportionate friction, implementing it may be the most effective path.

How do you communicate risk when the evidence is incomplete, particularly with AI systems whose behavior, dependencies, and attack paths can change faster than governance processes?

CISOs deal with incomplete evidence all the time during incident response. We may know there’s been an alert or an incident, but we often can’t tell leadership the root cause or even the extent of the damage until the investigation is complete.

That experience teaches you to be honest about uncertainty. I explain what we know, what we don’t know, how confident we are, what we’re doing, and what evidence could change our assessment. Transparency gives leadership enough information to make decisions without pretending the evidence is more complete than it is.

I take the same approach with AI to separate demonstrated capability from speculation. There are alarming headlines about agents escaping sandboxes and bot swarms potentially bringing down the internet. I focus conversations with boards on the technical reality behind those headlines: what has actually been demonstrated, what conditions made it possible, and how far that evidence is from the scenario being predicted.

Then I bring the discussion back to the company’s actual exposure: what access the agent has, which actions it can take, how large the blast radius could be, and whether we can detect and stop unexpected behavior.

Because technology is changing faster than traditional governance cycles, governance can’t be limited to periodic reviews. It requires continuous monitoring and clear triggers for reassessing risk when components such as models, tools, data, permissions, or connectivity change.

Noma addresses AI risk across discovery, development, and runtime. Where do you see the biggest disconnect between how enterprises conceptualize AI risk and how that risk actually manifests operationally?

The biggest disconnect is between the fear that autonomous agents will escape our control and the operational reality: agents using the access and tools we gave them.

Noma Labs demonstrated this with GitLost. An attacker placed malicious instructions in a public GitHub issue. An agentic workflow with access to private repositories followed those instructions and published private repository data in a public comment. The agent simply used the access and tools it had been given.

The example shows why the full lifecycle matters. Discovery should identify the agent, its tools, and its permissions. Development testing should determine how it responds to malicious instructions. Runtime protection should recognize and stop the movement of data from a private repository to a public comment.

Agent risk combines familiar identity, access, application, data, and supply-chain risks with non-deterministic behavior and machine-speed execution. That’s why discovery, development testing, and runtime protection have to work together.

Having worked on both sides of the security market, has your vendor-side experience changed how you evaluate cybersecurity products? What do you probe for first?

Absolutely. Working vendor-side taught me how different the go-to-market narrative can be from the operational ground truth.

The first thing I probe is materiality. What data will the product access? Which systems will it connect to? How dependent will the business become on it, and what would happen if it failed or was compromised? That determines the depth of the evaluation.

For a lower-impact SaaS product, basic diligence may be appropriate. We’ll review certifications, investigate the company’s history and reported security issues, and run a trial to verify that the product operates as expected.

If the product will become core to the business, certifications and product demonstrations are only a starting point. That requires much deeper technical and operational validation.

What happens when security favors a product, but engineering sees integration debt, finance questions the economics, or leadership sees overlap with existing tools? How do you reconcile those views?

Security doesn’t choose a product in isolation. If engineering sees integration debt, finance sees poor economics, or leadership sees duplication, those are risks too.

I bring everyone back to the outcome: What risk are we reducing, what capability do we need, and can an existing tool provide it? Then we compare total cost, integration effort, and measurable risk reduction. If the answer isn’t clear, we run a time-bound pilot with agreed success criteria.

The decision should deliver the best overall outcome for the business, not simply security’s preferred product.

When do you bring practitioners into a vendor evaluation, and what can they uncover that executive discussions and product demonstrations cannot?

I bring practitioners in as soon as we determine that a product will handle sensitive data, integrate deeply into our environment, or become operationally important, and always before the commercial decision is locked in.

Executive discussions explain what the product is supposed to do, and demonstrations show a controlled happy path. Practitioners can determine what it will actually take to deploy, secure, and operate it. They’ll examine the architecture, data flows, identity model, dependencies, tenant isolation, logging, failure modes, integration effort, and day-to-day operational burden.

For an important product, I want someone with strong secure SDLC or application-architecture expertise to speak directly with the vendor’s engineering team. I also want evidence, not generic questionnaire responses. Don’t just say that you perform threat modeling or remediate vulnerabilities. Show us a recent example, how the issue moved through your process, and how you verified the fix.

Practitioners can uncover fragile integrations, insecure defaults, missing telemetry, and operational costs that won’t appear in an executive conversation or a polished demonstration.

You’ve been recognized as a Global Cyber Security Hall of Fame inductee and leading security voice. Has that recognition changed how you approach public conversations on emerging security issues?

Not fundamentally. I’m grateful for the recognition, but it doesn’t change my responsibility to be accurate, keep learning, and improve how I work and communicate.

Finally, what is one cybersecurity problem that technology alone will never solve, no matter how sophisticated the technology becomes?

The intersection of people and technology. People still make decisions about how to respond, what actions to take, and whom to trust.

Social engineering is the clearest example. Attackers exploit urgency, fear, and helpfulness, not just technical vulnerabilities. Technology can filter messages and support identity verification, but it can’t entirely eliminate the need for human judgment.

That’s why it’s so important to give people the knowledge, support, and systems they need to make better decisions and recover when something goes wrong.


About Diana Kelley

Diana Kelley is a globally recognized cybersecurity and AI security leader, CISO, board director, advisor, author, educator, and keynote speaker. She is Chief Information Security Officer at Noma Security and has held senior executive and technical leadership roles across some of the most influential organizations in security, including Protect AI, Microsoft, IBM Security, Symantec, Burton Group (now Gartner), KPMG, SecurityCurve, and SaltCybersecurity.

A trusted voice at the intersection of cybersecurity, AI, risk, and ethics, Diana serves on the boards and advisory bodies of leading industry organizations, including WiCyS, The Executive Women’s Forum, InfoSec World, CyberFuture Foundation, TechTarget Security Editorial, and DevNet AI/ML. She is also Chair of the EWF Annual Conference, helping shape one of the industry’s premier forums for women leaders in cybersecurity, risk, and technology. Her volunteer leadership has included roles with the ACM Ethics & Plagiarism Committee, CompTIA Cybersecurity Committee, Sightline Security, WOPLLI Technologies, Bridgewater State University’s Bartlett College of Science and Mathematics Advisory Council, and the RSAC US Program Committee.

Her industry recognition includes being named one of EM360Tech’s Top 10 Security Analysts of 2026, one of AuditBoard’s Top 25 Resilient CISOs in 2024, a 2023 Global Cyber Security Hall of Fame inductee, an EWF Executive of the Year, an SCMedia Power Player, and one of Cybersecurity Ventures’ 100 Fascinating Females Fighting Cybercrime.

More about Diana:

About Noma Security

Noma Security gives enterprises one control plane to discover, control, and protect AI agents across endpoint, SaaS, and homegrown environments. The platform identifies agents and their connected MCP servers, skills, models, tools, and data; controls what agents and users can access and do; continuously tests AI applications for weaknesses; and stops risky behavior at runtime. Noma’s Runtime Context Engine connects identity, intent, prompts, tool activity, data, destinations, and behavior across each session, helping security teams distinguish legitimate actions from dangerous sequences. Through Open Enforcement, Noma applies policy across the architecture customers already use, including hooks, gateways, SDKs, and direct APIs. Noma secures more than 2.6+ million agents, is trusted by Fortune 500 companies, and is backed by more than $132 million in funding. Noma unifies AI security and governance wherever agents operate. Named a Leader in the 2026 Gartner Emerging Market Quadrant for AI Application Security, Noma secures more than 2.6+ million agents, is trusted by Fortune 500 companies, and is backed by more than $132 million in funding.

Learn more at noma.security.

Cybersecurity CISO Cyber Risk AI Risk Management Information Security Cyber Defense Cyber Resilience