“An AI system can draw on an engineer’s knowledge, but when it comes to applying that knowledge, it doesn’t have an engineer’s judgment.”
In a refinery or a pharmaceutical facility, a cyber incident can quickly become an operational problem. For Dustin Johnson, that makes one gap in AI impossible to ignore: it can draw on an engineer’s knowledge, but it lacks an engineer’s judgment in applying it. That raises practical questions about which information an AI system can access, what authority it has and when a person must review its work, and Johnson argues that teams need to be able to examine, approve and control changes to AI-assisted analysis before relying on it in operation.
Johnson is CTO and Founding Partner at Seeq, and his career has taken him from aerospace and network analysis to industrial AI. In this conversation, he explains why security must protect the ability to make sound decisions as well as systems and data, which problems belong in product architecture rather than governance, and what a strong working relationship between a CTO and security leadership looks like. He also sets out the evidence he asks vendors for, what breaks a tie between credible vendors, and why extensibility and open APIs are important enough to be veto-worthy.
From aerospace and network analysis to industrial AI, your career has put you close to some very different engineering environments. Which experiences most shaped how you think about security today?
Building software for industrial environments makes the consequences of engineering decisions very tangible. When a recommendation directly impacts a physical outcome, confidence in the recommendation has to come from something engineers can examine and verify. That experience puts a premium on understanding how a system might fail, limiting what it can affect and preserving enough history to investigate a problem. AI makes those engineering disciplines even more valuable because the speed of generating a solution can easily outpace the work required to validate it.
At Seeq, operational data, human expertise, and AI converge to influence industrial decisions. What new security considerations emerge when those three come together?
An AI system can draw on an engineer’s knowledge, but when it comes to applying that knowledge, it doesn’t have an engineer’s judgment. This creates important questions about which information the system can access, what authority it has and when a person needs to review the system’s work. At Seeq, the emphasis on making AI-assisted analysis inspectable reflects how consequential those decisions can become. Teams need a way to examine a proposed approach, approve a specific version and control subsequent changes, before relying on it in operation.
In a refinery or pharmaceutical facility, a cyber incident can become an operational problem very quickly. What does “security” need to protect in these environments beyond systems and data?
Security needs to account for the people, equipment and production processes relying on those systems. An incident that compromises the information behind an operating decision can have consequences, even if the software remains available. Engineers need to know whether the analysis they are using still reflects actual conditions and if anything has changed since it was reviewed. Protecting the ability to make sound decisions, rooted in human engineering expertise, is essential to keeping an industrial operation safe and reliable.
From a CTO’s seat, which security problems belong in the product and architecture, and which should be solved through governance or operations? Where is that line often blurred?
The product’s architecture should have a robust authorization model, an industry standard authentication model, and provide the controls to constrain what capabilities users and AI agents have access to. On the other hand, governance should establish who can grant that authority, which decisions require approval and what level of risk the organization is prepared to accept. Operations must then monitor if those arrangements are working as intended. The line becomes blurred when companies rely on employees to follow security rules that the software itself should enforce automatically.
Security leadership and the CTO often approach risk from different angles. What does a strong working relationship between the two look like when the stakes are high?
A strong relationship between security leadership and CTO starts with a shared understanding of the intended benefit of a system, and the consequences of its failure. In instances where the stakes and risks are raised, for instance giving AI more responsibility, both leaders need to agree on what makes that risk worth taking and how much authority is appropriate. A limited deployment with explicit permissions and review points gives them something concrete to evaluate together. As evidence accumulates, they can make a decision together about whether broader use is justified, and what additional controls it would require.
A vendor claims its platform can reduce security risk while also improving operational efficiency. What evidence would you ask for to separate a genuine business case from a well-packaged pitch?
The vendor should be able to identify the specific risk being reduced and what work becomes easier as a result. An evaluation should establish the initial conditions, measure the change, and include the effort required to configure, supervise and maintain the platform. For an AI-enabled product, this also means examining what access and autonomy it needs to deliver the promised benefit. A business case is much more credible when the customer is able to verify results in a limited deployment under its own operating constraints.
Imagine you have three technically credible vendors and no obvious winner. What final question or piece of evidence would you want before putting your name behind the decision as CTO?
I often look at the strength of the vendor’s team and the type of relationship they’re selling. A strong team looking to establish a partnership will often outcompete an offer from a larger team selling a transactional deal.
What would make you personally veto a security purchase, even if the security team, procurement, and business stakeholders were already leaning toward the vendor?
Extensibility and open APIs are more important in the AI era than ever before, and I believe they’re important enough to raise to the veto-worthy level.
What’s a green flag in a security vendor that rarely gets enough attention during enterprise evaluations? Conversely, what’s a red flag that can look perfectly reasonable during the sales process?
A vendor that clearly and candidly explains the limits of its product is a strong positive signal. Providing this initial clarity for customers helps them understand where their own expertise, processes and systems still need to contribute. A red flag is promising minimal human involvement and leaving the review and approval process vague. Reducing routine work is valuable, but the evaluation should establish whether people will still have enough information and control to exercise judgment when it matters.
