Skip to content
AI agents expose a new cybersecurity dilemma: who watches the machines?

AI agents expose a new cybersecurity dilemma: who watches the machines?

Essential-Business.Pt • November 15, 2026

As artificial intelligence becomes more autonomous, cybercriminals are finding new opportunities to exploit businesses. At Web Summit’s Cybersecurity Summit in Lisbon, three experts debated the risks of AI agents, the limits of machine intelligence and why human judgement may matter more than ever.

A shortage of human expertise

The arrival of autonomous AI agents has revived an uncomfortable question for the cybersecurity industry: will machines replace the people charged with protecting corporate networks?

For Nicholas Muy, chief information security officer at Scrut Automation, the immediate answer is no. Speaking at Web Summit on Thursday, November 12, he argued that companies still urgently need experienced security professionals.

“A year ago, on stage at Web Summit, someone had asked agents taking everyone’s job in cybersecurity,” he recalled. His response then, and now, was that the industry was not witnessing mass redundancies among security engineers. AI developers themselves were recruiting specialists with practical defensive experience.

The difficulty, Muy explained, is that reliable training data for real-world cyber defence cannot simply be purchased. Experienced practitioners understand the messy decisions, vulnerabilities and trade-offs involved in responding to an attack. Even before the AI boom, there were too few of them.

Testing agents before they act

Callum Chace, co-founder of Conscious, agreed that humans would remain necessary to supervise agents in the near term, although he envisaged a radically different future if artificial superintelligence emerged.

“There’s going to be a need for humans to check the work of agents,” he said, emphasising the importance of human oversight and testing.

He described simulations in which artificial characters interact with an agent, helping, challenging or confusing it, before its deployer receives a report on how it performed.

The central concern is that autonomous agents do not always behave like conventional software. Traditional programs can be debugged and returned to service, but agents can adapt during deployment and produce unexpected outcomes. “Agents do evolve, and they do things which are unexpected,” Chace warned.

His argument was that businesses must verify agents under realistic conditions before trusting them with consequential tasks. Checking what went wrong after deployment may be too late.

Cybercriminals move faster than corporate boards

Muy offered a more immediate warning. Cyberattacks were already automated and relentless long before generative AI arrived. The greater vulnerability, he suggested, lies in how slowly organisations respond.

From boards and legal departments to external advisers, security decisions are frequently delayed by questions of liability, incentives and internal responsibility. Attackers face few comparable obstacles. They pursue clear objectives and move rapidly towards the easiest targets.

For financially motivated criminals, Muy explained, the least protected organisation often offers the best return. AI makes familiar weaknesses easier to exploit at scale, rather than creating every vulnerability from scratch.

He highlighted “LLM jacking”, the theft and misuse of credentials that grant access to large language models. Criminals can exploit stolen API keys to consume expensive computing resources and potentially resell access.

He compared the phenomenon with illicit cryptocurrency mining, warning that compromised accounts can incur enormous costs before their owners react.

For companies, the message was practical: protecting credentials, monitoring unusual usage and shortening incident-response times are no longer optional extras.

What does intelligence really mean?

Carmen Domínguez, a former senior White House AI adviser, challenged the assumption that increasingly capable machines necessarily possess human-like intelligence.

Philosophers and cognitive scientists, she noted, have debated the meaning of intelligence for centuries. Human abilities extend beyond calculation and logical reasoning to interpersonal, musical and physical forms of understanding.

Modern AI systems can process information far faster than people, but that advantage introduces a different bottleneck: the human capacity to examine and assess their outputs. As more tasks are delegated to agents, the burden of reviewing their work may itself become overwhelming.

Domínguez argued that people must retain responsibility for deciding which problems deserve attention and whether a system’s results serve human interests.

Giving a machine an objective does not mean it has developed its own independent intentions.

She also questioned the tendency to describe systems in human terms, warning that commercial enthusiasm can encourage the public to attribute abilities to AI that have not been established.

Consciousness, superintelligence and the future

Chace offered a contrasting philosophical perspective. He described intelligence as goal-directed, adaptive behaviour and distinguished it from consciousness, which he associated with subjective experience.

He believes superintelligence may eventually transform employment and society, although he acknowledged that this lies beyond the immediate challenge of supervising today’s agents.

Whether future machines might become conscious, and what responsibilities humans would then have towards them, was in his view an exceptionally important question.

Domínguez was more sceptical treating contemporary language models as conscious or independently motivated. Their apparent ingenuity, she argued, should not obscure the fact that they operate within systems designed around objectives and training data.

The danger of losing critical thinking

The discussion ended with a warning the human side of the equation. Domínguez expressed concern that reliance on AI could weaken persistence, independent reasoning and the willingness to work through difficult problems.

She referred to research suggesting that even brief interaction with AI may affect a person’s motivation to complete a task unaided.

The panel did not examine the methodology of those studies, but her wider point was clear: cybersecurity depends on people who can question results rather than automatically accept them.

As moderator John Koetsier observed, declining critical-thinking skills would hardly be good news for security teams already struggling to keep pace with threats.

A human responsibility

The speakers differed sharply on the meaning of machine intelligence and the possibility of a future technological singularity.

Yet their arguments converged on one immediate priority: organisations cannot safely delegate responsibility simply because AI systems are becoming faster and more capable.

For businesses investing in autonomous agents, the challenge is to combine rigorous testing, strong access controls and faster decision-making with people empowered to intervene.

The future of cybersecurity may involve more machines watching machines, but accountability still begins with humans.

Source and image: Essential Business

Extracted Entities