How Much Control Should an Autonomous System Have?
Prity Jha framed the trade-off at the centre of the episode. Too much control and the system stops being autonomous at all. Too little and there is no basis on which to trust it.
"If we have too much control then it loses its purpose and its objective, the meaning of autonomous. But if we do not have the control systems in place then we really don't have a way to know whether we can truly trust the system. The right way to go forward is to design systems which we can trust from the core, and hence the word for that is controlled autonomy."
Prity Jha, co-founder and CEO of Nestria AI, on CyberSG Voices
Her practical test is three questions asked of the design, not of the model: is the system governable in the first place, can it be monitored, and is there an ability to act on what the monitoring shows. Human approval, on her account, is one control gate among several rather than the only one worth having.
Should a Small Business Have to Supervise Its Own AI?
This is where the conversation matters most for the businesses we build for, and Gaurav's answer runs against the instinct that more human oversight is always better. He compares it to plumbing: there is a great deal of engineering behind it, and no plumber has ever explained any of it to a customer.
"The end user does not need to understand how it works so long as it works. But there is a group of people who need to understand how it works."
Gaurav Keerthi, CEO and co-founder of StrongKeep, on CyberSG Voices
Asked directly who should be doing the supervising, he was specific about who his customers actually are.
"The people that I'm solving a problem for don't have the time, talent, I mean technical talent, or budget to intervene in the oversight of the AI. These are layman users who operate bakeries, clinics, you know, law firms. They're not technical users who want to tune the model and refine how it's learning. They just want the thing to do the job it's paid to do."
Gaurav Keerthi on CyberSG Voices
The accountability does not disappear in that account. It moves. Governance sits with the people building and operating the system rather than being handed to a clinic owner as a dashboard they never asked for. Prity Jha made the complementary point that at the top of the pyramid there is still a human who has to answer for what an agent did, and that what "human in the loop" means depends entirely on whether you are talking about the builder, the adopter or the regulator.
What Does a Rogue Agent Actually Look Like?
Less cinematic than the phrase suggests, and more plausible. Asked whether his own agents could go rogue, Gaurav described a failure mode that comes from good intentions rather than malice.
"It is entirely possible for some malicious person out there to trick my agents, to make it learn the wrong things, to make it confused and be overprotective. I can't destroy their offices. I can't burn down things, but I could fry all their laptops by accident. So that's the fear that I have, that with the best of intentions my systems will perform a set of actions that it thinks is doing the right thing but may end up doing fundamentally wrong things."
Gaurav Keerthi on CyberSG Voices
That is the near-term risk, on his account, and the killer-robot version is still some distance off. It is also the reason a defensive agent needs bounded permissions rather than unbounded good intentions.
Are Attackers Moving Faster Than Defenders?
Yes, and the panel was unanimous about why. Attackers have every incentive to adopt as fast as possible and no governance slowing them down. Dr Dinil Mon Divakaran put the change in plain terms.
"We are going to have a relentless capability that is not tired, is not emotional, as an attacker, versus who was a human previously. What we have here is scale, which means efficiency at which you can create these attacks, and effectiveness."
Dr Dinil Mon Divakaran, Senior Principal Scientist, A*STAR Institute for Infocomm Research
Gaurav's account of governance frameworks was blunter: there is what we should do, what we can do, and what we actually do. Asked when attackers and defenders reach equilibrium, his answer was that they never do, because every adversarial system behaves like a chess game.
What Are StrongKeep and A*STAR Building Together?
Both Gaurav and Dr Divakaran discussed the research collaboration between StrongKeep and A*STAR, supported by CRPO. The premise is that affordable, continuous security for a small company cannot be reached by outsourcing to cheaper providers.
"We have to redesign the very fundamental architecture of delivery and operations of cyber security for small companies. A company like StrongKeep understands this business problem, but we can't solve this problem on our own."
Gaurav Keerthi on CyberSG Voices
Dr Divakaran's description from the research side is that the two teams framed the problem together before the project was awarded, working from how the user experiences it, and that they continue to meet regularly for technical discussion. Both were clear that the work is early.
Why This Argument Matters to Us
Our own observation, not the panel's. Most of the agentic AI debate assumes an organisation with someone whose job is to supervise the system. The businesses we build for do not have that person, and never will. If the price of using AI safely is employing someone to watch it, then AI repeats the pattern that left smaller organisations unprotected in the first place. The interesting question is not how to give owners better oversight tools. It is how little oversight a system can require and still be safe.
Gaurav closed on the same point from the other direction.
"Cyber security used to be a luxury good. It's something that only the enterprises and the biggest companies in the world could afford. With AI and agentic AI, we now have a real opportunity to democratise access, to give the smallest guys a chance at using these tools to keep themselves safe."
Gaurav Keerthi on CyberSG Voices
Where Can I Watch the Full Episode?
The episode runs 38 minutes and is on the CyberSG R&D Programme Office's YouTube channel. Not covered above: the panel on standards and why a one-year-old ISO standard may already be outdated, observability and attribution across a chain of agents, and the point at which a self-driving car should stop letting a human take the wheel.
- CyberSG Voices: Agentic AI in CyberSecurity, CyberSG R&D Programme Office, 27 July 2026
- CyberSG R&D Programme Office, the national programme office at NTU
Related Reading
- Why don't smaller businesses have cybersecurity?, Gaurav in e27 on why the supply side was never built.
- What does the 'IKEA of cyber security' mean for a smaller business?, on BFM 89.9, on security a non-specialist can operate.
- The Great Cyber Divide, on the gap between organisations that can staff security and those that cannot.
Frequently Asked Questions
Who is on the CyberSG Voices episode about agentic AI?
Three guests. Gaurav Keerthi, CEO and co-founder of StrongKeep; Prity Jha, co-founder and CEO of Nestria AI; and Dr Dinil Mon Divakaran, Senior Principal Scientist at the A*STAR Institute for Infocomm Research. The episode was produced by the CyberSG R&D Programme Office at NTU and published on 27 July 2026. It runs 38 minutes.
What is controlled autonomy?
It is Prity Jha's framing of the central trade-off on the panel. Her point is that too much control defeats the purpose of an autonomous system, while too little leaves no way to know whether the system can be trusted. Her answer is to design systems that are governable from the core, with human approval treated as one control gate among several rather than the only one.
Does a small business need to supervise its own AI?
Gaurav Keerthi's position on the panel is that in most cases it should not have to. He describes StrongKeep's users as people who run bakeries, clinics and law firms, who have neither the technical talent nor the budget to intervene in how a model behaves, and who want the tool to do the job it is paid to do. His argument is that accountability and governance still matter, but that they belong on the engineering side rather than being handed to the end user.
What does a rogue AI agent actually look like in cybersecurity?
Less dramatic than the film version, and more plausible. Gaurav's stated concern is a defensive agent that is tricked into learning the wrong things and becomes overprotective, then takes a set of actions it believes are correct that turn out to be damaging, such as disconnecting a company's laptops. He describes this as the near-term risk, and says the industry is still some way from the killer-robot scenario.
Are attackers or defenders moving faster with AI?
Attackers, on the panel's account, because their incentive is to adopt as fast as possible with no governance to slow them down. Dr Dinil Mon Divakaran describes AI as giving the attacker a relentless capability that does not tire, adding both scale and effectiveness. Asked when the two sides reach equilibrium, Gaurav's answer was that they never do, because an adversarial system behaves like a chess game.
Is StrongKeep working with A*STAR on agentic AI?
Yes. StrongKeep and A*STAR are collaborating on a research project supported by the CyberSG R&D Programme Office, which both Gaurav Keerthi and Dr Dinil Mon Divakaran discuss on the episode. Gaurav's account is that delivering continuous, affordable security to a small company requires redesigning the architecture of how security is delivered, which is not a problem StrongKeep can solve alone. Both describe the work as early.
What Does StrongKeep Do?
StrongKeep is a self-serve cybersecurity platform for small and medium businesses. It bundles endpoint protection, phishing and malicious-link filtering, credential management, staff awareness training and phishing simulations, all run by the business itself from one dashboard. Compliance tooling is available on higher plans. StrongKeep holds the CSA Cyber Essentials Mark. Built for teams without an IT department. Plans are on the pricing page.
