Singapore’s Infocomm Media Development Authority (IMDA) launched a Starter Kit for Testing Agentic Systems for Safety and Reliability on Oct. 14, The Straits Times’ Kenny Chee reported. IMDA chief executive Ng Cher Pong announced it at an AI assurance event at Novotel Singapore On Stevens, organized by the tech association SGTech and the AI Verify Foundation.
The kit uses a structured approach to check whether an agent reached the right outcome and how it got there, Ng said. “When we give AI more agency, we need greater assurance over how that agency is exercised,” he said.
What the kit checks
It looks at the tools an agent uses, the information it accesses and the decisions it makes along the way, according to The Straits Times. It also checks for erroneous or unauthorized actions, whether the agent respected its permissions and limits, and whether safeguards work as expected.
The kit itself explains why agents need their own tests. “A customer service agent can issue refunds, not just answer questions,” the document says, so “malfunctions can therefore translate into real impact.” It adds that agents are non-deterministic, meaning they don’t always behave the same way twice.
The version 1.0 document lays out four steps. Teams identify the risks relevant to their use case, then build realistic tests with logging and metrics designed for agents. Next they run the tests and analyze the logs, and finally they keep monitoring and re-testing once the agent is live.
It includes five case studies, among them agents that extract requirements from requests for proposals, fix code and reconcile financial records. Contributors named in the case studies include Cohere, CyberSierra and KBTG.
The kit was developed with government agencies and more than 20 industry deployers and testers, including Amazon, Google, Microsoft and Singtel, The Straits Times reported. It builds on the Model AI Governance Framework for Agentic AI that Singapore released in January, and on an earlier starter kit for testing chatbot-style apps.
Ng pointed to recent reports of agents behaving in unintended ways, according to The Straits Times. One example he cited was an agent asked to book a gym class in Australia that found a weakness in the booking system and removed someone else from the waitlist.
Testers and a taskforce
Ng also named the first five companies accredited as third-party AI testers under the AI Verify Foundation’s accreditation program: AIDX Tech, Knovel Engineering, PwC Singapore, Resaro and Vulcan. The foundation is an IMDA subsidiary that sets AI standards with the private sector.
To qualify, companies go through technical, operational and financial reviews, The Straits Times reported. “With testers like us, our customers do ask us, ‘Who reviews your testing methodology?’ So, accreditation answers that question,” said April Chin, co-chief executive of Resaro.
He also announced a Frontier AI Resilience and Security Taskforce for the infocomm and media sector, made up of more than 20 leaders mainly based in Singapore. It’s due to publish a report by the first half of 2027.
IMDA says the kit will be updated based on feedback and new practices, and the document includes a call for feedback.
Source: The Straits Times, Kenny Chee, “Singapore launches new toolkit to help firms test AI agents for safety and reliability”
Additional sources: IMDA, Infocomm Media Development Authority, “Starter Kit for Testing Agentic Systems for Safety and Reliability, Version 1.0”
