¡°AI safety is a collective problem¡±, Dr. Qinghua Lu, Member of the Independent International Scientific Panel on Artificial Intelligence
Dr. Qinghua Lu is a Group Leader at Australia¡¯s Commonwealth Scientific and Industrial Research Organisation (CSIRO) and an expert in Artificial Intelligence (AI) Engineering and Safety. She is also a member of the Independent International Scientific Panel on AI (/independent-international-scientific-panel-ai/en) and recently contributed to the Panel¡¯s preliminary report (/independent-international-scientific-panel-ai/en/preliminary-report).
Today, artificial Intelligence has far-reaching impacts on peace and security. Rather than viewing AI through an either optimistic or pessimistic lens, Dr. Lu focuses on understanding emerging risks, building effective safeguards, and stresses that international cooperation will be essential to ensuring AI develops safely.
Question: When we talk about Artificial Intelligence creating new security risks, what does it mean in practice? Could you explain what an AI system can do today that would make a cyberattack, disinformation campaign, or other harmful activity easier?
Qinghua Lu: Artificial Intelligence can amplify existing security threats, be used to attack others, and become a target of attack itself. As AI shifts from chatbots that answer questions to agents able to make plans, take actions, such as using tools and running code without human oversight, these risks are growing. In other words, AI agents can take unwanted actions that no human explicitly requested or anticipated.
Another example is how AI can intensify threats by increasing their speed and scale. During an election campaign, it could generate a false video, image or statement about a candidate and spread it rapidly across social media, adapting the content to influence different groups of voters.
AI can also cause harm more directly. The United Kingdom¡¯s AI Security Institute1 recently reported, during a cybersecurity evaluation, an AI agent tried to insert malicious code into a public software project. It even created false online identities to persuade the project¡¯s developer to accept the code.
Finally, AI systems can also be tricked into doing harmful things. Imagine asking an AI assistant to visit a travel website and book a hotel for you. The website could contain a hidden instruction telling the assistant to send your private emails, payment details, or other personal information to someone else. If the assistant mistakes this for a legitimate instruction, it may follow it without your knowledge.
Question: The preliminary report of the Independent International Scientific Panel on Artificial Intelligence raises particular concerns about AI-enabled biological threats. Can you elaborate on these concerns?
Qinghua Lu: One of the concerns is that as much as AI can benefit humanity, it may also cause harm. For example, scientists can use AI to review thousands of research papers and understand how virus work, supporting new treatments or vaccines. However, someone with bad intentions could use similar tools to change a virus and make it more dangerous and easier to spread. In the past, someone needed to have a special scientific knowledge to do this. But now this important barrier to misuse is progressively being lowered. AI is making complex information easier to find, combine and understand.
Many current AI systems already include safeguards to reduce risks, such as refusing dangerous requests, monitoring for misuse and limiting access to certain tools or information. As AI capabilities grow, these protections will need to become more robust. This could include multiple independent supervisory models, tighter controls over tools and data, and human approval for high-risk actions.
Question: These days, more than one billion people use conversational AI routinely. At the same time, the adoption of AI is uneven around the world - with the Global North concentrating infrastructure and models. What could be the impact for countries lagging behind?
Qinghua Lu: AI could bring major benefits around the world, including better healthcare, education, public services and economic productivity. However, these benefits are unlikely to be shared evenly. Countries with good internet access, powerful computers, skilled workers and money to invest are likely to benefit first.
At the same time, countries that benefit less may still experience its negative impact. Their workers may lose jobs or see their job change because of changing labour markets. And new businesses, productivity gains and tax revenues may be concentrated in richer countries.
There is also a security problem. AI can be used for fraud, disinformation and cyberattacks and these threats can easily cross borders. But not every country has the expertise or tools to detect and respond to them. Countries with weaker cybersecurity systems may therefore be more vulnerable.
For these reasons, international cooperation is important. Countries with fewer resources should receive support to develop local talent, build AI infrastructure, conduct testing in different languages and access cybersecurity tools. They should also have a voice in deciding how AI is developed and governed, rather than simply using technology designed by other countries.
Question: What should governments and the international community do now, given that AI is developing faster than many existing rules and safeguards?
Qinghua Lu: As AI develops rapidly, it is important to identify and address potential risks early and to develop appropriate governance alongside the technology. Reasonable safety measures should be put in place, especially for high-risk systems. Controls over agents and software should be established and access to tools and data should be limited when necessary.
Transparency can help us identify emerging risks, understand where existing safeguards are falling short, and determine where they may need to be strengthened.
Another message to the international community is that it should cooperate on how AI safety is assessed. AI safety is a collective problem because no single country or organisation can identify all the risks on its own. Therefore, countries need to cooperate and share evidence. Establishing common methods for testing AI and collecting data on its safety would allow countries to learn from one another, while protecting privacy, national security and commercial information.
Question: Lastly, what are the next steps for the Scientific Panel of Experts on AI?
Qinghua Lu: The Panel¡¯s next step is to produce evidence-based thematic briefs on emerging or fast-moving issues such as AI safety. These briefs will build toward the Panel¡¯s next annual report, which will inform the second Global Dialogue on AI Governance in New York in May 2027.