UK Prime Minister Andy Burnham recently announced in the United Nations General Assembly plans for the creation of a new National Centre for Information Defence to counter the risk of disinformation and deepfakes from hostile actors. In his speech, the Prime Minister emphasised that security starts in every home, a statement that resonates strongly with us at the Institute for Digital Security and Behaviour (IDSB). Although disinformation is nothing new, the BBC 4 Today interview with Wes Streeting (23.09.26 at 02.12hrs) highlights that this renewed focus is needed to counter the increased level of threat seen in the current technological and geopolitical environment. This includes his own experience of being targeted by deepfakes.
We agree that people need to be supported to effectively navigate the threats that they face in the online environment and working together across disciplines, sectors and groups is the best way to achieve this. This is why we are actively exploring how we can engage with concepts such as upstream public engagement when researching deepfakes and security threats. It is vital that any national effort to address this challenging issue is enacted with transparency at its core, empowering people to effectively identify fact from fiction and protect themselves in their everyday lives.
The recent technology trends report completed by members of the IDSB and published by the UKRI-funded CRANE network highlights the importance of addressing threats related to the post-trust information ecosystem. AI-generated content, verification challenges, and the increasingly automated targeting of influence attempts, present substantial issues within the online space, with the potential to reduce public trust, impact political stability and create pathways to radicalisation. Threats also include the use of deepfake technology to create impersonated content designed to influence people’s opinions or understanding (e.g., seen in the deepfake presidential address), demean others (e.g., seen in the Grok AI scandal), or to fraudulently extract money or information (e.g., seen in targeted family emergency or CEO scams).
Research within the IDSB has highlighted how online spaces can be used by those with nefarious intent to influence others. Findings in recent work, suggests that LLM-generated content can be difficult to detect and has the potential to be more persuasive than human communications. Research also shows that it can lead to the production of more radicalising content, exposing people to potentially extreme information and connections even when they use relatively benign prompts to explore a topic. Whilst AI can bring potential benefits in aiding the detection of disinformation and deepfakes, research shows that it also has the potential to make such communications harder to spot and to automate complex influence operations.
There is a growing requirement for evidence-based recommendations that consider how best to protect people from online risks related to AI-generated content. However, research can struggle to adequately involve end users in identifying what can most help people to build their resilience, ensuring that the ambitions set out by the UK Prime Minister can be effectively met. Using collaboration and co-creation is vital to ensure that any approaches developed will be relevant to people’s everyday experiences, actionable and have the best chance of delivering the intended outcome. Within the IDSB, we hope to continue working in collaboration with others to collectively shape a safe and secure online future for all.
Respond
