Head of Information & Cyber Security

Through this article, Ayo Adebayo, Head of Information & Cyber Security, explores the dual role of artificial intelligence (AI) in cybersecurity, acting as a defence tool and a threat. Adebayo draws a metaphor between the complexities of AI risks and the labyrinthine design of the Winchester House, highlighting concerns such as unintended consequences, ethical dilemmas, and risk ownership. He emphasises the need for comprehensive strategies, including policy frameworks, cross-functional collaboration, and training, to navigate AI risks and ensure it is a force for positive change in cybersecurity and beyond.

“Risk comes from not knowing what you are doing” Waren Buffett.

Technology is evolving rapidly, with artificial intelligence (AI) emerging as both a powerful tool and a formidable challenge in cybersecurity. As cybersecurity practitioners, we witness firsthand the transformative impact AI has on our ability to defend against increasingly sophisticated cyber threats. AI serves a dual role in cybersecurity: it acts as a shield and a sword. On one hand, AI-powered tools enhance our defensive capabilities, enabling us to detect and respond to threats more swiftly and accurately. On the other hand, cybercriminals are leveraging AI to develop more advanced and elusive attack methods.

Early this year, the story of “The Winchester House” was brought to my attention. For those unfamiliar with it, it is a large, historic mansion located in San Jose, California, renowned for its architectural oddities and mysterious past. Architectural features of this house include staircases that lead to walls and ceilings and doors that open out into nothing, among many other peculiarities.

“The Winchester House”, with its labyrinthine layout and perplexing design choices, is a testament to the anxieties and uncertainties of its builder, Sarah Winchester. Still, it can also be seen as a metaphor for Artificial Intelligence (AI) risks and cyber security concerns.

"AI presents a labyrinth of risks, concerns, and threats that require careful navigation, but with a proactive approach, we can harness its potential while safeguarding against its pitfalls."

As AI and associated technologies continue to be developed and deployed across sectors, industries and companies on a global scale, there is no doubt that while businesses and organisations seek value and benefits, some anxiety exists about AI's risks. The anxiety arises from risks such as security, transparency, privacy, accountability, bias and fairness, with many jurisdictions worldwide rushing to develop laws and regulations to address them. This is no small feat as technology is advancing rapidly, and countries struggle to come to grips with the wide-reaching impact of AI.

That said, the European Union (EU) has developed the AI Act, which came into force on 1^st August 2024. The act adopts a risk-based approach, aiming to have AI-based systems classified into risk categories with different degrees of regulation applied.  It is important to note that the EU AI Act applies not only to the AI developed by organisations but also to those procured by the organisation.

So, just as this historic house embodied the physical manifestation of its owner’s anxieties, it could also be viewed as a symbol of anxiety that could be present from the various concerns surrounding AI.

The Labyrinth of AI Risks

The Winchester House, with its staircases leading to nowhere and doors opening into walls, can represent the unpredictable and often opaque nature of AI deployments and development. Among the main issues typically associated with AI are:

1. Unintended Consequences

Just as Sarah Winchester's architectural choices resulted in a house filled with baffling features, AI solutions and systems can produce unforeseen outcomes. For example, poorly designed AI systems might inadvertently create vulnerabilities that cyber attackers could exploit. These unintended consequences can significantly impact safety, data subject rights and cybersecurity.

2. Ethical Dilemmas

The Winchester House is understood to echo its creator's personal fears; similarly, broader societal values and dilemmas can be reflected in AI. These ethical quandaries may reflect issues such as a lack of transparency, bias in AI algorithms, the loss of privacy, and even the potential for AI to be used in harmful ways to people. These challenges in ethics and values, which need to be carefully considered in all AI development and deployment, present some anxiety among stakeholders.

3. Risk Ownership

The Winchester House expanded in seemingly endless directions without an apparent functional blueprint or qualified design. The breadth of AI risk, which spans more than the cybersecurity domain, may result in a lack of coordinated approach concerning effective risk management. This poses the question of who should own AI risk in an organisation.

Some organisations have since ascribed this to a “Chief AI Officer” type role, with others embedding this in the role of “Chief Risk Officer” or “Chief Ethical Officer”, etc. However, the most common approach appears to be that this is left to information technology or the cyber security function, with many organisations considering AI simply a technical or data issue. It is evident that while IT and cyber security practitioners have a role to play in the identification, treatment and monitoring of some AI risks, a multidisciplinary or cross-functional approach is necessary to address this effectively.

Addressing AI Anxiety

A comprehensive and proactive approach must be adopted to mitigate AI risks and navigate its complexities. Here are some strategies to consider,

1. Establish Policy and Oversight Framework

Just as the Winchester House would have benefited from a master architect's guidance, a policy for developing, procuring, and deploying AI in the organisation must be established. The policy sets the tone and foundation for AI development, deployment and use. In addition to communicating management expectations and intent concerning AI, the policy must make clear the general principles and governance requirements for AI in the organisation.  

2. Collaboration and Cross-Functional Engagement

With AI projects impacting various aspects of an organisation, a holistic approach towards managing AI risk is crucial. To this end, a collaboration between teams, such as risk management practitioners, procurement, data scientists, software engineers, legal and compliance officers, data protection, security teams, and business leaders, should be established. This ensures all perspectives are considered, risks mitigated, and AI solutions responsibly developed and deployed.

3. Training and Awareness

Given that AI systems can be rather complex and their impact far-reaching, all stakeholders must be educated and informed about their responsibilities and the risks involved in a successful and ethical adoption of AI. Proper training and awareness will assist those involved in AI development and use to become aware of the moral implications, including transparency, accountability, and fairness and help prevent unethical or irresponsible applications of AI. By fostering a culture of continuous learning, good outcomes can be experienced with AI.

4. Standards and Frameworks

To ensure that the development or deployment of AI is consistent with established policies, clear guidelines, standards and principles must be created and communicated. By creating and embedding this framework into the fabric of AI systems' design, procurement and deployment, we can help ensure that these technologies enhance rather than compromise enterprise goals and objectives.

The Winchester Mystery House symbolises the complexities and fears that can shape human endeavours. Similarly, the growing rise of AI presents a labyrinth of risks, concerns, and threats that require careful navigation. By drawing lessons from the past and adopting a proactive approach, we can build a secure framework that harnesses the potential of AI while safeguarding against its pitfalls. In doing so, we will address AI's cybersecurity challenges and ensure that organisational goals are met, paving the way for a future where AI serves as a force for positive change and innovation.