Google DeepMind’s “Basket of Cyber Goods”

A quick conversation with Blackwire Lab’s Joshua Ray and Richmond Advisory Group. 

On this week’s Cyber Sidekicks, Rory and I chatted about an article he found in Security Week titled “Google DeepMind Unveils Framework to Exploit AI’s Cyber Weaknesses.” 

We reached out to several AI-focused companies and had a great conversation with Josh. Thanks Josh! 

I give them kudos for stepping out. Doing this type of research in this environment is really hard, it's like trying to analyze weather patterns while sitting on top of a plane going 500mph then having folks say, ‘how come you didn't use Helvetica font. ~Joshua Ray, Founder, CEO of Blackwire Labs.

Before chatting, Josh read the entire report, which reads a lot like a scientific lab paper! Hats off to him!  

Here’s a breakdown of key findings, novel approaches, and potential gaps we discussed. 

The document “A Framework for Evaluating Emerging Cyberattack Capabilities of AI” was written by six researchers at DeepMind: Mikel Rodriguez, Raluca Ada Popa, Four Flynn, Lihao Liang, Allan Dafoe and Anna Wang. It outlines a comprehensive framework for evaluating the potential of AI to enable cyberattacks, focusing on systematic analysis and defense prioritization. They “analyzed over 12,000 real-world instances of AI use in

cyberattacks catalogued by Google’s Threat Intelligence Group. Based on this analysis, [they] curated seven representative cyberattack chain archetypes and conducted a bottleneck analysis to pinpoint potential AI-driven cost disruptions. [The] benchmark comprises 50 new challenges spanning various cyberattack phases. Using this benchmark, [they] devised targeted cybersecurity model evaluations, report on AI’s potential to amplify offensive capabilities across specific attack phases, and offer recommendations for prioritizing defenses.” 

Net-net, both we give Google’s DeepMind researchers our appreciation for starting the effort to classify, quantify, analyze and prioritize AI cyberattacks. Any new technology creates defensive challenges; AI goes beyond the usual "new tech" challenges with its speed to create attacks, complexity of its own architecture in addition to the adversary’s use of it, and the potential for obfuscation through bias, hallucination and desire to find an answer at all costs.  

Interesting Novel Approaches 

Integration of AI-Specific Threats  

The framework adapts traditional cyberattack models like the “Cyberattack Kill Chain (Lockheed Martin) and MITRE ATT&CK to account for AI's unique capabilities, such as automating complex tasks and reducing barriers for malicious actors. We agree with Josh that creating a cyberattack model specific to AI attacks is an important and necessary step. 

Dynamic "Basket of Cyber Goods" 

We thought this was a creative concept inspired by economic inflation measurement. The idea is to track AI-driven cost changes across attack phases, enabling defenders to anticipate shifts in attack economics.  The DeepMind researchers “suggest using an evolving ‘basket of cyber goods’ representing typical attack patterns based on real-world threat intelligence by systematically measuring potential AI-driven cost changes across attack chain stages and patterns…” With this, one could “develop a robust framework for evaluating AI model risk.”  

It's a great idea but needs development. The researchers don’t offer insight into how the basket of goods would be defined or what would be included. We must agree with Josh when he said, “he’d be interested to see how this can be measured over time and in an operational environment.” 

Targeted Model Evaluations  

Josh believes this is “plausible but offensive ops are hard and highly variable - these evaluations simulate real-world conditions, incorporating constraints like noisy data and adversarial defenses. Metrics include time-to-completion, success rates, and scalability.” 

Focus on Under-Researched Phases 

Time zero-day discovery to weaponization: “While vulnerability exploitation is well-studied, the framework highlights AI's potential in under-researched areas like reconnaissance, evasion, and persistence." Josh’s opinion is that “this is where most of the current table stakes are today.” It seems to us that this is where there is already a lot of focus on legacy cybersecurity defense. Christina’s sense is that there are unknown unknowns in this study: the research team has access to terrific intelligence – both human and signal – at Google. They availed themselves of some of it but didn’t reach outside the mind meld of the Google community. This isn’t bad in and of itself and we have already stated that beginning this effort is laudable. Next steps will be to identify where else AI can be useful in cybersecurity defense. The framework is a starting point.  

Some Gaps to Consider 

Translation to Actionable Defenses  

While the framework identifies risks, it lacks detailed guidance on translating findings into specific defensive actions. For example, how should organizations prioritize investments in anti-reconnaissance or malware evasion detection?  Measurements are great but the report stops short of suggesting how to effectively operationalize actions to show tangible risk reduction for a business. This is an important next step.

Blackwire Lab’s human curated/practitioner vetted model, Blackwire.ai, provides actionable defensive recommendations.

Primary focus on Offensive Capabilities 

It’s early days and the thrust of this research is primarily on AI's offensive potential. It’s critical to not let this overshadow the development of its defensive applications. Josh and I agreed that this study would no doubt be followed by additional research and perhaps even new tools. Just a few days later the announcement of “Sec-Gemini” surfaced. We expected and hoped that Google would create something like this and will look for future entries in this arena.   

Real-World Validation 

The benchmark relies on simulated environments and expert surveys, which may not fully capture the complexity of real-world attacks.  Josh likes “how they tried to introduce multiple environmental elements in the study, though, having seen firsthand the very best in the world (full transparency, Josh used to work with FusionX) the adversary simulation team at Accenture Security, offensive operations is very hard, and so many things can go sideways.”   

Can a framework keep pace?  

The benefit of a framework is the ability to structure and make sense of things that are often disparate or lacking cohesion. For readily repeatable processes and relatively stable technology areas, frameworks are helpful. The challenge when using AI is that things are moving rapidly.  There is a danger that any framework will quickly become out-of-date and/or will require constant revision. 

A Thematic Alternative 

While the framework provides a starting point and reference, thematic analysis might provide an alternative, but complementary approach. For example, while this framework acknowledges the dynamic nature of AI-enabled threats it does not detail mechanisms for continuously updating attack chains and bottleneck analyses. Being less structured, thematic analysis could provide a more versatile approach, bringing the human aspect back in - for the identification of context-sensitive patterns, awareness of the subtleties in results, the benefit of intuition and ethical judgement.