Attackers are using GenAI to enhance their operations, but there is an opportunity for defenders to increase their capabilities using it too.

Speaking at Infosecurity Europe in London, Tal Zamir, CTO of Perception Point, said that employees are using GenAI, and this has led to the roll out of desktop apps and browser extensions – and increased the risk of data leaks.

“When you use any GenAI app, they let human reviewers view sensitive data,” Zamir said, pointing at the OpenAI breach where conversations were publicly available to other users.

Rogue AI Use

Zamir said that GenAI agents look across the enterprise, and can find details via keyword, as well as looking for matching documents and files that are exposed across the work population.

Also, he warned of fake GenAI apps, which are hard to determine from the real apps, and could steal an API key if used.

“The biggest GenAI models out there are available to attackers in an easy way, and they can grab the model and run it locally on a laptop,” he said, warning that attackers can use AI to increase productivity, increase the scale of attacks and target any industry, and bring junior attackers up to speed.

He said that the “highest bar is full automation of the attack process” where websites can be attacked with one-day vulnerability.

On the Defense

Zamir said on the defensive side, GenAI can allow protection to be enhanced with better detection, and also eliminate incident response work.

Also, Zamir talked about better detection of Business Email Compromise (BEC) emails, especially those with no links or email domain displayed in the body, saying in one instance Perception Point were able to understand a zero-hour phishing attempt, and expose a new set of attacks without any human intervention.

Speaking to SC UK, Zamir said this is done automatically now, “but we provide our customers with another layer of human service as even if our system is great at stopping all of the BECs, the customer is so convinced that it is a human element we add as well.”

This article first appeared in SC Media UK, written by Dan Raywood on June 5, 2024.