Detecting and Preventing AI-Based Phishing Attacks: 2024 Guide

What Is a Gen AI-Powered Phishing Attack? 

A gen AI-powered phishing attack uses generative artificial intelligence to craft highly personalized and convincing phishing messages. These attacks rely on AI algorithms to analyze vast amounts of data from social media, corporate websites, and other public sources to mimic legitimate communication styles and content. 

Unlike traditional phishing attacks that often rely on generic templates, GenAI-powered attacks can generate context-specific messages targeting individuals or specific departments within an organization. This level of personalization makes it challenging for recipients to distinguish between genuine messages and fraudulent ones, increasing the success rate of phishing attempts.

Generative AI allows cybercriminals to bypass traditional security measures that screen for known phishing indicators, making these attacks more difficult to detect and prevent with standard cybersecurity tools.

This is part of a series of articles about AI security.

How Phishing Has Changed with AI 

Now that generative AI has become more commonly used by threat actors, traditional phishing indicators, such as poor grammar and generic greetings, are no longer reliable signs of a fraudulent message. Generative AI enables cybercriminals to produce well-crafted, personalized messages that mimic legitimate communication from trusted sources. 

The integration of generative AI into phishing schemes has also made attacks more scalable. Cybercriminals can now automate the generation of convincing messages at a scale previously unattainable, targeting large numbers of potential victims with minimal effort. 

The shift to gen AI to launch phishing attacks requires a reevaluation of anti-phishing strategies. Organizations now need advanced detection tools that can analyze deeper aspects of messages, such as their context, the behavior patterns of senders, and indicators of the use of large language models (LLMs), rather than just superficial indicators.

Putting AI-Based Phishing in Context: Other Phishing Trends 

The use of AI and LLMs is only the latest sophisticated measure adopted by attackers. It comes on the heels of multiple other trends that are making phishing attacks more difficult to detect and prevent. 

Quishing

Quishing involves the use of QR codes to redirect individuals to malicious websites. When a QR code is scanned, it often leads the user to a counterfeit login page, such as a fake Microsoft login. This tactic has seen a significant rise, with a 427% increase detected from August to September 2023 alone. Attackers exploit the fact that most email security systems fail to scan embedded QR codes, allowing them to bypass traditional email filters. 

2-Step Phishing

2-step phishing helps attackers evade detection by embedding the malicious payload within a seemingly authentic service. A user might receive an email claiming their password has expired and prompting them to click a link to reset it. This link redirects to a legitimate-looking site that hosts a hidden malicious link. Attackers exploit over 400 commonly used platforms like Salesforce, SharePoint, and Adobe. 

Text Obfuscation

Text obfuscation involves altering familiar phishing templates to evade detection. Hackers insert invisible characters between letters, making phishing texts appear normal at first glance but malicious upon closer inspection. For example, a single letter might be composed of multiple unicode characters, transforming simple words into complex strings that evade traditional text filters. 

Browser in the Browser

This technique creates a convincing fake browser window within the actual browser, using HTML and CSS to mimic legitimate sites. For example, a user might see a Netflix login page in a pop-up window that appears genuine but is actually a fraudulent domain. This method can deceive users into entering their credentials, believing they are on a secure site. It avoids detection by not displaying any favicon, which is used by security filters to verify site legitimacy. 

Archive in the Browser

This tactic leverages the .zip domain to trick users into thinking they are opening a file directly in their browser. Users click on links like “invoice pdf,” which instead download a malicious executable file disguised as a legitimate document. This method bypasses many security filters because there are no obvious indicators like download buttons or clickable prompts. 

Encoded HTML Files

Encoded HTML files can bypass security measures that typically flag URLs. Attackers use techniques like Base64 encoding combined with AES encryption to hide malicious payloads within HTML files. These files often evade detection due to their benign appearance and the fact that files do not carry the same reputational risks as URLs. 

Captchas, Geofencing and Redirects

Hackers use CAPTCHAs, geofencing, and redirection tactics to create an illusion of legitimacy and bypass security filters. For example, they might use CAPTCHAs to prevent automated security systems from accessing the final malicious payload. Geofencing restricts access to the malicious site based on the user’s location, blocking major security firms. Redirection tactics complicate detection by directing users to benign pages initially, redirecting them to malicious sites later. 

Phone Scams 

Phone scams have evolved to include fake renewal alerts from well-known services like McAfee, Norton, and PayPal. These scams trick users into calling a number provided in a fake invoice, connecting them to call centers that guide them through installing remote access software like TeamViewer. This allows attackers to gain control over the victim’s device, stealing sensitive information without relying on email-based methods. 

In the age of AI, phone scams are taking a new direction. Deepfake technology is making it possible to create fake audio files or even videos that impersonate a real person’s voice and appearance. In a widely publicized case, a finance worker at a large corporation participated in a video call with senior executives from his company, and approved a payment of $25 million, only to discover that the other participants on the call were fabricated using deepfake video.

How to Defend Against and Detect Gen AI Phishing Attacks

Here are some of the measures that organizations can take to defend themselves against AI-enabled phishing attacks.

1. Anti-Phishing Solutions 

To combat the increasing sophistication of generative AI-powered phishing attacks, organizations must adopt advanced anti-phishing solutions that themselves leverage artificial intelligence. These technologies analyze patterns and behaviors that are indicative of phishing attempts, going beyond traditional signature-based detection methods. 

Advanced anti-phishing solutions use LLMs to identify signs of generative AI in phishing messages and determine the likelihood of a sophisticated phishing attack. By combining traditional heuristics, sender and domain analysis, and generative AI-based detection, these solutions can identify and stop the new wave of phishing attacks. 

2. Advanced Email Filtering 

Advanced email filters use algorithms and machine learning to analyze incoming emails for signs of phishing, including unusual sender behavior, malicious attachments, and links to known phishing sites. By examining the context and content of each message, these filters can identify and quarantine emails that exhibit characteristics of phishing attempts, even if they do not match known patterns.

These systems are also capable of learning from new threats, adapting their detection mechanisms as attackers evolve their tactics.

3. Context-Based Defenses

Context-based defenses against gen AI phishing attacks involve deploying security measures that understand the context in which communications occur. They use artificial intelligence and machine learning to analyze not just the content of messages, but also their context, timing, and the relationship between sender and recipient. 

By understanding typical communication patterns within an organization, these systems can flag anomalies that may indicate a phishing attempt. For example, if an email requesting sensitive information is sent at an unusual time or from a suspicious domain, a context-based system would identify it as a potential threat.  

In addition, email authentication protocols like DMARC (Domain-based Message Authentication, Reporting & Conformance), DKIM (DomainKeys Identified Mail), and SPF (Sender Policy Framework) help verify the authenticity of emails, reducing the likelihood of spoofed messages reaching end-users.  

4. Multi-Layered Security 

A multi-layered security approach combines various security measures at different levels within the organization to create a comprehensive defense strategy. It includes deploying advanced endpoint protection to detect and block malicious activities on individual devices, network security solutions to monitor and control traffic, and email security systems that use advanced filtering and authentication techniques.

Incorporating behavior analysis tools can also help identify unusual patterns that may indicate a phishing attempt, such as an employee accessing sensitive data at odd hours. By integrating these diverse layers of security, organizations can ensure that even if one defense mechanism fails, others are in place to mitigate the risk.  

5. End-User Training 

By educating employees on the latest phishing tactics, including how generative AI is used to create highly convincing scams, organizations can empower their workforce to act as the first line of defense. Training should cover the identification of phishing emails, safe practices for handling suspicious messages, and the importance of reporting potential threats to the IT security team.

Incorporating regular simulations of phishing attacks can help reinforce learning and gauge employee readiness. These simulations mimic real-life phishing scenarios, providing practical experience in spotting and responding to sophisticated attempts. 

AI-Based Phishing Protection with Perception Point

Perception Point uses AI to fight AI to protect the modern workspace by uniquely combining an advanced AI-powered threat prevention solution with a managed incident response service. By fusing GenAI technology and human insight, Perception Point protects the productivity tools that matter the most to your business against any threat. 

Patented AI-powered detection technology, scale-agnostic dynamic scanning, and multi-layered architecture intercept all social engineering attempts, file & URL-based threats, malicious insiders, and data leaks. Perception Point’s platform is enhanced by cutting-edge LLM models to thwart known and emerging threats.

Reduce resource spend and time needed to secure your users’ email and workspace apps. Our all-included 24/7 Incident Response service, powered by autonomous AI and cybersecurity experts, manages our platform for you. No need to optimize detection, hunt for new threats, remediate incidents, or handle user requests. We do it for you — in record time.

Contact us today for a live demo.