Malware Detection: 7 Methods and Security Solutions that Use Them

What Is Malware Detection? 

Malware detection is the process of identifying malicious software (malware) on a computer or network. This is usually done through the use of anti-virus software or other security tools that are designed to detect, quarantine, and remove malware. The goal of malware detection is to protect computer systems and networks from harm caused by malicious software such as viruses, Trojans, ransomware, and other forms of malware.

7 Malware Detection Techniques

1. Application Allowlisting

Application allowlisting is a technique that involves specifying a list of approved applications that are allowed to run on a computer or network. Any other application that is not on the allowlist is automatically blocked from running. This method of malware detection operates under the assumption that most malware is delivered through software that is not explicitly approved by the system administrator or user. By only allowing known and trusted applications to run, the risk of malware infection is greatly reduced.

Allowlisting can be implemented through software such as endpoint security products, firewalls, or other security tools. The allowlist can be updated periodically to include new applications and remove outdated ones, and can also be configured to automatically block new applications until they have been reviewed and approved. Application allowlisting is considered a proactive and effective method of malware detection, as it blocks malicious software before it can run and cause harm to the system.

2. Signature-Based Detection

Signature-based detection is a technique for detecting malware that involves comparing the characteristics of a potential threat to a database of known malware signatures. The signature is a unique identifier for a specific type of malware and can be made up of various elements such as file size, hashes, strings, or other identifying characteristics. Anti-virus software, intrusion detection systems, and other security tools use signature-based detection to scan for malware on a computer or network.

When a potential threat is detected, the anti-virus software compares its characteristics to the database of known malware signatures. If a match is found, the software will either quarantine or remove the malicious software from the system. Signature-based detection is considered to be a reactive method of malware detection, as it can only detect malware that it has a signature for, and new or previously unknown malware may go undetected.

Despite its limitations, signature-based detection is still widely used as a primary method of malware detection, as it is relatively fast, efficient, and can provide a high level of accuracy when used in conjunction with other techniques, such as heuristics-based detection or reputation-based detection.

3. Enhanced Signature Analysis: Checksumming

Checksumming is an improved version of signature analysis, focusing on the calculation of CRC (Cyclic Redundancy Check) checksums. This approach was created to address the primary drawback of signature analysis, namely the large database it requires and the frequent occurrence of false positives.

To bypass traditional detection methods, attackers often employ polymorphic malware in their malicious advertising campaigns, making them harder to detect. Polymorphic viruses have the ability to alter their “body” during replication, eliminating any constant search strings. This means that even when a signature is identified, this kind of malware lacks a consistent fragment of virus-specific code to be detected. Polymorphism is typically achieved by incorporating non-constant keys containing random sets of decryption commands into the main virus code or by modifying the executable virus code. Since a variable code has no signature, alternative techniques must be employed to identify the malicious code:

Reduced Masks

By examining elements within the encrypted body of the virus, researchers can isolate the encryption key and obtain a static code. The signature, or mask, can then be identified within the resulting static code.

Known Plaintext Cryptanalysis

This technique involves using a system of equations to decode an encrypted virus body, similar to solving a classical cryptographic problem where one decodes an encoded text without keys (with some differences). In cryptanalysis, the system reconstructs the keys and the decryption algorithm. The encrypted virus body is then decoded by applying this algorithm to the encoded fragment.

Statistical Analysis

The system can evaluate the frequency of processor commands used and utilize this information to determine whether a file is infected or not.

Heuristics

Malware researchers analyze large volumes of data, searching for suspicious activity and patterns. This method involves looking for malicious code associated with suspicious behavior, such as code being served to thousands of users within a short time frame. Researchers would take note of such activity and investigate further.

4. Malware Detection Using Deep Learning

This approach uses deep neural networks, which are complex models that are designed to learn patterns and relationships within large datasets, to analyze and classify software as either benign or malicious.

In deep learning-based malware detection, the neural network is trained on a large dataset of known malware and benign software. During training, the network is fed examples of malware and benign software, and it learns to identify the key features and patterns that differentiate the two. After training, the network can then be used to analyze new software and make predictions about its nature, based on the learned patterns.

The advantage of deep learning-based malware detection is that it can detect new and unknown malware that may not have been seen before. This is because the neural network is not limited to the fixed set of signatures or rules used in traditional signature-based or rule-based malware detection methods. Instead, it can learn to identify new and evolving threats, making it a powerful and effective tool for detecting malware.

5. Recursive Unpacking

Recursive unpacking is the process of discovering threats at any nesting level within content such as files and URLs attached to emails, shared via cloud collaboration tools, or stored on cloud storage platforms.

This capability is crucial for defending against malware, as attackers often hide malicious content deep within files as an evasion technique, relying on the fact that many security solutions do not scan embedded content.

6. Analyzing Files on Different Operating Systems

As attackers become more sophisticated, they seek exploits and vulnerabilities specific to particular operating systems (OS). For instance, attackers may search for MS Office vulnerabilities in Word or Excel files that exist only on macOS environments.

This is another evasion technique, as attackers are aware that detonating files and dynamically scanning them on both Windows and macOS is a capability that most security solutions lack or cannot afford due to inefficiency and slow processing times.

Thus, while many focus on scanning for Windows OS threats, attackers exploit macOS vulnerabilities to launch successful malware attacks.

7. Next-Generation Sandboxing

Traditional sandboxing is a behavior-based detection method that relies on application-level checks, which attackers have often learned to bypass.

An alternative, modern approach to traditional sandboxing involves using CPU-level analysis. This next-generation sandboxing technique also runs files dynamically within a virtual machine but focuses on analyzing content to reveal core exploit techniques earlier in the kill chain, pre-malware release, in a deterministic, non-behavioral manner.

This method takes advantage of Intel PT (Processor Trace) to access the full execution flow of the potentially malicious artifact and analyze it using a complete “trace” alongside examining changes to virtual memory during execution.

Since files are analyzed at the CPU/memory level rather than the application level, this next-generation sandboxing approach is significantly faster compared to traditional sandboxes, which may take minutes to scan each piece of content.

Learn more in our detailed guide on how to prevent malware.

Types of Malware Detection and Protection Solutions 

There are several types of malware detection and protection solutions designed to identify, prevent, and remediate malware threats. These solutions often employ various techniques and technologies to provide comprehensive security against different types of malicious software. Some of the most common types of malware detection and protection solutions include:

• Antivirus software: Antivirus software is designed to scan, detect, and remove known viruses, worms, and other types of malware from computer systems. It uses signature-based detection to compare files on the system to a database of known malware signatures.
• Anti-malware software: Anti-malware software extends beyond antivirus software to detect and remove a wider range of malicious software, including Trojans, spyware, adware, and ransomware. It employs a combination of signature-based and behavior-based detection techniques.
• Intrusion Detection and Prevention Systems (IDS/IPS): IDS monitors network traffic for suspicious activities and alerts administrators of potential security breaches, while IPS actively blocks or prevents detected threats. Both can be host-based (HIDS/HIPS) or network-based (NIDS/NIPS).
• Endpoint Protection Platforms (EPP): EPP solutions provide centralized management and protection of endpoint devices, such as desktops, laptops, and mobile devices. They typically include antivirus, anti-malware, and other security features like device control, application control, and data loss prevention (DLP).
• Security Information and Event Management (SIEM): SIEM systems collect and analyze security events and log data from various sources to provide a comprehensive view of an organization’s security posture. They help detect and respond to potential threats by correlating events and identifying patterns indicative of a security incident.
• Sandboxing: Sandboxing solutions provide a secure, isolated environment for analyzing and testing suspicious files or applications without risking harm to the main system. They can help detect previously unknown malware by monitoring the behavior of the suspicious file and identifying any malicious actions.
• Email Security Solutions: Email security solutions are designed to protect organizations and individuals from email-based threats, such as phishing, spear-phishing, and malware delivered through email attachments or embedded links. By employing a combination of signature-based and behavior-based detection techniques, email security solutions can identify and block malware delivered via email. You can see an example of this in the Evasive RAT attack example below:

Related content: Read our guide to malware protection.

Preventing Malware with Perception Point

Perception Point delivers one platform that prevents malware from reaching your end users, as well as other types of cyber attacks including phishing, ransomware, APTs and zero-days.

Advanced Email Security is an integrated cloud email security solution (ICES) that can replace SEGs. The solution cloud-native SaaS solution protects your organization against all threats using 7 layers of advanced threat detection layers to prevent malicious files, URLs, and social-engineering based techniques.

Advanced Browser Security adds enterprise-grade security to your organizations native browsers. The managed solution fuses browser protection technology with multi-layer advanced threat prevention engines which delivers the unprecedented ability to detect and remediate all malicious threats from the web, including phishing, ransomware, malware, APTs, and more. Multi-layered static and dynamic detection capabilities instantly detect and block access to malicious/phishing websites and prevent malicious file downloads of ransomware, malware, and APTs.

Advanced Threat Protection for Cloud Collaboration, File Sharing and Storage Applications, such as Microsoft 365 applications (OneDrive, SharePoint, Teams), Google Drive Box, AWS S3 buckets, Zendesk, Salesforce, and any of the other hundreds of apps out there, protects your organization with near real-time dynamic scanning. It does not tamper with files and does not impede on productivity.

An all-included managed Incident Response service is available for all customers 24/7 with no added charge. Perception Point’s team of cybersecurity experts will manage incidents, provide analysis and reporting, and optimize detection on-the-fly. The service drastically minimizes the need for internal IT or SOC team resources, reducing the time required to react and mitigate web-borne attacks by up to 75%.

Get a demo today!