If your organization uses a Sandbox as a solution for determining whether files are malicious or benign, and you rely heavily on its analysis results, in this post we’ll look at why you should never fully trust Sandbox results.

It doesn’t matter whether you use commercial Sandboxes such as Any.Run, Hybrid Analysis, and Joe Sandbox, Sandboxes integrated with EDR solutions, or any other type of Sandbox. The important thing to keep in mind is that trusting the result of a Sandbox analysis and allowing a file into an Enterprise environment solely based on that result can have serious consequences and may ultimately lead to a major security incident.

Why…? 🤔
The answer is pretty simple. What does a Sandbox actually do? Under ideal conditions, assuming the Sandbox environment is suitable for the file we’re about to analyze, it executes the file in an isolated environment, monitors its behavior, and reports things such as API calls, process creations, network activity, Registry changes, file system changes, and so on.

I’ve seen organizations whose Sandbox infrastructure is designed specifically for Windows tools and can therefore only execute PE files. Then, they happily throw Linux and Android files into it and are pleased when they get a Clean result! 🙂

So here’s the question:
What percentage of today’s Malware actually executes its main Payload immediately after the file is launched?

If we look at this topic very broadly, a Malware may be designed in a way that:

  1. It only activates on a specific Target system or in a specific environment. One example of Malware whose behavior was heavily dependent on the characteristics and conditions of the Target environment was Stuxnet, which I think most of us are at least somewhat familiar with.
  2. It needs to receive a specific parameter or satisfy a specific condition. This parameter could be generated and obtained locally, entirely Offline, or retrieved Online through communication with an external infrastructure. Until it receives this parameter, the Malware may exhibit completely normal behavior.
  3. It may make execution of its main Payload dependent on a specific Event. This Event could be a specific time, the creation or modification of a file, the execution of a particular Process, or pretty much anything else you can think of. It could even be directly related to a Business process within the organization. In such cases, the Malware may remain on the system for months without showing any malicious behavior until the Event it is waiting for actually occurs.
  4. Another category of Malware makes execution of its main Payload dependent on the presence of specific hardware, a Peripheral, or particular hardware conditions. As a result, if the expected hardware or conditions are not present, the Malware may simply continue executing normally without exhibiting any malicious behavior.
  5. And so on…
So, executing a file in a Sandbox does not necessarily mean that we have observed the Malware’s real behavior. The file may have executed successfully, and the Sandbox may have monitored everything, but what we actually observed could simply be the behavior the Malware chose to exhibit — not its actual behavior…

Now let’s put ourselves in the shoes of the person or team developing a Malware specifically for a particular organization…

One of the things we would definitely consider is implementing techniques that prevent the organization’s security Solutions — such as Antivirus, EDR, and Sandbox — from detecting our Malware, while making detection as costly and difficult as possible.

If the organization’s security Solutions are going to detect the Malware right from the start, then what kind of Malware would it be in the first place? 🤠

Of course, Antivirus, EDR, and other Solutions are not the focus of this post. We’re only going to talk about Sandboxes here…

How Can We Prevent a Sandbox from Detecting the Malware’s Real Behavior?

As mentioned earlier, many modern Malware samples inspect their environment before executing their Payload. If they detect that they are running inside an analysis environment, Sandbox, or lab system, they may choose not to execute their main Payload or may exhibit completely normal and harmless behavior instead of their actual behavior.

In the following sections, we’ll dive a little deeper into this topic and examine some of the techniques Malware uses to identify these environments:

Many Sandboxes run on VMware, VirtualBox, Hyper-V, or KVM. Therefore, there’s a possibility that Malware can detect this by collecting and analyzing various parameters such as BIOS data, system specifications, Devices, CPU information, Drivers, Registry keys, DMI Vendor/Model information, and so on.

Of course, we need to distinguish between Virtualization Detection and Sandbox/Analysis Environment Detection. Being a VM is just one of the Indicators that Malware can use to identify an analysis environment.

An analysis/Sandbox VM may have an unusual number of CPU cores, amount of RAM, or disk space compared to the organization’s real Endpoints. By checking parameters such as the number of Processors, available Memory (for example, using GetMemoryStatusEx), and available disk space (for example, using DeviceIoControl / GetDiskFreeSpaceEx), Malware can potentially detect this.

For example, the Blitz Malware, which was first identified in late 2024, attempted to identify analysis environments and Sandboxes by checking the number of CPU cores, screen resolution, and installed Drivers, and would alter its behavior when running in such environments.

A Sandbox typically doesn’t have an active physical monitor. So, Malware can simply use functions such as GetSystemMetrics and EnumDisplayMonitors to determine the number of monitors and their resolutions.

The name and MAC Address of a Network Adapter can be an Indicator for detecting a Hypervisor or VM environment. For example, VirtualBox MAC Addresses typically start with 08:00:27, while QEMU/KVM MAC Addresses use 52:54:00 by default. VMware commonly uses 00:0C:29 and 00:50:56 by default.

It’s also important to keep in mind that in a targeted attack, the person or team developing the Malware will usually have prior information about the organization’s security Solutions, including the Sandbox being used. This makes designing and implementing these detection mechanisms much easier and more targeted.

Some Sandboxes execute a file immediately after Boot, meaning the System Uptime may only be a few seconds, whereas this is not typically the case on real systems. Therefore, Malware can check the Uptime and, for example, if it’s less than 60 minutes, either wait or simply avoid executing its main Payload altogether.

Human Interaction can be considered one of the most significant differences between a real environment and a Sandbox.

A real user moves the Mouse, uses the Keyboard, likely has data in the Clipboard, has various Applications installed and actively uses them, has a Browser filled with History and Cookies, has previously opened a collection of files (Recent Documents), and switches between windows in ways that are different from what you would typically see in an Automated or Sandbox environment — and there are many other things like this.

Malware can therefore examine various parameters to look for these indicators and decide, based on the results, whether to execute its main Payload or not. In many cases, Malware is also designed to completely call ExitProcess if it doesn’t find these indicators, Sleep for a certain period and wait, or simply exhibit normal and harmless behavior.

For example, a Malware called LummaC2 used geometric and trigonometric calculations to analyze Mouse movement patterns and attempted to determine whether the Mouse was being moved by a real user or by a script by analyzing the angles between movements.

There are certain files and paths whose presence can be an Indicator that the system is running in an analysis environment. For example, the following paths are associated with VMware and VirtualBox:

C:\Windows\System32\drivers\vm*
C:\Program Files\VMware\
C:\Windows\System32\VBox*

Some Sandboxes run analysis and monitoring tools directly inside the environment. The presence of these tools can itself be used as an Indicator of an analysis environment.

Many Malware samples change their behavior as soon as they detect tools such as x64dbg, Procmon, or similar tools running.
In many cases, if the Malware only checks the Process name, this mechanism can be bypassed by renaming the tool. For example, instead of running procmon.exe, we could rename it to prokman.exe.
Naturally, this approach doesn’t always work and depends on how the detection mechanism is implemented. The Malware may check the Path, Hash, Window Class, Loaded Modules, Drivers, Mutexes, Services, or other Artifacts instead of simply looking at the Process name.
(In this post, we’re not going to discuss ways to bypass these Evasion techniques. We’ll leave that topic for future posts.)

Some Malware samples don’t just check for basic Connectivity; they also expect DNS Resolution, HTTP/TLS, Proxy configuration, specific Network services, Routing, and the expected Response from their infrastructure. If these conditions aren’t met, they may avoid executing their main Payload.

If you’re thinking to yourself at this point, “Well, this and many of the previous techniques can be easily bypassed!”, you’re absolutely right. Just be patient — we’ll get to that too.

Many Sandboxes have specific Hostname, Username, and Domain Name values by default. Malware can easily detect that it is running in an analysis environment by checking these values.

These techniques interact directly with CPU Instructions. Since these checks are directly related to the behavior of the Virtualization Layer, some of them can be harder to hide or emulate compared to typical OS-level Artifacts.

For example, one technique is to send 0x40000000 to CPUID as the Leaf. In environments where the Hypervisor Vendor Interface is exposed, executing CPUID with Leaf 0x40000000 can return the Hypervisor’s Vendor ID.

Feature and Interface Discovery

For example, VMware has a proprietary interface called the VMware Backdoor for communication between the Guest and the Hypervisor. Malware can send a request to the I/O port 0x5658 along with the value VMXh to check whether it receives a Response. If it does, there’s a high probability that it’s running inside VMware.

Another example is the use of the Trap Flag (the TF bit in the EFLAGS/RFLAGS register). When this bit is enabled, the CPU generates a Debug Exception (#DB) after executing each Instruction. The interesting part is that this behavior can have subtle differences between a real environment and a Hypervisor. Malware can leverage these subtle differences to detect whether it is running inside a VM.

This technique was reported and subsequently addressed by Palo Alto Networks in 2021 in connection with the behavior of certain VMs.

In some specific scenarios, Malware can use information from hardware sensors such as CPU temperature, Fan Speed, or Power Management status as part of a set of Indicators for identifying its execution environment.

On a real system, there is typically an expected relationship between CPU Load, temperature increase, and changes in Fan Speed. However, in some Sandboxes, these sensors may not be available, may report unrealistic values, or may otherwise behave in ways that don’t match what would normally be expected. Malware can detect these inconsistencies.

Another technique Malware can use alongside other methods is checking the system’s current Wallpaper.

Many Sandboxes and analysis environments use the default Windows Wallpaper or other well-known Wallpapers. Malware can therefore inspect the current Wallpaper and compare it against known values or patterns to obtain an Indicator that it may be running in an analysis environment.

Of course, Malware typically doesn’t rely on a single Indicator. Instead, it uses multiple techniques together and ultimately calculates a Score representing the likelihood that it is running in an analysis environment. Something like:

VM Detection Score:
  CPUID +10
  No USB Devices +5
  No Browser History +5
  VM MAC Address +10
  Low System Uptime +5
  No User Files +10
  Default Wallpaper +5

Total Score > 25

=> Sandbox / Analysis Environment

Well, to keep this post from getting too long, let’s stop this category of techniques here for now…

All the techniques we’ve discussed so far are actually just a small and relatively simple subset of the techniques used to detect Sandboxes and analysis environments. Since the goal of this post wasn’t to dive into highly technical details, we’ve focused mainly on the basic and common techniques.

You’ve probably thought to yourself while reading through some of the techniques above: “Can’t we just change this in the Sandbox to make it look like a real system?” Yes, exactly…

One of the reasons for learning about these techniques is precisely that: to design our analysis environment to resemble a real environment as closely as possible, making it harder and more costly for Malware to detect that it is running inside a Sandbox.

Imagine a Malware sample specifically designed to run inside your organization and programmed not to execute its main Payload until it is certain that it is actually inside your organization. Now, let’s assume we have the most advanced commercial Sandbox available. How closely does the environment of that Sandbox, in its default configuration, actually resemble our organization’s real environment? Closely enough to convince the Malware to execute its Payload?

That is exactly the point of this post. The whole idea can be summed up like this: having the best Sandbox from a technical perspective is only half the story. The other, equally important half is making the Sandbox environment resemble our organization’s real environment as closely as possible.

Having an advanced Sandbox without making its execution environment resemble the Target organization as closely as possible simply cannot give us the results we expect. If we can get our Sandbox to that point, then we can place more confidence in its results — although, once again, never with complete certainty.

If you want to dive deeper into this topic, Evasive Malware is one of the best resources published on the subject, and I highly recommend reading it.

If you don’t want to start from Chapter 1 and would rather jump straight into Sandbox detection and Evasion techniques, I recommend starting with Chapter 4.

Another question you may have is that these days, a large portion of an organization’s services run on VMs. So if a Malware decides not to execute its Payload simply because it detects that it’s running inside a VM, wouldn’t it end up missing the real Target as well?

That’s absolutely true. But as I mentioned earlier, in targeted attacks, Malware is specifically developed for a particular organization and, based on its knowledge of that organization’s infrastructure, determines when and under what conditions to execute its Payload.

Another point worth mentioning is that while detecting the execution environment is an advantage for Malware, it isn’t always an advantage. In fact, the more techniques a Malware uses to identify its execution environment, the higher the chance that the Malware itself will be detected.
Of course, when Malware is developed specifically for a particular organization, it is usually tested against the Security Solutions used by that organization — such as Antivirus, Sandbox, and EDR — before being deployed, to make sure the Malware remains as difficult to detect as possible.

Let’s get back to the main topic:

Is Using a Sandbox Useless?

Absolutely not. Taking all the points we’ve discussed so far into account, a Sandbox is still one of the most important and useful tools for Malware analysis. In fact, even if an organization has all the other Security Solutions in place but doesn’t have a Sandbox, it is missing an important part of the Malware analysis and detection chain.

So what’s the solution?

We should try to make the Sandbox environment resemble a real system in the organization as closely as possible, reducing the likelihood of Sandbox detection and, consequently, Evasion.

If you think the Sandbox currently deployed in your organization is already very good and comprehensive, I recommend taking a look at the techniques discussed in Evasive Malware before putting too much trust in it. You might change your mind a little after reading it!

How Can We Address Sandbox Weaknesses?

We should make whatever changes are possible to the Sandbox environment, both at the OS level and in the configurations provided by the Vendor, to achieve two goals:

First, prevent Malware from easily detecting that it is running in an analysis environment.

Second, make the Sandbox environment resemble the organization’s real systems as closely as possible.

In advanced commercial Sandboxes, the part related to preventing analysis-environment detection is already covered to a large extent, so our job is easier in that regard. However, simulating the organization’s real environment is still up to us.

Well, trying to find every Sandbox Detection technique one by one and then checking whether our Sandbox is vulnerable to each of them isn’t really practical. It takes a lot of time, and we would constantly have to keep looking for new techniques.

The skill of a Malware developer is finding ways to detect the analysis environment. New techniques are discovered every year. We find one technique and make the Sandbox resistant to it, then a while later another technique is discovered, and the cycle continues…

A quick way to approach this is to use a set of ready-made tools designed to assess the environment and check some of the Indicators associated with Virtualization and Sandbox Detection. Tools such as Al-Khaser, Pafish, VMwareCloak, and VBoxCloak can be useful for this purpose.

What we need to do is run these tools inside the analysis environment so they can identify and list the Indicators that could allow Malware to detect that it is running in a Virtualized or analysis environment. We can then address or Mitigate these Indicators in the analysis environment as much as possible.

A sample output from one of these tools looks like this:

Sample Al-Khaser Output

So far, let’s assume we’ve managed to make the Sandbox environment resemble a real system to a large extent. But how can we make it resemble our organization’s real environment?

At this point, we’re no longer talking about Anti-VM or Anti-Sandbox configurations. We need to identify the characteristics of the systems in our organization that Malware could use to determine its execution environment, and replicate those characteristics in the Sandbox as much as possible. Things like the OS version and Build, installed Software, Browsers, Security Agents, Services, system configurations, Domain, Network topology, Proxy and DNS settings, and even User Behavior.

The goal isn’t to Clone a specific system. The goal is to turn the Sandbox into a representative Endpoint of the organization’s real environment.

The greater this similarity, the smaller the gap between what the Malware sees in the Sandbox and what it will encounter inside the organization. As a result, it becomes more difficult for the Malware to distinguish between the two environments.

A Proposed Approach for File Analysis in an Organization

If we’re dealing with a relatively large organization, we obviously can’t send every file that is about to enter the organization to the Malware Analysis team for manual analysis. So, what’s the solution?

A Risk-Based Scoring approach can be used here. The idea is to combine the results from multiple layers of analysis and ultimately derive a Risk Score, for example:

Multi-AV + Static Analysis (Automated) + Sandbox Analysis → Risk Score

If the Risk Score is low, we can allow the file to enter the organization based on the organization’s Policy and sensitivity level. However, if the final Score exceeds a defined Threshold, the file can be sent to the Malware Analysis team for deeper analysis.

This approach allows us to leverage Automation to handle a high volume of files while separating more suspicious files for specialized analysis.

No matter how good our Sandbox is, analyzing Malware in an Interactive Behavioral Analysis Environment can still provide the Analyst with more accurate and deeper insights.
An environment where the Analyst can interactively examine the Malware’s behavior during execution, modify the execution conditions, and test different hypotheses based on what they observe in real time.
We’ll talk about this in more detail in future posts.

Summary

One more important point: it’s not the case that we can send a file to the Malware Analysis team and expect them to tell us that the file is definitely safe.
The results from different layers, combined with the Malware Analysis findings, come together to help us make a more informed decision about the file.

Stay safe and keep learning! ❤️

Why You Shouldn’t Trust Sandbox Results was originally published in System Weakness on Medium, where people are continuing the conversation by highlighting and responding to this story.