In 2019, we visited The MITRE Corporation and took part in the MITRE ATT&CK® Evaluation Round 2. During this very in-depth 3-day assessment, our EDR solution, together with our MDR service, were tested against emulated attack techniques of the APT29 threat group. For those unfamiliar with the rules, conditions and detection categories of the Evaluations, we'd recommend reading our overview of MITRE methodology first.
With the results of our Evaluation now announced, we'd like to sum these up, give our analysis, and look at some interesting examples of individual test runs.
High-level overview: Tactics
Adversary techniques in ATT&CK Evaluations are grouped by tactics, or stages of the attack. In general, our EDR product, together with our MDR service, showed good coverage of most of the techniques tested. The table below shows the ATT&CK tactics and techniques for APT29 detected by the Kaspersky solution:
We can see that Kaspersky’s solution demonstrated its highest levels of visibility in the Execution, Persistence, Privilege Escalation, and Lateral Movement attack stages. Good visibility of these tactics, i.e. the ability of the security solution to display the events with sufficient completeness and detail, significantly increases the probability of successfully detecting an attack.
First, it’s important to see the initial stages of the attack (Execution, Persistence), while it's still possible to take necessary measures to stop the attack from developing.
Our solution also demonstrated high-quality detection in the later stages - a significant factor for Threat Hunting in systems where intrusion has already taken place. Most crucial of all is the detection of those types of malicious activity that are hard to fight with preventive technologies (Privilege Escalation, Lateral Movement).
A closer look: Techniques
Now let's look more closely at the attack methods simulated in the ATT&CK Evaluation. Our solution demonstrated the best level of visibility for these techniques:
This table shows confident detection of the key techniques frequently used in modern targeted attacks. First, there are technics of early stages of the attack, when adversaries will try to execute malicious code (Powershell, Command-Line Interface, Rundll32, Windows Management Instrumentation, Service Execution, User Execution), to maintain their foothold in the attacked system (Windows Management Instrumentation, Event Subscription, Create Account), to gain higher-level permissions (Bypass User Account Control) and to evade defenses (Masquerading, NTFS File Attributes, Valid Account, Component Object Model Hijacking). The detection of these techniques means the attack can be blocked before any serious damage is done.
At the later stages, we demonstrated the effective detection of the lateral movement-related techniques (including the tricky one – Pass the Ticket) and of Collection stage techniques (Input Capture, Screen Capture, Data Staged, Email Collection).
Our solution also demonstrated good visibility for data exfiltration (Data Compressed technique - all attempts to pack the data and pull it out of the system should be clearly seen by the security officer); as well as Commonly Used Port technique (Command and Control tactic), which is quite important, because the communication between a malicious tool and its command and control centers is actually the most popular way of exfiltrating data from the infected system.
In depth: Test Runs
Now let’s look at exactly how our solution, Kaspersky EDR, detected some of the most important attack techniques during Evaluation Round 2.
Step 1.A.2
Adversaries used a special Unicode symbol - right-to-left override character – to mislead the user: he would see the file name rcs.3aka3.doc, while the real name of this file was cod.3aka.scr. Clicking on this file, the user would think he was opening a Word document, while in fact he would run a screensaver file (.scr), which is executable.
Our solution successfully detected these events as the attack technique T1036 Masquerading:
As you can see in the screenshot above, KEDR correctly labelled the technique in IoA Tags (yellow label) and revealed both names for the file: the name seen by the user (rcs.3aka3.doc) and the name with the real extension (.sсr). The security officer now has all the necessary data to recognize and understand this attempt to hide the real file extension.
The detection categories gained in this test run: Technique, MSSP, Telemetry.
Step 3.A.2
A special file monkey.png was uploaded to the infected endpoint. This file was a picture, but a tricky one: steganography was applied to hide some malicious payload in this picture, which would be used to develop the attack (second stage payload). This trick is often used by modern hackers because steganography helps to hide not only the data (as does cryptography) but also the very fact of a data transfer. In this particular test case, the payload was restored from the container by a Powershell script. Kaspersky EDR successfully detected this malicious activity:
As shown on this screenshot, the security officer could see all the events of the attack connected in one graph. The malicious object restored from the container (binary payload) was detected by our EPP solution on the endpoint, and its verdict (HEUR:HackTool.Win32.Inject.heur) was sent to Kaspersky EDR. Meanwhile, the suspicious Powershell activity was detected by two IoA rules with corresponding verdicts (more about IoA rules here). Red labels indicate a high level of danger and, based on this, the Kaspersky EDR dashboard displays a notification on its Alerts page.
Our solution also indicates where this malicious file was uploaded from, and shows details of the Powershell commands used to restore the payload.
Detection categories gained on this run: Technique, MSSP, Telemetry.
Step 3.B.2
In this test run, adversaries tried to bypass User Account Control and run the Powershell payload. Our solution labelled this activity as T1088 Bypass User Account Control technique:
In this screenshot, we see the execution of sdctl.exe utility which is often used for BypassUAC techniques: this is due to the value of autoElevate (set to «true») in the manifest of this application. The yellow label shows the correct name of the detected attack technique.
As the attack was developing, our solution also detected the elevated Powershell execution — red labels mean a high level of danger.
And again, KEDR provided the security officer with useful data: visibility not just of the fact that suspicious Powershell was running, but into exactly how this was done.
Detection categories gained: General (with Alert modifier), MSSP, Telemetry.
Steps 6.А.*
Here’s a group of interesting test cases: adversaries used a special application to extract the encrypted credentials from Chrome SQL DB, and this application was renamed as accesschk.exe. Our solution correctly showed that was going on:
According to the attack graph above, Powershell was used to run accesschk.exe, and this application, in turn, created a passwordDB file. Our EDR interface allows the retrieval of this file from the attacked endpoint by the security officer, just by clicking on the file name and selecting the «Get file» command:
When analyzing the retrieved passwordDB file, the security officer may want to know its origins - what application created this file? The name of this app is provided in the telemetry (accesschk.exe), but there’s a reason why it doesn't ring any bells - the file name has been deliberately changed to masquerade it to the legitimate Sysinternals tool. The security officer can retrieve this file from the endpoint, too, and explore it manually. But what if the attackers have already removed accesschk.exe?
In this case, our Threat Intelligence Portal can help (by the way, any user of Kaspersky EDR can make 1,000 free requests to this service). When we search for the MD5 hash of accesschk.exe in this portal, we can find the real name of the program: chrome-passwords.exe.
In the results of this test run, MITRE experts pointed out that our solution connected all these findings: «Detection Notes: Telemetry showed file creation PasswordsDB (Chrome Passwords database) as well as accesschk.exe execution (which indeed is renamed chrome-passwords.exe). PasswordsDB creation is correlated with general detection of Powershell»
Detection categories gained in this test run: MSSP, Telemetry.
Step 9.B.7
The main goal of most APT attacks is to steal sensitive data. So the data must somehow be pulled out of the infected system. Usually, attackers try to pack the data to an archive before the exfiltration: it's easier to grab everything as one file, and compression means less time taken for the transfer.
This method was used in the 9.B.7 step of the evaluation: adversaries applied additional compression to the collected archive and encrypted it with a password. Let's see how Kaspersky EDR detected this activity.
Our solution not only labelled the techniques correctly (T1002 Data Compressed and T1022 Data Encrypted) but also provided the password for the encrypted archive (fGzq5yKw). With this knowledge, the security officer can retrieve the archive from the endpoint, unpack it and see what data has been stolen. This should greatly help in investigating the breach.
The detection categories gained on this run: Technique, MSSP.
Step 11.А.2
Among the most interesting cases on the second day of the ATT&CK Evaluation, was the one where commands from ADS (Alternate Data Stream) were executed via Powershell. This is a popular way to add malicious payload to legitimate files: ADS streams are not seen in common file managers.
In the test, Kaspersky EDR correctly displayed the events and labelled them with the appropriate techniques:
It's also worth mentioning that the Launch Parameters section showed the full command used to run ADS: the stream was read from the file 2016_United_States_presidental_election_-_Wikipedia.html
Detection categories gained on this run: Technique, MSSP, Telemetry.
Step 11.A.10
As you may have noticed, the ATT&CK adversaries used Powershell many times during the evaluation. That’s quite understandable: the functionality of this application is very broad, and it should be detected with great care — Powershell is a system utility that can be run by legitimate users and sysadmins. This can result in multiple false positives which can then dramatically increase the Total Cost of Ownership (TCO) of the security solution, as considerable resources are wasted double-checking events that have nothing to do with security incidents.
For the same reason, other system utilities are also popular in modern targeted attacks. In the test case 11.A.10, the adversaries used the legitimate application certutil.exe to decode the malicious payload from the file kxwn.lock. Here’s how our solution detected this activity:
Kaspersky EDR has clearly warned the security officer about the suspicious usage of certutil.exe to decode the payload, and has also demonstrated the full command line used to run the decoding (see Launch Parameters section).
As a result, our detect gained Technique category with Alert modifier, as well as MSSP and Telemetry categories.
Step 16.C.1
Lateral movement is a crucial stage of APT attacks - the first endpoint compromised is not usually the ultimate goal of the attack, so the hacker has to move from one machine onto others. In the test case 16.C.1, adversaries used a legitimate tool (Powershell) and legitimate service (WinRM) to move within the attacked infrastructure, by establishing the WinRM connection to the domain controller NewYork (port 5985). This event was detected by Kaspersky EDR:
Our solution displayed the connection from Powershell to the 10.0.0.4 address, detected as powershell_with_network_activity - the red label means that an Alert was issued. WinRM use has been identified based on the “wsman” service name and the use of the 5985 port – we also showed this info in the test.
Detection categories gained: General, MSSP, Telemetry.
Step 16.D.2
The last case we'd like to look at in this article is an attempt to dump a KRBTGT password hash by running the hacking tool Mimikatz on the domain controller NewYork. Our solution successfully detected this technique:
Here you can see two red-label verdicts (mimikatz_commands_patterns и credentials_dumping_tools_services_or_processes) that correspond to the T1003 Credential Dumping technique. The security officer can also see the full command line used to run Mimikatz: the m.exe file is well-known to security experts.
This detection was given the highest category - Technique with Alert modifier, as well as MSSP.
Conclusion
The results of MITRE ATT&CK Evaluation Round 2 allows security professionals to see how EDR solutions from different vendors detect and visualize different steps of a possible APT29 attack which has been carefully emulated in a test environment. It also helps to assess the performance of the vendor’s Managed Detection and Response (MDR) team.
The Evaluation results show that, despite the test limitations, Kaspersky EDR demonstrates a high quality of detection and good visibility of adversary actions, especially for the key techniques applied at the most crucial stages of attacks launched by APT29 and other contemporary threat actors.
To achieve this, Kaspersky offers its clients a comprehensive solution that combines a fully automated multi-layered security product, and an automated manual Threat Hunting service provided by our professional SOC team. The importance of this combination has also been demonstrated in the ATT&CK Evaluation: even though many attack methods are well detected by our EDR product automatically, there are also techniques that require human expertise to uncover (MSSP Detects).
Participation in the ATT&CK Evaluation Round 2 was an interesting challenge and a valuable experience for us, and we’re already using this experience in further developing our solution, and continuing to raise the quality of our detection and usability. We look forward to participating in the Evaluation Round 3, and to consolidating our success as we detect the advanced attacks of another significant actor - Carbanak / FIN7.
To learn more about the ATT&CK use in our products, see Kaspersky in MITRE ATT&CK