Best Practices
Unique and Specific Procedures¶
For VECTR to track results of Purple Team activities over time, some way is needed to uniquely identify a specific test procedure. The MITRE Technique or Sub-Technique ID is not specific enough to accomplish this. For example, there may be many unique test procedures for T1047 - Windows Management Instrumentation.
It's important to create individual tests confirming detection and response for each of the different attack scenarios that may be categorized by a specific ATT&CK Technique. To accomplish this, VECTR's Test Cases map to the MITRE Enterprise ATT&CK "Procedure" concept, and VECTR uses Test Case Templates to uniquely identify a test procedure. These templates can be managed in the Library section of VECTR, as described below.
Local vs Template Data¶
VECTR's Assessments are located in an Environment. Each Environment represents a logical testing space where you may track related tests over time.
When creating Test Cases and measuring detection, we refer to this as "Local" data in contrast to the templates mentioned in the previous paragraph. A local Test Case in VECTR represents a recorded execution of an attack procedure and its defense response at a specific instance in time.
Local Test Cases¶
Local data in VECTR is contained in a test Environment. All Test Cases are performed in a Campaign which in turn is inside an Assessment.
Test Case Templates¶
Test Case templates are contained in the Library section of the application. Each of these is a description of a specific attack procedure that you may perform once or multiple times when testing an environment. Heed the note on the importance of naming below. Aim for as much specificity as possible when naming Test Case templates.
You should almost always be creating your local Test Cases from templates or mapping them to existing templates.
Naming¶
Naming is very important! Test Case templates should have very specific, detailed names that are unique to what they do. There are currently many ways to perform almost each MITRE Enterprise ATT&CK Technique and sub-technique.
There will continue to be new Techniques identified in the wild as adversaries evolve, and there will continue to be new Procedures identified that map to each Technique. Name them carefully so you can clearly identify what you're testing and understand where your gaps in detection and prevention exist.
Assessments should also be clearly named to describe the activity for easier reporting.
Example Assessment Names:
- Good Name: Manually call schtasks.exe to create a Scheduled Task
- Bad Name: Create a Scheduled Task
The 2nd example is a bad name because there are a lot of ways to create scheduled tasks beyond just running schtasks.exe. There's a footnote in the detection area of the ATT&CK framework for this that says "Remote access tools with built-in features may interact directly with the Windows API to perform these functions outside of typical system utilities. Tasks may also be created through Windows system management tools such as Windows Management Instrumentation and PowerShell, so additional logging may need to be configured to gather the appropriate data."
It may be beneficial to create additional Test Case procedures for those other methods or scheduled task creation to determine if your detection, prevention, and log aggregation measures are properly tuned to catch those as well
Tagging¶
Tag everything. Use tags for fix priority. Use tags for things you want to retest. Tag anything you want to follow up on. Tag with specific people's names. Tag anything that you had trouble reproducing or want to document.
Tags make reporting and sorting your Test Data a lot easier once you have a lot of it.
Ordering Data¶
Attack Lifecycles and Phases¶
Anywhere in the system where multiple Attack Lifecycles could be aggregated for display will use the Phase's Library Order. This includes all Reporting screens with non-fixed Phase lists. Because multiple Assessments could be displayed and therefore mulitple Attack Lifecycles could be in use, these views use the Phase Library Order for display. In VECTR if you navigate to Library -> Phases you can see an icon to configure this order.
This is not the same order that affects the Escalation Path shown for your Campaigns during active testing. Continue reading below.
Escalation Path¶
Attack Lifecycle information is used by Assessments to allow custom ordering of Phases during testing and evaluation. Each Assessment is associated with one Attack Lifecycle. The Attack Lifecycle for each Assessment defaults to the Default Attack Lifecycle which includes all MITRE Enterprise ATT&CK Tactics and some additional Phases (Tactics). The Phase order within an Attack Lifecycle can be configured in the Library -> Attack Lifecycle section of VECTR.
This order will affect the order of phases shown on the Escalation Path diagram for Campaigns being tested.