Understanding the Samples Sources and Methodology Used in Advanced Threat Defense Testing
December 21, 2015
By Jack Walsh

Samples Sources

A number of malicious sample sources feed into ICSA Labs’ Advanced Threat Defense (ATD) testing.

One source is the spam ICSA Labs collects. ICSA Labs collects hundreds of thousands of spam messages every day through its spam honeypots. Specific attention is paid to spam with attachments.  If the attachments are malicious, they are sample source candidates for ATD testing.

Another sample source is malicious URLs.  Some of these come from the spam mentioned above.  In addition to URLs in the spam, ICSA Labs also receives malicious URL feeds from a number of sources.  Additionally, ICSA Labs sees if there is a malicious file on the other end of this URL -- either as a direct file link or by following a series of steps (e.g. a drive-by attack with a multi-stage download process) leading to it.

False-Positive Testing

ICSA Labs mixes legitimate applications and their associated activity into its ATD testing.  False-positive test runs using innocuous applications and their activity help ensure vendor solutions aren’t tuned to simply identify everything as malicious.

Pre-Use Analysis and Sample Modification

Before malicious samples are utilized in testing, they go through a pre-use analysis process.  This helps filter, select and prioritize the samples that ultimately end up in the detection effectiveness testing queue. The pre-use analysis process includes both static and dynamic analysis techniques utilizing fuzzy hash clusters, anti-virus scanner detections and behavioral activities to name a few. 

Following pre-use analysis, ICSA Labs changes a large percentage of the test samples to make them new and different. These modified samples are then fed back into the pre-use analysis process.

The pre-use analysis process helps ensure that ATD testing avoids dead samples, a flood of duplicates of the same attack, and other undesirable conditions.

How ICSA Labs Verifies Samples Used are Malicious

As part of the pre-use analysis process, behavioral analysis is performed utilizing sandboxes (note that ICSA Labs has its own internal sandbox) to identify suspicious events (including network traffic) which are used both as part of the selection process and to target behavior while performing attacks.

Anti-virus scanner detection results are also used to help filter already known samples, weed out potentially unwanted applications (PUAs) and to classify candidate samples for modification before use in testing.

Next Week

Next week’s post is the fourth and final entry in this blog series.  It will explain how enterprises benefit from this regularly recurring testing. 


Post new comment

  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>

More information about formatting options

This question is for testing whether you are a human visitor to prevent automated spam submissions.
Enter the characters shown in the image.