In Network IPS Testing, Are We An Extension of Vendor Quality Assurance?
June 4, 2010
By Jack Walsh

On a visit last year with a network IPS product developer, I happened to mention that to some degree ICSA Labs is like an extension of each IPS developer’s QA department.  A representative from the network IPS vendor who was present disagreed with my conclusion.  

Perhaps I should have said it another way:  Suppose during ICSA Labs network IPS testing a product initially misses coverage protection for 40, or 20, or a dozen, or even half-a-dozen vulnerabilities.   Then these holes in the product's network security coverage protection were either overlooked or missed by the company that developed the security device.  ICSA Labs often later finds such deficiencies and reports them to the vendor.  The question is: What does that make us?  

More recently a representative from a different network security product developer  wondered aloud during a conversation what ICSA Labs could find that his much larger company – with an obscenely vast QA staff by the way – could not?  The vendor initially thought it would be unlikely for us to find anything wrong with their product.  The truth though is that no product has EVER successfully completed ICSA Labs network IPS testing without correcting some number of shortcomings. This fact often surprises product developers: with each network IPS product, ICSA Labs can and does find a surprisingly large number of problems that need to be corrected. 

Ultimately, vendors don’t uncover the same weaknesses that we do.  ICSA Labs helps vendors through multiple testing iterations as they try to fix one or more flaws (i.e., they attempt to fix it, we test the fix, they attempt to fix it again, we test it again, etc.).   If that is not an extension of QA, it sure seems like it to me.

Comments

Post new comment

  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>

More information about formatting options

CAPTCHA
This question is for testing whether you are a human visitor to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.