What We Can Learn From Science Regarding Industry-Sponsored Testing of Security Products

Robert Cialdini, a Psychology professor at Arizona State University, stated his top-notch book, Influence Science and Practice:

"Take the case of the medical controversy surrounding the safety of calcium-channel blockers, a class of drugs for heart disease. One study discovered that 100 percent of the scientists who found and published results supportive of the drugs had received prior support (free trips, research funding, or employment) from the pharmaceutical companies; but only 37 percent of those critical of the drugs had received any such prior support. "

His statement was based on a scientific paper published in The New England Journal of Medicine in 1998. Details of the research can be found here:
Conflict of Interest in the Debate over Calcium-Channel Antagonists

Wow, these results are staggering. 37% of doctors were critical of a particular form of drug. But when some form of support is involved all doctors became in favor of such drug. This and other related research cited at the end of this article scientifically proves (at least from a psychology and medicine perspective) that industry-supported evaluation or testing of security related products such as antiviruses, IPS's, etc. have an influence on the quality and outcomes of their results.

Examples of such type of research studies in the security industry are:
1. Symantec funded an antivirus testing by PassMark: Consumer Antivirus Performance Benchmarks
2. Symantec sponsored another antivirus evaluation by Dennis Technology Labs: PC Anti-Virus Protection 2011
3. Trend Micro sponsored an antivirus testing by NSS Labs:(debatable) http://trendmicro.mediaroom.com/index.php?s=43&item=749
4. Microsoft sponsored two NSS Labs tests for comparing the security of IE8 with other browsers:
http://arstechnica.com/microsoft/news/2009/08/microsoft-sponsors-two-nss-reports-ie8-is-the-most-secure.ars
5. Trend-Micro commissioned West Coast Labs Anti-Spam comparison tests: http://it.trendmicro.com/imperia/md/content/uk/whitepaper/wp06_wclantispamrpt_090317us.pdf

The results of these studies are not surprising. Symantec was ranked first by Dennis Technology Labs and PassMark. Trend Micro was ranked first by NSS Labs. IE8 was shown to be far superior than its peers according to NSS Labs. Trend Micro topped the antispam comparison tests by West Coast Labs.

The reason I am blogging this, is because I have come across a lot of CIO's and security experts who still believe and take into granted the results published by such kind of studies. Its even a pity to see security gurus from notable organizations such as SANS fall into this and cite these results.

For more information please see:

1. Study: Industry-Sponsored Research Yields Favorable Results a Majority of the Time: http://www.doctorpundit.com/index.php/2010/08/03/study-industry-sponsored-research-yields-favorable-results-a-majority-of-the-time
2. The uncertainty principle and industry-sponsored research: http://www.ncbi.nlm.nih.gov/pubmed/10968436
3. Pharmaceutical industry sponsorship and research outcome and quality: systematic review http://www.bmj.com/content/326/7400/1167.full
4. Source of funding and outcome of clinical trials - Journal of General Internal Medicine http://www.springerlink.com/content/r654521305u8547k/

2010 Cairo Security Camp

I gave off a presentation at the 2010 Cairo Security Camp at Cairo, Egypt about 2 months ago. The event was held at Nile University Smart Village. My presentation was on rootkits detection and removal. I have also talked about ADS and MBR infections.
All praise to God, according to the attendees evaluation, I was voted as both, the most liked speaker, and best event topic.