"Take the case of the medical controversy surrounding the safety of calcium-channel blockers, a class of drugs for heart disease. One study discovered that 100 percent of the scientists who found and published results supportive of the drugs had received prior support (free trips, research funding, or employment) from the pharmaceutical companies; but only 37 percent of those critical of the drugs had received any such prior support. "His statement was based on a scientific paper published in The New England Journal of Medicine in 1998. Details of the research can be found here:
Conflict of Interest in the Debate over Calcium-Channel Antagonists
Wow, these results are staggering. 37% of doctors were critical of a particular form of drug. But when some form of support is involved all doctors became in favor of such drug. This and other related research cited at the end of this article scientifically proves (at least from a psychology and medicine perspective) that industry-supported evaluation or testing of security related products such as antiviruses, IPS's, etc. have an influence on the quality and outcomes of their results.
Examples of such type of research studies in the security industry are:
1. Symantec funded an antivirus testing by PassMark: Consumer Antivirus Performance Benchmarks
2. Symantec sponsored another antivirus evaluation by Dennis Technology Labs: PC Anti-Virus Protection 2011
3.
4. Microsoft sponsored two NSS Labs tests for comparing the security of IE8 with other browsers:
http://arstechnica.com/microsoft/news/2009/08/microsoft-sponsors-two-nss-reports-ie8-is-the-most-secure.ars
5. Trend-Micro commissioned West Coast Labs Anti-Spam comparison tests: http://it.trendmicro.com/imperia/md/content/uk/whitepaper/wp06_wclantispamrpt_090317us.pdf
The results of these studies are not surprising. Symantec was ranked first by Dennis Technology Labs and PassMark.
The reason I am blogging this, is because I have come across a lot of CIO's and security experts who still believe and take into granted the results published by such kind of studies. Its even a pity to see security gurus from notable organizations such as SANS fall into this and cite these results.
For more information please see:
- Study: Industry-Sponsored Research Yields Favorable Results a Majority of the Time: http://www.doctorpundit.com/index.php/2010/08/03/study-industry-sponsored-research-yields-favorable-results-a-majority-of-the-time
- The uncertainty principle and industry-sponsored research: http://www.ncbi.nlm.nih.gov/pubmed/10968436
- Pharmaceutical industry sponsorship and research outcome and quality: systematic review http://www.bmj.com/content/326/7400/1167.full
- Source of funding and outcome of clinical trials - Journal of General Internal Medicine http://www.springerlink.com/content/r654521305u8547k/