Software security testing is perhaps the hardest facet of Software Quality Assurance to realise and security faults are often not discovered until such time and their abuse is discovered. There can be no hard and fast rule on the exit criteria for any given project. Each project must be risk assessed and the effort put behind security testing must be importation to the risks defined.
Software security tests goes beyond the normal realm of software testing, no longer are we concerned with the software functioning correctly, but the potential consequences of when it does not. Software failure can and de enviable lead to vulnerabilities that can later be exploited. Secondly we are concerned with any weakness in the form of functionality that may be exposed and therefore be available for abuse. Often it is not the software that is under attack but the systems, weaknesses in the software can facilitate such an attack. These elements may never be software issues for normal users, but none the less may leave the system open to abuse, and present a risk to both users and system owners.
We can never be sure that any given piece of software is free of potential bugs that may used in an attack; “If a library of known exploits fails to find a gap in our application’s armour, we have only the illusion of security”. (Thompson, 2003) Consequently key to the discussion of exit criteria for software testing is risk assessment and risk management. Exit criteria must therefore be defined appropriately based on the nature of the project and the risks concerned. The economic reality is, in line with all things testing, that there has to be a compromise between the time and money available and the level of assurance achieved.
Software security does not come from putting up a password page on the front of application. Comprehensive security must be built in for the start, Lipner goes further and suggests that security should be incorporated into the software development lifecycle it’s self (Lipner, 2004) This would be reasonable since we earlier established that the cost of fixing a given bug is dramatically reduced the earlier it is caught. Thus establishing a security aware development process is likely to be the cost effective and productive approach. Further the process it’s self and the milestone defined will assist in defining the exit criteria.
We can never be certain that any given piece of software is totally secure, indeed history has shown us that it is more than probable that security flaws exist and sooner or later will be exploited. Security testing is therefore an economic compromise aimed at reducing risk. It should be remember that risk management does not seek to eliminate risk, it can’t, but seeks the reduce the probability of risk to acceptable levels.
LIPNER, S. (2004) The Trustworthy Computing Security Development Lifecycle. Proceedings of the 20th Annual Computer Security Applications Conference.[online] DIO: 10.1109/CSAC.2004.41 (Accessed: 22-3-2009)
THOMPSON, H. H. (2003) Why security testing is hard. Security & Privacy, IEEE, 1, 83-86.[online] DIO: 10.1109/MSECP.2003.1219078 (Accessed: 22-3-2009)