Testing Tools, A Report on what is Commercially Available
Once an application has been developed, the developers must demonstrate that it performs the tasks for which it was designed accurately, reliably and with adequate performance. For this to be fulfilled extensive testing must be carried out and tools have been built to assist with this process. Developers have built different types of tool for addressing different aspects of the same general problem. The importance of proper testing to detect as many errors as feasibly possible has been driven by the increase of malicious or criminal intent on the part of developers that produce applications with functions that facilitate fraud or other criminal activity (an especial risk to the financial industry). This problem has been addressed by European Community Legislation, increasing the onus on software developers to show that they took all reasonable steps to ensure an application was free of defects and suitable for the purpose for which it was developed. Failure to do so could leave the developer liable to be sued by anyone have has incurred a loss in any business as a result of software collapse. The main types of tool that have resulted as a partial result of this are described below.
Testing Tool Categories
There are a large number of testing tools that are available, but they all work in very different ways. The main types of testing categories are described below.
Tools that analyse source code without executing test cases, but in deriving test cases for the software to be tested. There are three different types used in industry that are described below:
Code based testing tools accept source code as input and perform a number of analyses that result in the generation of test cases. This type of automated tool can broken down in to four further categories. The first are Code analysers that evaluate test modules automatically for proper syntax; statements are then highlighted where the syntax is wrong, if construction is error prone or if an item is undefined. The second category is Structure checkers where modules are submitted as input and a graph generated, depicting the hierarchy of modules and tools check for structural flaws, for example, determining the location of loops and branches and how they are used within the system. The third type are Data analysers which review data structures, data declarations and module interfaces, and notes improper linkage between modules, conflicting data definitions and illegal data usage. The final type are Sequence checkers where sequences of events are checked and marked if coded in wrong sequence.
Specialised testing languages enable a software engineer to write detailed test specifications that describe each test case and the logistics for its execution. An example of one of these languages is Prolog, that is specifically used for test case generation.
Requirements based testing tools isolate specific user requirements and suggest test cases (or classes of tests) that will exercise the requirements.
Tools that analyse source code during execution of test cases by interacting with a program as it is executing and checking the path coverage, test assertions about the value of specific variables and otherwise instrumenting the execution flow of the program. They can be either intrusive or non-intrusive. An intrusive tool changes the software to be tested by inserting extra instructions or ‘probes’ that perform the activities mentioned above. A non-intrusive tool uses a separate hardware processor that runs in parallel with the processor containing the program that is being tested.
Systems can be difficult to test because several parallel operations are being carried out concurrently, which is especially true for real-time systems. Therefore it is difficult to anticipate the conditions and generate representative test conditions. However, dynamic test tools can capture a state of events during the execution of a program and so are often called program monitors, because they watch and report the behaviour of the program. The functions of the monitor are to list the number of times a submodule is called or a line of code is executed. These statistics tell testers if the test cases have statement coverage. Another function is to report on whether a decision point has