Main concept of mutation testing is to see the behaviour of the application after injecting some defect. This phenomenon of injecting defect is called defect seeding where intentionally defect is introduced to a product to check the detection rate. It is mostly done during unit testing where a certain change in code is done to see the behaviour of application. We do it to check that our testcases are smart enough to catch the defect.
In this technique we mutate or change certain statement in the source code and check the test code is able to find errors . These mutants are run with an input data from a given test set. If a test set can distinguish a mutant from the original program i.e., it produces different execution result, the mutant is said to be killed. Otherwise, the mutant is called as a live mutant.
A mutant remains live because it is equivalent to the original program i.e., it is functionally identical to the original program or the test data is inadequate to kill the mutant. If a test data is inadequate, it can be improved by adding test cases to kill the live mutant. A test set which can kill all non-equivalent mutants is said to be adequate (mutation score).
Following are the steps to execute mutation testing:
Step 1: Faults are introduced into the source code of the program by creating many versions called mutants. Each mutant should contain a single fault, and the goal is to cause the mutant version to fail which demonstrates the effectiveness of the test cases.
Step 2: Test cases are applied to the original program and also to the mutant program. A test case should be adequate, and it is tweaked to detect faults in a program.
Step 3: Compare the results of original and mutant program.
Step 4: If the original program and mutant programs generate the same output, then that the mutant is killed by the test case. Hence the test case is good enough to detect the change between the original and the mutant program.
Step 5: If the original program and mutant program generate different output, Mutant is kept alive. In such cases , more effective test cases need to be created that kill all mutants.
Mutation Testing Types:
Value Mutations: An attempt to change the values to detect errors in the programs. We usually change one value to a much larger value or one value to a much smaller value. The most common strategy is to change the constants.
Decision Mutations: The decisions/conditions are changed to check for the design errors. Typically, one changes the arithmetic operators to locate the defects and also we can consider mutating all relational operators and logical operators (AND, OR , NOT)
Statement Mutations: Changes done to the statements by deleting or duplicating the line which might arise when a developer is copy pasting the code from somewhere else.
Mutation testing is based on two hypothesis :
Competent programmer hypothesis : This hypothesis states that most software faults introduced by experienced programmers are due to small syntactic errors.It states that programmers are competent, which implies that they tend to develop programs close to the correct version. As a result, although there may be faults in the program delivered by a competent programmer, we assume that these faults are merely a few simple faults which can be corrected by a few small syntactical changes. Therefore, in Mutation Testing, only faults constructed from several simple syntactical changes are applied, which represents the faults that are made by "competent programmers".
Coupling effect: The coupling effect asserts that simple faults can cascade or couple to form other emergent faults.The Coupling Effect Hypothesis is that complex faults are coupled to simple faults in such a way that a test data set that detects all simple faults in a program will detect a high percentage of the complex faults.
In this technique we mutate or change certain statement in the source code and check the test code is able to find errors . These mutants are run with an input data from a given test set. If a test set can distinguish a mutant from the original program i.e., it produces different execution result, the mutant is said to be killed. Otherwise, the mutant is called as a live mutant.
A mutant remains live because it is equivalent to the original program i.e., it is functionally identical to the original program or the test data is inadequate to kill the mutant. If a test data is inadequate, it can be improved by adding test cases to kill the live mutant. A test set which can kill all non-equivalent mutants is said to be adequate (mutation score).
Following are the steps to execute mutation testing:
Step 1: Faults are introduced into the source code of the program by creating many versions called mutants. Each mutant should contain a single fault, and the goal is to cause the mutant version to fail which demonstrates the effectiveness of the test cases.
Step 2: Test cases are applied to the original program and also to the mutant program. A test case should be adequate, and it is tweaked to detect faults in a program.
Step 3: Compare the results of original and mutant program.
Step 4: If the original program and mutant programs generate the same output, then that the mutant is killed by the test case. Hence the test case is good enough to detect the change between the original and the mutant program.
Step 5: If the original program and mutant program generate different output, Mutant is kept alive. In such cases , more effective test cases need to be created that kill all mutants.
Mutation Testing Types:
Value Mutations: An attempt to change the values to detect errors in the programs. We usually change one value to a much larger value or one value to a much smaller value. The most common strategy is to change the constants.
Decision Mutations: The decisions/conditions are changed to check for the design errors. Typically, one changes the arithmetic operators to locate the defects and also we can consider mutating all relational operators and logical operators (AND, OR , NOT)
Statement Mutations: Changes done to the statements by deleting or duplicating the line which might arise when a developer is copy pasting the code from somewhere else.
Mutation testing is based on two hypothesis :
Competent programmer hypothesis : This hypothesis states that most software faults introduced by experienced programmers are due to small syntactic errors.It states that programmers are competent, which implies that they tend to develop programs close to the correct version. As a result, although there may be faults in the program delivered by a competent programmer, we assume that these faults are merely a few simple faults which can be corrected by a few small syntactical changes. Therefore, in Mutation Testing, only faults constructed from several simple syntactical changes are applied, which represents the faults that are made by "competent programmers".
Coupling effect: The coupling effect asserts that simple faults can cascade or couple to form other emergent faults.The Coupling Effect Hypothesis is that complex faults are coupled to simple faults in such a way that a test data set that detects all simple faults in a program will detect a high percentage of the complex faults.
No comments:
Post a Comment