If any release contains only the modules which has been changed, it is called to be delta release.Also known as partial release , delta release is one that includes only those areas within the release unit that have changed or are new since the last full release or delta release.
Wednesday, 10 February 2016
MT - 71 - What is test log?
The IEEE Std. 829-1998 defines a test log as a chronological record of relevant details about the execution of test cases. It's a detailed view of activity and events given in chronological manner. Testlog is a repository to store passed and failed testcases. Test log is document which records all test result of different activities which are both pass and fail criteria
The following figure shows a test log and is followed by a sample test log.
The following figure shows a test log and is followed by a sample test log.
MT - 70 - Testing Standards !!
Some widely used standards related to Quality Assurance and Testing :
ISO/IEC 9126
This standard deals with the following aspects to determine the quality of a software application:
- Quality model
- External metrics
- Internal metrics
- Quality in use metrics
This standard presents some set of quality attributes for any software such as:
- Functionality
- Reliability
- Usability
- Efficiency
- Maintainability
- Portability
ISO/IEC 9241-11
This standard proposed a framework that describes the usability components and the relationship between them. In this standard, the usability is considered in terms of user performance and satisfaction. According to ISO 9241-11, usability depends on context of use and the level of usability will change as the context changes.
ISO/IEC 25000:2005
ISO/IEC 25000:2005 is commonly known as the standard that provides the guidelines for Software Quality Requirements and Evaluation (SQuaRE). This standard helps in organizing and enhancing the process related to software quality requirements and their evaluations. In reality, ISO-25000 replaces the two old ISO standards, i.e. ISO-9126 and ISO-14598.
SQuaRE is divided into sub-parts such as:
- ISO 2500n - Quality Management Division
- ISO 2501n - Quality Model Division
- ISO 2502n - Quality Measurement Division
- ISO 2503n - Quality Requirements Division
- ISO 2504n - Quality Evaluation Division
The main contents of SQuaRE are:
- Terms and definitions
- Reference Models
- General guide
- Individual division guides
- Standard related to Requirement Engineering (i.e. specification, planning, measurement and evaluation process)
ISO/IEC 12119
This standard deals with software packages delivered to the client. It does not focus or deal with the clients’ production process. The main contents are related to the following items:
- Set of requirements for software packages.
- Instructions for testing a delivered software package against the specified requirements.
Miscellaneous
Some of the other standards related to QA and Testing processes are mentioned below:
| Standard | Description |
|---|---|
| IEEE 829 | A standard for the format of documents used in different stages of software testing. |
| IEEE 1061 | A methodology for establishing quality requirements, identifying, implementing, analysing, and validating the process, and product of software quality metrics. |
| IEEE 1059 | Guide for Software Verification and Validation Plans. |
| IEEE 1008 | A standard for unit testing. |
| IEEE 1012 | A standard for Software Verification and Validation. |
| IEEE 1028 | A standard for software inspections. |
| IEEE 1044 | A standard for the classification of software anomalies. |
| IEEE 1044-1 | A guide for the classification of software anomalies. |
| IEEE 830 | A guide for developing system requirements specifications. |
| IEEE 730 | A standard for software quality assurance plans. |
| IEEE 1061 | A standard for software quality metrics and methodology. |
| IEEE 12207 | A standard for software life cycle processes and life cycle data. |
| BS 7925-1 | A vocabulary of terms used in software testing. |
| BS 7925-2 | A standard for software component testing. |
MT - 69 - Regression testing and its types !
Regression Testing is a testing of a program which was previously tested and went through any modification. It is done to ensure that the defects have not been introduced or new defects has not been uncovered in the other areas of the program. It is performed when a software or its environment is changed. Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes. They are subcategorised into 3 types :
Unit regression testing:
Testing the only one particular fixed unit or feature .
Region regression testing:
Testing the changes and the impact regions is called Regional Regression testing. In this type of regression testing only select modules are tested which are connected to the changed modules
Full regression testing:
Testing the changes and all the remaining features of the system.It is normally done when the changes are done in the root of the product or whenever the modifications or changes are more in the product or a larger area of the product is impacted.
Excerpts :
If any modification is done in one module and if the module is some where linked with the other two modules , then in this case regional regression is preferred over the full regression testing.If modifications are done in many modules in the project and if its difficult to recognize the effected modules then we do full regression testing rather than taking risk.
Unit regression testing:
Testing the only one particular fixed unit or feature .
Region regression testing:
Testing the changes and the impact regions is called Regional Regression testing. In this type of regression testing only select modules are tested which are connected to the changed modules
Full regression testing:
Testing the changes and all the remaining features of the system.It is normally done when the changes are done in the root of the product or whenever the modifications or changes are more in the product or a larger area of the product is impacted.
Excerpts :
If any modification is done in one module and if the module is some where linked with the other two modules , then in this case regional regression is preferred over the full regression testing.If modifications are done in many modules in the project and if its difficult to recognize the effected modules then we do full regression testing rather than taking risk.
MT - 68 - Testware in terms of testing !
The artifacts that are produced during the testing process which are required to plan, design, and execute tests is called Testware. It includes artifacts produced during the test process required to plan, design, and execute tests, such as documentation, scripts, inputs, expected results, set-up and clear-up procedures, files, databases, environment, and any additional software or utilities used in testing.In short the testware includes all the materials used to perform a test like test cases, testplan, scripts , test results, etc.
Tuesday, 9 February 2016
MT - 67 - Relationship between Use case , Include and Extend in terms of testing !!
These two type of relationships are used in use case models .The UML( Unified Modelling Language) describes the relationship between usecases , <<include>> and <<exclude>> relationship. Before seeing the relationship between these three terms lets see about the use case modelling. Use Cases describes a logical task that may be performed by the system. The Use Case description describes the interaction between the system and the outside world. UML is used to create specification for the system based on the use cases.Use Case corresponds to a logical unit of work, so that as a rule of thumb it is typically,
performed by one person, in one place, at one time,
leaves the business data in a consistent state and
generates some business value.
The Use Case steps are usually arranged into a Basic Flow and Alternative Flows.The Basic Flow has other names, such as the Main Success Scenario or Happy Day scenario. It describes the steps that are required for a straightforward (but not always the simplest) execution of the Use Case. It sets aside the panoply of possibilities within the Use Case so that we can see the essence of the thing.
The Alternative Flows are also known by other names, such as exceptions or extensions. Each one describes some variation from the steps in the Basic Flow. Alternative Flows can also be overlaid on top of other Alternative Flows. Within a single execution of the Use Case – a single Scenario – any number of Alternative Flows may be invoked
Lets take an example of use case and relate it to the <<Include>> and <<extend>> relationship :
Eg : Use Case: “Take Customer Order”
Basic Flow:
1. Actor enters Customer details
2. Actor enters code for product required
3. System displays Product details
4. Actor enters quantity required
5. Actor enters Payment details
6. System saves Customer Order
Alternative Flows:
[multiple products]
After step 4, when the Actor enters the quantity required,
Repeat steps 2 to 4 for additional Products
Resume at step 5, to enter Payment details
This diagram defines the use case with the basic flow.
Now here comes the concept of <<include>> relationship, The «include» relationship allows us to include the steps from one Use Case into another.
This is valuable when the included steps occur as a recognisable sequence in many
different contexts.
In the above example , we need to capture the Customer’s details. After further analysis, we may find that there are other Use Cases where we also need to record the Customer’s details.Furthermore, we will find that there are a number of ways of doing this, for example, we may need to capture them as a new customer or recall them as an existing customer. So long as the steps required to capture the Customer’s details are not trivial, it becomes more economic to extract them into a separate flow, which we may call “Identify Customer”. See the picture below :
Now , the seperate use caseUse Case: “Identify Customer” will be as
Basic Flow:
1. Actor enters search criteria, surname and postcode
2. System displays matching Customers
3. Actor selects Customer
4. System displays Customer details
5. Actor confirms Customer
Alternative Flows:
[new customer]
After step 2, when the System does not display the required Customer, Actor creates new
Customer,
1. Actor selects to add new Customer
2. Actor enters Customer details
Resume at step 5, to confirm Customer
We now modify the base Use Case “Take Customer Order” to refer to the included Use Case “Identify Customer”. The very first use case will look like
Use Case: “Take Customer Order”
Basic Flow:
1. Actor records Customer details, include (Identify Customer)
2. Actor enters code for Product required
3. System displays Product details
4. Actor enters quantity required
5. Actor enters Payment details
6. System saves Customer Order
Alternative Flows:
[multiple products]
After step 4, when the Actor enters the quantity required,
Repeat steps 2 to 4 for additional Products
Resume at step 5, to enter Payment details
So when we execute the Use Case “Take Customer Order” step 1 tells us to execute all the steps of “Identify Customer”, including the possibility of all its Alternative Flows. The name of the included Use Case should be highlighted by placing it in parentheses or quotation marks.
Now lets discuss <<extend>> relationship
The «extend» relationship allows us to modify the behaviour of the base Use Case. Suppose we want to sell products that are made to order and require a degree of customer specification. For these products we will need to record the customer’s additional requirements, such as size and colour. In this case we are adding something extra to the flow of the base Use Case.We could do this as an Alternative Flow, thus: the steps will be like :
Use Case: “Take Customer Order”
Basic Flow:
1. Actor records Customer details include (Identify Customer)
2. Actor enters code for product required
3. System displays product details
4. Actor enters quantity required
5. Actor enters payment details
6. System saves Customer Order
Alternative Flows:
[multiple products]
After step 4, when the Actor enters the quantity required,
Repeat steps 2 to 4 for additional products
Resume at step 5, to enter payment details
[customer specified product]
At step 3, when the System displays the Product details, if the product requires customer
specified features,
1. Actor enters customer specified requirements, such as size and colour
Resume at step 4, to enter quantity required, until step 6 where the Customer Order is saved.
At this step the additional customer-specific product details must also be saved
The «extend» relationship says that we execute the base Use Case but when we get to a
specified point in the flow, if the right conditions are met, we perform some different steps.
Clearly this is very similar to an Alternative Flow.
When to use «include» and when to use «extend»
1. Do the additional steps constitute a single contiguous set of steps?
Yes, you are adding steps at one place in the flow – use «include».
No, you need to modify the base Use Case in more than one place – use «extend».
2. Do the additional steps make sense on their own?
Yes – use «include».
No – use «extend».
Summary:
Include = reuse of functionality
Extends = new and/or optional functionality
Extends is used when you understand that your use case is too much complex. So you extract the complex steps into their own "extension" use cases.
Include is used when you see common behaviour in two use cases. So you abstract out the common behaviour into a separate "abstract" use case.
Extend is optional from user point of view, it may or may not occur.
performed by one person, in one place, at one time,
leaves the business data in a consistent state and
generates some business value.
The Use Case steps are usually arranged into a Basic Flow and Alternative Flows.The Basic Flow has other names, such as the Main Success Scenario or Happy Day scenario. It describes the steps that are required for a straightforward (but not always the simplest) execution of the Use Case. It sets aside the panoply of possibilities within the Use Case so that we can see the essence of the thing.
The Alternative Flows are also known by other names, such as exceptions or extensions. Each one describes some variation from the steps in the Basic Flow. Alternative Flows can also be overlaid on top of other Alternative Flows. Within a single execution of the Use Case – a single Scenario – any number of Alternative Flows may be invoked
Lets take an example of use case and relate it to the <<Include>> and <<extend>> relationship :
Eg : Use Case: “Take Customer Order”
Basic Flow:
1. Actor enters Customer details
2. Actor enters code for product required
3. System displays Product details
4. Actor enters quantity required
5. Actor enters Payment details
6. System saves Customer Order
Alternative Flows:
[multiple products]
After step 4, when the Actor enters the quantity required,
Repeat steps 2 to 4 for additional Products
Resume at step 5, to enter Payment details
Now here comes the concept of <<include>> relationship, The «include» relationship allows us to include the steps from one Use Case into another.
This is valuable when the included steps occur as a recognisable sequence in many
different contexts.
In the above example , we need to capture the Customer’s details. After further analysis, we may find that there are other Use Cases where we also need to record the Customer’s details.Furthermore, we will find that there are a number of ways of doing this, for example, we may need to capture them as a new customer or recall them as an existing customer. So long as the steps required to capture the Customer’s details are not trivial, it becomes more economic to extract them into a separate flow, which we may call “Identify Customer”. See the picture below :
This picture denotes shared common steps for the use case
Basic Flow:
1. Actor enters search criteria, surname and postcode
2. System displays matching Customers
3. Actor selects Customer
4. System displays Customer details
5. Actor confirms Customer
Alternative Flows:
[new customer]
After step 2, when the System does not display the required Customer, Actor creates new
Customer,
1. Actor selects to add new Customer
2. Actor enters Customer details
Resume at step 5, to confirm Customer
We now modify the base Use Case “Take Customer Order” to refer to the included Use Case “Identify Customer”. The very first use case will look like
Use Case: “Take Customer Order”
Basic Flow:
1. Actor records Customer details, include (Identify Customer)
2. Actor enters code for Product required
3. System displays Product details
4. Actor enters quantity required
5. Actor enters Payment details
6. System saves Customer Order
Alternative Flows:
[multiple products]
After step 4, when the Actor enters the quantity required,
Repeat steps 2 to 4 for additional Products
Resume at step 5, to enter Payment details
So when we execute the Use Case “Take Customer Order” step 1 tells us to execute all the steps of “Identify Customer”, including the possibility of all its Alternative Flows. The name of the included Use Case should be highlighted by placing it in parentheses or quotation marks.
Now lets discuss <<extend>> relationship
The «extend» relationship allows us to modify the behaviour of the base Use Case. Suppose we want to sell products that are made to order and require a degree of customer specification. For these products we will need to record the customer’s additional requirements, such as size and colour. In this case we are adding something extra to the flow of the base Use Case.We could do this as an Alternative Flow, thus: the steps will be like :
Use Case: “Take Customer Order”
Basic Flow:
1. Actor records Customer details include (Identify Customer)
2. Actor enters code for product required
3. System displays product details
4. Actor enters quantity required
5. Actor enters payment details
6. System saves Customer Order
Alternative Flows:
[multiple products]
After step 4, when the Actor enters the quantity required,
Repeat steps 2 to 4 for additional products
Resume at step 5, to enter payment details
[customer specified product]
At step 3, when the System displays the Product details, if the product requires customer
specified features,
1. Actor enters customer specified requirements, such as size and colour
Resume at step 4, to enter quantity required, until step 6 where the Customer Order is saved.
At this step the additional customer-specific product details must also be saved
When to use «include» and when to use «extend»
1. Do the additional steps constitute a single contiguous set of steps?
Yes, you are adding steps at one place in the flow – use «include».
No, you need to modify the base Use Case in more than one place – use «extend».
2. Do the additional steps make sense on their own?
Yes – use «include».
No – use «extend».
3. Do the additional steps occur in more than one base Use Case?
Yes – use «include».
No – use «extend».
Summary:
Include = reuse of functionality
Extends = new and/or optional functionality
Extends is used when you understand that your use case is too much complex. So you extract the complex steps into their own "extension" use cases.
Include is used when you see common behaviour in two use cases. So you abstract out the common behaviour into a separate "abstract" use case.
Extend is optional from user point of view, it may or may not occur.
Wednesday, 3 February 2016
MT - 66 - Test Effectiveness and Test Efficiency
Test Efficiency :
The ratio of the effective or useful output to the total input in any system.Therefore, efficiency is an attribute which means to maximize the useful output for a given input reducing wastage or losses. Efficiency cannot be more than a 100%, in a sense that a 100% efficient system will have zero losses. Test efficiency is not only about test execution alone, but all or most of test activities, like test planning, comprehension, test cases creation, review, execution, defect tracking and closure. TE is not just one single derivation but a number of calculations at each phase and activity of testing. What activities or phases one is interested in, what is particularly measured, depends on lot of other things, the type of project, complexity, availability of resources, the situation, customer requirement etc.
Test Efficiency = (Test Defects / (Test Defects + Acceptance Defects)) * 100
OR
Testing Efficiency = (No. of defects Resolved / Total No. of Defects Submitted)* 100
Test defects = Unit + Integration + System defects
Acceptance Defects = Bugs found by the customer
Test Effectiveness :
Effectiveness means the capability of producing an effect.Test effectiveness is to ensure quality and close the two quality gaps, namely producer’s quality gap and customer’s quality gap. As definition of quality goes, quality is both process and product quality which is meeting customer requirements and conformance to product specification.
Test Effectiveness = ((Defects removed in a phase) / (Defect injected + Defect escaped)) * 100
OR
Test effectiveness = Number of defects found divided by number of test cases executed.
OR
Test Effectiveness = Loss due to problems / Total resources processed by the system
Example :
The table shows the defect origin, on the X, where a defect was injected, where he belongs to? and on the Y, where the defect was detected.
Let us calculate the test effectiveness of the Integration testing activity:
Total number of defects of all origin found during Integration testing activity = 17
Total number of defects existing while entering in IT = (6+15+28) – (35) = 14
Total number of defects injected in the current stage = 14
Effectiveness of IT test phase = Total defects found in this phase/(No of defects existing + injected)
Effectiveness = 17/(14+14 ) * 100 = 60.7%
Difference Between these two :
- Efficiency - productivity metric and Effectiveness - quality metric!
- Efficiency is a productivity metrics meaning how fast one can do something. Hence Testing efficiency metric can be "No. of test cases executed per hour or per person day".Effectiveness is a quality metrics meaning how good a person is at testing. Hence Testing effectiveness metrics can be "No. of bugs identified by a tester in a given feature / Total no. of bugs identified in that feature".
-Effectiveness - How well the user achieves the goals they set out to achieve using the system (process).
Efficiency - The resources consumed in order to achieve their goals.
The ratio of the effective or useful output to the total input in any system.Therefore, efficiency is an attribute which means to maximize the useful output for a given input reducing wastage or losses. Efficiency cannot be more than a 100%, in a sense that a 100% efficient system will have zero losses. Test efficiency is not only about test execution alone, but all or most of test activities, like test planning, comprehension, test cases creation, review, execution, defect tracking and closure. TE is not just one single derivation but a number of calculations at each phase and activity of testing. What activities or phases one is interested in, what is particularly measured, depends on lot of other things, the type of project, complexity, availability of resources, the situation, customer requirement etc.
Test Efficiency = (Test Defects / (Test Defects + Acceptance Defects)) * 100
OR
Testing Efficiency = (No. of defects Resolved / Total No. of Defects Submitted)* 100
Test defects = Unit + Integration + System defects
Acceptance Defects = Bugs found by the customer
Test Effectiveness :
Effectiveness means the capability of producing an effect.Test effectiveness is to ensure quality and close the two quality gaps, namely producer’s quality gap and customer’s quality gap. As definition of quality goes, quality is both process and product quality which is meeting customer requirements and conformance to product specification.
Test Effectiveness = ((Defects removed in a phase) / (Defect injected + Defect escaped)) * 100
OR
Test effectiveness = Number of defects found divided by number of test cases executed.
OR
Test Effectiveness = Loss due to problems / Total resources processed by the system
Example :
The table shows the defect origin, on the X, where a defect was injected, where he belongs to? and on the Y, where the defect was detected.
Let us calculate the test effectiveness of the Integration testing activity:
Total number of defects of all origin found during Integration testing activity = 17
Total number of defects existing while entering in IT = (6+15+28) – (35) = 14
Total number of defects injected in the current stage = 14
Effectiveness of IT test phase = Total defects found in this phase/(No of defects existing + injected)
Effectiveness = 17/(14+14 ) * 100 = 60.7%
Difference Between these two :
- Efficiency - productivity metric and Effectiveness - quality metric!
- Efficiency is a productivity metrics meaning how fast one can do something. Hence Testing efficiency metric can be "No. of test cases executed per hour or per person day".Effectiveness is a quality metrics meaning how good a person is at testing. Hence Testing effectiveness metrics can be "No. of bugs identified by a tester in a given feature / Total no. of bugs identified in that feature".
-Effectiveness - How well the user achieves the goals they set out to achieve using the system (process).
Efficiency - The resources consumed in order to achieve their goals.
Monday, 1 February 2016
MT - 65 - Difference between "Defect" , "Failure" , "Bug" , "Mistake" , "Error" , "Fault"
Bug: If the Error found by testers are accepted as error by Developers. Then the error will called Bug.
Error: This is cause due to human actions like code is not following the standard, there is some mistake in syntax, or there is mistake in invocation of variable or might be there is some mistakes in which database connectivity code is faulty.This can be a misunderstanding of the internal state of the software, an oversight in terms of memory management, confusion about the proper way to calculate a value, etc.
Fault: When the product/software successfully launched in the market and running properly but due to any reason if it works unexpectedly is called Fault.
Failure: If the product fails to full fill the requirement, then it is called Failure. It is a deviation of software from its expected delivery or service.Failure are caused by environment or sometime due to mishandling of product.
Mistake : human interaction which produce an incorrect result which is also called error.
Defect: Any issue currently caught in that application that are deviating the actual result from the requirement, will taken as Defect.
“A mistake in coding is called error ,error found by tester is called defect, defect accepted by development team then it is called bug ,build does not meet the requirements then it Is failure.”
Error: This is cause due to human actions like code is not following the standard, there is some mistake in syntax, or there is mistake in invocation of variable or might be there is some mistakes in which database connectivity code is faulty.This can be a misunderstanding of the internal state of the software, an oversight in terms of memory management, confusion about the proper way to calculate a value, etc.
Fault: When the product/software successfully launched in the market and running properly but due to any reason if it works unexpectedly is called Fault.
Failure: If the product fails to full fill the requirement, then it is called Failure. It is a deviation of software from its expected delivery or service.Failure are caused by environment or sometime due to mishandling of product.
Mistake : human interaction which produce an incorrect result which is also called error.
Defect: Any issue currently caught in that application that are deviating the actual result from the requirement, will taken as Defect.
“A mistake in coding is called error ,error found by tester is called defect, defect accepted by development team then it is called bug ,build does not meet the requirements then it Is failure.”
Subscribe to:
Posts (Atom)