UTP11 — Test Objective issue

Test Objective
1. In the OMG Testing Profile (OTP) specification a Test Objective is defined both as a stereotype on a dependency, «TestObjective», and as an artifact elsewhere. The «TestObjective» stereotype does not apply to this artifact, just to the dependency.

2. The Test Objective dependency relationship is intended to show the dependency from a Test Context or a Test Case to a model element that contains the textual string stating the objective of the test, the test objective artifact.

Resolution:

Test Objective as a dependency:

a. Change Suggestion 1 - No longer use the stereotype «TestObjective» for this dependency relationship. A dependency relationship stereotype name is typically a verb, like verify, satisfy, refine, and copy. Using a noun phrase like “TestObjectiveon” to name a dependency does not provide the declaration that is needed, i.e. subject verb object, e.g. “BlockA satisfies requirement 1”, or “Use Case A refines requirement 1”.

b. Change Suggestion 2 – Use the verb “Validate” as the name of the stereotype that is associated with a dependency. This stereotyped dependency can be used from a Test Case or Test Context to a Test Objective but, can also be used between a test case and model elements that state objectives/goals/capabilities of the item being validated.

c. Why use validate? – Others were considered such as satisfy, realize and implement.

i. The validation definition used by test is “Validation is the set of activities ensuring and gaining confidence that a system is able to accomplish its intended use, goals and objectives (i.e. meet stakeholder requirements) in the intended operational environment” , or “Did we build the right thing?” These system uses, goals and objectives are represented in the model in the form of use cases, MOEs, TPMs, etc.

ii. There is a clear distinction between verification and validation. Based on this, there will be unique Test Contexts for verification and validation.

iii. Just as we need to show what test cases will be verifying what requirements we will also need to show what test cases will be validating objectives and goals of the system (or the thing being validated). Therefore, this relationship will ultimately be needed just as verify is used today to verify requirements.

iv. This relationship can also be used to show what test cases are validating the objectives stated for a Test Context. It is the same relationship pattern.

Test Objective part 2
1. A Test Objective artifact is a textual statement stating a goal or objective of a test context. The OTP specification doesn’t define the type of model element to use for this artifact.

Resolution:

1. Test Objective artifact suggestion –

a. We have selected a comment stereotyped «TestObjective» to capture the Test Objective artifact for a Test Context. However, other model elements could be used, such as an attribute of the Test Context. The advantage of a comment is that it can be reused across test contexts

Test Objective as a dependency:
a. Change Suggestion 1 - No longer use the stereotype «TestObjective» for this dependency relationship. A dependency relationship stereotype name is typically a verb, like verify, satisfy, refine, and copy. Using a noun phrase like “TestObjectiveon” to name a dependency does not provide the declaration that is needed, i.e. subject verb object, e.g. “BlockA satisfies requirement 1”, or “Use Case A refines requirement 1”.
b. Change Suggestion 2 – Use the verb “Validate” as the name of the stereotype that is associated with a dependency. This stereotyped dependency can be used from a Test Case or Test Context to a Test Objective but, can also be used between a test case and model elements that state objectives/goals/capabilities of the item being validated.
c. Why use validate? – Others were considered such as satisfy, realize and implement.
i. The validation definition used by test is “Validation is the set of activities ensuring and gaining confidence that a system is able to accomplish its intended use, goals and objectives (i.e. meet stakeholder requirements) in the intended operational environment” , or “Did we build the right thing?” These system uses, goals and objectives are represented in the model in the form of use cases, MOEs, TPMs, etc.
ii. There is a clear distinction between verification and validation. Based on this, there will be unique Test Contexts for verification and validation.
iii. Just as we need to show what test cases will be verifying what requirements we will also need to show what test cases will be validating objectives and goals of the system (or the thing being validated). Therefore, this relationship will ultimately be needed just as verify is used today to verify requirements.
iv. This relationship can also be used to show what test cases are validating the objectives stated for a Test Context. It is the same relationship pattern.
Test Objective part 2
1. A Test Objective artifact is a textual statement stating a goal or objective of a test context. The OTP specification doesn’t define the type of model element to use for this artifact.
Resolution:
1. Test Objective artifact suggestion –
a. We have selected a comment stereotyped «TestObjective» to capture the Test Objective artifact for a Test Context. However, other model elements could be used, such as an attribute of the Test Context. The advantage of a comment is that it can be reused across test contexts
Resolution:
The RTF discussed a lot on the test objective issue and we agreed entirely that a dependency as test objective is quite unintuitive, confusing and misleading. We had a long way to go until the RTF agreed on a resolution that suits all and is most accepted in the testing area in the industry. Finally, we decided to change the metaclass of the current «TestObjective» from Dependency to Class and to put tag definitions into the stereotype that are pertinent to test management, test planning and test schedule.
The new «TestObjective» is conceptually compatible with the concept Objective from the Business Motivation Metamodel (BMM), although BMM does not provide a UML profile yet and is therefore not a normative reference for the UML Testing Profile.