UML Testing Profile 2 Avatar
  1. OMG Specification

UML Testing Profile 2 — All Issues

  • Acronym: UTP2
  • Issues Count: 47
  • Description: All Issues
Closed All
All Issues

Issues Summary

Key Issue Reported Fixed Disposition Status
UMLTP22-4 Harmonize illustration of property modifiers in the spec UTP2 1.0b1 $issue.fixedSpecification.name Deferred closed
UMLTP22-9 Group abstract syntax diagram in separate sub-section for better changeability of the spec UTP2 1.0b1 $issue.fixedSpecification.name Deferred closed
UMLTP22-11 Non-navigable associations ends shall be removed from the spec UTP2 2.0 $issue.fixedSpecification.name Deferred closed
UMLTP22-14 UTP should incorporate concepts to speciy decision tables for allowing decision table testing UTP2 2.1 $issue.fixedSpecification.name Deferred closed
UMLTP22-12 Generators for and Queries on Test Logs UTP2 2.0b2 $issue.fixedSpecification.name Deferred closed
UMLTP22-1 LoginExample is not in synch with latest Profile Specification UTP 2.0b1 $issue.fixedSpecification.name Deferred closed
UMLTP22-8 Cleanup Figure 8.15 - Test-specific actions Overview UTP2 1.0b1 $issue.fixedSpecification.name Resolved closed
UMLTP22-40 Correction of typografical and editorial formatting on specification documents UTP2 2.1 $issue.fixedSpecification.name Resolved closed
UMLTP22-39 Syntactical correction of machine-readable files for UTP 2.2 submision UTP2 2.1 $issue.fixedSpecification.name Resolved closed
UMLTP22-7 Change bar in section 8.6.1 Data Specifications need to be removed UTP2 1.0b1 $issue.fixedSpecification.name Closed; No Change closed
UMLTP22-10 Arbitration specification binding mechanism restricted to only one arbitration specification UTP2 1.0b1 $issue.fixedSpecification.name Resolved closed
UMLTP22-6 Last paragraph of Clause 8.6.2 Data Values needs to be removed UTP2 1.0b1 $issue.fixedSpecification.name Resolved closed
UMLTP22-2 Erroneous Figure 9.3 The UTP auxiliary library UTP2 2.0 $issue.fixedSpecification.name Resolved closed
UMLTP22-5 Unnecessary linebreak between first and second paragraph of Clause 8.7.1 Arbitration Specifications UTP2 1.0b1 $issue.fixedSpecification.name Closed; No Change closed
UMLTP21-19 Correct XMI files and URI of machine-readable files for UTP 2.1 UTP2 1.0b1 UTP2 2.1 Resolved closed
UMLTP21-2 Abstract TestDesignDirective concepts into abstract TestDirective UTP 2.0b1 UTP2 2.1 Resolved closed
UMLTP21-3 UTP test log facility ought to enable logging on procedural element level UTP 2.0b1 UTP2 2.1 Resolved closed
UMLTP21-1 LoginExample is not in synch with latest Profile Specification UTP 2.0b1 UTP2 2.1 Deferred closed
UMLTP2-24 Explicit arbitration specification binding does not scale UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-57 Corrections to resolutions summarized by the FTF errata sheet UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-46 ProcedureInvocation should specialize AtomicProceduralElement UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-12 Non-unique Rule Identifier UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-11 Clarification of vocabularies UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-8 Some references to "UML 2.5" in the constraints in the Profile Specification need to be cleaned-up. UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-6 The definitions of test level and test type are useless/not understandable. UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-53 Enumeration ProcedurePhaseKind is not listed in the specification UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-30 TestObejctive.specification and TestRequirement.specification should be ValueSpecifications UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-33 Multiplicity of ID for TestSet and TestContext should be 0..1 UTP2 1.0b1 UTP2 2.0 Resolved closed
UMLTP2-37 TestContext.testLevel and TestContext.testType ought to allow for an upper bound of * UTP2 1.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-3 Formal specification of ChoiceOfValues incomplete UTP 2.0a1 UTP2 2.0 Resolved closed
UMLTP2-2 Clean-up description of AnyValue UTP 2.0a1 UTP2 2.0 Resolved closed
UMLTP2-1 Include changes specified by errata UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-14 Predefined verdicts are not always reusable UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-13 Arbitration Specification Semantics UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-9 Some references to "UTP" in the description in the Profile Specification need to be cleaned-up. UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-7 The definitions of test level and test type are useless/not understandable. UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-4 Invalid links UTP 2.0b1 UTP2 2.0 Closed; No Change closed
UMLTP2-25 A stereotype Procedure should be incorporated as superclass of TestCase, TestExecutionSchedule and TestProcedure UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-27 TestLog.verdict should be extracted into a dedicated ArbitrationResult concept UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-28 TestLog.instanceOf should allow for many TestLogStructures UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-29 Clarification of the sets expectedElement, forbiddenElement and ignoredElement in ExpectResponseAction UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-16 OCL code and natural language description of Constraints inconsistent UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-15 Predefined verdict fault has no meaning UTP 2.0b1 UTP2 2.0 Duplicate or Merged closed
UMLTP2-10 LoginExample is not in synch with latest Profile Specification UTP 2.0b1 UTP2 2.0 Deferred closed
UMLTP2-5 Formal specification of ComplementedValue incomplete UTP 2.0b1 UTP2 2.0 Resolved closed
UMLTP2-26 UTP test log facility ought to enable logging on procedural element level UTP 2.0b1 UTP2 2.0 Deferred closed
UMLTP2-23 Abstract TestDesignDirective concepts into abstract TestDirective UTP 2.0b1 UTP2 2.0 Deferred closed

Issues Descriptions

Harmonize illustration of property modifiers in the spec

  • Key: UMLTP22-4
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The current document generation process used to produce the UTP 2 specs is limited with respect to the illustration of property/association end modifiers (i.e., unique, ordered, redefines, subsets, read-only, union). According to UML, those modifiers shall be illustrated after der name and type and multiplicity of a property. The following grammar rule (taken from UML 2.5, Clause 9.54) specifies the textual notation of properties:
    <property> ::= [<visibility>] [‘/’] <name> [‘:’ <prop-type>] [‘[‘ <multiplicity-range> ‘]’] [‘=’ <default>] [‘

    {‘<prop-modifier > [‘,’ <prop-modifier >]* ’}

    ’]

    In the UTP 2 document generation framework the illutration of modifiers was circumvented in a way that is is close to, but different to the notation prescribed by UML. Although, the aligned notation of UTP 2 is okay from a readers point of view, it appears that the modifiers have not been consistenly illustrated. See two examples from the spec:

    In section 8.3.2.7.17 TestDesignDirective , TestDesignDirective modifies are displayed before the name of the propery:

    {read-only, union} subDirective : TestDesignDirective [*]

    In section 8.4.2.5 TestConfigurationRole, modifiers are displayed after the name of the property:
    /roleConfiguration {read-only, union}

    : RoleConfiguration [*]

    Even though both ways are still good enough to comprehend, from an editorial point of view, one way should be preferred to other.

  • Reported: UTP2 1.0b1 — Fri, 10 May 2019 10:31 GMT
  • Disposition: Deferred — $issue.fixedSpecification.name
  • Disposition Summary:

    Deferred due to lack of time

    The RTF agreed that this a valid, but pure editorial issues. As explained in the issue description, the reason for these varying property modifiers lies in the document generation framework. Since all required information is available, though, the RTF agreed to resolve this issue in a future RTF.

  • Updated: Fri, 5 Jan 2024 20:27 GMT

Group abstract syntax diagram in separate sub-section for better changeability of the spec

  • Key: UMLTP22-9
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    In the current spec, the abstract syntax diagrams reside in dedictated sub-sections directly underneath the conceptual context section (e.g., 8.7.2. Test Logging) they belong to. Stereotype definitions of ths conceptual context are grouped in a dedicated sub-section 'stereotype definitions'.

    In UTP 2.1 RTF turned out that this structure does not support maintainability and modifiability of the spec. Everytime an additional diagram is added to the conceptual context section (maybe due to enhacements, clean-ups, better seperation and illustration of complex diagrams), the entire stereotype definitions need to be renumbered, because the respective diagram sub-sections and the 'stereotype specification' sub-section reside on the same level.

    If the abstract syntax diagrams would be sub-sections of a section called 'Abstract Syntax', there would be no need to renumber stereotypes.

    Therefore, I propose to make the spec more maintainable and stable by incoporating the following structure for every concepual context section in the spec.

    8.7. Test Evaluation
    8.7.1 ArbitrationSpecifications
    8.7.2 Test Logging
    8.7.2.1 Abstract Syntax
    8.7.2.2 Stereotype Specfications

  • Reported: UTP2 1.0b1 — Fri, 10 May 2019 10:15 GMT
  • Disposition: Deferred — $issue.fixedSpecification.name
  • Disposition Summary:

    Deferred due to lack of time

    The issues requests a pure editorial change for better maintainability of the specifcation. The RTF agreed that the proposed change makes sense, but due to lack of time, the resolution of that issue will be resolved in a future RTF.

  • Updated: Fri, 5 Jan 2024 20:27 GMT

Non-navigable associations ends shall be removed from the spec

  • Key: UMLTP22-11
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    In the current spec, non-navigable association ends (i.e., association ends that shall belong to the association) are still listed in the section 'Associations' of the stereotype on the other side.

    See for example in section 8.4.2.5 TestConfigurationRole the following entry in the sub-section 'Associations':
    : TestConfiguration

    The spec should be cleaned up in that regard.

  • Reported: UTP2 2.0 — Fri, 10 May 2019 10:05 GMT
  • Disposition: Deferred — $issue.fixedSpecification.name
  • Disposition Summary:

    Deferred due to lack of time

    The RTF agreed that this is indeed a valid, yet pure editorial issue. The reason for showing non-navigable association ends in the stereotype specifications lies in the document generation framework that is used to produce the specification document. It is, at the moment, not possible to spare non-navigable association ends from the generation process. In the next RTF, it will be resolved by either enhancing the document generation framework in a way so that it can spare non-navigable association ends or by adding an editorial explanation to clause 6.2 Typographical conventions.

  • Updated: Fri, 5 Jan 2024 20:27 GMT

UTP should incorporate concepts to speciy decision tables for allowing decision table testing

  • Key: UMLTP22-14
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Decision table testing is a major and frequently used black-box test design technique. In system and acceptance testing as well as testing of business processes, decision table testing is used to identify the test conditions within business rules by representing these rules in a tabluar format.

    The Decision Modeling Notation (DMN) provides a MOF-based metamodel for modeling decision tables. Within the DMN, decition tables are an isolated part, well suited for being extracted as profile as part of UTP.

    Incorporating a decision table profile into UTP, inspired by the DMN MOF-based metamodel, would improve UTP in the area of business rule testing and make UTP more complete regarding specification-oriented test design techniques.

  • Reported: UTP2 2.1 — Mon, 14 Dec 2020 21:24 GMT
  • Disposition: Deferred — $issue.fixedSpecification.name
  • Disposition Summary:

    Deferred due to lack of time

    The RTF agreed that there should be a tighter integration between decision tables and utp2 to perform decision table testing. Due to lack of time, the RTF agreed to defer the resolution of this issue to a subsequence RTF

  • Updated: Fri, 5 Jan 2024 20:27 GMT

Generators for and Queries on Test Logs

  • Key: UMLTP22-12
  • Status: closed  
  • Source: KnowGravity Inc. ( Mr. Markus Schacher)
  • Summary:

    Curently, UTP2 supports capturing actual results of a Test Set or a Test Case execution via its Test Log capability. A TestLog is an instance of a Test Log Structure that shall specify, what exacty is to be logged. However, the current UTP2 specification does not specify, how such a Test Log Structure is to be expressed. Furthermore, when one or more Test Logs have been instantiated (after executing some Test Sets or Test Cases), the tester may whish to extract only those parts of an existing (potentially huge) Test Log that are relevant to investigate a specific problem. This is not supported by the current UTP2 specification.

  • Reported: UTP2 2.0b2 — Tue, 18 Dec 2018 15:50 GMT
  • Disposition: Deferred — $issue.fixedSpecification.name
  • Disposition Summary:

    Deferred due to lack of time

    RTF agreed that this issues is a helpful extension to the language. It will, thus, be addressed in a future revision due to lack of time.

  • Updated: Fri, 5 Jan 2024 20:27 GMT

LoginExample is not in synch with latest Profile Specification

  • Key: UMLTP22-1
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Double-check that the LoginExample copes with the latest version of the spec.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:17 GMT
  • Disposition: Deferred — $issue.fixedSpecification.name
  • Disposition Summary:

    Deferred due to lack of time

    This issue is still valid, but will be addressed in a future RTF due to lack of time.

  • Updated: Fri, 5 Jan 2024 20:27 GMT

Cleanup Figure 8.15 - Test-specific actions Overview

  • Key: UMLTP22-8
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    "Figure 8.15 - Test-specific actions Overview" needs to be cleaned up. Stereotype shapes are overlapping.

  • Reported: UTP2 1.0b1 — Wed, 19 Jun 2019 13:46 GMT
  • Disposition: Resolved — $issue.fixedSpecification.name
  • Disposition Summary:

    Replace Figure 8.15 with new figure

    RTF agreed to correct this pure editorial issue.

  • Updated: Fri, 5 Jan 2024 20:12 GMT
  • Attachments:

Correction of typografical and editorial formatting on specification documents

  • Key: UMLTP22-40
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The specification documents (both ptc/23-04-02 and ptc/23-04-03) that were submitted with the UTP 2.2 RTF report (doc#: ptc/23-04-01) contained some issue with editiorial content. Some words that were set in (bold) blue do not represent links, but the typographical convention for these words are not given in the specification. Furthermore, the newly added clauses made not use of those links at all.

  • Reported: UTP2 2.1 — Fri, 21 Jul 2023 08:47 GMT
  • Disposition: Resolved — $issue.fixedSpecification.name
  • Disposition Summary:

    Complementary editorial/typographical changes from AB comments

    The following resolution and the editing instructions represent complementary editorial corrections of the already submitted specification documents (both ptc/23-04-02 and ptc/23-04-03) because of the Ab review comments recieved after submission of the UTP 2.2 in May 2023 (doc# ptc/23-04-01).
    The editing instructions describe only the delta of changes, thus, all the changes reported by the editing instructions of the formert report (doc# ptc/23-04-01) are present in the new specifications documents. In addition, the new submitted specification documents (ptc/24-07-02 and ptc/23-07-03) contain editing instructions as a result of the AB review of the former submission (ptc/23-04-01).

  • Updated: Fri, 5 Jan 2024 20:12 GMT

Syntactical correction of machine-readable files for UTP 2.2 submision

  • Key: UMLTP22-39
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The machine-readable files submitted with UTP 2.2 RTF report (doc#: ptc/23-04-01) contained some syntactical issues that prevented the import of the machine readable files into compliant tools.

  • Reported: UTP2 2.1 — Fri, 21 Jul 2023 08:43 GMT
  • Disposition: Resolved — $issue.fixedSpecification.name
  • Disposition Summary:

    Minor editorial changes to the XMI

    The machine-readable file for the UTP2 metamodel (utp2.xm, ptc/23-04-06) contained the following issues that prevent import/validity:

    Issue in Line 1091
    <ownedEnd xmi:type="uml:Property" xmi:id="_1CSfwPAuEeW2VZXfgcC2cA" name="referencedBy" type="_omHdQKJVEeS8gLY2208_PA" isDerived="true"association="_1CR4sPAuEeW2VZXfgcC2cA">

    Corrected to:
    <ownedEnd xmi:type="uml:Property" xmi:id="_1CSfwPAuEeW2VZXfgcC2cA" name="referencedBy" type="_omHdQKJVEeS8gLY2208_PA" isDerived="true" association="_1CR4sPAuEeW2VZXfgcC2cA">

    Issue in line 1231:
    <ownedLiteral xmi:type="uml:EnumerationLiteral" xmi:id="_MPcSIFeHEeihnpddrZbjug" name="implicitExpect"i/>

    Corrected to:
    <ownedLiteral xmi:type="uml:EnumerationLiteral" xmi:id="_MPcSIFeHEeihnpddrZbjug" name="implicitExpect"/>

    Issue in Line 1566:
    <ownedEnd xmi:type="uml:ExtensionEnd" xmi:id="KAmGUHJKEemyx_XW_ukewg" name="extension_TestTechniqueStructure" type="_90Z0HJJEemyx_XW_ukewg" aggregation="composite"association="_KAc8YHJKEemyx_XW_ukewg"/>

    Corrected to:
    <ownedEnd xmi:type="uml:ExtensionEnd" xmi:id="KAmGUHJKEemyx_XW_ukewg" name="extension_TestTechniqueStructure" type="_90Z0HJJEemyx_XW_ukewg" aggregation="composite" association="_KAc8YHJKEemyx_XW_ukewg"/>

  • Updated: Fri, 5 Jan 2024 20:12 GMT
  • Attachments:

Change bar in section 8.6.1 Data Specifications need to be removed

  • Key: UMLTP22-7
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    There is an obvious misplaced change bar marker in the beginning of section "8.6.1 Data Specification". It needs to be removed.

  • Reported: UTP2 1.0b1 — Wed, 19 Jun 2019 13:50 GMT
  • Disposition: Closed; No Change — $issue.fixedSpecification.name
  • Disposition Summary:

    Changebar not present anymore in UTP 2.1 official document

    This pure editotial issue has been already fixed with the release of the UTP 2.1 document

  • Updated: Fri, 5 Jan 2024 20:12 GMT

Arbitration specification binding mechanism restricted to only one arbitration specification

  • Key: UMLTP22-10
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP 2 defines for each element that may produce a verdict (i.e. test set, test case and all procedural elements), default arbitration specifications that prescribe how to calculate that verdict. The default arbitration specifications are optimized for functional testing, though. In case of testing non-functional aspects such as fault tolerance, performance or security, the rules to determine functional verdicts do usually not suffice anymore. Therfore, UTP 2 enables the definiton of proprietary arbitration specification (mainfested as stereotype <<ArbitrationSpecification>>) to grant the hightest flexibility possible to the user.

    However, the current mechanism to associate a test set, test case or procedural element with a dedicated arbitration specification is restricted to only one arbitration specification. Even though this would be sufficient in many situations, the problem with the current binding mechanism is that the test set, test case or procedural element is literally changed by the binding. This may lead to situations where elements are copied only for the sake of associating a different arbitration specification to a that element. This is a situation that should be avoided under all circumstances.

    We suggest adding a loosely coupled arbitration specification binding mechanism that keeps the elements and the arbitration specifications that ought to be bound to that element separated from each other. This would bring the benefit that test sets, test cases and procedural elements stay clean of any arbitration specification. This would also foster reusability of these elements.

    The loosely coupled mechanism to bind arbitration specifications to the test sets, test cases and procedural elements could be based on a new concept 'ArbitrationDirective' , similar to the 'TestDesignDirective' concept that already exists in the language and proved its applicability. By doing so, users could define and assign an arbitrary number of arbitration specifications to a test set, test case or procedural element without technically modifying any of these elements.

  • Reported: UTP2 1.0b1 — Fri, 5 Oct 2018 08:48 GMT
  • Disposition: Resolved — $issue.fixedSpecification.name
  • Disposition Summary:

    Introducing an arbitration specification binding mechanism

    The RTF agreed that the current arbitration facility is not flexile enough and will result in a number of duplicated model elements just for the purpose of assigning different pass/fail criteria. The RTF has developed an arbitration specification binding mechanism that loosely couples arbitration targets with arbitration specifications. Bindings can be overridden in a cascading manner that fosters reuse of existing arbitration specification.

  • Updated: Fri, 5 Jan 2024 20:12 GMT
  • Attachments:

Last paragraph of Clause 8.6.2 Data Values needs to be removed

  • Key: UMLTP22-6
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The last paragraph of Clause 8.6.2 Data Values is an unintended left over from the revised submission. In the revised submission, there was the attempt to precisely define the semantics of matching ValueSpecifications. This attempt ought to be entirely removed. Thus, this paragraph should be removed.
    For the sake of clarity:

    Both stimuli and expected responses yield data values for distinct signature elements. A signature element is defined
    as instance of either a Parameter or Property (i.e., this specification introduces a virtual metaclass SignatureElement that is the joint superclass of Property and Parameter and has at least the following attributes: type : UML::Type,
    lower : Integer, upper : UnlimitedNatural). Given by UML [UML25], a "... Type specifies a set of allowed values
    known as the instances of the Type." This specification denotes this set in the context of a SignatureElement
    expressed as type(se), with type(se) as SignatureElement.type, and use T as abbreviation for type(se).
    We specify
    with se instance of SignatureElement and lower(se) as SignatureElement.lower and denote it by SE type.
    A ValueSpecification V as an argument for a SignatureElement is specified as
    These basic definitions are further used for the specific ValueSpecification matching mechanism extensions
    introduced by UTP.

  • Reported: UTP2 1.0b1 — Wed, 19 Jun 2019 13:56 GMT
  • Disposition: Resolved — $issue.fixedSpecification.name
  • Disposition Summary:

    Remove last paragraph

    RTF agreed to remove the last paragraph as it represents a relict from the initial submission.

  • Updated: Fri, 5 Jan 2024 20:12 GMT
  • Attachments:

Erroneous Figure 9.3 The UTP auxiliary library

  • Key: UMLTP22-2
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Figure 9.3 shows a contained package "UTP Types Library" which is a relict from the initial submission. The UTP Types Library is a stand-alone library, technically unrelated to the UTP Auxiliary Library. This is also reflected in the machine readable files. Figure 9.3 needs to be corrected in that regard.

  • Reported: UTP2 2.0 — Thu, 20 Jun 2019 16:34 GMT
  • Disposition: Resolved — $issue.fixedSpecification.name
  • Disposition Summary:

    Replace Figure 9.3 with new figure

    Task force agreed that this minor, pure editorial issue should be resolved by replacing the erroneous figure with a corrected one

  • Updated: Fri, 5 Jan 2024 20:12 GMT
  • Attachments:

Unnecessary linebreak between first and second paragraph of Clause 8.7.1 Arbitration Specifications

  • Key: UMLTP22-5
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    This linebreak can be removed.

  • Reported: UTP2 1.0b1 — Wed, 19 Jun 2019 13:59 GMT
  • Disposition: Closed; No Change — $issue.fixedSpecification.name
  • Disposition Summary:

    Issue not present anymore

    The reported issue is not anymore present in the document with number formal/20-08-05.

  • Updated: Fri, 5 Jan 2024 20:12 GMT

Correct XMI files and URI of machine-readable files for UTP 2.1

  • Key: UMLTP21-19
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    This is an urgent issue, raised by the AB with respect to the correctness of the XMI files as well as the URI of the XMI files and the correct URIs mentioned on the cover page of the UTP 2.1 specification.

  • Reported: UTP2 1.0b1 — Thu, 20 Jun 2019 15:11 GMT
  • Disposition: Resolved — UTP2 2.1
  • Disposition Summary:

    Correct XMI files and URI of machine-readable files for UTP 2.1

    As discussed during the AB meeting at the Amsterdam meeting, the XMI file needed some cleanup and also the URI of the machine-readable files need to be corrected on the cover page.

  • Updated: Tue, 8 Oct 2019 18:00 GMT

Abstract TestDesignDirective concepts into abstract TestDirective

  • Key: UMLTP21-2
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The current TestDesignFacility og UTP 2 is based on the stereotypes TestDesignDirective, TestDesignTechnique, TestDesignDirectiveStructure and TestDesignTechniqueStructure. These elements are tightly integrated by means of the Profile's abstract syntax. So far, this is fine for the test design purposes. As such, the offered concepts support the definition of platform-independent instructions how to derive test cases, test data, test sets, test procedures and/or test execution schedules. These instruction can be put into effect by either a manual test designer or translated into the inpurt for a test generator.

    UTP 2, however, is capable of describing test automation architectures (ISTQB TAA) or the architecture of automated testware - both terms are considered to be synonyms. In a test automation architecture, further activities need to be specified such as the generation of executable test code, the configuration and automated setup of the test execution environment and tool (setup and teardown of the automated test environment; compilation of test scripts, automated scheduling of test cases in a test set according to a given test execution schedule etc.), the configuration of the logging facility (e.g., depths and structure of logged information), the configuration of the arbitration specification (which arbitration specifications shall be used, linked to test cases, test sets and procedural elements, etc.).

    All these configurations are essentials parts of an automatedt testware and can be used in combination or separately from each other (e.g., specifying a specific arbitration specification for verdict calculation on already executed set of test cases in a post-execution manner).

    UTP is currently too restrictive and only offers concepts for configuring the test design activities. As a test modelling language designed in particular for test automation, UTP should offer also concepts for the other activities that need to be performed in an test automation solution.

    Therfore, UTP should abstract from TestDesignDirective and TestDesignTechnqiue into abstract classes TestArchitectureDirective and TestArchitectureCapability (and the respective structures). That would, one hand, enables user-specific extensions in order to cope with the aforementioned, not yet supported concepts of test script generation, test execution, test logging and verdict arbitration, on the other hand prepare UTP 2 for potential improvements in that area in further revisions of the language.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 10:00 GMT
  • Disposition: Resolved — UTP2 2.1
  • Disposition Summary:

    Introduce an abstract TestDirective facility

    The RTF agreed that an integration of an abstract test directive concepts (similar to the already existing and proved test design directive facility) enables test engineers to capture information relevant for carrying out certain test-related activities such as test design, test execution and/or arbitration. Allowing test engineers to model these kind of information opens the door for automating them.

  • Updated: Tue, 8 Oct 2019 18:00 GMT
  • Attachments:

UTP test log facility ought to enable logging on procedural element level

  • Key: UMLTP21-3
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP 2 offers currently rudimentary test log capabilities. It is possible to capture the minimal test execution information for test cases and test sets. These information are sufficient for the sake of coverage calculation or high-level test reporting, however, the details of a test case execution (e.g., the execution of CreateStimulusActions or ExpectedResponseAction) are completely omitted. This wastes a lot of potential UTP could offer for building test automation architectures like post-execution comparison/verdict calculation, incorporation of test execution traces from heterogeneous test tools in order to process the captured information solely on model level (raising the level of abstraction for automated results checking), comprehensibility or high-level failure analysis and test execution comparison (trends etc.).

    As high-level test modelling language, UTP ought to provide at least a facility to enable sophisticated activities on fine-grained test execution logs by capturing the execution on procedural element level.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 20:30 GMT
  • Disposition: Resolved — UTP2 2.1
  • Disposition Summary:

    Enable logging on procedural element level

    The task force agreed that the ability of abstracting from technical test logs into platform-independent test logs opens the door for UTP 2 to integrate the execution details of different test results potentially created by different test exeuction tools, as well as additional information such as information obtained from dynamic analysis tools by means of a single platform-independent language. UTP 2 may use these information for many purposes such as visualizing the executed test cases in a unique way; post-processing the execution results to calculate an overall verdict; perform further analysis like timing violations etc.
    Having a single, integrated view on the results of test executions may become very conventient particulary in situations with heterogeneous testing landscapes where the eventual verdict of a test execution log has to be computed by a comprehensive analysis of diverse test execution logs.
    UTP 2 can be seen as a kind of test information integration language that enables the specification and implementation of analyses over a set of originally seperated test execution logs.

  • Updated: Tue, 8 Oct 2019 18:00 GMT
  • Attachments:

LoginExample is not in synch with latest Profile Specification

  • Key: UMLTP21-1
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Double-check that the LoginExample copes with the latest version of the spec.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:17 GMT
  • Disposition: Deferred — UTP2 2.1
  • Disposition Summary:

    Deferred due to lack of time

    This issue is still valid, but will be addressed in a future RTF due to lack of time.

  • Updated: Tue, 8 Oct 2019 18:00 GMT

Explicit arbitration specification binding does not scale

  • Key: UMLTP2-24
  • Status: closed   Implementation work Blocked
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP offers a concept to bind arbitration specifications (henceforth, the acronym AS is used) to test sets, test cases and procedural elements. The binding comes in three flavors (see page 136):

    • default arbitration specification realized by an implicit binding
    • static arbitration specification realized by a static binding directly by the respective test set, test case or procedural element
    • dynamic arbitration specification (dynamic binding) overrides any previously existing binding in the context of a procedure invocation

    Albeit these three binding kinds makes definitely sense (and should not be removed from the spec) the problem is the technical representation of how bindings shall be expressed, i.e., directly by the test set, test case, procedural element or procedure invocation for dynamic binding. The main issue is that the technical binding mechanism does not scale for large test sets that exhibits exhaustive modularization and reuse of test procedures. In theory, there can be an unlimited number of arbitration specifications set for a test set, test case or procedural element. The classification into static and dynamic binding is a mere synthetic (and unnecessarily complex) concept. In fact, there can only be one static binding being present at any point in time and any further binding must be made via dynamic binding, which in turn requires the use of a <<ProcedureInvocation>> and, thus, a TestExecutionSchedule to actually invoke a test set/test case to dynamically bind the respective AS. This imposes a certain methodology on the user, which is too prescriptive.

    The entire mechanism becomes more complicated when procedural element AS shall be dynamically bound. This can only be achieved by simply listing procedural element AS in the set of 'ProcedureInvocation.dynamicArbitrationSpecification' and the use of the boolean field 'transitiveOverride'. It is not possible to explicitly bind a certaint procedural element AS to just a single procedural element (or a subset thereof). Furthermore, it is not easily possible to specify whether a transitiveOverride shall be applied to all deeply nested ProcedureInvocations or not.

    Eventually, it must be stated that the approach to bind different AS to the very same test set/test case/procedural element is a good idea (that should be preserved), but as it is realized now, not usable and excessively complex.

    A possible solution to that inflexible and overly complex binding mechanism could be to extract the binding from test set/test case/procedural element into a dedicated concept that allow for expressing these bindings external to the test set/test case/procedural element. This would have two advantages if done properly:

    1. The test set/test case/procedural element would remain clean of any AS information and would only represent the information required to eventually execute a test case. Test sets/test cases/procedural elements would be easier to maintain after all.

    2. An external binding mechanism would be able to explicitly specify which AS shall be set for which element in a certain context. This holds true even for deeply nested test procedures and procedural elements, which is not the case with the current binding mechanism. A kind of ArbitrationDirective would enable a flexible and unrestricted mechanism to define how the verdict for a given test set/test case/procedural element shall be calculated without imposing certain design decision (like the use of TestExecutionSchedules and ProcedureInvocations) on the test designer. This is in turn another fact that positively impacts maintainability of existing test specifications. Practically, an unlimited number of ArbitrationDirectives could be defined for any test set/test case, which would make the AS mechanism most flexible.

    If extracted, there would only be two kinds of binding remain:

    • implicit binding: if no additional AS information is specified for a certain test set/test case/procedural element, the default AS would take over (like it is right now)
    • explicit binding: (Parts of) the AS bindings for test sets/test cases and procedural elements are explicitly bound and can be put into effect by using a certain ArbitrationDirectve. Additionally, other ArbitrationDirectives can be defined to declare further AS bindings, without the need to change (or even touch), the existing test set/test case/procedural elements.
  • Reported: UTP 2.0b1 — Wed, 24 Jan 2018 09:58 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Arbitration specification bindings will be cleaned up from technical debts

    The FTF agree that the mechanism to bind multiple Arbitration Specifications to test cases, test sets and procedural elements is incomplete and blocking implementations. Parts of the specification needs to be cleaned up from techhnical debts.

    Furthermore, the FTF agreed that UTP 2 would benefit from a mechanism that offers consistent and scalable way to bind different arbitration specifications for different analysis purposes. Idealy, the elements that shall be arbitrated and the corresponding arbitration specifications shall be loosely coupled by a dedicated and reliable UTP 2 capability for arbitration specification binding.

    Adding such a new capability is outside the scope of the FTF. Therefore, the FTF will submit another issue for the upcoming RTF to incorporate that specific functionality.

    Since dynamic binding of arbitration specifications is not anymore part of the specification, for it was never finished properly, editorial parts of the specification that deals with the different binding mechanisms must be revised to remove any description about the dynamic arbitration specification binding.

  • Updated: Fri, 5 Oct 2018 08:49 GMT
  • Attachments:

Corrections to resolutions summarized by the FTF errata sheet

  • Key: UMLTP2-57
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    A few mistakes were made during the elaboration of the issue resolutions that led to a situation where editing instructions were fogotten to be stated, however, the changes have been applied to the language. The FTF had created an errata sheet to keep track of these supplementing editing instructions. Due to OMG policies, the changes described by the errata sheet need to be voted on as well. This was requested by the AB reviewer.
    Therefore, the following list summarizes incomplete editing instructions of already resolved FTF issues (the errata sheet that was submitted with the first FTF submission is attached to the issues):

    • Issue UMLTP2-14
      Summary of the correction: The predefined verdict 'fault' was removed from the UTP 2 Types Library. The corresponding element 'Fault' of the conceptual model (Clause 7.6 Arbitration Specification) was not removed by the resolution. This concept must be removed to keep the conceptual model and the profile specification in synch.
    • Issue UMLTP2-24
      Summary of the correction: An obsolete entry 'Detailed Semantics' of 8.7.1.1.2.2 ArbitrationSpecification needs to be removed from the specification. Additionally, the Figure 8.1 Language Architecture need to reflect the removal of the UTP Arbitration library (old Clause 9.3, which is removed by the resolution of Issue UMLTP2-24)
    • Issue UMLTP2-27
      Summary of the correction: The newly introduced stereotype "ArbitrationResult" needs to be listed in the profile summary table in Clause 8.2 Because of the introduction of a new stereotype, the numbering of the sub-clauses in clause 8.7.1.1.2 Stereotype Specifications needed to be updated. The new numbers that
      mentioned by the original issue resolution had a typo and were not correct. This is corrected by this errata document.
    • Issue UMLTP2-29
      Summary of the correction: The newly introduced attribute 'expectationKind' of Clause 8.5.3.4.4 ExpectResponseAction was omitted by the issue resolution.
  • Reported: UTP 2.0b1 — Thu, 14 Jun 2018 09:29 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Corrections of incomplete editing instructions listed by the FTF errata sheet

    Apply corrections for incomplete editing instructions of the following issues:

    • Issue UMLTP2-14
      Summary of the correction: The predefined verdict 'fault' was removed from the UTP 2 Types Library. The corresponding element 'Fault' of the conceptual model (Clause 7.6 Arbitration Specification) was not removed by the resolution. This concept must be removed to keep the conceptual model and the profile specification in synch.
    • Issue UMLTP2-24
      Summary of the correction: An obsolete entry 'Detailed Semantics' of 8.7.1.1.2.2 ArbitrationSpecification needs to be removed from the specification. Additionally, the Figure 8.1 Language Architecture need to reflect the removal of the UTP Arbitration library (old Clause 9.3, which is removed by the resolution of Issue UMLTP2-24)
    • Issue UMLTP2-27
      Summary of the correction: The newly introduced stereotype "ArbitrationResult" needsto be listed in the profile summary table in Clause 8.2. Because of the introduction of a new stereotype, the numbering of the sub-clauses in clause 8.7.1.1.2 Stereotype Specifications needed to be updated. The new numbers that
      mentioned by the original issue resolution had a typo and were not correct. This is corrected by this errata document.
    • Issue UMLTP2-29
      Summary of the correction: The newly introduced attribute 'expectationKind' of Clause 8.5.3.4.4 ExpectResponseAction was omitted by the issue resolution.
  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

ProcedureInvocation should specialize AtomicProceduralElement

  • Key: UMLTP2-46
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The stereotype ProcedureInvocation allows fro invoking test-specific procedures such as TestExecutionSchedules, TestCases and TestProcedures, but also simple libraries.

    It is currently specializes CompoundProceduralElement, which seems to be erroneous. The invokation of a test-specific procedure is an atomic instruction, and as such, it should specialize AtomicProceduraleElement instead.

    The corresponding ProcedureInvocationArbitrationSpecification specializes AtomicProceduralElementArbitrationSpecification, though. So, what ever decision will be made with respect to ProcedureInvocation, the corresponding arbitration specification might require some re-work, too.

  • Reported: UTP 2.0b1 — Tue, 24 Apr 2018 18:11 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Change ProcedureInvocation to specialize AtomicProceduralElement instead

    The FTF decided that ProcedureInvocation is not a CompoundProceduralElement, but an AtomicProceduralElement. This is already reflected in the editorial parts of the specification (i.e., the conceptual model and the arbitration specification section of clause 8), but was obviously incorrectly stated in the diagrams of both the conceptual model and the profile specification (maybe as a relic from the initial submission).

    This issues will be resolved together with proposal #54 for issue #24

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Non-unique Rule Identifier

  • Key: UMLTP2-12
  • Status: closed  
  • Source: KnowGravity Inc. ( Mr. Markus Schacher)
  • Summary:

    Rules DRTC01 (should be "DRTR01") and DRTC01 have the same identifier.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 10:32 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Rename rule DRTC01 to DRTR01

    The rule in clause 7.2.1 has accidently named the same way as a rule in clause 7.3.1.1. This must be fixed simply by renaming the rule.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Clarification of vocabularies

  • Key: UMLTP2-11
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP should better distinguish the origin of the terms in the vocabulary. E.g., there are concepts and definitions, which are solely listed in the conceptual model or the profile implementation vice versa. The UTP vocabulary should indicate where this term originated from.

    A potential solution is to add a introductory sentence to section 4 to explain that the terms in section 4 are a summary of the conceptual model of UTP 2 (section 7). Furthermore, improve section 6.2 (Typographical Conventions) with respect to the kids of cross references in the spec (to the definition, to the conceptual model to stereotypes)

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:46 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Clarify the nature of Clause 4

    This issue will be merged with the issue #6 and resolved in the course of the resolution of issue #6 with is done with the proposal #47.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Some references to "UML 2.5" in the constraints in the Profile Specification need to be cleaned-up.

  • Key: UMLTP2-8
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    This issue was raised by Manfred Koethe as UTP 2 revised submission reviewer

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:13 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    All OCL constraints were removed from the specification

    During the resolution of UMLTP2-16, it was decided to firstly removed the entire OCL code from the specification for it was highly erroneous and written in a rush. This means that also the incorrect references to UML 2.5 will removed in the course of the resolution of issue #16, thus, this issue will be resolved by proposal #UMLTP2-39.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

The definitions of test level and test type are useless/not understandable.

  • Key: UMLTP2-6
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    This issue has been raised by Manfred Koethe as UTP 2 revised submission reviewer.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:07 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Clarify the nature of Clause 4

    The terms and definitions found in clause 4 are a summary of the conceptual model that is defined in clause 7. Examples and further details can be found in the respective sub-section of clause 7.

    Therfore, the FTF agreed to add an introduction to clause 4 that helps the reader to understand the nature of clause 4 and refer to clause 7 for further examples and details.

    Furhter, the FTF decided that the definitions of test type and test level suffice as a summary, for further examples and details for both definitions are given in clause 7. Clause 6.1 "How to read this document" shall be updated accordingly.

    In the course of this discussion, the FTF found out that the definition of test objective can be improved for clarity.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Enumeration ProcedurePhaseKind is not listed in the specification

  • Key: UMLTP2-53
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The description of the enumeration ProcedurePhaseKind is not part of the specification.

  • Reported: UTP 2.0b1 — Thu, 26 Apr 2018 10:49 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Incorporate ProcedurePhaseKind into the specification

    The FTF decided that is an editorial issue that has to be fixed.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

TestObejctive.specification and TestRequirement.specification should be ValueSpecifications

  • Key: UMLTP2-30
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The tag definition specification of a TestObejctive and TestRequirement is currently typed by String. Although this would most likely suffice in many situation, it prevents tools to specify other forms than plain text (e.g., structured text with formal semantics such as ETSI's TPLan). If this tag definition would be of type ValueSpecification, tools/implementers could decide to go for a more capabale representation of a 'specification' such as OpaqueExpressions or Expression. Eventually, plain text is again supported by the use of LiteralString.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 10:05 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Replace String attributes 'specification' with ValueSpecification

    As the issue description mentiones, replacing the String-typed attribute 'specification' of TestRequirement and TestObjective with ValueSpecification widens the possibilities of how to express these information.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

Multiplicity of ID for TestSet and TestContext should be 0..1

  • Key: UMLTP2-33
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The ID attribute offered by the TestSet and TestContext should be 0..1 similar to TestConfiguration, TestCase etc.
    The reason is that those IDs shall act as traceability element in context of a testing tools landscapes, in particular for synchonization with test management and test reporting tools.
    If such a traceability link is not required, models shall not become invalid.

  • Reported: UTP2 1.0b1 — Mon, 19 Feb 2018 14:22 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Change multiplicity of attribute ID of TestSet and TestContext to 0..1

    As recommended by the issue description, the mutliplicity of the attribute 'ID' should be changed to 0..1.
    This proposal incorporates also the resolution of proposal UMLTP2-38. As agreed in that proposal, the tag definitions 'testLevel' and 'testType' of TestContext ought to have an upper bound of *.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

TestContext.testLevel and TestContext.testType ought to allow for an upper bound of *

  • Key: UMLTP2-37
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The two tag definitions 'testLevel' and 'testType' determine the corresponding test item level and the test item's quality criteria that are targeted by the artifacts of the test context. Currently, this is set to an optional multiplicity of 1, but ought to be 0..*

  • Reported: UTP2 1.0b1 — Mon, 26 Mar 2018 11:47 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Change upper bound to 0..*

    The issue is correct; it is easily possible that tester want to reuse the test artifacts assembled by the test context for more than just one test level or test type.

    In fact, the way it is written in the spec (plural), it seems that it was intended from the very beginning to assign an upper bound of * to both tag definitions.

    The resolution of this issue was done in conjunction with proposal UMLTP2-34.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Formal specification of ChoiceOfValues incomplete

  • Key: UMLTP2-3
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The formal specification of ChoiceOfValues is incomplete. These paragraphs are relics from an approach to provide a precise semantics of data value extensions.

    Suggestion: Remove these paragraphs.

  • Reported: UTP 2.0a1 — Mon, 15 Jan 2018 13:42 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Remove incomplete formal specification of ChoiceOfValue

    The paragraphs mentioned in the issue descriptions are relics of the attempt to state the semantics of some ValueSpecifications extensions more precise during the revised submission phase. It shall simply be removed from the description.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

Clean-up description of AnyValue

  • Key: UMLTP2-2
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The last two paragraphs of AnyValue description are incomplete.

    Suggest resolution: Remove those two paragraphs

  • Reported: UTP 2.0a1 — Mon, 15 Jan 2018 13:25 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Remove incomplete formal specification of AnyValue

    The paragraphs mentioned in the issue descriptions are relics of the attempt to state the semantics of some ValueSpecifications extensions more precise during the revised submission phase. It shall simply be removed from the description.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Include changes specified by errata

  • Key: UMLTP2-1
  • Status: closed  
  • Source: KnowGravity Inc. ( Mr. Markus Schacher)
  • Summary:

    Most changes specified in the errata to the beta specification (ad/17-06-01) have not yet been included (except for the changes on the cover page). These changes must be included in the final specification.

  • Reported: UTP 2.0b1 — Mon, 15 Jan 2018 11:45 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Resolve errata issues that are still valid

    The FTF decided to check each errata issue for validity and to resolve them in case they have not yet been resolved in course of the production of the beta specification.

    The following proposal compares the errata (OMG doc# ad/2017-06-01) issues (by number) with the published document with doc# ptc/2017-09-29. In case the errata issue has already been resolved during the production of the Beta specification (ptc/2017-09-29) na action will be taken and the corresponding errata issue will declared as obsolete.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

Predefined verdicts are not always reusable

  • Key: UMLTP2-14
  • Status: closed   Implementation work Blocked
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP 2 provides the predefined verdict instances none, pass, inconclusive, fail, error. These instances are currenty represented by LiteralStrings of type 'verdict'. Using LiteralStrings is fine for the UTP test logging facility for TestCaseLog and TestSetLog simply refers to ValueSpecifications of type 'verdict'. But in situations where those predefined verdict instances shall be used in a different context e.g. in the context of types of Properties of a StructuredClassifier and InstanceSpecifications thereof, LiteralStrings cannot be really reused for Slot values. Therefore, the LiteralString should be reworked into InstanceSpecifications of type 'verdict' with their specification set to LiteralStrings (of type 'verdict') with name and value set to the 5 predefined verdicts.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 13:10 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Replace LiteralString verdict instances with InstanceSpecification verdict instances

    As the issue description indicates the use of LiteralStrings as predefined verdict instances may prevent reuse of these predefined verdict instances in certain situations.
    If one of the predefined verdict instances shall be used in a context other than associating test logs with these predefined verdicts, it happens that reuse is not possible because of composite associations between a UML metaclass and ValueSpecification.
    For example, if there is a user-defined complex verdict, expressed as StructuredClassifer with Properties and a predefined verdict shall be assigned to one of the Properties in the context of an InstanceSpecification's Slot, it happens that Slot.value is a composite association to ValueSpecification. This means that the Slot physically owns the ValueSpecification, which is not possible then in the context of a predefined (i.e., read-only) model library.
    Therefore, the proposal is to use InstanceSpecifications instead to express the predefined verdicts. By doing so, these verdict instances can be reused by means of InstanceValues.
    The specification shall not, however, prevent using other ValueSpecifications as verdict instances in case of user-specific test models. It must be clearly stated that the use of InstanceSpecifications is simply the default UTP way to express verdict instances.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

Arbitration Specification Semantics

  • Key: UMLTP2-13
  • Status: closed  
  • Source: KnowGravity Inc. ( Mr. Markus Schacher)
  • Summary:

    The relationship between "8.7.1 Arbitration Specifications" and "9.3 UTP Arbitration" is not clear. Probably, 9.3 is only the (informal?) semantics of 8.7.1?

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 12:52 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Natural language description of default arbitration specification suffices for the moment

    During the meeting on 19th April, the group agreed to do the following:

    • Remove complete clause 9.3 UTP Arbitration from the spec for it represents an informative part of the spec. The classes provided by that library were never meant to be implemented by a vendor; they serve the mere purpose to unambiguously define the semantics of the default arbitration specifications.
    • Provide a natrual language decscription of the default arbitration specification as normative part of the stereotype descriptions

    Removal of clause 9.3 is the easiest way to solve this issue. Having an executable specification is a nice add-on, especially for tool vendors. But this should be a self-contained clause of mere informative nature. It is therefore heavily recommended and indorsed by the FTF to re-incorporate the precise yet executable semantics of the default arbitration specifications back into the document.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Some references to "UTP" in the description in the Profile Specification need to be cleaned-up.

  • Key: UMLTP2-9
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    This issues was raised by Manfred Koethe as UTP 2 revised submission reviewer

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:14 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    All OCL constraints were removed from the specification

    During the resolution of UMLTP2-16, it was decided to firstly removed the entire OCL code from the specification for it was highly erroneous and written in a rush. This means that also the incorrect references to UTP stereotypes in the OCL constraints were removed, thus, this issue was resolved by proposal UMLTP2-39.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

The definitions of test level and test type are useless/not understandable.

  • Key: UMLTP2-7
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    This issue was raised by Manfred Koethe as UTP 2 revised submission reviewer.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:08 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Duplicated submission - see UMLTP2-6

    This issue is a duplicated submission of UMLTP2-6

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Invalid links

  • Key: UMLTP2-4
  • Status: closed  
  • Source: KnowGravity Inc. ( Mr. Markus Schacher)
  • Summary:

    Most of the intra-document links (the blue underlined words) do not refer to the correct target. This seems to be caused by the transformation process between the original specification model generated out of the model and the final PDF document published as the beta version of UTP.

  • Reported: UTP 2.0b1 — Mon, 15 Jan 2018 14:03 GMT
  • Disposition: Closed; No Change — UTP2 2.0
  • Disposition Summary:

    Obsolete without manual transformation of generated document

    This issue should become obsolete without manual transformation of generated document

  • Updated: Wed, 3 Oct 2018 14:16 GMT

A stereotype Procedure should be incorporated as superclass of TestCase, TestExecutionSchedule and TestProcedure

  • Key: UMLTP2-25
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The concpetual model (Clause 7) and profile specification (Clause 8) uses the term 'procedure' as parent concept for the test-specific procedures introduced by UTP, which are TestCase, TestExecutionSchedule and TestProcedure.

    While the test-specific procedures are reflected by stereotype definitions in Clause 8, the rather abstract concept 'procedure' is not represented as stereotype.

  • Reported: UTP 2.0b1 — Wed, 24 Jan 2018 08:36 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    No added value by a stereotype Procedure

    Having a stereotype <<Procedure>> would not add any value to the language.
    The UML representation of the concept 'Procedure' is given by Behavior. A stereotype <<Procedure>> would not add any additional property. Therefore, the FTF decided to close this issue with no change.

    Instead of creating a new stereotype, the FTF decided to introduce an additional derived tag definition into <<ProcedurInvocation>> of type Behavior that retrieves the invoked Behavior from the underlying UML metaclasses.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

TestLog.verdict should be extracted into a dedicated ArbitrationResult concept

  • Key: UMLTP2-27
  • Status: closed   Implementation work Blocked
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP allows for capturing the most important test execution information for test cases and test sets. Those information are the minimum required information to document and reasoning over coverage of test requirements or test objectives.

    In theory and practice, it is possible and required to calculate verdicts by different verdict arbitrations for the very same test case or test set. A test log should only contain the mere information about the exchange of message between the test environment and the test item. Information about comparison or verdict arbitration should not be part of a test log, but of a dedicated ArbitrationResult concept.

    The extraction of verdict into a dedicated ArbitrationResult is also necessary to avoid introducing redundant information into the model by copying the test log elements for the mere purpose of verdict association. Currently, it would not introduce much redundancy by copying just a single TestLog element (and the respective stereotype, of courde), but it is expected that UTP will introduce a fine-grained log model in a future version that is capable of recording all relevant information of test executions. If verdict still remained to test log or the test log entries later on, it would be necessary to potentially copy entire (and long) test log information with dozens of actual procedural elements, just for the sake of additional verdict arbitration. This situation would not scale at all for large test projects.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 20:11 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    ArbitrationResult is an important concept to avoid redundancy

    The concept ArbitrationResult will be integrated into the Profile Specification to avoid redundancy in case the very same test case shall be checked by different ArbitrationSpecifications which may produce different verdicts.
    No change of the conceptual model required, for the ArbitrationResult is simply a technical solution to avaoid redundancy, not a conceptual change.
    The solution will be based on UML's InstanceSpecification concept, for an ArbitrationResult is actually an instance of the execution of an ArbitrationSpecification.
    ArbitrationResult will contain at least the attribute ‘verdict’. Additional information might be added by means of the pre-defined auxiliary library (later on)

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

TestLog.instanceOf should allow for many TestLogStructures

  • Key: UMLTP2-28
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Structural aspects other than the predefined ones for the stereotype TestLog can be added by means of a <<TestLogStructure>> Classifier. Currently, UTP restricts a TestLog to be classified by at most one TestLogStructure. But it may easily happen that a TestLog is supposed to capture the information form more than just one <<TestLogStructure>> Classifier. Therefore, the multiplicity of TestLog.instanceOf should be 0..*.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 19:56 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Allow for multiple <<TestLogStructure>> Classifiers

    The abstract stereotype <<TestLog>> has a derived association to TestLogStructure. It is derived from the set of <<TestLogStructure>>Classifiers of the underlying InstanceSpecification (see InstanceSpecification.classifier). It should be possible to have more than just one <<TestLogStructure>>Classifier for any concrete <<TestLog>> to remain flexible and extendible for user needs.

    This issue will be resolved in the course of resolving issue UMLTP2-27 with proposal UMLTP2-36.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

Clarification of the sets expectedElement, forbiddenElement and ignoredElement in ExpectResponseAction

  • Key: UMLTP2-29
  • Status: closed   Implementation work Blocked
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    When designing test cases, the formalization of expected responses of the SUT in accordance to previously sent stimuli are of utmost importance for eventual verdict calculation. The expectations of the SUT's behavior can be expressed by at least two sets, the set of expected responses and the set of forbidden elements. In some situations it accelerates the test design if the expected responses are specified (when X is submitted Y is expected), in some situations it is easier to simply specify what is not expected (when X is submitted everything else but not Y is expected). Which set shall rather be used cannot be answered and depends heavily on the context of the test case. These two sets are complemented by the set of responses that can be ignored as a response of a stimulus. Ignoring elements (i.e., excluding them from verdict arbitration) is an important capability in automated testing.

    Currently, ExpectResponseAction explicitly defines the set of expected and forbidden elements. The set of ignored elements is implicitly represented by the complement set of the union of expected and forbidden elements.

    The current semantics of forbiddenElements says that elements that are forbidden (i.e., that expected to not being sent by the SUT at all) need to be explicitly declared, and if no elements are specified, no elements are forbidden. This definition has a big disadvantage for testing, for it is prone to result in 'false negatives' during test execution. Let's assume that a tester makes a mistake while specifying the set of forbidden elements. Response Y should actually be forbidden, but the tester forgot to explicitly assign Y to the set of forbidden elements. Let's further assume that during test case execution Y occurs in a situation where it ought to not occur. Since it is implicitly (but erroneously) accepted as at least ignored element, the test case would conclude with a 'false negative' verdict. That means that an existing failure of the SUT was not detected as such. 'False negatives' are to be avoided under all circumstances for they provide a wrong assumption about the quality of the SUT.

    Therefore, UTP should remove the set of forbidden elements from ExpectResponseAction and replace it by the set of ignored elements. By doing so, a tester would have to explicitly specify what is expected and what shall be ignored. All other elements, i.e., the complement set of the union of expected and ignored elements, are implicitly treated as forbidden elements. This would prevent the creation of implicit 'false negatives', for everything that the tester does not explicitly allow or ignore is considered to be forbidden.

    A second alternative could be to enable the specification of all three sets for ExpectResponseAction and to introduce an enumeration ImplicitExpectationKind with literals ImplicitIgnore, ImplicitForbid (default) and ImplicitExpect. This enables the tester to define exactly those sets that are most appropriate for the respective ExpectResponseAction.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 10:40 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    The three sets shall be made explicit

    Flexibility is of utmost importance for a standard like UTP 2. Since it is not possible to foresee which way is the ideal way to deal with the three different sets, UTP 2 will make the three sets explicit.

    In addition, to these three sets, a mechanism will be introduced that determines which set will be implicitly merged (union) with all other received responses. By default, the set of forbidden element will be implicitly extended. This prevents the occurrence of false negative results in case an actually forbidden element was forgotten to be specified as such.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

OCL code and natural language description of Constraints inconsistent

  • Key: UMLTP2-16
  • Status: closed   Implementation work Blocked
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Many Constraints of stereotypes are inconsistent with respect to OCL and natural language.

    See section 8.5.1.2.2 TestCase:

    • Constraint 'Use of BehavioredClassifier' is inconsistent
    • Constraint /* DRTC02: It is necessary that each test execution schedule requires at most one preconditon. */ does not belong to the stereotype test case at all.

    This is the case for most of the Constraints in section 8. A thorough check would be required to clean that up.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 15:35 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Remove Constraints and OCL syntax of natural langaue description if necessary

    The formalization of natural language constraints into OCL code was rather done in a rush for the revised submission. As a result, many OCL constraints were not correctly written. In these situation, the OCL code shall be removed in the first place.
    In some cases, entire Constraints were too restrictive and will be removed.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Predefined verdict fault has no meaning

  • Key: UMLTP2-15
  • Status: closed   Implementation work Blocked
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The predefined verdict fault and error are synonyms. Fault shall be removed. Error is more common in the test automation community to describe situations where some technical exception occured in the test environment.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 13:13 GMT
  • Disposition: Duplicate or Merged — UTP2 2.0
  • Disposition Summary:

    Remove predefined 'fault' verdict

    The predefined verdict 'fault' has no clear meaning to the TF. Therefore, it shall be removed from the specification entirely.

    This issue shall be resolved by the resolution of issue UMLTP2-14.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

LoginExample is not in synch with latest Profile Specification

  • Key: UMLTP2-10
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    Double-check that the LoginExample copes with the latest version of the spec.

  • Reported: UTP 2.0b1 — Tue, 16 Jan 2018 09:17 GMT
  • Disposition: Deferred — UTP2 2.0
  • Disposition Summary:

    Correct the details Login example in a later RTF

    The example is based on an earlier submission (i.e., the initial submission) of the UTP 2. Even though the example is not anymore in synch with the current version of the profile specification, it still helps understanding how to apply the UTP ina dynamic test process.

    Due to limited time, it was decided to defer the correction of the Login example to a later RTF.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Formal specification of ComplementedValue incomplete

  • Key: UMLTP2-5
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The last two paragraphs represents an incomplete formal specification of the semantics of ComplementedValue. It should be removed for does not have an added value in the current state.

  • Reported: UTP 2.0b1 — Mon, 15 Jan 2018 15:50 GMT
  • Disposition: Resolved — UTP2 2.0
  • Disposition Summary:

    Remove incomplete formal specification of ComplementedValue

    The paragraphs mentioned in the issue descriptions are relics of the attempt to state the semantics of some ValueSpecifications extensions more precise during the revised submission phase. It shall simply be removed from the description.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments:

UTP test log facility ought to enable logging on procedural element level

  • Key: UMLTP2-26
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    UTP 2 offers currently rudimentary test log capabilities. It is possible to capture the minimal test execution information for test cases and test sets. These information are sufficient for the sake of coverage calculation or high-level test reporting, however, the details of a test case execution (e.g., the execution of CreateStimulusActions or ExpectedResponseAction) are completely omitted. This wastes a lot of potential UTP could offer for building test automation architectures like post-execution comparison/verdict calculation, incorporation of test execution traces from heterogeneous test tools in order to process the captured information solely on model level (raising the level of abstraction for automated results checking), comprehensibility or high-level failure analysis and test execution comparison (trends etc.).

    As high-level test modelling language, UTP ought to provide at least a facility to enable sophisticated activities on fine-grained test execution logs by capturing the execution on procedural element level.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 20:30 GMT
  • Disposition: Deferred — UTP2 2.0
  • Disposition Summary:

    Will be addressed by a later RTF due to timing reasons

    During the F2F in Santander, the FTF agreed that this capability is an important aspect for UTP 2. It will be resolved in a subsequent RTF.

  • Updated: Wed, 3 Oct 2018 14:16 GMT

Abstract TestDesignDirective concepts into abstract TestDirective

  • Key: UMLTP2-23
  • Status: closed  
  • Source: Fraunhofer FOKUS ( Mr. Marc-Florian Wendland)
  • Summary:

    The current TestDesignFacility og UTP 2 is based on the stereotypes TestDesignDirective, TestDesignTechnique, TestDesignDirectiveStructure and TestDesignTechniqueStructure. These elements are tightly integrated by means of the Profile's abstract syntax. So far, this is fine for the test design purposes. As such, the offered concepts support the definition of platform-independent instructions how to derive test cases, test data, test sets, test procedures and/or test execution schedules. These instruction can be put into effect by either a manual test designer or translated into the inpurt for a test generator.

    UTP 2, however, is capable of describing test automation architectures (ISTQB TAA) or the architecture of automated testware - both terms are considered to be synonyms. In a test automation architecture, further activities need to be specified such as the generation of executable test code, the configuration and automated setup of the test execution environment and tool (setup and teardown of the automated test environment; compilation of test scripts, automated scheduling of test cases in a test set according to a given test execution schedule etc.), the configuration of the logging facility (e.g., depths and structure of logged information), the configuration of the arbitration specification (which arbitration specifications shall be used, linked to test cases, test sets and procedural elements, etc.).

    All these configurations are essentials parts of an automatedt testware and can be used in combination or separately from each other (e.g., specifying a specific arbitration specification for verdict calculation on already executed set of test cases in a post-execution manner).

    UTP is currently too restrictive and only offers concepts for configuring the test design activities. As a test modelling language designed in particular for test automation, UTP should offer also concepts for the other activities that need to be performed in an test automation solution.

    Therfore, UTP should abstract from TestDesignDirective and TestDesignTechnqiue into abstract classes TestArchitectureDirective and TestArchitectureCapability (and the respective structures). That would, one hand, enables user-specific extensions in order to cope with the aforementioned, not yet supported concepts of test script generation, test execution, test logging and verdict arbitration, on the other hand prepare UTP 2 for potential improvements in that area in further revisions of the language.

  • Reported: UTP 2.0b1 — Tue, 23 Jan 2018 10:00 GMT
  • Disposition: Deferred — UTP2 2.0
  • Disposition Summary:

    Wil be addressed by a later RTF due to time reasons

    The FTF agreed that this is an important capability of which UTP 2 would benefit from. But it is not a implementation bocking issue. It would also not break backward compatibilitry, therefore, it will be deferred to a later RTF.

  • Updated: Wed, 3 Oct 2018 14:16 GMT
  • Attachments: