UMLTP2 — Explicit arbitration specification binding does not scale

UTP offers a concept to bind arbitration specifications (henceforth, the acronym AS is used) to test sets, test cases and procedural elements. The binding comes in three flavors (see page 136):

• default arbitration specification realized by an implicit binding
• static arbitration specification realized by a static binding directly by the respective test set, test case or procedural element
• dynamic arbitration specification (dynamic binding) overrides any previously existing binding in the context of a procedure invocation

Albeit these three binding kinds makes definitely sense (and should not be removed from the spec) the problem is the technical representation of how bindings shall be expressed, i.e., directly by the test set, test case, procedural element or procedure invocation for dynamic binding. The main issue is that the technical binding mechanism does not scale for large test sets that exhibits exhaustive modularization and reuse of test procedures. In theory, there can be an unlimited number of arbitration specifications set for a test set, test case or procedural element. The classification into static and dynamic binding is a mere synthetic (and unnecessarily complex) concept. In fact, there can only be one static binding being present at any point in time and any further binding must be made via dynamic binding, which in turn requires the use of a <<ProcedureInvocation>> and, thus, a TestExecutionSchedule to actually invoke a test set/test case to dynamically bind the respective AS. This imposes a certain methodology on the user, which is too prescriptive.

The entire mechanism becomes more complicated when procedural element AS shall be dynamically bound. This can only be achieved by simply listing procedural element AS in the set of 'ProcedureInvocation.dynamicArbitrationSpecification' and the use of the boolean field 'transitiveOverride'. It is not possible to explicitly bind a certaint procedural element AS to just a single procedural element (or a subset thereof). Furthermore, it is not easily possible to specify whether a transitiveOverride shall be applied to all deeply nested ProcedureInvocations or not.

Eventually, it must be stated that the approach to bind different AS to the very same test set/test case/procedural element is a good idea (that should be preserved), but as it is realized now, not usable and excessively complex.

A possible solution to that inflexible and overly complex binding mechanism could be to extract the binding from test set/test case/procedural element into a dedicated concept that allow for expressing these bindings external to the test set/test case/procedural element. This would have two advantages if done properly:

1. The test set/test case/procedural element would remain clean of any AS information and would only represent the information required to eventually execute a test case. Test sets/test cases/procedural elements would be easier to maintain after all.

2. An external binding mechanism would be able to explicitly specify which AS shall be set for which element in a certain context. This holds true even for deeply nested test procedures and procedural elements, which is not the case with the current binding mechanism. A kind of ArbitrationDirective would enable a flexible and unrestricted mechanism to define how the verdict for a given test set/test case/procedural element shall be calculated without imposing certain design decision (like the use of TestExecutionSchedules and ProcedureInvocations) on the test designer. This is in turn another fact that positively impacts maintainability of existing test specifications. Practically, an unlimited number of ArbitrationDirectives could be defined for any test set/test case, which would make the AS mechanism most flexible.

If extracted, there would only be two kinds of binding remain:

• implicit binding: if no additional AS information is specified for a certain test set/test case/procedural element, the default AS would take over (like it is right now)
• explicit binding: (Parts of) the AS bindings for test sets/test cases and procedural elements are explicitly bound and can be put into effect by using a certain ArbitrationDirectve. Additionally, other ArbitrationDirectives can be defined to declare further AS bindings, without the need to change (or even touch), the existing test set/test case/procedural elements.

Arbitration specification bindings will be cleaned up from technical debts

The FTF agree that the mechanism to bind multiple Arbitration Specifications to test cases, test sets and procedural elements is incomplete and blocking implementations. Parts of the specification needs to be cleaned up from techhnical debts.

Furthermore, the FTF agreed that UTP 2 would benefit from a mechanism that offers consistent and scalable way to bind different arbitration specifications for different analysis purposes. Idealy, the elements that shall be arbitrated and the corresponding arbitration specifications shall be loosely coupled by a dedicated and reliable UTP 2 capability for arbitration specification binding.

Adding such a new capability is outside the scope of the FTF. Therefore, the FTF will submit another issue for the upcoming RTF to incorporate that specific functionality.

Since dynamic binding of arbitration specifications is not anymore part of the specification, for it was never finished properly, editorial parts of the specification that deals with the different binding mechanisms must be revised to remove any description about the dynamic arbitration specification binding.