UMLTP21 — UTP test log facility ought to enable logging on procedural element level

UTP 2 offers currently rudimentary test log capabilities. It is possible to capture the minimal test execution information for test cases and test sets. These information are sufficient for the sake of coverage calculation or high-level test reporting, however, the details of a test case execution (e.g., the execution of CreateStimulusActions or ExpectedResponseAction) are completely omitted. This wastes a lot of potential UTP could offer for building test automation architectures like post-execution comparison/verdict calculation, incorporation of test execution traces from heterogeneous test tools in order to process the captured information solely on model level (raising the level of abstraction for automated results checking), comprehensibility or high-level failure analysis and test execution comparison (trends etc.).

As high-level test modelling language, UTP ought to provide at least a facility to enable sophisticated activities on fine-grained test execution logs by capturing the execution on procedural element level.

Enable logging on procedural element level

The task force agreed that the ability of abstracting from technical test logs into platform-independent test logs opens the door for UTP 2 to integrate the execution details of different test results potentially created by different test exeuction tools, as well as additional information such as information obtained from dynamic analysis tools by means of a single platform-independent language. UTP 2 may use these information for many purposes such as visualizing the executed test cases in a unique way; post-processing the execution results to calculate an overall verdict; perform further analysis like timing violations etc.
Having a single, integrated view on the results of test executions may become very conventient particulary in situations with heterogeneous testing landscapes where the eventual verdict of a test execution log has to be computed by a comprehensive analysis of diverse test execution logs.
UTP 2 can be seen as a kind of test information integration language that enables the specification and implementation of analyses over a set of originally seperated test execution logs.