The RTC steps table of Test Junction 005 should contain a Continue event at the last place in steps 2-4.

In RTC step 4 the event pool should be reversed and it should be a trivial RTC step to discard CE(S1.2) as the note suggests: "The completion event occurrence generated for S1.2 is lost, since the state has no completion transition available".

The graphical notation is unusual in the following cases:

• The filled triangles in Figs. 9.65 and 9.66 (lower symbol) are supposed to be choice pseudostates.
• The upper symbol, a circle segment in Fig. 9.66 should be a junction pseudostate.

UML defines the graphical symbols as follows: a junction is represented by a small filled circle, a choice Pseudostate is shown as a diamond-shaped symbol.

This problem is also present in Junction tests.

The 3rd RTC step should not contain T1.1 as fired transition since it is not present in the generated trace and the note says "the initial pseudo state and its outgoing transition are not traversed".

The purpose of Test Event 001 is to show that, upon creation, the tested state machine immediately starts its execution.

The expected execution trace only contains a single part, the exit behavior of the initial state wait. The outgoing transition T2 (triggered by Start event) is not included in the trace, nor is the received Start event. Therefore I am not sure that the test could actually verify that the execution was started immediately and the exit behavior was executed when the state exited, and not earlier since only the exit behavior prints to the trace.

Another concern is that the test cannot verify that the initial compound transition is executed during initialization (requirement Event 001, p. 243), since initial transition T1 does not print to the trace.

Is the order of transitions significant in the Fired transition(s) column of RTC steps?

E.g., in Test Fork 001 the 3rd RTC step fires [T2(T3.1, T3, T4)]. The transitions here are not sorted alphabetically, which suggests the order matters. However, the trace shows a different execution order of transition effects: T3(effect), ..., T3.1(effect), ..., T4(effect), ...
The orders of fired transitions and the trace also differ in Test Entering 011.

On the contrary, in the following tests the fired transition order matters and it determines the order of:

• transition effects in Fork 002,
• entry behaviors in Entry 002-B, Terminate 001 and Terminate 002.

It would be beneficial to clarify whether the order in the column is relevant, and if not, sort the transitions alphabetically.

Do the following steps form a single atomic step: firing a transition, the target state's entry behavior then completion?

E.g., in Test Transition 019 the 6th RTC step simultaneously fires T1.2 and T2.2 transitions from [S1[S1.1, S2.1]] configuration.
The traces show that which one of the parallel transitions (T1.2, T2.2) is fired first determines which target state (S1.2, S2.2) completes and puts completion event into the event pool first, because all traces are either:

• ..., T1.2(effect), ..., T2.2(effect), T1.3(effect), T2.3(effect) or
• ..., T2.2(effect), ..., T1.2(effect), T2.3(effect), T1.3(effect).

This would imply that all of the following is an atomic step and they cannot overlap with their counterparts in the other region: transition effect, entering the target state, state completion, putting CE(target) into the event pool. This rule is never mentioned in PSSM and it contradicts Test Entering 011 where ..., S1.2(entry), ..., S1.1(entry), ... trace order and CE(S1.1), CE(S1.2) completion event order are present.

I think the traces should be all the possible 12 execution order of this activity diagram.

This means that currently these 6 traces are missing, where the order of the last two effects is switched:

• S1.1(exit)::T1.2(effect)::S2.1(exit)::T2.2(effect)::T2.3(effect)::T1.3(effect)
• S1.1(exit)::S2.1(exit)::T1.2(effect)::T2.2(effect)::T2.3(effect)::T1.3(effect)
• S1.1(exit)::S2.1(exit)::T2.2(effect)::T1.2(effect)::T1.3(effect)::T2.3(effect)
• S2.1(exit)::S1.1(exit)::T2.2(effect)::T1.2(effect)::T1.3(effect)::T2.3(effect)
• S2.1(exit)::S1.1(exit)::T1.2(effect)::T2.2(effect)::T2.3(effect)::T1.3(effect)
• S2.1(exit)::T2.2(effect)::S1.1(exit)::T1.2(effect)::T1.3(effect)::T2.3(effect)

The atomicity problem is also present in:

• Entry 002-B, where fired transition order determines the order of entry behaviors and the order of CEs in the event pool,
• Terminate 001 and Terminate 002, where RTC steps transition order determines the order of entry behaviors and completion events.

In Test Transition 011-D the entry behavior of states S1.1 and S2.1 in the two regions should be concurrent, therefore the following two alternative traces are missing:

• S2.1(entry)::S1.1(entry)::T3(effect)::S1.1(exit)::S2.1(exit)::S1(exit)
• S2.1(entry)::S1.1(entry)::T3(effect)::S2.1(exit)::S1.1(exit)::S1(exit)

State S1.2 in Fig. 9.16. has an entry behavior that prints S1.2(exit) to the trace. It should print S1.2(entry).
This problem is also present in the test suite XMI.

S1(exit) is missing from the end of the trace in Transition 011-A. (It is executed after T1.4 and before T3 transitions, which do not print to the trace.)
The whole trace should be the following: S1.1(entry)::S1.1(exit)::T1.3(effect)::S1.2(entry)::S1(exit)

Start event is missing from Received event occurrence(s) of Test Transition 007 and Event 017-B. It should be listed there since the SemanticTest class always sends Start signal to the target (and the tester) (PSSM p. 61).

Currently the specification says: "Completion occurs when, after a run-to-completion step, there are no more event accepter registered for the doActivity with its DoActivityContextObjectActivation".

However, cannot a long-running action still be executing even when there is no accepter registered?

For example:

init -> fork -> accept -> join -> final
-> action with computation

The accept can accept an event and move to the join (run-to-completion step performed for the doActivity), but the action with computation can still be running on its own thread (according to the Threading Model section in the fUML specification). What guarantees that there are no more concurrent parts still executing when there is no accepter registered (there is no "default" accepter for a doActivity as far as I understand).

Two alternative execution of test Behavior 003-B is missing. Since "a doActivity Behavior ... is executed on its own thread of execution" (PSSM p. 39), slow doActivity execution is possible.

If the doActivity is running slowly, dispatching AnotherSignal and the internal transition might come first. The resulting trace is: S1(entry), T3(effect), S1(doActivityPartI), S1(doActivityPartII).

If the doActivity is running even more slowly, dispatching and discarding Continue event can precede firing accept(Continue) AcceptEventAction in the doActivity to register EventAccepter. This results in a deadlock. In this case a possible trace is the following: S1(entry), T3(effect), Continue event dispatched and discarded, S1(doActivityPartI), accept(Continue) in doActivity registers EventAccepter, deadlock. See similar issue of Test Behavior 003-B.

Another related issue: testEnd effect of transition T4 is missing from Figure 9.10.

An alternative execution of test Behavior 003-B is missing. Since "a doActivity Behavior ... is executed on its own thread of execution" (PSSM p. 39), slow doActivity accepter registration is possible, i.e., Continue event is received earlier than the accept(Continue) AcceptEventAction in the doActivity is fired to register EventAccepter. In this case the Continue event is discarded in the lack of a matching accepter, then the doActivity and thereby the whole state machine wait for a new Continue event indefinitely, which results in a deadlock.
The following trace describes the sequence of events: S1(entry), Continue event dispatched and discarded, S1(doActivityPartI), accept(Continue) in doActivity registers EventAccepter, deadlock.

For SM_ObjectActivation the figure (Figure 8.4) displays that it has a register( in completionEventOccurrence: CompletionEventOccurrence [1]) operation.
However, the text below refers to a registerCompletionEvent operation ("The registerCompletionEvent operation is used to add").
The semantics xmi also contains register as an operation name. However, registerCompletionEvent might be a better name, as it is similar to the registerDeferredEvent operation name.

Figure 8.9 (doActivity Behavior Execution) shows that a DoActivityContextObject has exactly 1 DoActivityExecutionEventAccepter.

This multiplicity should be unlimited since

• a doActivity can register multiple EventAccepters, ("While a doActivity Behavior is executing, it may need to register EventAccepters for specific EventOccurrences.")
• which are encapsulated into DoActivityExecutionEventAccepters, and ("When an executing doActivity Behavior registers an EventAcceptor ..., it is wrapped in a DoActivityEventAccepter...")
• a DoActivityExecutionEventAccepter encapsulates only a single EventAccepter (1 encapsulatedAccepter in the figure).

The text uses two names for the same class, DoActivityExecutionEventAccepter and DoActivityEventAccepter.

Test Behavior 003-A describes a scenario, where the doActivity is aborted. There are no alternative traces specified, only the one, where the doActivity is aborted after printing S1(doActivityPartI) and waiting for the Continue signal.

Why is there no alternative trace listed, where the doActivity is aborted before executing the printing of S1(doActivityPartI)? From the current test it seems that the doActivity can be aborted only when it is waiting for a signal. However, Test Event 017-B and Terminate 002​ explicitly lists traces where the doActivity is aborted before printing anything or completing its initial first RTC step.

The Received event occurrence part lists "AnotherSignal – received when in configuration S1". From this, it seems to me that AnotherSignal can be processed any time and S1 can be exited before executing any part of the doActivity.

Moreover:
1) It would be helpful to clarify abortion e.g. in 8.5.5 StateActivation exit. In which phases of its execution can the doActivity aborted? At the boundaries of RTC steps? Or even inside an RTC step?
2) A possible test case to showcase these questions could be the following. One simple state with a doActivity, entering the state is triggered by Start, exiting is triggered by Continue. The doActivity has two WriteStructuralFeatureAction, that writes 1 and 2 to an integer value owned by the statemachine, which is initialy 0 (there is neither timing or waiting in the doActivity). Received events are Start, Continue. List of traces could show, whether the doActivity can be aborted in such ways, that in the end the integer is 0, 1 or 2.

In S1: "/do Activity exit" shoud be "/exit Activity exit"

The alternative execution traces part lists the following traces:
"2. T2(effect)::S1(entry)::T2.2(effect)::T3.2(effect)::S3.1(doActivity)::T3.1.2(effect)
3. T2(effect)::S1(entry)::T2.2(effect)::T3.2(effect)::T3.1.2(effect)::S3.1(doActivity)"

I think these traces are not valid. T3.2 is triggered by the completion event of S3.1. However, as per "StateActivation completion" on page 37, "The StateActivation can only generate a CompletionEventOccurrence when all RegionActivations for Regions of the composite state have completed and the doActivity Behavior has completed.". Therefore T3.2(effect) cannot be printed before S3.1(doActivity).

The precedence relations are the followings. T2 -> S1; S1 -> T2.2; S1 -> S1(do); S1 -> T3.1.2, S1(do) -> T3.2, T3.1.2 -> T3.2

The full list of valid traces are the followings (some of them are missing from the current list)

T2 S1 T2.2 T3.1.2 S3.1(do) T3.2
T2 S1 T2.2 S3.1(do) T3.1.2 T3.2
T2 S1 T3.1.2 T2.2 S3.1(do) T3.2
T2 S1 T3.1.2 S3.1(do) T2.2 T3.2
T2 S1 T3.1.2 S3.1(do) T3.2 T2.2
T2 S1 S3.1(do) T2.2 T3.1.2 T3.2
T2 S1 S3.1(do) T3.1.2 T2.2 T3.2
T2 S1 S3.1(do) T3.1.2 T3.2 T2.2

"T3.1.2 is triggered by the dispatching of CE(S3.1.2)" -> CE(S3.1.1)

There is no S3.1.2 in the state machine. The RTS steps part correctly refers to [CE(S3.1.1)] in the event pool.

The text of some of the test cases reference wrong figures. For example:

• 9.3.3.9 Transition 015 (page 86): "The doActivity for state S1 state is exactly the same as the one presented in Figure 9.8.". The doActivity in Figure 9.8 contains an accepts and prints doActivityPartI and doActivityPartII. According to the PSSM test suite xml, the doActivity for Transition 015 just prints "S1(doActivity)" and does not contain any accept.

• 9.3.6.3 Exiting 002 (page 132):
• “The state machine that is executed for this test is presented in Figure 9.8.” it is Figure 9.48.
• “The doActivity behavior of S1 has exactly the same behavior as the one presented in Figure 9.37” it is Figure 9.8.

The correct notation is for example:

entry/Activity entry

All the diagrams in PSSM show instead

/entry Activity entry

This is confusing and should get changed.
Also the text in all examples of the UML specification is left justified. It is not mentioned as a requirement, but I think most tools follow this convention. In the PSSM specification the text is centered. I suggest to change it to left justified.
The effect of Transitions is notated with a colon:

/Activity: effect.

I think that should also be consistent. Either remove the colon, or use it with state behaviors as well.
As an additional suggestion: In most cases it is not relevant for the test case, that an activity is called. The string "Activity" could be left out to keep the diagram less cluttered.

The behavioral part of the PSSM semantic model is specified using Java syntax. The subset of Java that is used does not always conform to the mapping rules defined in Annex A of fUML between Java and Activities.

Examples:

1. Usage of index starting from 0 instead of 1 in StateActivation::hasCompleted operation.
2. Constructor call with arguments in StateMachineEventAccepter::accept operation.
3. Usage of an iterative for loop instead a parallel for loop in StateActivation::enterRegion operation.

Any test in the PSSM test suite with a test driver that sends multiple signals to the model being tested may not currently be properly allowing all possible execution traces. This is because itt cannot, in general, be presumed that event occurrences are received in the order they are sent, even if they are all sent from the same thread. This was always true in fUML (per the statement of “the semantics of inter-object communications mechanisms” in subclause 2.4 of the spec), but it is completely, formally clear in fUML 1.3, in which EventOccurrence is an active class, such that all event occurrences are sent concurrently with each other.

For example, consider test Transition 007 (described in subclause 9.3.3.2 of the PSSM beta spec). The tester behavior for this test sequentially sends three signal instances: AnotherSignal, Continue and Continue again. However, while these signals are sent sequentially, there is no guarantee they will be received by the tested state machine in the same order. For example, one of the Continue signal instances could be received before the AnotherSignal instance, which would cause the state machine (as shown in Fig. 9.12) to take transition T3 to S2 and never get to S3.

Rather than try to capture all the possible traces that should be allowed by the such tests a s currently modeled, it would be better to modify the tests so that they should only produce the trace that is currently suspected. This can be done by having the state machine under test send signals back to the tester, in order to coordinate the sending of sequential signals. For example, in the case of Transition 007, the state machine could send signals back to the tester as part of the doTraversial behaviors for transitions T1 and T2. The test behavior would then have to include accept event actions in order to wait between the send signal actions. (Of course, to allow the test to send signals back to the tester, either the Tester/Target association in the test architecture would need to be made bidirectional, or some signaling mechanism would need to be provided through the SemanticTest class.)

Section 9.2.2.2.3 states: "The classifier behavior of a semantic test has the TestCase stereotype applied."

The source of the TestCase stereotype is not mentioned. Presumably, it is UTP, but the relationship of PSSM and UTP is not further described except a list of definitions from the UTP specification including Test Case on page 61.

The test case stereotype requires a return parameter of type VerdictKind. That is not implemented by PSSM.

Regarding the general OMG requirement for OMG specifications to reuse other specifications if possible, I propose to integrate the test case stereotype including the verdict concept from UTP.

The text for the semantics of exiting the source of a local transition appears incomplete; the first paragraph provides a condition under which the source cannot be exited, but if that condition is not met, does not describe how the source should be exited.