Parallel test threads¶
Parallel test threads:
are test step sequences run in parallel (to main sequence)
are also referred to as concurrent test cases
can communicate among each other by exchanging messages while running
Start and End¶
Start
The start of a parallel test thread is accomplished by means of the parallel (also called asynchronous) call of a package. That is done by the test step “parallel package reference”.
“Parallel package reference” is inserted into the test case, similarly to the classical package reference, by means of a context menu command of the workspace explorer
The classical package call is also referred to as synchronous call.
As with a synchronous call, parameters can be given to the calling package. In the caller it is possible to define the target variables into which the return values of the called package are to be inscribed.
Runtime and termination
Asynchronously called packages end:
after execution of their last test step which is determined by the control flow
after execution of the test step “return”
in case of a test abort
Asynchronously called packages cannot be aborted forcibly (by the caller) except in case of a test abort; they always end by themselves. However, they can be sent a demand for termination (see Exchange of messages between parallel test threads).
Note: If an asynchronously called package has in turn started parallel subpackages, it will wait after execution of its last test steps, except in case c) for the termination of all subpackages before coming back. That means subpackages executed in parallel never live longer than the superordinate parent package. That behavior is referred to as implicit join of a package with its subpackages.
Synchronization and access to return values of an asynchronously called package
The return values of a package are not available until the end of its execution, and only then are they inscribed into the target variables of the caller. So the caller of a parallel package has to wait for the end of that package before it can access its return values. That is done automatically as soon as a test step of the caller reads one of the target variables or as part of the variable dump used for the trace analysis when executing a stop trace step. Then the reading access blocks the caller until the end of the subpackage (also referred to as explicit join of a package with its subpackage).
The writing access to a target variable fails as long as the asynchronous execution of the subpackages is not terminated. The same is true for the new assignment of a target variable in a second asynchronous call, as long as the first call is not terminated.
Reporting¶
The presentation of asynchronous calls of packages in a test report is made in the same way as that of synchronous ones. However, the color marking of the calls is different. Furthermore, subpackages executed in parallel cannot be presented in line (i.e. interlaced with the test steps of the parent package) because of the time overlap with their parent package.
Exchange of messages between parallel packages¶
Parallel packages can exchange messages among each other. Messages can be any kind of Python objects (numbers, strings, etc.). For transmitting messages, a package needs so-called outboxes, and inboxes are required for receiving messages. These are special variables which can be generated in the variable container of a package. Inboxes and outboxes are combined into unidirectional communication channels in the dialog of the parallel package reference test step, similarly to parameter and return value assignment. Messages are sent and received exclusively by means of the test steps “Read from inbox” and “Write to outbox”.
Outbox and test step “Write to outbox”¶
An outbox is the endpoint of the unidirectional communication channel between two parallel packages on the sender side. Outboxes can be added to a package, like ordinary variables, through the variable manager. The option Discard messages if not connected is relevant when the attempt is made to send messages by way of an outbox which is NOT connected with an inbox. If the option is deactivated (default), that will cause an error. If the option is activated, messages to be sent will be lost without any error message.
Particular attention has to be paid to marking an outbox as an OUT parameter by means of a corresponding tick in the variable manager. That marking determines the possibilities of interconnection (see chapter Relays).
Messages are sent by means of the test step Write to outbox that is to be found on an test steps tab in the category Data operations. The outbox concerned has to be inserted or chosen in the field Outbox. Any kind of Python expression can be inserted into the field Message, and its evaluation result will be sent as a message during execution.
Note: Messages can be sent only by means of the test step Write to outbox. An ordinary assignment to an outbox variable is not possible.
Inbox and “Read from inbox”¶
An inbox is the endpoint of the unidirectional communication channel between two parallel packages on the receiver side. It works according to the FIFO principle: messages arriving first in the inbox will be delivered first. Inboxes can be added to a package, like ordinary variables, through the variable manager. Inboxes possess the following specific parameters:
Size: Number of messages an inbox can buffer without overflowing.
Standard time-out: Standard time, in milliseconds, of waiting for incoming messages when an empty inbox is called. After expiration of that time without messages coming in the call will return ~not present~. The standard time-out can be overwritten in the test step Read from inbox.
The two options of the inbox determine its behavior in case of overflow. An inbox overflows at the exact moment when the number of messages exceeds the size of the box. If the option Delete the oldest message(s) in case of an overflow is activated then exactly so many of the oldest messages are deleted that the inbox remains just below overflowing when new messages arrive. If, in addition to that, the option Report panel size overflows is active a warning is sent to the console before messages are discarded. However, if only the option Report panel size overflows is active there will also be a warning, but in that case the messages will not be lost because the size of the inbox grows beyond the set limit so that newly arriving messages can still be buffered. If none of the two options is activated, the delivery of messages will fail with a hard error.
Tip: With Value 1 and option Delete the oldest message(s) in case of an overflow
you can produce a kind of “live return value” which is available immediately in the calling package
and is continually updated by the parallel package.
Particular attention has to be paid to marking an inbox as an IN parameter by means of a corresponding tick in the variable manager. That marking determines the possibilities of interconnection (see chapter relays).
Messages are collected from an inbox by the test step read from inbox which can be found on the test steps tab in the category Data operations. That test step has two modes - Collect and Check. In the Collect mode the oldest message is taken out of the inbox, in the Check mode the message remains in the box. The check mode is activated by means of the option Keep message in inbox (Caution: If old messages are not collected you will have no access to the newer ones, and the inbox may overflow).
The test step also allows, in addition to inputs and/or selection of an inbox, the definition of a time-out (in milliseconds) for waiting for incoming messages when the inbox is empty. After expiration of that timeout the call will return ~not present~. Time-out options are Default time-out (as parameterized in the inbox variables), Unlimited time-out as well as a special time-out from the respective test step in Read from inbox.
The test step further provides settings for evaluating and saving a checked message. Those are similar to the settings of a reading or calculation test step.
Note: Messages can be collected only by means of the test step “Read from inbox”. An ordinary access to the inbox variable (for example by using a Python expression) will cause errors.
Linking inboxes and outboxes¶
In order to ensure that messages to be sent via an outbox will be delivered to an inbox, the outbox must be linked to an inbox before transmission. That is done on the tab Inboxes and outboxes of the editing dialog belonging to the test step Parallel package reference. In the left column the inboxes and outboxes of the called package are shown. By means of a drop-down menu the inboxes and outboxes of the caller can be called up and assigned in the right column.
In order to ensure that the inboxes of a called package are shown on the left side, the option IN parameter has to be activated. A similar condition is true for the outboxes of the called package; if those are to be shown on the left side, the option OUT parameter has to be activated.
The rule for inboxes and outboxes to be assigned to the caller is: for those boxes to appear in the selection list, the options IN and OUT parameter must not be activated. If that rule is violated by subsequently activating or deactivating the IN and OUT marking, there will be runtime errors.
The background of this strict rule is the fact that the IN/OUT parameter marking declares the inboxes and outboxes concerned as interfaces of the corresponding packages whose interconnection is the task of the caller. The attempt to establish an interconnection by the called package would invariably lead to an error because at that moment the boxes are already being used.
Note: One outbox can always be linked to one inbox and vice versa (which is ensured by parameterization). The re-use of outboxes and inboxes in further asynchronous package calls is possible. However, the parallel package must be terminated before the inbox/outbox can be linked to another parallel package in a new call. Otherwise there will be a runtime error.
Relays - Exchange of messages between parallel subpackages¶
So far we have looked only at the direct interconnection of inboxes and outboxes. That, however, only serves to establish communication channels between caller and called package. More complex communication structures require relays. Relays are connecting pieces with two ends for interconnection, an IN end and an OUT end. The IN end can be linked to an outbox, the OUT end to an inbox. Alternatively, the ends can also be linked to further relays. That leads to possible link structures of the following kinds:
outbox–>IN-relay-OUT–>inbox,
outbox–>IN-relay1-OUT–>IN-relay2-OUT–>inbox
etc.
If, for example, an exchange of messages is intended between the two parallel subpackages S1 and S2 of a main package H, the main package gets a relay R(H), the subpackage S1 an inbox In(S1) and the subpackage S2 an outbox Out(S2). When S1 is called its inbox In(S1) is linked to the relay (R(H), and when then S2 is called the outbox Out (S2) is also linked to the relay R(H). That brings about the structure Out(S2) –> R(H) –> In(S1) so that the subpackage S2 can send messages to the subpackage S1.
Relays are, like inboxes and outboxes, special variables that can be added to the variable container. They serve exclusively for forwarding messages, i.e. they can be used neither for reading nor writing. Relays possess, like inboxes, a message buffer. That buffer serves to store messages temporarily when the relay is already connected with an outbox but not yet with an inbox and the sender is already transmitting messages. The size of the message buffer can be configured in the setting dialog of a variable as well as its behavior in case of overflow. If the option Error at overflow is deactivated, only a warning will be shown on the console without triggering a (hard) error. In this case no message will be lost because the buffer is dynamically enlarged.
Being marked as IN/OUT parameters is as relevant for relays as it is for inboxes and outboxes. Relays in a package to be called asynchronously appear, depending on their marking, as IN or OUT parameters on the left side of the inbox or outbox section. Unmarked relays cannot be interconnected by the caller and therefore do not appear in the display.
Complementary restrictions concern the registration of the relays of the calling package in the selection lists of the right column. If a relay is marked as an OUT parameter its OUT end may not be linked by the package itself. Thus the relay does not figure in the selection lists of the inbox section. The same is true for marking as IN parameter which leads to removing the relay from the selection lists of the outbox section. If a relay is marked as both IN and OUT parameter it does not figure in any of the selection lists.
Event-oriented message treatment (REACT test step)¶
A frequent demand in test cases with several parallel test threads is to wait simultaneously and repeatedly for the occurrence of different events (e.g. the arrival of messages from different parallel packages). Moreover, depending on the kind of occurring event, specific test steps are to be carried out. This is what the REACT test step serves for. It liberates the test case generator of the necessity to sequentially browse, in a loop, all desired sources of events and allows a very compact presentation of the test steps to be carried out with every event. The REACT test step can be found in the category Control flow of the test steps tab.
The Edit dialog shows the list of events and/or conditions the REACT test step waits for. Initially that list is empty. New events can be defined by means of the button add. The user has the possibility to wait for
the arrival of messages in an inbox (inbox condition),
the arrival of a return value from an asynchronously called package (future condition) or
the expiration of a time-out (after condition).
In accordance with the condition selected,
the name of the inbox concerned,
the name of the target for the package return value
or a Python expression for the time-out
has to be specified in the second column.
The order in which the sources of events are browsed can be determined by means of the up/down buttons. Occurred events are treated sequentially, i.e. during the treatment of one event no other events can be treated. They are, however, saved for later treatment.
For each of the events defined in the REACT dialog, a correspondingly marked container test step is generated in a test case below the REACT test step. Any other test steps to be executed on the occurrence of the corresponding event can be inserted into that container test step.
Use BREAK test step for finishing event treatment and quitting the REACT test step. That option can be included in the event treatment test threads. Without BREAK (or RETURN) the REACT test step and thus the surrounding package would never end. The BREAK test step can also be found in the category Control flow on the test steps tab.