Provided generators¶

Reference test package generator¶

Use cases and general description¶

The Reference test package generator can compare the signal traces of two trace files (reference and test trace) qualitatively with each other. Simple signal comparisons can be performed using a Trace Analysis Calculation Step. For more complex analyses, it is recommended to use either the trace analysis from an analysis package or package or to define the trace analysis individually (code-based). Signals with different identifiers in each trace can also be associaded by signal name matching using search patterns. If signals in the test trace have been added or removed in comparison to the reference trace, the resultig verdict can be configured. Additionally, irrelevant signals can be explicitly excluded by means of filters.

The parameterization of the Reference test package generator can be either done GUI-supported or code-based. For the code-based parameterization the following interfaces are available as builtins: api, user, tools The Reference test package generator tools are specific to this generator.

For easier handling of the trace signals they are provided as Object API conform Signal-objects.

Operation and settings¶

The following tabs are available in the dialog for further settings:

Common¶

It is possible here to influence whether a single package with all signal comparisons or a separate package is to be generated for each signal comparison. It is also possible that a single package is generated, which is then called via parameter sets with the different signal pairs.

The following methods of generation can be selected to analyze the n signal pairs:

Method	Generated artifacts	Evaluation	Execution Speed
Single package	One package with n analysis steps	Evaluation is less differentiated because the entire package fails if one signal comparison fails	Fastest
Multiple packages	One package per signal pair	Most differentiated evaluation possibilities	Slowest
Parameter sets	One package with n parameter sets	Compromise between speed and analysis detail	Moderate

Signal filtering¶

Filters can be defined here to include or exclude signals. The specified filters are applied to the signals of both the Reference and the Test Trace.

Package preparation¶

Definition of an individual package name during generation.

Signal matching¶

Information on the assignment of the signals from the reference and the test trace.

Trace analysis¶

This is where the actual trace analysis is defined, which is applied to the signal pairs. A trace analysis is generated where a calculation trace step is used to compare the reference signal and the test signal.

Preview¶

Preview of the executed code. This view is for debugging purposes only.

Examples¶

Example of signal filtering

The following example shows a special case in which a single selected reference signal is applied to all filtered test signals.

def FilterSignals(refSignals, testSignals):
    """
    Use this method to filter the signals you want to be analyzed.

    :param refSignals:   list of all signals within reference trace file
    :type refSignals:    list<string>
    :param testSignals:  list of all signals within test trace file
    :type testSignals:   list<string>
    :return:             tuple of signals to be analyzed
    :rtype:              tuple(refSignals, testSignals)
    """

    # filter test signals
    testSignals = [testSignal for testSignal in testSignals if testSignal.name.startswith("Sig")]

    # filter reference signal list. The result should contain only one element.
    refSignals = [refSignal for refSignal in refSignals if refSignal.name.startswith("Ref")]

    # The first element will be the reference signal for all test signals
    singleRefSignal = refSignals[0]
    refSignals = [singleRefSignal]*len(testSignals)

    return (refSignals, testSignals)

Example package preparation

The following example shows how an individual name is created for each generated single package according to the signal pairs assigned to each other. In addition, offset synchronization of test and reference traces is configured.

def PreparePackage(packageObject, traceAnalysisObject, signals):
    """
    Within this method the 'packageObject' and 'traceAnalysisObject' can be customized according to
    the Object API to fit individual needs. For instance, attributes or the name of the test package
    can be adjusted. Also the trace analysis can be preconfigured by adding episodes that can be
    used within 'CreateAnalysis()' by 'tools.GetOrCreateEpisode(traceAnalysisObject, episodeName)'.
    e.g. packageObject.Attributes.SetValue('MyTestcaseAttribute','theAttributeValue')

    This method is executed *once* per package.

    :param packageObject:        object of the generated reference test package
    :type packageObject:         :class:`.PackageFacade`
    :param traceAnalysisObject:  the trace analysis object
    :type traceAnalysisObject:   :class:`.TraceAnalysisFacade`
    :param signals:              list of tuples of 'refSignal' and 'testSignal'. One of those can be
                                 'None' if no matching signal was found.
    :type signals:               list<tuple(refSignal<Signal>, testSignal<Signal>)>
    :return:                     name of the package
    :rtype:                      string
    """
    api.countPreparePackageCalls += 1
    packageObject.Attributes.SetValue('MyTestcaseAttribute','theAttributeValue')

    refSignal = signals[0][0]
    testSignal = signals[0][1]

    ### custom synchronization
    syncConfig, refRecordingGroup, testRecordingGroup = tools.GetSynchronizationConfiguration(packageObject, signals)
    # configure synchronization according to Object API
    syncConfig.SetMaster(refRecordingGroup)
    offsetSync = syncConfig.AssignOffsetSynchronization(testRecordingGroup, refRecordingGroup)
    offsetSync.SetOffset('-0.4')
    syncConfig.SetActive()

    ### custom package name
    return f'MyPackage_{refSignal} vs. {testSignal}'

Example of trace analysis

In addition to an envelope curve analysis, this code also generates a plot for each analyzed signal pair.

def CreateAnalysis(traceAnalysisObject, refName, testName):
    """
    Customizable method to add individual TraceAnalysisSteps to the 'traceAnalysisObject'
    for a corresponding pair of signals. Use the Trace Analysis API, which is already available
    as 'traceAnalysisApi', to create custom TraceSteps for your analysis.
    This method is executed *once* for every matched signal pair (refName, testName).

    :param traceAnalysisObject:  the trace analysis object
    :type traceAnalysisObject:   :class:`.TraceAnalysisFacade`
    :param refName:  signal name of the reference trace file
    :type refName:   string
    :param testName: signal name of the test trace file
    :type testName:  string
    """

    episode = tools.GetOrCreateEpisode(traceAnalysisObject)

    calculation = traceAnalysisApi.CreateCalculation(f'Comparison {refName} vs. {testName}')
    calculation.SetExpression(f'HoseAbsolute({refName}, {testName}, 1, 5)')

    episode.AppendTraceStep(calculation)
    tools.EnablePlot(calculation, plotName=testName)

    return traceAnalysisObject

Parameter set generator: Multiple package runs¶

Description¶

This parameter set generator allows to run a package multiple times, which is useful for stress testing and for assessing the stability of a test through multiple runs.

Configuration¶

The parameter set generator can be configured via the corresponding dialog. It contains two fields:

Name of runs: Display name of the package runs during execution and in the test report. It will be appended with the number of the run, e.g. ‘Run #1’, ‘Run #2’ etc.
Number of runs: Number of package runs