Object detection¶

Object detection services enable the extraction of objects such as vehicles or people, patterns, sub-images, etc. within images using the Multimedia API.

By default, ecu.test supports simple algorithms for recognizing sub-images within an image (template matching and feature-based matching). It is possible to integrate additional algorithms for object detection using UserPyModules. For example, AI models such as YOLO, which can detect different object classes such as people, vehicles, animals, etc.

User-defined object detection¶

Object detection in ecu.test uses an object detector to identify individual objects within the image and output them as a MatchList. To do this, a Python module must be provided. In order for this to be recognized as a valid UserObjectDetector, several requirements must be met:

The module must be located in the UserPyModules folder of the current workspace.
The variable MODULE_TYPE = ‘ObjectDetector’ must be defined at the top level of the script.
An ObjectDetector class with any name must be implemented that fulfills the following interface:
- __init__
- GetId
- GetDocumentation
- DetectObjects
- GetAvailableParameters
- GetParameter
- SetParamter
No two object detectors with the same ID may exist.

Further details on the interface can be found in the example in the following section.

Newly added or modified object detectors are recognized by ecu.test after the user libraries have been updated. This is done in the main menu “Extras”.

To use a user-defined object detector via the internal multimedia API, an instance can be created via

api.Multimedia.GetObjectDetector

The implemented methods of the interface can be called directly on the returned object, e.g.

api.Multimedia.GetObjectDetector('ExampleObjectDetector').GetParamter('objects').

Minimal example for object detection¶

An example implementation of a custom object detector can be found in the ecu.test installation directory:

Templates/DefaultData/UserPyModules/UserObjectDetector/ExampleObjectDetector.py

The minimal example in ExampleObjectDetector.py contains the following code. Further information on the individual methods can be found in the doc strings.

from __future__ import annotations

from typing import TYPE_CHECKING, Any

import numpy as np

if TYPE_CHECKING:
    import numpy.typing as npt

MODULE_TYPE = 'ObjectDetector'


class ExampleObjectDetector:

    def __init__(self):
        """
        The object detector can be created when *GetObjectDetector(detectorId)* is called on
        the internal multimedia API, where *detectorId* is the identifier returned by *GetId()*.
        """
        self._params: dict[str, Any] = {
            param['name']: param['default'] for param in self.GetAvailableParameters()
        }

    def DetectObjects(self, img: npt.NDArray) -> list[dict[str, int | float]]:
        """
        Calls the detection algorithm of you object detector. This example detector creates
        (pseudo-random) object matches in the image dimensions and returns it.

        :param img: Numpy image (width x height x channels)
        :return: A list of dictionaries that describe the (pseudo-random) objects in the image with
                 position, dimensions and confidence.
        """
        objects = []
        # retrieve image dimensions
        height, width = img.shape[:2]
        # create pseudo random number generator with fixed seed
        gen = np.random.default_rng(seed=42)
        # create objects at random positions. The number of objects is defined in the 'objects'
        # parameter
        for _ in range(self._params.get('objects', 10)):
            objWidth = gen.random() * 100
            objHeight = gen.random() * 100
            x = gen.random() * (width - objWidth)
            y = gen.random() * (height - objHeight)
            objects.append(
                {
                    'x': x,
                    'y': y,
                    'width': objWidth,
                    'height': objHeight,
                    'confidence': gen.random(),
                }
            )
        return objects

    @staticmethod
    def GetDocumentation() -> str:
        """
        Returns a short documentation for your ObjectDetector class.

        :return: A documentation string.
        """
        return (
            'This is an example object detector that detects a parameterized number of objects at '
            'random positions.'
        )

    @staticmethod
    def GetId() -> str:
        """
        The identifier of the object detector which is used to create the detector via the
        multimedia api.

        :return: Identifier of the object detector.
        """
        return 'ExampleObjectDetector'

    def GetAvailableParameters(self) -> list[dict[str, Any]]:
        """
        Returns a list of all configureable parameters. Each parameter must be described by a unique
        name and a default value. A short description is optional.

        :return: List of all available paramters
        """
        return [
            {
                'name': 'objects',
                'default': 10,
                'description': 'Number of objects to detect',
            }
        ]

    def GetParameter(self, name: str) -> Any:
        """
        Get the current value of a given parameter.

        :param name: Name of the parameter. Available parameters can be queried via
                     GetAvailableParameters() method.
        :return: Value of the given parameter.
        """
        return self._params.get(name, None)

    def SetParameter(self, name: str, value: Any) -> None:
        """
        Set a new value to the given parameter.

        :param name: Name of the parameter. Available parameters can be queried via
                     GetAvailableParameters() method.
        :param value: The new value of the given parameter.
        """
        self._params[name] = value