Text recognition¶
Using an optical character recognition (OCR) service, text contents can be extracted from images. Thus, an image can be compared with a string during evaluation in the Image Read test step. Also, text can be located within an image by using the internal Multimedia API. By default ecu.test supports text recognition by using the Tesseract OCR engine. Additionally, it is possible to integrate user-defined OCR services using UserPyModules. The OCR service can be configured in the test configuration’s media access tab.
User-defined text recognition¶
The text recognition capability of ecu.test uses an OCR service either to convert an image into a string (e.g. by concatenating all recognized text in lexicographic order) or to precisely locate individual text passages within an image. Besides the pre-integrated service by Tesseract OCR, users may extend ecu.test with their own user-defined version. A user-defined OCR service must be implemented as a Python module fulfilling several requirements:
The module needs to be located in the
UserPyModulesdirectory of the current workspace.On top level of the script a variable MODULE_TYPE = ‘UserOCRService’ must be specified.
A OCR service class must be implemented satisfying the following interface:
__init__
GetServiceId
ImageToText
FindTextPositions
GetAvailableParameters
GetParameter
SetParamter
There must not be two services with the same service Id.
Further details on the interface can be found in the example in the following section.
Newly added or modified services are available after refreshing the user libraries which can be done in ecu.test in the menu “Extras”.
To utilize a user defined OCR service in the test case via the internal Multimedia API, an instance can be created by calling
api.Multimedia.GetOCRService or
api.Multimedia.GetDefaultOCRService.
The implemented methods of the interface can be called directly on the returned object, e.g.
api.Multimedia.GetOCRService('ExampleOCR').GetParamter('threshold').
Minimal example for text recognition¶
An example implementations of a user-defined OCR service can be found in the ecu.test installation directory:
Templates/DefaultData/UserPyModules/UserOCRService/ExampleOCRService.py
The minimal example in ExampleOCRService.py contains the following code. Further remarks to
the individual methods can be found in the corresponding docstrings.
from __future__ import annotations
import logging
from typing import Any, TYPE_CHECKING
MODULE_TYPE = 'OCRService'
if TYPE_CHECKING:
import numpy.typing as npt
class ExampleOCR:
def __init__(self):
"""
The service can be created when *GetOCRService(serviceId)* is called on the internal
multimedia API, where *serviceId* is the identifier returned by *GetServiceId()*.
Further, if the service is selected in the test configuration's media access tab,
the class is instantiated
- when an image is compared to a string during evaluation in image read and
- when GetDefaultOCRService() is called on the Multimedia API via
*api.Multimedia.GetDefaultOCRService()*.
"""
self.ocrMode = 'dummy'
self.canDetectText = False
self.threshold = 0.4
@staticmethod
def GetServiceId() -> str:
"""
The identifier of the OCR service as displayed in the test configuration's media access tab
and used to create service via the multimedia api (see __init__).
:return: Identifier of the service
"""
return 'ExampleOCR'
def ImageToText(self, img: npt.NDArray) -> str:
"""
Converts the image to a string representation, e.g. by concatenating all found text in
lexicographic order.
:param img: Numpy image (width x height x channels)
:return: String representation of the image
"""
height, width = img.shape[:2]
return f'The image has dimension ({height}, {width}) and maybe some text.'
def FindTextPositions(self, img: npt.NDArray) -> list[dict[str, int | float | str]]:
"""
Returns a list of all text matches found in the image.
Each text match is a dictionary that must contain information about
- its location in terms of a bounding box with upper left corner (x,y) and
dimensions (width, height),
- its content (text) and
- the confidence of the individual match (confidence).
:param img: Numpy image (width x height x channels)
:return: List of dictionaries describing all text matches
"""
h, w, __ = img.shape
return [
{
'x': int(w / 4.0),
'y': int(h / 4.0),
'width': int(w / 2),
'height': int(h / 2),
'text': 'Some large imaginary text in the middle of the image.',
'confidence': 0.42,
}
]
def GetAvailableParameters(self) -> list[dict[str, Any]]:
"""
Returns a list of all configureable parameters. Each parameter must be described by a unique
name and a default value. A short description is optional and will be displayed in a
configuration dialog (tcf, media access tab).
:return: List of all available paramters
"""
return [
{
'name': 'ocrMode',
'default': 'dummy',
'description': 'This text describes what can or cannot be done.'
},
{
'name': 'canDetectText',
'default': False
},
{
'name': 'threshold',
'default': 0.5,
'description': 'Matches with confidence below this value are ignored.'
},
]
def GetParameter(self, name: str) -> Any:
"""
Returns the value of the parameter addressed by name.
:param name: Name of the parameter
:return: Value of parameter addressed by name
"""
try:
return getattr(self, name)
except AttributeError:
logging.getLogger('ecu.test').info(
'{}: No such parameter: {}'.format(self.GetServiceId(), name)
)
def SetParameter(self, name: str, value: Any) -> None:
"""
Sets the value of the parameter addressed by name.
:param name: Name of the parameter
:param value: Value of the parameter
"""
if hasattr(self, name):
setattr(self, name, value)
else:
logging.getLogger('ecu.test').info(
'{}: No such parameter: {}'.format(self.GetServiceId(), name)
)