Remote photoplethysmography (rPPG) with time-of-flight (ToF) camera at the BEZ

Background and Motivation

One of the major challenges in biometrics is ensuring robustness against presentation attacks. Such attacks can take many forms, including facial masks or falsified fingerprints, and their complexity is increasing in light of recent advances in artificial intelligence. Moreover, security assessments are seldom transferable between systems, meaning that resilience against circumvention must typically be re-evaluated whenever software or hardware components are modified.

In view of these factors, regular evaluation and improvement of existing PAD technologies and the development of new PAD technologies are essential.

The BEZ is currently pursuing two PAD strategies:

• Near-infrared presentation attack detection (NIR-PAD)
• Three-dimensional presentation attack detection (3D-PAD)

In the field of 3D forgery detection, the ToF camera represents a promising approach due to its performance and integrability.

Research Objective in the BEZ Context

The aim of the study is to evaluate and develop 3D ToF cameras for presentation attack detection in facial biometrics and to integrate them into existing systems.

Brief description of the technology

ToF cameras are special 3D camera systems based on the time-of-flight principle. The scene (in this case, the face) is illuminated with modulated infrared light, and a precise depth image is calculated from the time of flight of the reflected light. In addition, ToF systems measure the amount of light reflected back from the scene and output this as an amplitude image. The amplitude images allow conclusions to be drawn about the reflective properties of the skin.

Methodology

In the context of PAD technology in facial biometrics, the ToF camera is used for remote photoplethysmography (rPPG) – a method that allows contactless heart rate measurement.

At the BEZ, the measurement setup is integrated into the BEZ's long-term test series, in which test subjects test biometric systems on site several times a week. This enables regular tests with real, authorised users (bona fide tests). In contrast, outside of the long-term test series, additional targeted attacks with masks made of different materials are investigated (PAD testing).

An industrial 3D ToF camera is used for the measurements, recording depth and amplitude images in each test run. At the same time, reference data on heart rate is recorded using a finger pulse oximeter. The rPPG signals from the ToF camera are extracted from facial areas (forehead, cheeks) and then evaluated in the frequency range.

Presentation attacks are detected not only through the analysis of movement patterns, but also by assessing the quality and presence of physiological signals characteristic of genuine presentations. Methods employed for this purpose include the spectral pulse peak prominence ratio (SPR), a metric that quantifies the ratio between the pulse frequency signal and the residual spectrum. In addition, depth-related movements are analysed by calculating the standard deviation of region-of-interest (ROI) depth values, enabling the identification of distinctive patterns such as micro-movements indicative of attacks.

The simultaneous use of 2D (amplitude) and 3D data makes it possible to extract both geometric features and physiological signals and combine them for PAD analysis. This special combination aims to significantly increase the robustness of biometric systems against attacks.

In further stages of development, the integration of an RGB camera is planned in order to obtain additional comparison and validation data.

Student Projects and Practical Projects

• Non-contact measurement of pulse rate using a camera
• Material characterisation with a 3D camera using scattering properties
• 3D-Presentation-Attack-Detection