On November 13th Mathias Ibsen defended his PhD thesis on Secure Face Recognition against Digital and Physical Manipulations. Congratulations to the excellent grade (summa cum laude)!
Thesis Abstract:
Biometric systems, particularly face recognition (FR) systems, have become ubiquitous and are used in many real-world authentication scenarios, ranging from unlocking smartphone devices to automated border control systems at airports. FR systems offer a seamless and convenient authentication method, as faces can be captured at a distance. Furthermore, advancements in deep learning-based methods and accessibility to large training datasets have meant that state-of- the-art FR systems achieve near-perfect biometric performance on several challenging benchmarks. However, the high generalisability of FR systems has also meant that these systems are vulnerable to different digital and physical face manipulations, which can impair their security. For instance, alterations to a face might be performed by a malicious actor in the physical domain (e.g., using makeup or silicone masks) or in the digital domain (e.g., using morphing or face swapping techniques) with the intention to impersonate another target identity and, as such, gain fraudulent access to a system. To mitigate this security risk, methods for detecting various digital and physical face manipulations have been proposed. However, most of these approaches consider only a few well-studied types of digital face manipulations or physical face manipulations and struggle to generalise beyond the data they were trained on. Motivated by the need to enhance the security of FR systems towards digital and physical face manipulations, this thesis investigates in more detail the impact of different digital and physical face manipulations on FR systems. Moreover, this thesis proposes novel algorithms for detecting digital and physical face manipulations by both developing methods focusing on detecting specific types of digital face manipulations or physical face manipulations and by developing highly generalisable methods for the unified detection of attacks on FR systems across both the digital and physical domains. Furthermore, methods for adapting existing pre-trained FR systems to be more resilient towards digital and physical face manipulations are proposed, and it is investigated how algorithms might be used in collaboration with human examiners to detect digital face manipulations. Comprehensive and realistic experimental evaluations demonstrate the capabilities of the proposed algorithms for detecting different digital and physical face manipulations and for making FR systems more resistant towards such manipulations.