Skip to content
Publicly Available Published by Oldenbourg Wissenschaftsverlag January 14, 2020

VPID: Towards Vein Pattern Identification Using Thermal Imaging

  • Sarah Faltaous

    Sarah Faltaous is a PhD student in the human-computer interaction group at the University of Duisburg-Essen. She holds a master degree in cognitive systems from the University of Ulm and a bachelor in digital media engineering and technology from the German University in Cairo.

    EMAIL logo
    , Jonathan Liebers

    Jonathan Liebers is a PhD student in the human-computer interaction group at the University of Duisburg-Esen. He holds a bachelor and master degree in Applied Computer Science - Systems Engineering from the University of Duisburg-Essen. His research interest lies mainly within the field of usable security and biometrics.

    , Yomna Abdelrahman

    Yomna Abdelrahman is a postdoctoral researcher at the Research Institute CODE of the Bundeswehr University, Munich. In her work, she explores thermal imaging to build novel interactive systems and improve existing ones. She received a PhD from the University of Stuttgart.

    , Florian Alt

    Florian Alt is a professor for Usable Security and Privacy at the Resarch Institute CODE of the Bundeswehr University, Munich. Florian looks at the role of humans in security critical systems, focusing on topics related to behavioral biometrics, physiological security, social engineering, and usable security in novel application areas, such as smart homes and VR. Florian is a subcommittee chair for CHI 2020, TPC chair of Mensch und Computer 2020 and was general chair of the 2018 Conference on Mobile and Ubiquitous Multimedia. He holds a PhD in computer science from the University of Stuttgart and a diploma in Media Informatics from LMU Munich.

    ORCID logo
    and Stefan Schneegass

    Stefan Schneegass is an assistant professor of computer science at the University of Duisburg-Essen. His research focuses on the crossroads of human-computer interaction and ubiquitous computing. This includes novel interaction techniques for mobile, wearable, and ubiquitous devices as well as novel approaches for implicit authentication using such devices. He holds a PhD from the University of Stuttgart.

From the journal i-com

Abstract

Biometric authentication received considerable attention lately. The vein pattern on the back of the hand is a unique biometric that can be measured through thermal imaging. Detecting this pattern provides an implicit approach that can authenticate users while interacting. In this paper, we present the Vein-Identification system, called VPID. It consists of a vein pattern recognition pipeline and an authentication part. We implemented six different vein-based authentication approaches by combining thermal imaging and computer vision algorithms. Through a study, we show that the approaches achieve a low false-acceptance rate (“FAR”) and a low false-rejection rate (“FRR”). Our findings show that the best approach is the Hausdorff distance-difference applied in combination with a Convolutional Neural Networks (CNN) classification of stacked images.

1 Introduction

Previous work recognized the limitations of current authentication mechanisms. A survey of the password habits among American consumers, for example, indicates that the average consumer takes minimal action to secure their online accounts [12]. Prior work further showed that only few users use a PIN for their mobile device [28] and passwords are frequently re-used and forgotten [15]. Abdulwahid et al. conclude that robust authentication mechanisms are needed that operate in a transparent, continuous, and user-friendly fashion [4]. In search for usable and secure authentication mechanisms, a large number of approaches have been proposed. A promising direction is biometric authentication [23]. Biometric authentication identifies a user using physiological or behavioral characteristics [19], [24], [32], [35]. One of its major advantages is that there is no need to remember a PIN, a password, or another secret.

Figure 1 
            Examples of false-color images which interfere with the skeleton extraction.
Figure 1

Examples of false-color images which interfere with the skeleton extraction.

In current systems, biometric authentication mechanisms typically require the user to perform an explicit action. Fingerprint authentication, for example, requires to put a finger on a fingerprint sensor and an iris scanner requires looking into a camera. Another biometric authentication approach is the use of vein patterns of the palm dorsal. Using vein patterns underneath the human skin has several advantages. Every person has a unique pattern of veins, which is stable from the age of ten and unique even for twins [1]. Unlike other biometrics (e. g., fingerprint), the extraction and duplication of vein patterns needs massive interventions. As a result of being less fraud-prone, such approaches are supposed to provide a higher security level for any system.

As veins have a different temperature than the surrounding skin [11], thermal cameras can capture these patterns from a distance without interfering with the user’s current task. Figure 1 shows two examples of possible application scenarios with interactive system. The first example is for tabletop multi-user identification, as shown in Figure 1a, where users could be authenticated based on their captured vein patterns from a thermal camera mounted to cover the field of interaction. This could be applied to permit access and usage in a shared office setup or to customize settings for smart households. Another example is laptop authentication. Having a thermal camera that faces a keyboard, as shown in Figure 1b, the system would be able to track who is using a laptop and can reject access for unknown users or to specific information. In both cases, user identification and authentication happens continuously and implicitly and, thus, does not provide an additional burden to the user. As soon as a user starts to interact, the system automatically recognizes the user and grants or denies access to it.

In accordance with Crawford [10] we define “implicit authentication” analogously to a “zero-interaction authentication” [8] that is transparent to the user. This kind of authentication bears the inherent advantage that a user is not bothered by the authentication process, which traditionally requires an explicit interaction with the authentication system (e. g., such explicit interaction additionally interrupts the interaction with the underlying systems thus negatively impacts usability). Our presented system eliminates the need for explicit interaction and is subsequently especially suited for the usage in public or open spaces, with public displays, in offices or in many other scenarios, where spontaneous interactions can occur.

We present Vein Pattern Identifier (VPID), a system that uses thermal images for vein pattern authentication. We implemented six different approaches including standard computer vision algorithms and deep learning methods which we compared against each other. We conducted a controlled lab study with 12 participants, where we collected data to evaluate the algorithms. The results show that the Hausdorff distance [21] as a shape comparison algorithm with skeleton stacking and a CNN for classification performs best.

The contributions of this paper are as follows:

  1. A processing pipeline[1] that extracts the vein pattern from thermal images and identifies users based on it.

  2. A study comparing the accuracy provided by six different computer vision algorithms along with Convolutional Neural Network (CNN).

2 Related Work

We identified two strands of related work to our research: (1) seamless user authentication and identification using biometric information and (2) thermal imaging as a sensing technology.

2.1 Biometric Authentication

The uniqueness of biometrical features has been widely explored to identify and authenticate users [9], [23], [32]. Biometrics are classified into behavioral and physiological biometrics [41]. Behavioral biometrics rely on behavioral cues to authenticate users, for instance, their touch and keystroke behavior [13], [16], [33]. Physiological biometrics are based on “something you are” and include a person’s physiological information such as iris, face, voice, fingerprint, and hand geometry [37]. An example is Bodyprint [19] that uses body parts like ear, finger, fist, and palm prints, obtained via a mobile phone touchscreen as a biometric trait.

Veins patterns are another biometric feature that has been proposed for identification and authentication. Since vein patterns are unique for each users and stable for a lifetime [1], they are well suited as a biometric. For example, veins can be captured under IR-lights [5], using VGA [39], or near IR imaging [17]. While these technologies provide good input, they either require external sources of illumination [38], have high costs, or require multiple sensors [14].

Researchers also explored using thermal (far IR) imagining to extract the veins on the palm [27], [40]. As veins have different temperature than the surrounding skin [11], they are visible to thermal cameras without additional illumination. Vein triangulation and knuckle shapes are used to differentiate between users. From this we learn that veins are a powerful means for authentication. At the same time, approaches are required that allow authentication using vein patterns to be seamlessly integrated with interactive systems.

2.2 Thermal Imaging for Sensing Interaction

Due to the advantages that thermal imaging provides, including robustness to illumination and color changes, it has recently been proposed as a sensing technology for interactive systems [3], [29], [34]. This is achieved by integrating thermal imaging and existing computer vision techniques to improve touch or gestural interaction. The advantages of thermal imaging are utilized to overcome common RGB and depth cameras’ limitations, such as light dependence and lack of visible differences between veins and skin. Larson et al. used heat traces caused by fingers touching a surface and detected pressure for interaction on surfaces [29]. Sahami Shirazi et al. proposed thermal reflection to expand the interaction space beyond the camera’s direct field-of-view [34]. Abdelrahman et al. investigated the material properties supporting both touch and mid-air gestural interaction using thermal imaging as sensing technology [3]. Furthermore, Woo Cho et al. [7] introduced a new system to detect the thermal handprint for forensic identification.

Thermal imagers have been used in security contexts before. Abdelrahman et al. presented an approach to reveal PIN and lock patterns from thermal images of a user’s mobile device [2]. Overall, thermal cameras are promising sensors for a range of interactive systems e. g., multiple user interaction, tabletop interactions [34] that would naturally benefit from seamless identification and authentication.

3 Vein Identification Approaches

In the following, we present the VPID system that uses thermal imaging to identify users. VPID consists of a recognition pipeline and an authentication part. The recognition pipeline consists of nine steps starting with the raw camera data and resulting in the user’s vein pattern that can be fed into the authentication system. For the authentication part, we propose six different vein-based authentication algorithms.

3.1 Vein Pattern Construction Pipeline

For extracting the hand veins, we are using the OpenCV library[2] for image processing and features extraction. We apply in total nine steps to a thermal image stream to construct the vein pattern of each user that we later use to authenticate a user.

1. Image extraction: First, we extract frames of raw data from the live-stream of the thermal camera (cf., Figure 2a).

2. False-color image transformation: We transform the acquired raw thermal data, represented by an array of unsigned short integers, into a false-color image using an iron color palette scaled from minimumtemperature −7 and maximumtemperature +1. This palette boosts the color difference of temperatures across the shading palette without enhancing or omitting any potential temperature [18] (cf., Figure 2b). This insures that all small details are captured. Afterwards, we apply a color-based threshold to remove any background noise from the hand by setting it to black (cf., Figure 2c).

Figure 2 
              The first half of the steps in our pipeline that would result in the region of interest.
Figure 2

The first half of the steps in our pipeline that would result in the region of interest.

Figure 3 
              Starting with the region of interest, the second half of the image processing pipeline is presented. Specially, the area from the region of interest is processed into a skeleton.
Figure 3

Starting with the region of interest, the second half of the image processing pipeline is presented. Specially, the area from the region of interest is processed into a skeleton.

3. Hand-detection: Then, we use the data from step 2 to detect the warmest area in the image. A convex hull is then drawn to connect all the local maxima of the warm points. Next, we detect the hand informalities (i. e., fingers and knuckles) to verify a hand shape. We measure the defects, which are the points with the largest distance from the convex set (cf., Figure 2d). If no defects are detected, the shape is not identified as a hand.

4. Region of interest (ROI): As the vein pattern is located on the back of the hand, we define the ROI as the circle with the largest diameter enclosed in the hand’s contour (cf., Figure 2e). For that, we try all possible combinations of circles across the hand area (i. e., warmest area). To reduce random noise and artifacts at the border of the circle, the diameter is hence multiplied with a constant factor of 0.95 and we extract that mask (cf., Figure 2f).

5. Extraction of the vein pattern skeleton: We transfer the circular mask depicting the ROI to grayscale image with a normalized histogram. Then, we use an adaptive-threshold based approach to detect the local maxima and assigning them to be part of the vein skeleton (cf., Figure 3a).

6. Noise removal: Afterwards, we use median blurs followed by an opening morphological operation to eliminate noise and small, single artifacts from the image (cf., Figure 3b).

7. Topological skeleton: The image obtained from step 6 results in thick lines or broad areas (i. e., one or multiple connected lines with a width of more than one pixel). Hence, we used the thinning approach of Zhang Suen [42], which reduces the size of each area until it is exactly one pixel thin (cf., Figure 3c).

8. Skeleton affine transformations: Using the center of the ROI obtained in step 4, we obtain the center point of the defects in step 3 and plot a vector. This vector is used to rotate any captured image upwards, in the direction of a positive y-axis (cf., Figure 3d). To further unify the metrics of the logged data, we scale the ROI so that the diameter reaches the maximum height of the image (cf., Figure 3e).

9. Skeletons stacking: In this final step, the vein skeletons obtained from several images of the same video sequence (i. e., same hand) are stacked over each other to assure the best quality to the finest detail. We refined the first 14 frames of each video and stacked them as a final step.

3.2 Vein Pattern Authentication

The authentication process is done through comparing the similarities between vein pattern skeletons. We implemented six different approaches for shape comparison, namely, Hausdorff distance, Shape Context, Jaccard distance, and three different versions of Hu moments. In addition, we refine our results by machine learning algorithms.

Hausdorff and Shape Context distances based on the work of Huttenlocher, Klanderman and Rucklidge [21], and Belongie et al. [6], respectively. These two approaches depend mainly on comparing the distance-difference between two skeletons. Hence, a user is authenticated if the comparison of two skeletons, S1 and S2 results in distance(S1,S2)Vthreshold.

Jaccard distance . The Jaccard index [22] defines a metric for similarity between two sets (e. g., A and B). Where A consists of all white pixels of the first skeleton, and B consists of all white pixels of the second skeleton. Each pixel is defined as a tuple of its X- and Y-coordinates. The Jaccard index is then defined as: J(A,B)=ABAB, where 0J(A,B)1. In other words, the size of the overlapping area between two images is divided by the area of the union of both images, where the area is defined by the presence of the white pixels in the binary image. To provide a comparable metric the Jaccard distance is used. It is defined as: JDistance(A,B)=1J(A,B).

Hu moments based on the work of Hu [20]. These are numerical values retrieved from image characteristics (e. g., lie on the same plane, has non-zero values, etc.). These numerical values then can be compared to each other. Hu proposed three different ways, namely:

  1. I1(A,B)=i=1...7|1miA1miB|

  2. I2(A,B)=i=1...7|miAmiB|

  3. I3(A,B)=maxi=1...7|miAmiB||miA|

where miA=sign(hiA)·log(hiA) and miB=sign(hiB)·log(hiB) are the Hu moments hiA, hiB of A and B respectively.

Convolutional Neural Network . Recent work using deep learning has shown that Convolutional Neural Network (CNN) can be used to classify images [25], [30]. A trained CNN can match one new, unknown image onto one of many learned classes of images. When used as standalone mechanism, the input data would always be identified and classified to any of the registered classes. Therefore, we deployed the CNN with the other comparison and matching algorithms. In our case, this can be used to train the system with the skeleton images of the user’s vein pattern. For our work, we tested various CNNs (i. e., LeNet [31], AlexNet [26], and VGG-16 [36]) with sample data collected across several pilot studies. Finally, we decided to use “AlexNet” for our evaluation, as it does not require long training time and has high accuracy. The network was trained for 30 epochs, with a constant random seed, stochastic gradient descent as the solver and a base learning rate of 0.001.

4 Data Collection

Figure 4 
            The setup of the controlled study, which was used for data collection. The camera was mounted on a height of 37cm37\hspace{0.1667em}\text{cm}.
Figure 4

The setup of the controlled study, which was used for data collection. The camera was mounted on a height of 37cm.

For testing the approaches, we used an Optris PI450[3] contactless thermal camera. It has a 62°×48° field-of-view, 382 px × 288 px optical resolution, and temperature sensitivity of 40 mK. The camera was mounted on a tripod at a fixed height of 37cm, facing towards a table to capture the participant’s hand as depicted in Figure 4. The room temperature conditions were kept constant for all sessions.

For capturing thermal images, we have solely used the “iron” colouring palette scheme, as it provides an infliction point in its gradient over the relative colour intensity. This infliction point helps in separating the “warm” colour values per pixel from the “cold” colour values. To set the exposure parameters (i. e., maximal and minimal temperature to scale the colouring palette scheme) for the thermal imager, we have automatically determined the warmest point of temperature within the image, and added a constant offset of +1 °C for the maximum observed temperature and −5 °C for the minimum temperature. Our tests showed that this offset is ideal to capture the participant’s hand as a whole with all of its contents.

4.1 Participants and Procedures

We recruited 12 participants (4 female, 8 male, μ=28, σ=8.5) via university mailing lists. The participants have backgrounds in different majors. After welcoming the participants, we described the goal of the study and handed out consent forms as well as demographic questionnaires. We asked the participant before recording to wash his/her hands under normal tab water. This procedure was done at the beginning to unify the temperature of participant’s hands. The study was conducted over two sessions on the same day, separated by 23mins break. In every session, each hand (i. e., left or right) was recorded for 20secs. We defined the area and direction in which the hand should be placed on and pointing to, so the camera could start recording.

Collecting biometric information from participants raised the need for a secure procedure, in order to protect their privacy. As the always existing, possible risk of a data breach leaves participants with no way of changing or revoking their biometric information, a failure to comply with a secure procedure to protect their data is highly unethical. Subsequently, we gathered the informed consent of all our participants and assigned them an anonymous identifier. Directly upon finishing the acquisition of data per participant, we deleted the connecting attribute of their private information to the anonymous identifier, so that no connection can be re-established in the future.

4.2 Evaluation

Table 1

The observed criteria that can characterize the content of a false-colour thermal image. The Thermal Image Quality Rating, (“TIQR”) is the sum of its conditions, but at least zero.

Nr. Condition title Condition description Figure Rating
1 High contrast The user’s vein pattern is clearly visible and distinguishable from the surrounding area. 5a 1
2 Medium contrast The user’s vein pattern is barely visible. This condition can apply in conjunction with condition Nr. 1. 5b 1
3 Fingers visible The user’s fingers are clearly visible in the image and not cropped off or dark. 5e 1
4 Complete contour The shape of the hand within the false-color image resembles the true form of a regular hand. No parts are cropped off. The hand is not twisted in any way. 5c 1
5 Dark vein pattern The vein pattern appears dark instead of bright in the false-colour image. 5f −4
6 Wrong ROI Another warm object in the field of view of the thermal imager was falsely identified as a hand, thus the region of interest is not the user’s back of the hand. 5d −4

Figure 5 
              Examples of the six image-quality rating criteria mentioned in Table 1.
Figure 5

Examples of the six image-quality rating criteria mentioned in Table 1.

To start with the evaluation process, we had to first categorise the received images. One major aspect that directly influences the obtained results is the quality of the images captured by the thermal camera. In general, we noticed that certain circumstances exist that result in a “good” hand thermal-image recording, which contain a very visible vein pattern. In contrast to other circumstances, a “bad” thermal recording with the vein pattern being only barely visible. The quality of a captured image is defined by the visibility of the vein pattern in the thermal image, which can be measured by its contrast towards the surrounding area of tissue. One main contributing factor is the angle of the hand towards the thermal imager. If the hand is twisted or curled while being captured by the thermal imager, the vein pattern might not be visible at all. One of the core assumptions of our vein-detection algorithm is that the vein pattern would be in general warmer (i. e., brighter) than the surrounding area. Hence, a quantifying quality metric had to be set to categorize any input. For that we used the manual Thermal Imagining Quality Rating (TIQR) algorithm, where we defined six main criteria. These are based on observations from several pilot studies, where we captured one or two hands and extracted the skeletons. Then we matched visually what we obtained compared to the original input. Each criterion is assigned a numerical value. Thus, the TIQR value would be the summation of all the existing criteria ratings. Also, if the final TIQR value was found to be negative, it would be rounded up to 0.

By doing that, we ended up with four main categories of images, where only the categories having a TIQR of “3” or “4” is furthermore processed for vein pattern detection. All the data, that was marked for processing was subsequently classified into one of three groups. The first group is the “registered” data set. The data in this set was provided to the system and used to train an “AlexNet”. It consists of, in total, twelve distinct skeletons, where each skeleton originates from a different person’s hand. The second group is the “testing” data set. It consists of six elements, where each one has a counterpart in the “registered” data set. These are the skeletons extracted from the second recording session of the same registered hand. The third group is formed only of the skeletons of not-registered hands and is used also for testing, referred to as the “unknown-testing” data set.

Figure 6 
              An activity chart depicting all three variants of the authentication system.
Figure 6

An activity chart depicting all three variants of the authentication system.

For the evaluation, the “testing”- and “unknown-testing”-skeletons are tested against the authentication system. Dependent on the input and the system’s output, the true positive rate (“TPR”), true negative rate (“TNR”), false positive rate (“FPR”) and false negative rate (“FNR”) are determined. Ideally, all skeletons in the “unknown-testing” data set should be rejected by the system and become a true negative (“TN”), as they have no counterpart in the systems’ “registered” data set. A false negative (“FN”) is met, when an element of the “testing” data set is incorrectly rejected, since each element should have been accepted instead. A true positive (“TP”) is counted, if an element of the “testing” data set is correctly accepted by the system. Last but not least, a false positive (“FP”) is met, when an element of the “unknown-testing” data set is accepted by the system. As this data set does not have any counterparts in the “registered” data set, all of them should be rejected instead.

Figure 6 depicts the three evaluated variants of how the authentication system can be designed. The first variant (i. e., Variant I), depicted in Figure 6a, only uses a shape comparison algorithm as a measure of similarity between an input skeleton and any of its registered skeletons. The user’s identity is assumed as the registered skeleton with the closest distance to the input. If the measure of similarity, the “distance”, is below the threshold T1, the authentication attempt is successful, otherwise it is rejected.

The second variant (i. e., Variant II), which is illustrated in Figure 6b, additionally performs an image classification to detect the user’s identity. For the classification, the pre-trained AlexNet is utilized, which states a top prediction and an associated certainty. If this certainty is below T2, the user is rejected. Otherwise, the user’s identity is assumed to be the prediction of the neural network. The threshold T2 is set to a constant value of 75 %.

The third variant (i. e., Variant III), switches the process and exchanges the activities of image classification and shape comparison and is depicted in Figure 6c. Here, the neural network filters the registered data for a set of potential matches. Then, the shape comparison is applied to compare the input against the best matching image of this set only.

As a measure of comparison between the different approaches to utilize the system, we chose the accuracy (“ACC”, see Equation 1a). To find the best suitable values for the threshold T1, we used an optimizer, which calculates the accuracy for every present shape comparison value. The threshold value, which lead to the highest accuracy per method, was then set as the constant value of T1.

(1a)ACC=TP+TNP+N

4.3 Results

Figure 7 
              Depiction of the maximum accuracy per variant, when skeleton stacking is not applied.
Figure 7

Depiction of the maximum accuracy per variant, when skeleton stacking is not applied.

Figure 8 
              Depiction of the maximum accuracy per variant, when skeleton stacking is applied.
Figure 8

Depiction of the maximum accuracy per variant, when skeleton stacking is applied.

All three variants, in which the system can be deployed, are depicted in Figure 7, when skeleton stacking is not applied. The bar chart reports the accuracy per shape comparison algorithm. In contrast, Figure 8 depicts the variants, when skeleton stacking is applied.

Figure 9 
              The FAR/FNR-chart for the authentication system using Variant III, skeleton stacking and the Hausdorff distance as a shape comparison algorithm.
Figure 9

The FAR/FNR-chart for the authentication system using Variant III, skeleton stacking and the Hausdorff distance as a shape comparison algorithm.

Skeleton stacking improved the performance, when the Hausdorff distance was used as a shape comparison algorithm. Here, combining skeleton stacking and Variant III. leads to an accuracy of 100 %, i. e. the system did not make any errors, when the threshold T1 is set to a value between 9.48 to 10.63. In any other case, skeleton stacking did not improve the accuracy. When no skeleton stacking is applied, the shape comparison methods “Shape Context” and “Jaccard” worked best, yielding the highest accuracies in direct comparison to the Hu-methods and the Hausdorff distance. The Variant III, with the application of skeleton stacking and the Hausdorff distance as a shape comparison algorithm, was able to outperform any other combination, by reaching an equal error rate (“EER”) of zero. Out of the six skeletons in the “testing” data set, all were accepted, while all six skeletons in the “unknown-testing” data set were correctly rejected. When applying the same threshold boundaries of 9.48 to 10.63 to Variant I and II, utilizing skeleton stacking and the Hausdorff method, both reported between four and six false positives and no false negatives.

Variant II, which utilizes the CNN besides the shape comparison algorithms, increased the accuracy in any case, except for the Jaccard-distance shape comparison algorithm without the application of skeleton stacking. A single false positive made the difference between Variant II and Variant III, when the Hausdorff distance was applied without the stacking of skeletons.

As a consequence, the combination of CNNs with the traditional shape comparison algorithms is beneficial. They work in general best, if the shape comparison algorithm first performs a rejection (Variant II), but can exceed their performance, if this process is switched (Variant III). In any case, shape comparison algorithms are useful to tweak the performance of deep learning methods. CNNs also states a certainty value besides its predictions, which can be utilized as a rejection criterion similarly to the shape comparisons. From our data, we saw, that for a learned skeleton as an input, the certainty very often was above 90 % and many times close to 100 %. In contrast, unknown data, often ranged in a certainty interval of 40 % to 50 %. The feasibility of certainty values needs to be investigated in future research.

5 Limitations and Future Work

An important factor influencing the accuracy of the VPID system is the image quality. If the image is blurry, the system is unable to extract the vein pattern. Given that the VPID system is designed to implicitly and continuously identify and authenticate users, it can discard images that do not provide proper vein patterns. This, however, requires automatic detection of such images. While we manually rated the images, future work needs to investigate how this can be automated.

External influences such as the room temperature or physical activity influence the clarity of the vein pattern. In cases in which the vein pattern is not fully visible, authentication becomes challenging. Understanding the influence of the context on the clarity of the vein pattern is an important aspect that can make vein-based authentication feasible. Using a machine learning model that takes these differences into account might further improve authentication accuracy.

During the controlled study, we have tested the system with only one thermal imager, which limited the position and orientation the participant’s hand could have in the recorded frame. In order to allow a greater degree of freedom for the participant’s hand, the system could be used with multiple thermal imagers, that cover more angles. An implementation and evaluation of an authentication system with more than one thermal imager needs to be performed in future work.

6 Conclusion

Relying on biometric features as replacement of passwords and tokens enhances the convenience of authentication. Users usually interact with devices like laptops and interactive tabletops without occluding the back of their hand. Thus, deploying a thermal camera, which currently becomes affordable and small, to capture their veins pattern allows user identification and authentication in a contactless and continuous manner.

Such implicit identification and authentication provides advantages, particularly from a usable security point of view. This is due to the fact that the additional identification or authentication step is not loaded onto the user and, thus, the user can interact with the system without such intermediate step.

To achieve this, we presented the VPID system consisting of a recognition pipeline and an authentication mechanism. We conducted a user study with 12 participants to evaluate the pipeline and different strategies for authentication. The results of the conducted study, highlight the potential of using veins patterns. We show that it is possible to distinguish users using their vein pattern. For that we used six different algorithms. Our results shows that the best used algorithm was the Hausdorff distance after applying CNN of stacked images.

About the authors

Sarah Faltaous

Sarah Faltaous is a PhD student in the human-computer interaction group at the University of Duisburg-Essen. She holds a master degree in cognitive systems from the University of Ulm and a bachelor in digital media engineering and technology from the German University in Cairo.

Jonathan Liebers

Jonathan Liebers is a PhD student in the human-computer interaction group at the University of Duisburg-Esen. He holds a bachelor and master degree in Applied Computer Science - Systems Engineering from the University of Duisburg-Essen. His research interest lies mainly within the field of usable security and biometrics.

Yomna Abdelrahman

Yomna Abdelrahman is a postdoctoral researcher at the Research Institute CODE of the Bundeswehr University, Munich. In her work, she explores thermal imaging to build novel interactive systems and improve existing ones. She received a PhD from the University of Stuttgart.

Florian Alt

Florian Alt is a professor for Usable Security and Privacy at the Resarch Institute CODE of the Bundeswehr University, Munich. Florian looks at the role of humans in security critical systems, focusing on topics related to behavioral biometrics, physiological security, social engineering, and usable security in novel application areas, such as smart homes and VR. Florian is a subcommittee chair for CHI 2020, TPC chair of Mensch und Computer 2020 and was general chair of the 2018 Conference on Mobile and Ubiquitous Multimedia. He holds a PhD in computer science from the University of Stuttgart and a diploma in Media Informatics from LMU Munich.

Stefan Schneegass

Stefan Schneegass is an assistant professor of computer science at the University of Duisburg-Essen. His research focuses on the crossroads of human-computer interaction and ubiquitous computing. This includes novel interaction techniques for mobile, wearable, and ubiquitous devices as well as novel approaches for implicit authentication using such devices. He holds a PhD from the University of Stuttgart.

References

[1] Vein recognition in Europe, Biometric Technology Today (2004).Search in Google Scholar

[2] Yomna Abdelrahman, Mohamed Khamis, Stefan Schneegass and Florian Alt, Stay cool! Understanding thermal attacks on mobile-based user authentication, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, pp. 3751–3763, ACM, New York, NY, USA, 2017.10.1145/3025453.3025461Search in Google Scholar

[3] Yomna Abdelrahman, Alireza Sahami Shirazi, Niels Henze and Albrecht Schmidt, Investigation of material properties for thermal imaging-based interaction, in: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 15–18, ACM, New York, NY, USA, 2015.10.1145/2702123.2702290Search in Google Scholar

[4] Abdulwahid Al Abdulwahid, Nathan Clarke, Steven Furnell, Ingo Stengel and Christoph Reich, The current use of authentication technologies: an investigative review, in: 2015 International Conference on Cloud Computing (ICCC), pp. 1–8, April 2015.10.1109/CLOUDCOMP.2015.7149658Search in Google Scholar

[5] Jose Anand, T. G. Arul Flora and Anu Susan Philip, Finger-vein based biometric security system, International Journal of Research in Engineering and Technology2 (2013), 197–200.10.15623/ijret.2013.0212035Search in Google Scholar

[6] S. Belongie, J. Malik and J. Puzicha, Shape matching and object recognition using shape contexts, IEEE Trans. Pattern Anal. Mach. Intell.24 (2002), 509–522.10.1109/34.993558Search in Google Scholar

[7] K. W. Cho, F. Lin, C. Song, X. Xu, F. Gu and W. Xu, Thermal handprint analysis for forensic identification using Heat-Earth Mover’s Distance, in: 2016 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA), pp. 1–8, Feb 2016.10.1109/ISBA.2016.7477241Search in Google Scholar

[8] Mark D. Corner and Brian D. Noble, Zero-interaction authentication, in: Proceedings of the 8th Annual International Conference on Mobile Computing and Networking, MobiCom ’02, pp. 1–11, ACM, New York, NY, USA, 2002.10.1145/570645.570647Search in Google Scholar

[9] Lynne Coventry, Antonella De Angeli and Graham Johnson, Usability and biometric verification at the ATM interface, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’03, pp. 153–160, ACM, New York, NY, USA, 2003.10.1145/642611.642639Search in Google Scholar

[10] Heather Anne Crawford, A framework for continuous, transparent authentication on mobile devices, Ph.D. thesis, University of Glasgow, 2012.Search in Google Scholar

[11] J. M. Cross and C. L. Smith, Thermographic imaging of the subcutaneous vascular network of the back of the hand for biometric identification, in: Institute of Electrical and Electronics Engineers 29th Annual 1995 International Carnahan Conference on Security Technology, 1995. Proceedings, pp. 20–35, Oct 1995.Search in Google Scholar

[12] CSID, Consumer Survey: Password Habits, A study among American consumers, 2012.Search in Google Scholar

[13] Alexander De Luca, Alina Hang, Frederik Brudy, Christian Lindner and Heinrich Hussmann, Touch me once and I know it’s you!: Implicit authentication based on touch screen patterns, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pp. 987–996, ACM, New York, NY, USA, 2012.10.1145/2207676.2208544Search in Google Scholar

[14] Roemi Fernández and Manuel Armada, Multisensory system for the detection and localization of peripheral subcutaneous veins, Sensors17 (2017), 897.10.3390/s17040897Search in Google Scholar PubMed PubMed Central

[15] Dinei Florencio and Cormac Herley, A large-scale study of web password habits, in: Proceedings of the 16th International Conference on World Wide Web, WWW ’07, pp. 657–666, ACM, New York, NY, USA, 2007.10.1145/1242572.1242661Search in Google Scholar

[16] Lishoy Francis, Keith Mayes, Gerhard Hancke and Konstantinos Markantonakis, A location based security framework for authenticating mobile phones, in: Proceedings of the 2Nd International Workshop on Middleware for Pervasive Mobile and Embedded Computing, M-MPAC ’10, pp. 5:1–5:8, ACM, New York, NY, USA, 2010.10.1145/1890791.1890796Search in Google Scholar

[17] I. Fujitsu, PalmSecure-SL, 2012.Search in Google Scholar

[18] Gtattersall, Choosing the appropriate colour palette for thermal imaging in animals, Mar 2016.Search in Google Scholar

[19] Christian Holz, Senaka Buthpitiya and Marius Knaust, Bodyprint: biometric user identification on mobile devices using the capacitive touchscreen to scan body parts, in: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 3011–3014, ACM, New York, NY, USA, 2015.Search in Google Scholar

[20] Ming-Kuei Hu, Visual pattern recognition by moment invariants, IRE Transactions on Information Theory8 (1962), 179–187.10.1109/TIT.1962.1057692Search in Google Scholar

[21] D. P. Huttenlocher, G. A. Klanderman and W. A. Rucklidge, Comparing images using the Hausdorff distance, IEEE Trans. Pattern Anal. Mach. Intell.15 (1993), 850–863.10.1109/34.232073Search in Google Scholar

[22] Paul Jaccard, The distribution of the flora in the alpine zone. 1, New Phytologist11 (1912), 37–50.10.1111/j.1469-8137.1912.tb05611.xSearch in Google Scholar

[23] Anil Jain, Lin Hong and Sharath Pankanti, Biometric identification, Commun. ACM43 (2000), 90–98.10.1145/328236.328110Search in Google Scholar

[24] Anil K. Jain, Arun Ross and Sharath Pankanti, Biometrics: a tool for information security, IEEE Transactions on Information Forensics and Security1 (2006), 125–143.10.1109/TIFS.2006.873653Search in Google Scholar

[25] Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton, ImageNet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems vol. 25 (F. Pereira, C. J. C. Burges, L. Bottou and K. Q. Weinberger, eds.), Curran Associates, Inc., 2012, pp. 1097–1105.Search in Google Scholar

[26] Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, pp. 1097–1105, 2012.Search in Google Scholar

[27] Ajay Kumar and K. Venkata Prathyusha, Personal authentication using hand vein triangulation and knuckle shape, IEEE Transactions on Image Processing18 (2009), 2127–2136.10.1109/TIP.2009.2023153Search in Google Scholar PubMed

[28] Stan Kurkovsky and Ewa Syta, Digital natives and mobile phones: A survey of practices and attitudes about privacy and security, in: 2010 IEEE International Symposium on Technology and Society (ISTAS), pp. 441–449, June 2010.10.1109/ISTAS.2010.5514610Search in Google Scholar

[29] Eric Larson, Gabe Cohn, Sidhant Gupta, Xiaofeng Ren, Beverly Harrison, Dieter Fox and Shwetak Patel, HeatWave: Thermal imaging for surface user interaction, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, pp. 2565–2574, ACM, New York, NY, USA, 2011.10.1145/1978942.1979317Search in Google Scholar

[30] Yann LeCun, Yoshua Bengio and Geoffrey Hinton, Deep learning, Nature521 (2015), 436.10.1038/nature14539Search in Google Scholar PubMed

[31] Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner andet al., Gradient-based learning applied to document recognition, Proceedings of the IEEE86 (1998), 2278–2324.10.1109/5.726791Search in Google Scholar

[32] Ken Pfeuffer, Matthias J. Geiger, Sarah Prange, Lukas Mecke, Daniel Buschek and Florian Alt, Behavioural biometrics in VR: Identifying people from body motion and relations in virtual reality, in: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, pp. 110:1–110:12, ACM, New York, NY, USA, 2019.10.1145/3290605.3300340Search in Google Scholar

[33] Napa Sae-Bae, Kowsar Ahmed, Katherine Isbister and Nasir Memon, Biometric-rich gestures: A novel approach to authentication on multi-touch devices, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pp. 977–986, ACM, New York, NY, USA, 2012.10.1145/2207676.2208543Search in Google Scholar

[34] Alireza Sahami Shirazi, Yomna Abdelrahman, Niels Henze, Stefan Schneegass, Mohammadreza Khalilbeigi and Albrecht Schmidt, Exploiting thermal reflection for interactive systems, in: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, CHI ’14, pp. 3483–3492, ACM, New York, NY, USA, 2014.10.1145/2556288.2557208Search in Google Scholar

[35] Stefan Schneegass, Youssef Oualil and Andreas Bulling, SkullConduct: Biometric user identification on eyewear computers using bone conduction through the skull, in: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, pp. 1379–1384, ACM, New York, NY, USA, 2016.10.1145/2858036.2858152Search in Google Scholar

[36] Karen Simonyan and Andrew Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014).Search in Google Scholar

[37] Sulochana Sonkamble, Dr Ravindra Thool and Balwant Sonkamble, Survey of biometric recognition systems and their applications, Journal of Theoretical & Applied Information Technology11 (2010).Search in Google Scholar

[38] Tran Van Tien, Pham T. H. Mien, Pham T. Dung and Huynh Quang Linh, Using near-infrared technique for vein imaging, in: 5th International Conference on Biomedical Engineering in Vietnam, Springer, pp. 190–193, 2015.10.1007/978-3-319-11776-8_46Search in Google Scholar

[39] Jian-Gang Wang, Wei-Yun Yau, Andy Suwandy and Eric Sung, Person recognition by fusing palmprint and palm vein images based on “Laplacianpalm” representation, Pattern Recognition41 (2008), 1514 – 1527.10.1016/j.patcog.2007.10.021Search in Google Scholar

[40] Lingyu Wang and Graham Leedham, A thermal hand vein pattern verification system, in: Pattern Recognition and Image Analysis, Springer, 2005, pp. 58–65.10.1007/11552499_7Search in Google Scholar

[41] Helen M. Wood, The use of passwords for controlled access to computer resources, 500, US Department of Commerce, National Bureau of Standards, 1977.10.6028/NBS.SP.500-9Search in Google Scholar

[42] T. Y. Zhang and Ching Y. Suen, A fast parallel algorithm for thinning digital patterns, Communications of the ACM27 (1984), 236–239.10.1145/357994.358023Search in Google Scholar

Published Online: 2020-01-14
Published in Print: 2019-11-18

© 2019 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 4.3.2024 from https://www.degruyter.com/document/doi/10.1515/icom-2019-0009/html
Scroll to top button