On the Viability of Skeleton Recognition

Anonymous ECCV submission

Paper ID ***

Abstract.

1 Introduction

Person identification has become a valuable asset, whether for means of authentication, personalization, or other applications. Previous work revolves around either physiological biometrics, such as face recognition, or behavioral biometrics such as gait recognition. In this paper, we propose using skeletal measurements as a new physiological biometric for recognition.

In recent years, advances in range cameras have given us access to increasingly accurate real-time depth imaging. Furthermore, the low-cost and widely available Kinect [1] has brought range imaging to the masses. In parallel, the automatic detection of body parts from depth images has led to real-time skeleton mapping. As the resolution and accuracy of range cameras improve, so will the accuracy and precision of skeleton mapping algorithms.

In this paper we show that skeleton mapping is accurate and unique enough in individuals to be used for person recognition. We make the following two contributions. First, we show that ground truth skeleton measurements can uniquely identify a person. We model how the accuracy of skeleton recognition decreases as simulated error increases, and find it is still possible to use for recognition. Second, we evaluate our hypothesis using real-world data collected with the Kinect. Our results show that skeleton recognition performs well, particularly in situations where face recognition cannot be performed.

Much of the prior work in person recognition focuses on data gathered from other sensors, such as face recognition with color cameras and voice recognition with microphones. In the realm of depth imaging, most of the work surrounds behavioral recognition, continuing work in gait recognition. The Xbox 360 [] does use the height inferred from the Kinect as part of its user identification algorithm, albeit in addition to other attributes including face recognition.

The paper is organized as follows. First we discuss prior methods of person identification, in addition to the advances in the technologies pertaining to skeleton mapping (Section 2). Next we use a dataset of actual skeletal measurements to show that identification by skeleton is feasible, even when we simulate the error of measuring skeletons with a Kinect (Section ??). Finally, we Lastly, we collect skeleton data with a Kinect in an uncontrolled setting and we apply preprocessing and classification algorithms to this dataset (Section ??). We evaluate the performance of skeleton recognition with varying group size and compare it to face recognition. We conclude by discussing challenges working with the Kinect and future work (Section 6).

2 Related Work

A biometric is a distinguishable trait that can be measured and used for identification. There are many different possible biometrics. Biometrics can be divided into physiological and behavioral traits. Physiological traits include faces, fingerprints, and irises; speech and gait are behavioral. Faces and gait are the most relevant biometrics for this paper as they both can be collected passively and involve image processing.

Approaches to gait recognition fall into two categories: silhouette-based and model-based. Silhouette-based techniques recognize gaits from a binary representation of the silhouette as extracted from each image, while model-based techniques fit a 3-D model to the silhouette to better track motion [?]. Some model-based techniques use features of the generated model to aid in gait recognition, but they are severely limited by the difficulty in generating a 3-D model from a 2-D image []. Furthermore, behavioral traits typically are more characteristic as opposed to unique, and are subject to change with time and based on the observed activity [?]. On the other hand, face recognition, as a physiological biometric, is a more static feature. Face recognition can be broken down into three parts: face detection, feature extraction, and classification; these three parts are studied both individually and together [2].

In this paper, we propose using skeleton measurements as a biometric trait separate from gait biometrics. According to Jain et al., a proper biometric has the following characteristics: (i) universality, everyone has it, (ii) uniqueness, it should be different between any two people, (iii) permanence, it does not vary with time, and (iv) collectability, it is measurable. Universality, permanence, and collectability are easily met with skeletons (skeletal changes that occur with age happen gradually). Uniqueness is also met, but we discuss this in more detail in Section 3.

By using skeleton as a biometric for recognition, we can formulate skeleton recognition in a similar way as we can face recognition. The equivalent parts would be figure detection, skeleton fitting, and classification. Figure detection and skeleton fitting map to silhouette extraction and model fitting in gait detection, but as previously noted, they are severely limited. However, Zhao et al. [3] perform gait recognition in 3-D using multiple cameras. By moving to 3-D, many of the problems related to silhouette extraction and model fitting are removed. Additionally, by moving to 3-D, we can take advantage of the wealth of research relating to motion capture [4]. Specifically, range cameras offer real-time depth imaging, and the Kinect [1] in particular is a widely available range camera with a low price point. Figure detection and skeleton fitting have also been studied in motion capture, mapping to region of interest detectors and human body part identification or pose estimation respectively in this context [5–7]. Furthermore, OpenNI [?] and Kinect SDK [?] are two commercial systems that perform figure detection and skeleton fitting for the Kinect. Given the maturity of the solutions, we will use implementations of figure detection and skeleton fitting. Therefore this paper will focus primarily on the classification part of skeleton recognition.

3 Skeleton uniqueness

The most obvious concern raised by trying to use skeletons to recognize people is their uniqueness. Are skeletons consistently and sufficiently pairwise distinct to have reasonable hope of using them for people recognition?

3.1 Face recognition benchmark

A good way to understand the uniqueness of a metric is to look at the performance it gives for the pair-matching problem. In this problem you are given two measurements of the metric and you want to decide whether they come from the same individual (matched pair) or from two different individuals (unmatched pair).

The Labeled Faces in the wild [8] database is specifically suited to study the face pair matching problem and has been used to benchmark several face recognition algorithms. Raw data of this benchmark is publicly available and has been derived as follows: the database is split into 10 subsets. From each of these subsets, 300 matched pairs and 300 unmatched pairs are randomly chosen. Each algorithm runs 10 separate leave-one-out cross validation experiments on these sets of pairs. Averaging the number of true positives and false positives across the 10 experiments for a given threshold then yields one point on the true-positive vs false-positive curve (also known as ROC).

3.2 Experiment design

In order to run an experiment similar to the one used in the face pair-matching problem, we use the Goldman Osteological Data Set [9]. This data set consists of osteometric measurements of 1538 skeletons dating from throughout the Holocene. We keep from these measurements the lengths of six bones (radius, humerus, femur, tibia, left coxae, right coxae). Because of missing values, this reduces the size of the dataset to 1191.

From this data set, 1191 matched pairs and 1191 unmatched pairs are generated. The exact measurements of the bones are never directly accessible, but are always perturbed by a noise whose variance depends on the collection protocol. This is accounted for by adding independent random Gaussian noise to each constituents of the pairs.

3.3 Results

The pair-matching problem is then solved by using a proximity threshold algorithm: for a given threshold, a pair will be classified as matched if the Euclidean distance of its two constituents is lower than the threshold and unmatched otherwise.

This algorithm does not require any training, so it is run on the whole set of pairs without doing cross-validation. Figure ?? shows the ROC of the proximity threshold algorithm for varying variance of the noise added to the data.

4 Classification Algorithm

5 Experiment

5.1 Experiment design

5.2 Results

6 Conclusion

References

1. : (Kinect for xbox 360)

2. Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: A literature survey. ACM Comput. Surv. 35 (2003) 399–458

3. Zhao, G., Liu, G., Li, H., Pietikainen, M.: 3d gait recognition using multiple cameras. In: Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, IEEE (2006) 529–534

4. Moeslund, T.B., Hilton, A., Krger, V.: A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104 (2006) 90 – 126

5. Plagemann, C., Ganapathi, V., Koller, D., Thrun, S.: Realtime identification and localization of body parts from depth images. In: Proc. International Conferences on Robotics and Automation (ICRA 2010). (2010)

6. Ganapathi, V., Plagemann, C., Koller, D., Thrun, S.: Real time motion capture using a single time-of-flight camera. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR). (2010)

7. Leyvand, T., Meekhof, C., Wei, Y.C., Sun, J., Guo, B.: Kinect identity: Technology and experience. Computer 44 (2011) 94–96

8. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst (2007)

9. Ruff, C., Auerbach, B.: Human body mass estimation: a comparison of morphometric and mechanical methods. American Journal of Physical Anthropology (125) 331–342