LABIC - Bioinformatics and Computational Intelligence Laboratory

Local Repository of Research Datasets

UTFPR-SBD: Soft Biometrics Dataset

1. Dataset

The number of surveillance cameras installed in public places has grown enormously during the past years due to the need to increase public security. This fact allowed tha accumulation of ahuge amount of images and videos in real time without much effort. Different types of problems can be solved by processing the data obtained by security cameras, such as the identification of individuals. Soft biometrics attributes can be useful to perform this task, since they provide information that can be used to differentiate one person from another without requiring their direct cooperation. However, this demands an exhaustive process of analysis to be carried by one or more human observers. Depending on the quantity of cameras, this could even become an impossible task for humans. Hence, computer vision methods could be a valid alternative to perform soft biometric classification in images or videos. Within this score, Deep Learning (DL) methods have risen recently, achieving state-of-the art performances for several computer vision tasks such as object recognition, object detection and image segmentation. This is possible due to their capability to learn both, features and classifier, at once, in order to solve a particular problem. Following this line, this dataset was created for empirical studies of the suitability of DL methods for classifying soft biometrics in images or videos.

2. Specifications Table

3. Value of the Data:

• There is a growing concern about security in places where a large number of people circulate, and this has increased the need for surveillance. Consequently, the omnipresence of surveillance cameras has enabled organizations to record and storage videos at an unprecedented scale and
  pace. Several computational methods for identifying people in security videos were proposed. Most of then rely on a large amount of annotated images and video clips to be trained. Soft biometrics are traits (physical or behavioural) capable of being used for the identication of individuals according to their appearance. However, there are few annotated (pre-classified) video/image datasets speci cally devised for soft biometrics.
• Data is provided primarily for training deep learning methods for soft biometrics applications. However, since the datasets are fully-labeled, they can be used for other purposes, such as multilabel classification.
• The datasets have both, video clips and still images extracted from the frames. Annotation regarding the type of clothing, as well other characteristics were done at the frame level.

4. Dataset Description

There are two datasets: UTFPR-SBD1 and UTFPR-SBD2, with different levels of complexity.

The UTFPR-SBD1 dataset was collected in Ciudad del Este (Paraguay). It comprises 264 images (1080 1920 pixels) taken from 33 videos (8 frames per video) of 11 individuals walking towards the camera in front a simple background. Figure 1 shows some samples of this dataset. The soft biometrics attributes of UTFPR-SBD1 are shown in Table 1. Observe that all, but the color labels, are binary. It is important to state that the classi cation of some the soft biometrics attributes encompass some subjectiveness. Colors, for instance, are considered the predominant primary color of clothing and hair.

UTFPR-SDBD2 is aimed at being more complex, and contains 360 short videos (25 frames per second and resolution 1920x1080 pixels), of 48 people using di erent clothes. Videos were taken at different times of day (different illumination conditions) and people were walking in four directions in a hall. Each video was labeled taking into account the individual the closest to the camera. Other people walking in the background were considered as noise. Figure 2 shows some samples of the dataset. Tables 3 and 4 present the soft biometrics attributes that were annotated for the dataset:

5. Link to the dataset:  (soon it will be available for download)

6. Related Papers:

•  Aquino, N.M.R., Deep Learning Approaches for Soft Biometrics Classification in Videos. MSc. Dissertation. Federal University of Technology Paraná (UTFPR), Curitiba, Brazil, 2018
• Aquino, N.M.R., Gutoski, M., Hattori, L.T., Lopes, H.S., Soft Biometrics Classification Using Denoising Convolutional Autoencoders and Support Vector Machines. In: Proc. XIII Brazilian Congress on Computational Intelligence, Niterói, RJ, 2017
• Aquino, N.M.R., Gutoski, M., Hattori, L.T., Lopes, H.S., The effect of data augmentation on the performance of convolutional neural networks. In: Proc. XIII Brazilian Congress on Computational Intelligence, Niterói, RJ, 2017
• Perlin, H.A., Lopes, H.S., Extracting human attributes using a convolutional neural network approach. Pattern Recognition Letters, v. 68, n. 2, p. 250-259, 2015