A facial expression database is a collection of images or video clips with facial expressions of a range of emotions. Well-annotated (emotion-tagged) media content of facial behavior is essential for training, testing, and validation of algorithms for the development of expression recognition systems. The emotion annotation can be done in discrete emotion labels or on a continuous scale. Most of the databases are usually based on the basic emotions theory (by Paul Ekman) which assumes the existence of six discrete basic emotions (anger, fear, disgust, surprise, joy, sadness). However, some databases include the emotion tagging in continuous arousal-valence scale.
In posed expression databases, the participants are asked to display different basic emotional expressions, while in spontaneous expression database, the expressions are natural. Spontaneous expressions differ from posed ones remarkably in terms of intensity, configuration, and duration. Apart from this, synthesis of some AUs are barely achievable without undergoing the associated emotional state. Therefore, in most cases, the posed expressions are exaggerated, while the spontaneous ones are subtle and differ in appearance.
Many publicly available databases are categorized here. Here are some details of the facial expression databases. DatabaseFacial expressionNumber of SubjectsNumber of images/videosGray/ColorResolution, Frame rateGround truthTypeFERG-3D-DB (Facial Expression Research Group 3D Database) for stylized characters Aneja, Deepali, et al. "Learning to generate 3D stylized character expressions from humans." 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018. angry, disgust, fear, joy, neutral, sad, surprise 439574 annotated examples ColorEmotion labelsFrontal poseRyerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) Livingstone & Russo (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. Speech: Calm, happy, sad, angry, fearful, surprise, disgust, and neutral. Song: Calm, happy, sad, angry, fearful, and neutral. Each expression at two levels of emotional intensity.24 7356 video and audio files Color1280x720 (720p)Facial expression labels Ratings provided by 319 human ratersPosedExtended Cohn-Kanade Dataset (CK+)P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar and I. Matthews, "The Extended Cohn-Kanade Dataset (CK+): A complete facial expression dataset for action unit and emotion-specified expression," in 3rd IEEE Workshop on CVPR for Human Communicative Behavior Analysis, 2010 neutral, sadness, surprise, happiness, fear, anger, contempt and disgust123 593 image sequences (327 sequences having discrete emotion labels)Mostly gray640* 490Facial expression labels and FACS (AU label for final frame in each image sequence)Posed; spontaneous smilesJapanese Female Facial Expressions (JAFFE)neutral, sadness, surprise, happiness, fear, anger, and disgust10213 static imagesGray256* 256Facial expression labelPosedMMI DatabaseM. Valstar and M. Pantic, "Induced disgust, happiness and surprise: an addition to the MMI facial expression database," in Proc. Int. Conf. Language Resources and Evaluation, 2010 431280 videos and over 250 imagesColor720* 576AU label for the image frame with apex facial expression in each image sequencePosed and SpontaneousBelfast DatabaseI. Sneddon, M. McRorie, G. McKeown and J. Hanratty, "The Belfast induced natural emotion database," IEEE Trans. Affective Computing, vol. 3, no. 1, pp. 32-41, 2012 Set 1 (disgust, fear, amusement, frustration, surprise)114570 video clipsColor720*576Natural EmotionSet 2 (disgust, fear, amusement, frustration, surprise, anger, sadness)82650 video clipsColorSet 3 (disgust, fear, amusement)60180 video clipsColor1920*1080Indian Semi-Acted Facial Expression Database (iSAFE)Happy, Sad, Fear, Surprise, Angry, Neutral, Disgust44395 clipsColor1920x1080 (60 fps)Emotion labelsSpontaneousDISFAS. M. Mavadati, M. H. Mahoor, K. Bartlett, P. Trinh and J. Cohn., "DISFA: A Spontaneous Facial Action Intensity Database," IEEE Trans. Affective Computing, vol. 4, no. 2, pp. 151–160, 2013 -274,845 video framesColor1024*768; 20 fpsAU intensity for each video frame (12 AUs)SpontaneousMultimedia Understanding Group (MUG)N. Aifanti, C. Papachristou and A. Delopoulos, The MUG Facial Expression Database, in Proc. 11th Int. Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Desenzano, Italy, April 12–14, 2010. neutral, sadness, surprise, happiness, fear, anger, and disgust861462 sequencesColor896*896, 19fpsEmotion labelsPosedIndian Spontaneous Expression Database (ISED)S L Happy, P. Patnaik, A. Routray, and R. Guha, "The Indian Spontaneous Expression Database for Emotion Recognition," in IEEE Transactions on Affective Computing, 2016, .sadness, surprise, happiness, and disgust50428 videos Color1920* 1080, 50 fpsEmotion labelsSpontaneousRadboud Faces Database (RaFD)Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H.J., Hawk, S.T., & van Knippenberg, A. (2010). Presentation and validation of the Radboud Faces Database. Cognition & Emotion, 24(8), 1377—1388. neutral, sadness, contempt, surprise, happiness, fear, anger, and disgust67Three different gaze directions and five camera angles (8*67*3*5=8040 images)Color681*1024Emotion labelsPosedOulu-CASIA NIR-VIS databasesurprise, happiness, sadness, anger, fear and disgust80three different illumination conditions: normal, weak and dark (total 2880 video sequences)Color320×240Posed FERG (Facial Expression Research Group Database)-DB for stylized charactersangry, disgust, fear, joy, neutral, sad, surprise 655767Color768x768Emotion labelsFrontal poseAffectNetneutral, happy, sad, surprise, fear, disgust, anger, contempt~450,000 manually annotated ~ 500,000 automatically annotatedColorVariousEmotion labels, valence, arousalWild settingIMPA-FACE3Dneutral frontal, joy, sadness, surprise, anger, disgust, fear, opened, closed, kiss, left side, right side, neutral sagittal left, neutral sagittal right, nape and forehead (acquired sometimes)38534 static imagesColor640X480Emotion labelsPosedFEI Face Databaseneutral, smile2002800 static imagesColor640X480Emotion labelsPosedAff-Wildvalence and arousal200~1,250,000 manually annotatedColorVarious (average = 640x360)Valence, ArousalIn-the-Wild settingAff-Wild2neutral, happiness, sadness, surprise, fear, disgust, anger + valence-arousal + action units 1,2,4,6,12,15,20,25458~2,800,000 manually annotatedColorVarious (average = 1030x630)Valence, Arousal, 7 basic expressions, action units for each video frameIn-the-Wild settingReal-world Affective Faces Database (RAF-DB)6 classes of basic emotions (Surprised, Fear, Disgust, Happy, Sad, Angry) plus Neutral and 12 classes of compound emotions (Fearfully Surprised, Fearfully Disgusted, Sadly Angry, Sadly Fearful, Angrily Disgusted, Angrily Surprised, Sadly Disgusted, Disgustedly Surprised, Happily Surprised, Sadly Surprised, Fearfully Angry, Happily Disgusted)29672 annotated examplesColorVarious for original dataset and 100x100 for aligned datasetEmotion labelsPosed and Spontaneous
References