Effects of the duration of expressions on the recognition of microexpressions

Effects of the duration of expressions on the recognition of microexpressions first-author corresp ‡Corresponding Author ‡Corresponding Author †E-mail:fuxl@psych.ac.cn †E-mail:fuxl@psych.ac.cn fuxl@psych.ac.cn State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China Objective: 2 The purpose of this study was to investigate the effects of the duration of expressions on the recognition of microexpressions, which are closely related to deception. Methods: 2 In two experiments, participants were briefly (from 20 to 300 ms) shown one of six basic expressions and then were asked to identify the expression. Results: 2 The results showed that the participants’ performance in recognition of microexpressions increased with the duration of the expressions, reaching a turning point at 200 ms before levelling off. The results also indicated that practice could improve the participants’ performance. Conclusions: 2 The results of this study suggest that the proper upper limit of the duration of microexpressions might be around 1/5 of a second and confirmed that the ability to recognize microexpressions can be enhanced with practice. Duration of expressions Microexpressions Microexpression training tool (METT) Introduction 1 During interpersonal communication, it is important to be aware of others’ emotions (Matsumoto et al., 2000 2000 ; Ekman, 2003b 2003b ). As a Chinese saying goes, “Look at the weather when you step out; look at men’s faces when you step in.” However, not everyone shows their feelings on their faces. Some individuals may suppress true emotion and express a false facial expression. There are also some situations in which social norms force people to conceal, mask, or inhibit true feelings in ways depending on politeness, context, culture or their status (Ekman, 1972 1972 ). Fortunately, the suppressed expressions can be expressed subconsciously in the form of microexpressions and therefore can be detected by a skilled observer (Ekman and Friesen, 1969 1969 ; Ekman, 2009 2009 ). A microexpression is a brief and subtle facial movement which usually lasts for from 1/25 to 1/5 of a second revealing an emotion a person is trying to conceal (Ekman and Friesen, 1969 1969 ; Ekman, 2001 2001 ; 2003b 2003b ). It expresses one of the six universal emotions: disgust, anger, fear, sadness, happiness, and surprise. This kind of facial expression usually occurs in high stake situations, where people have something valuable to gain or lose (Ekman et al., 1992 1992 ). According to Ekman et al. ( 1992 1992 ), a microexpression is considered to reflect a human’s real intent, especially one of a hostile nature. A microexpression, therefore, can be an essential behavioral clue for lie detection and can be employed as a means of detecting a dangerous demeanor (Metzinger, 2006 2006 ; Schubert, 2006 2006 ; Weinberger, 2010 2010 ). Some extreme actions like terrorist attacks around the world necessitate the use of various technologies to detect dangerous individuals and prevent such actions. Analysis of microexpressions was found to be a suitable approach (Weinberger, 2010 2010 ). Haggard and Isaacs ( 1966 1966 ) first discovered microexpressions which they termed “micromomentary” expressions, and Paul EKMAN then formally named them microexpressions (Ekman and Friesen, 1969 1969 ; Ekman, 2003a 2003a ). Today, there are two main research groups in this field: one led by Paul EKMAN and including Mark G. FRANK and Maureen O′SULLIVAN, and another led by Porter (Porter and ten Brinke, 2008 2008 ). Although few peer-reviewed studies about microexpressions have been published (Porter and ten Brinke, 2008 2008 ; Vrij et al., 2010 2010 ; Matsumoto and Hwang, 2011 2011 ), applications of microexpressions seem to be booming in the USA (Hoffman, 2008 2008 ; Mervis, 2009 2009 ). A microexpression and a macroexpression differ only in their duration (Ekman and Friesen, 1969 1969 ; Ekman, 2001 2001 ). However, there have been at least six estimates of the duration of a microexpression: (1) “A micro expression flashes on and off the face in less than one-quarter of a second” (Ekman, 2001 2001 ); (2) “Microexpressions (that last 1/3 sec. or less)” (Ekman and Rosenberg, 2005 2005 ); (3) “They last for less than half a second” (Frank et al., 2009 2009 ); “Microexpressions are extremely quick facial expressions of emotion that appear on the face for less than 1/2 a s” (Matsumoto and Hwang, 2011 2011 ); (4) “Microexpressions (1/25–1/5 of a second)” (Porter and ten Brinke, 2008 2008 ), and “perhaps expressions last well under a second-perhaps 1/5 to 1/25 of a second” (Ekman and Friesen, 1975 1975 ); (5) “The duration of a microexpression is from 1/3 to 1/25 seconds” (Polikovsky et al., 2010 2010 ); (6) “Participants were shown images for durations in the range of microexpressions (15 ms and 30 ms)” (Clark et al., 2008 2008 ). Thus, there is a lack of consensus about the time range of the duration of a microexpression. The difference in duration might not be large, but for a fleeting thing like a microexpression, the difference should be taken into account. Therefore, the appropriate upper and lower limits of duration of microexpressions should be clarified. For this purpose, we can ask participants in a study to distinguish between microexpressions and macroexpressions, which differ only in terms of duration (the former are short; the latter are long). Supposing that the accuracy of recognizing a microexpression is equal to that of recognizing a macroexpression of longer duration, for the participants there should be no difference between microexpressions and macroexpressions even when they are of different durations (for example, they may think that 1 s happiness is the same as 3 s happiness). Then why bother to use a different name for the same thing (i.e., why not call the happiness of 1 s and 3 s the same macroexpression)? We can use the duration at and beyond which the accuracy of recognition of a microexpression is not significantly different from that of a macroexpression as the critical time point to distinguish a microexpression from a macroexpression, and this duration can be regarded as a proper upper limit of duration of microexpressions. Based on the hypothesis of level of processing (Craik and Lockhart, 1972 1972 ), we can say that the longer is the duration of expression, the deeper is the processing, and therefore the higher is the accuracy of recognition. Thus, according to this hypothesis and the analysis above, for microexpressions of different durations, there should be a turning point after which there will be no significant difference between the recognition accuracy of a microexpression and a macroexpression (where the level of processing for a microexpression is not different from that of a macroexpression). Thus, after that time point, there will be a plateau phase of recognition accuracy in which it would be meaningless to differentiate microexpressions from macroexpressions. Therefore, we take this point as an upper limit for microexpressions. The lower limit of duration of microexpressions should be determined by physiological constraints (how fast the facial muscles can make a complete expression) and the threshold of conscious perception (how long an expression must last for people to consciously perceive it). Ekman and Friesen ( 1974 1974 ) developed a test called the brief affect recognition test (BART), in which one of the six emotions (happiness, disgust, anger, fear, surprise, and sadness) was presented for 1/100th to 1/25th of a second (it was processed subconsciously). In the present study, we employed this paradigm, which we called the BART condition, to measure a person’s ability to recognize brief expressions. According to Matsumoto et al. ( 2000 2000 ), this test has the drawback of producing afterimages that affect judgments and reduce the ecological validity of the test (it is unlike real life because there are no preceding or following expressions). In 2003, Ekman ( 2003c 2003c ) published a microexpression training tool (METT), in which he eliminated these defects by inserting an expression between two neutral expressions. We also employed this paradigm, which we called the METT paradigm. To our knowledge, there has been no empirical research that compares the two paradigms. We think that there should be an empirical test to find out whether they differ significantly. If participants respond identically to the two paradigms (e.g., in terms of the accuracy of recognition), then there is no need to add two neutral images. However, if they respond differently, then it would be important to find out whether the responses differ at all levels of duration of microexpressions or only at shorter levels of duration. And if the latter was the case, then it would be important to determine how short a duration is required to cause a difference in response. In addition, Warren et al. ( 2009 2009 ) noted that there has been no valid study of the METT. Thus, it is necessary to validate the METT paradigm and compare it with a good reliability and validity test such as BART, which can be used as a valid criterion for evaluating METT. Therefore, we employed the two paradigms in our study. However, the main aims of this study were to investigate the effect of the duration of expressions on the recognition ability of fleeting expressions (microexpressions) and to determine a proper upper limit of the duration of microexpressions. Experiment 1 1 To find out how the duration of expressions may affect the recognition of microexpressions, we conducted an experiment in which the duration of exposure to expression pictures was 40, 120, 200, or 300 ms. We employed the two paradigms mentioned above. Materials and methods 2 Participants 3 Eleven healthy university students (seven females, four males, mean age 23.55 years, standard deviation (SD) 1.75 years) attended the experiment and received 20 CNY for their participation. All the participants reported normal or corrected-to-normal vision. None was aware of the purpose of the experiment. Materials and procedure 3 A Lenovo computer with a 17-inch cathode-ray tube (CRT) monitor running at a refresh rate of 85 Hz and the software package E-Prime (version 1.1) were used for stimuli presentation and data collection. The target stimuli were pictures of six basic expressions posed by two human models which were selected from the DFAT-504 database (Kanade et al., 2000 2000 ) and trimmed to 192 pixels×220 pixels. All stimuli were presented on a uniform silver gray background, which remained silver gray throughout the experiment. Before the formal experiment, there were two practice runs. Each run contained six pictures of facial expressions including happiness, disgust, anger, fear, sadness, and surprise. Each picture was presented on the screen for 2 s. Participants were asked to identify each expression by selecting from among the six emotional labels. After giving their answers, participants were given feedback to inform them of the correct answer or to confirm a correct answer. At the end of each practice run, the program reported the total accuracy rate. The program would not continue until the accuracy rate of recognition of microexpressions reached 100%. Errors made by the participants had two sources: first, that the participants did not recognize the expression shown in the picture, or second, that they did not fully process the pictures during the brief exposure time. The practice runs eliminated the first possibility. After recognition practice, two experimental conditions (the BART and METT paradigm conditions) were tested. In the BART condition, a “+” as the fixation appeared at the center of the computer monitor screen for 1 s; then one of the six basic expressions was presented in the center of the screen for 40, 120, 200, or 300 ms. Participants were asked to select one of the six emotional labels using a mouse. Under the condition of the METT paradigm, a neutral expression was first presented in the center of the screen for 2 s. Then one of the six basic expressions flashed briefly for 40, 120, 200, or 300 ms. Exposures of from 40 to 200 ms are deemed to be microexpressions (Ekman, 2003b 2003b ; 2009 2009 ). Finally, the neutral expression was presented for another 2 s. At the end of the presentation the participants had to identify the fleeting expression by selecting one of the six emotional labels. After the practice runs involving recognizing the two models’ expressions, twelve trials were run without feedback on accuracy to familiarize the participants with the formal experimental procedures. Then, the formal experiments were run with the two conditions (BART and METT, blocked). No feedback was given during each formal experiment. Each of the two formal experimental blocks consisted of 384 trials (4 durations×2 models×6 basic emotions×8 replicates). Results 2 A 2 conditions (BART and METT paradigm)×4 durations (40, 120, 200, and 300 ms)×6 expressions (disgust, anger, fear, sadness, happiness, and surprise) repeated measures analysis of variance (ANOVA) was conducted with the participants’ accuracy score (%) entered as the dependent variable. The mean accuracies of the two conditions are shown in Fig. 1 1 . The main effect of condition was not significant [ F (1, 10)=0.962, P =0.350, η p 2 =0.088]. There was a significant main effect for duration [ F (3, 30)=109.027, P < 0.001, η p 2 =0.916]. The main effect of expression type was also significant [ F (5, 50)=8.698, P < 0.001, η p 2 =0.465]. The interactions of condition and duration, and duration and expression type were significant [ F (3, 30)=18.795, P < 0.001, η p 2 =0.653; F (15, 150)=2.076, P =0.014, η p 2 =0.172]. All other effects were not significant. The accuracy of recognition of expressions increased as the duration increased and the curve leveled off after a duration of 200 ms (Fig. 1 1 ). A post-hoc pairwise comparison of the four levels of duration showed that the participants’ performance showed no significant difference only between the durations of 200 and 300 ms ( P =0.444; all other P < 0.01). Furthermore, we tested the effect of condition separately at each duration level to find out the difference between the two conditions. The only significant effect was found at 40 ms duration ( P =0.006, all other P > 0.684). Chance performance is 16.7% (1/6) To investigate the degree to which the participants could recognize the six expressions, a confusion matrix of recognition of the six expressions was calculated (Table 1 1 ), which collapsed across the conditions of BART and METT. For the purpose of exploring the accuracy of each expression at each of the four levels of duration, the detailed accuracy data are given in Table 2 2 . The number in the diagonal line is the mean accuracy rate of recognition of each expression which collapsed across the conditions of BART and METT 2 Expression 6 Mean answer rate (%) Happiness Disgust Anger Fear Surprise Sadness Happiness 86.8 * * 2.1 1.3 7.7 3.3 1.8 Disgust 2.7 69.9 * * 21.4 8.4 4.2 19.0 Anger 1.1 13.4 63.8 * * 2.9 1.1 9.6 Fear 3.6 6.1 4.0 57.0 * * 7.3 5.8 Surprise 5.0 2.8 2.9 13.2 82.7 * * 5.5 Sadness 0.8 5.8 6.5 10.8 1.4 58.2 * * The average numbers are recognition accuracy scores which are averaged by durations (columns) or expressions (rows) 3 Expression 10 Accuracy of recognition (%) 5 BART 5 METT paradigm 40 ms 120 ms 200 ms 300 ms Average * * 40 ms 120 ms 200 ms 300 ms Average * * Happiness 89.3 89.3 93.8 94.9 91.8 69.4 80.9 85.9 91.5 81.9 Disgust 64.8 72.7 72.9 71.6 70.5 57.5 75.1 74.0 71.1 69.4 Anger 60.9 62.0 69.9 67.1 65.0 35.3 64.9 76.3 74.5 62.8 Fear 43.3 55.2 58.2 55.2 53.0 44.0 64.5 67.2 69.5 61.3 Surprise 83.6 85.9 87.1 92.8 87.4 60.3 78.5 85.9 88.1 78.2 Sadness 54.1 63.8 60.4 68.2 61.6 45.0 54.7 55.8 64.8 55.1 Average * * 66.0 71.5 73.7 75.0 51.9 69.8 74.2 76.6 In Table 1 1 , the accuracy of recognition of expressions of each participant was computed by calculating the ratio of the cases in which one facial expression was classified correctly as one of the six expressions and the total number of classifications for that facial expression. Then, the accuracy rates of all participants were averaged to gain the mean accuracy for each expression. For instance, in this experiment the number of times that one participant labeled the facial expression ‘happiness’ correctly (122) was divided by the total number of classifications for that facial expression (128). So, the recognition accuracy for happiness for this participant was 95.3%, and so on. Finally, the accuracy rates of all participants were averaged to give the mean accuracy of 86.8% for ‘happiness’ in Experiment 1. The percentages in the diagonal line are the mean accuracies of recognition for each expression across the two conditions. Discussion 2 This experiment investigated the effects of the duration of expressions on the recognition of microexpressions. In contrast to general “macroexpressions”, the participants’ performance was impaired by the short duration of the microexpressions. As their performance showed no significant difference only after durations of 200 and 300 ms, the duration of 200 ms appeared to be a critical turning point in defining microexpressions. The participants showed confusion between anger and disgust, fear and surprise, and sadness and disgust (Table 1 1 ). Happiness was rarely mistaken for other expressions and therefore was the easiest expression to recognize, while anger and fear were distinguished relatively poorly (Tables 1 1 and 2 2 ). Experiment 2 1 Ekman ( 2009 2009 ) reported that if people had as little as 40 min of microexpression training, they could, on average, improve their performance from about 30% to 40% from the pretest to the posttest in an METT test. In Experiment 2, to test the effects of practice and to find out whether people would still be able to identify microexpressions when the presentation duration was below the lower limit of the duration of microexpressions as defined by Ekman and Friesen ( 1975 1975 ), the participants were required to achieve 100% accuracy in two consecutive runs during a practice session to ensure intensive practice and recognition of expressions presented for 20 ms. Moreover, the experiment aimed to further test the effects of the duration of expressions on the performance of recognition of microexpressions and to find the proper upper limit of microexpressions. Eight levels of duration (20, 40, 80, 120, 160, 200, 240, and 280 ms) were employed in this experiment. Materials and methods 2 Participants 3 Another twelve healthy university students attended the experiments and each received 20 CNY for their participation. Participants (eight females, four males) had a mean age of 22.67 years (SD=3.03 years). All the participants reported normal or corrected-to-normal vision. None reported being aware of the purpose of the experiment. Materials and procedure 3 The materials and the procedures were similar to those used in Experiment 1, except for the following changes. Two different models’ (one male and one female) six basic expressions were used and the practice of expression recognition did not cease until participants had correctly identified all six basic expressions in two consecutive runs (once classification accuracy reached 100%, the program did not proceed to the next run immediately but repeated the previous run to eliminate the possibility of a correct guess). In addition, to make the practice more intensive, a block of mixed recognition practice was added, in which the two models’ expressions were mixed. Only when the participants had achieved 100% recognition of the model’s six expressions in two successive runs would the program advance. In both the BART and METT paradigm conditions, the levels of duration of the six basic expressions were changed to eight levels: 20, 40, 80, 120, 160, 200, 240, and 280 ms. Results 2 The mean accuracies of the two conditions are shown in Fig. 2 2 and Table 3 3 . A 2 conditions×8 durations×6 expressions repeated measures ANOVA was used to analyze the data. Each of the factors and the combination of factors had significant effects. There were significant main effects for condition, duration, and expression type [ F (1, 11)=51.932, P < 0.001, η p 2 =0.825; F (7, 77)=104.416, P < 0.001, η p 2 =0.905; F (5, 55)=13.022, P < 0.001, η p 2 =0.542]. The interactions between condition and duration, condition and expression type, duration and expression type, and the second-order interaction were also significant [ F (7, 77)=69.679, P < 0.001, η p 2 =0.864; F (5, 55)=4.578, P =0.001, η p 2 =0.294; F (35, 385)=4.358, P < 0.001, η p 2 =0.284; F (35, 385)=3.224, P < 0.001, η p 2 =0.227]. The method of pairwise comparisons was used for the eight levels of duration. The results indicated that there were no significant differences between pairs of levels of duration after 160 ms (including 160 ms, all P > 0.455). Furthermore, the simple effect analysis for the effect of duration in the BART condition showed that there were only two significant effects for pairs of 20 and 240 ms ( P =0.024), and 40 and 240 ms ( P =0.003). An analysis of simple effects of duration in the condition of the METT paradigm revealed no significant differences in pairs of duration after 120 ms (all P > 0.147). A simple effect analysis was also employed to investigate the effect of condition within each level of duration. There were significant simple effects before 160 ms (all P < 0.02), and no significant differences at 200, 240, or 280 ms (all P > 0.139). The two curves (BART and METT) were almost superimposed at 200, 240, and 280 ms (Fig. 2 2 ). Chance performance is 16.7% (1/6) The average numbers are recognition accuracy scores which are averaged across durations (columns) or expressions (rows) 2 Expression 9 Accuracy of recognition (%) 20 ms 40 ms 80 ms 120 ms 160 ms 200 ms 240 ms 280 ms Average * * BART  Happiness 98.0 98.0 97.9 99.0 99.0 96.0 100.0 98.0 98.2  Disgust 85.6 93.8 97.9 92.8 97.0 96.9 95.0 95.0 94.3  Anger 73.1 69.9 80.4 85.7 86.6 83.4 86.6 84.6 81.3  Fear 74.1 82.6 82.5 87.7 88.8 86.6 96.0 88.7 85.9  Surprise 99.0 98.0 98.0 96.9 98.0 99.0 100.0 99.0 98.5  Sadness 82.5 81.3 87.8 90.8 95.8 90.8 92.8 93.9 89.5  Average * * 85.4 87.3 90.8 92.2 94.2 92.1 95.1 93.2 10 METT paradigm  Happiness 28.3 66.9 86.7 90.8 91.8 96.9 98.0 100.0 82.4  Disgust 43.0 73.3 91.8 97.0 92.8 97.0 96.9 97.0 86.1  Anger 47.2 49.2 61.7 74.2 75.1 80.3 82.4 79.3 68.7  Fear 20.0 79.5 93.9 91.8 91.8 95.9 95.5 98.0 83.3  Surprise 79.3 99.0 99.0 100.0 99.0 100.0 98.9 100.0 96.9  Sadness 38.7 55.4 68.0 76.3 84.7 86.6 84.4 85.6 72.5  Average * * 42.7 70.5 83.5 88.3 89.2 92.8 92.7 93.3 To explore the effects of practice, we compared the results from the two conditions in Experiments 1 and 2 (Fig. 3 3 ). Under the condition of the METT paradigm, the accuracy rates for the levels of 20 ms (Experiment 2) and 40 ms (Experiments 1 and 2) were compared with the accuracy expected by chance, which was 1/6 (if the participants did not recognize anything, they would by chance obtain the correct answer one in six times simply by guessing one of the six expressions). An independent sample t -test was employed to calculate the values of t for the difference between the means for each of the three accuracy levels and the accuracy expected by chance. The results showed all the comparisons were significant [ t (11)=8.349, P < 0.001; t (10)=5.022, P =0.001; t (11)=20.638, P < 0.001]. (a) BART condition; (b) METT paradigm condition Discussion 2 Again, the duration of expressions had significant effects on the recognition of microexpressions. Furthermore, the participants’ performance improved with practice (Fig. 3 3 ). However, the increasing trends did not differ between the conditions of the two experiments. The results also showed that, with considerable practice, participants could identify microexpressions even when the duration of expressions was below the lower limit of microexpressions as defined by Ekman (40 ms). The results suggest that the perception of human facial expression information is very fast and can be tuned by experience. The pattern of recognition accuracy found in this experiment was similar to that found by Matsumoto et al. ( 2000 2000 ). However, the accuracy of participants in our study was much higher, perhaps because we employed fewer models (two). In general, the expression of surprise was the easiest to recognize, followed by that of happiness. Fear was the most difficult expression to recognize. These results were similar to those of McAndrew ( 1986 1986 ). It could be argued that our experiments employed pictures of small size which resulted in poor performance in recognizing the expressions. However, there is evidence to suggest that image size does not affect the recognition of expressions (Ekman et al., 1979 1979 ). General discussion 1 We explored mainly the upper limit of duration of microexpressions. In both experiments, the duration of the expressions affected the recognition of the microexpressions. Training in recognizing expressions also played an important role in the identification of the microexpressions. The results indicated that there was a turning point at 200 ms in Experiment 1 and at 160 ms in Experiment 2, suggesting that the critical time point differentiating microexpressions from macroexpressions was about 200 ms and most likely less than 200 ms. The results showed that there was a large difference in the accuracy of recognition of microexpressions between the conditions of BART and METT, particularly at short durations (40 ms in Experiment 1 and less than 160 ms in Experiment 2). The reason may lie in the neutral image under the condition of METT, which played the role of a mask, preventing the fleeting expression from remaining on the retina. Thus, for a short duration, the processing of microexpressions could be more impaired by a mask (the neutral images) under the condition of METT paradigm, compared to the condition of BART without a mask. With an increase in duration, the effects of afterimage of a brief expression may decrease (Breitmeyer and Ogmen, 2006 2006 ) and the information of the microexpression may already be transferred to more durable memory storage. Thus, the mask may then have little effect on the processing of microexpressions under the condition of the METT paradigm. Consequently, the performance under the two conditions showed no significant difference when the duration of the microexpressions was longer. The simple effect analysis showed that the two conditions were different only after a short exposure (40 ms) when there had been no extensive practising (in Experiment 1). When the extent of practice was increased (in Experiment 2), the difference between the two conditions persisted until 160 ms, suggesting that adding two neutral images, one before and one after the microexpression, was necessary when the duration was short. Thus, for the purpose of testing the ability to recognize microexpressions with a duration of less than 200 ms, the METT paradigm was appropriate because it was more sensitive to the change of duration which was the only difference between microexpressions and macroexpressions (it also had a wider range of accuracy). The ability to identify microexpressions is related to the detection of deception. The main aim of microexpression studies is to find indicators of deception via microexpressions. Unfortunately, evidence about the link between microexpressions and deception is far from conclusive. However, microexpressions may be the most promising approach to detecting deception (Ekman, 2009 2009 ). According to Weinberger ( 2010 2010 ), the Transportation Security Administration in the USA has already employed a technique called screening passengers by observation techniques (SPOT) which is largely based on the findings from microexpression research. The U.S. Department of Defense has funded David MASUMOTO, a longtime collaborator with Paul EKMAN, in his work on microexpressions, for what he calls “behavior-detection techniques” (Mervis, 2009 2009 ). Now there are at least 3 000 behavior detection officers deployed at 161 airports in USA (Lord, 2010 2010 ). Frank et al. ( 2009 2009 ) pointed out that there is a physiological basis for microexpressions: a pyramidal motor system drives voluntary facial actions and an extrapyramidal motor system drives the more involuntary and emotional facial actions [which can also be viewed as the physiological basis of the Darwin’s inhibition hypothesis (Porter and ten Brinke, 2008 2008 )]. Thus, it is reasonable to regard the analysis of microexpressions as a useful and promising tool for detecting deception. There has been some debate and criticism about microexpression analysis and its applications (Committee on Science, Space and Technology, 2011 2011 ). First, many scientists have argued that microexpressions and their applications have not been subjected to controlled scientific tests (Lord, 2010 2010 ; Vrij et al., 2010 2010 ; Weinberger, 2010 2010 ). Empirical studies are needed to establish the validation for linking microexpressions to deception. Also, the application of microexpressions to airport security is not very effective as less than 1% of suspects have subsequently been arrested (Weinberger, 2010 2010 ). An appropriate attitude to microexpressions may be similar to that proposed by Porter and ten Brinke ( 2010 2010 ), who concluded that nonverbal cues can only assist investigators to detect deception. Many aspects of microexpressions need to be elucidated in the future including the lower limit, individual differences in expression and recognition of microexpressions (Bond and DePaulo, 2008 2008 ; O′Sullivan et al., 2009 2009 ; Wang and Fu, 2009 2009 ; Warren et al., 2009 2009 ), effective methods of training and the retention of training effects (Hurley, 2010 2010 ; Matsumoto and Hwang, 2011 2011 ), and automatic brief facial expression analysis systems (Polikovsky et al., 2010 2010 ; Wu et al., 2010 2010 ). As for individual differences, in groups closely related to deception detection, such as crime interrogators, national security personnel, visa interviewers, sales personnel, negotiators, and mental health professionals, the expression and recognition of microexpressions might not be the same as for other people. While microexpressions have some valuable applications in the field of security, they may also have applications in healthcare and forensic contexts (Porter et al., 2009 2009 ). To use microexpressions widely, it is important to design and develop an automatic brief facial expression analysis system in the field of computer vision to aid trained or untrained people to detect microexpressions and improve hit rates. The questions and debates about microexpressions can be solved by conducting much more empirical research which will fill the gap between knowledge (production features, dynamic features, recognition of microexpressions filmed in real life situations, etc.) and applications (e.g. aviation security), and establish a more stable connection between microexpressions and deception (the main purpose of studies on microexpressions). In conclusion, this study explored the upper limit of duration of microexpressions and had two major findings. Firstly, to some degree, a short training program can improve lay individuals’ ability to recognize microexpressions. Secondly, the accuracy of recognition of microexpressions is a function of the duration of the expressions and reaches a turning point at 200 ms (perhaps less than 200 ms) and then levels off, suggesting that the proper upper limit of duration of microexpressions may be around 1/5 of a second. Much more research is needed to move towards a more complete understanding of microexpressions. References Jr CF Bond , BM DePaulo . Individual differences in judging deception: accuracy and bias . Psychol Bull , 2008 . 134 ( 4 ): 477 - 492 . 18605814 DOI: 10.1037/0033-2909.134.4.477 http://doi.org/10.1037/0033-2909.134.4.477 . Individual differences in judging deception: accuracy and bias B Breitmeyer , H Ogmen . Visual Masking: Time Slices Through Conscious and Unconscious Vision , 2006 . : New York, USA Oxford University Press , . Visual Masking: Time Slices Through Conscious and Unconscious Vision T Clark , P Winkielman , D McIntosh . Autism and the extraction of emotion from briefly presented facial expressions: stumbling at the first step of empathy . Emotion , 2008 . 8 ( 6 ): 803 - 809 . 19102591 DOI: 10.1037/a0014124 http://doi.org/10.1037/a0014124 . Autism and the extraction of emotion from briefly presented facial expressions: stumbling at the first step of empathy Committee on Science, Space and Technology Behavioral Science and Security: Evaluating TSA’s SPOT Program . 2011 . June 1, 2011 . Available from http://democrats.science.house.gov/hearing/behavioral-science-and-security-evaluating-tsa%E2%80%99s-spot-program http://democrats.science.house.gov/hearing/behavioral-science-and-security-evaluating-tsa%E2%80%99s-spot-program . Behavioral Science and Security: Evaluating TSA’s SPOT Program FIM Craik , RS Lockhart . Levels of processing: a framework for memory research . J Verb Learn Verb Behav , 1972 . 11 ( 6 ): 671 - 684 . DOI: 10.1016/S0022-5371(72)80001-X http://doi.org/10.1016/S0022-5371(72)80001-X . Levels of processing: a framework for memory research P Ekman . J Cole . Universals and Cultural Differences in Facial Expressions of Emotion . Nebraska Symposium on Motivation , 1972 . 19 : Lincoln, USA University of Nebraska Press , 207 - 283 . . Universals and Cultural Differences in Facial Expressions of Emotion P Ekman . Telling Lies: Clues to Deceit in the Marketplace, Marriage, and Politics , 2001 . 3rd Ed : New York, USA W.W. Norton , . Telling Lies: Clues to Deceit in the Marketplace, Marriage, and Politics P Ekman . Darwin, deception, and facial expression . Ann N Y Acad Sci , 2003 . 1000 ( 1 ): 205 - 221 . 14766633 DOI: 10.1196/annals.1280.010 http://doi.org/10.1196/annals.1280.010 . Darwin, deception, and facial expression P Ekman . Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life , 2003 . : New York, USA Times Books , . Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life P Ekman . MicroExpression Training Tool (METT) . 2003 . June 1, 2011 . Available from http://www.paulekman.com/micros http://www.paulekman.com/micros . MicroExpression Training Tool (METT) P Ekman . C Martin . Lie Catching and Microexpressions . The Philosophy of Deception , 2009 . : New York, USA Oxford University Press , 118 - 133 . DOI: 10.1093/acprof:oso/9780195327939.003.0008 http://doi.org/10.1093/acprof:oso/9780195327939.003.0008 . Lie Catching and Microexpressions P Ekman , W Friesen . Nonverbal leakage and clues to deception . Psychiatry , 1969 . 32 ( 1 ): 88 - 106 . 5779090 . Nonverbal leakage and clues to deception P Ekman , W Friesen . RJ Friedman , HM Katz . Nonverbal Behavior and Psychopathology . The Psychology of Depression: Contemporary Theory and Research , 1974 . : New York Wiley , 203 - 224 . . Nonverbal Behavior and Psychopathology P Ekman , WV Friesen . Unmasking the Face , 1975 . : New Jersey, USA Prentice Hall , . Unmasking the Face P Ekman , EL Rosenberg . What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS) , 2005 . : New York, USA Oxford University Press , . What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS) P Ekman , K Brattesani , M O′Sullivan , W Friesen . Does image size affect judgments of the face? . J Nonverb Behav , 1979 . 4 ( 1 ): 57 - 61 . DOI: 10.1007/BF00986913 http://doi.org/10.1007/BF00986913 . Does image size affect judgments of the face? P Ekman , E Rolls , D Perrett , H Ellis . Facial expressions of emotion: an old controversy and new findings: discussion . Philos Trans R Soc Lond B Biol Sci , 1992 . 335 ( 1273 ): 63 - 69 . 1348139 DOI: 10.1098/rstb.1992.0008 http://doi.org/10.1098/rstb.1992.0008 . Facial expressions of emotion: an old controversy and new findings: discussion MG Frank , CJ Maccario , V Govindaraju . FXS Paul Seidenstat . Behavior and Security . Protecting Airline Passengers in the Age of Terrorism , 2009 . : Santa Barbara, California Greenwood Pub Group , 86 - 106 . . Behavior and Security EA Haggard , KS Isaacs . LA Gottschalk , AH Auerbach . Micro-momentary Facial Expressions as Indicators of Ego Mechanisms in Psychotherapy . Methods of Research in Psychotherapy , 1966 . : New York Appleton-Century-Crofts , 154 - 165 . . Micro-momentary Facial Expressions as Indicators of Ego Mechanisms in Psychotherapy J Hoffman . Exhibition: how faces share feelings . Nature , 2008 . 452 ( 7186 ): 413 DOI: 10.1038/452413a http://doi.org/10.1038/452413a . Exhibition: how faces share feelings CM Hurley . The Effects of Motivation and Training Format on the Ability to Detect Hidden Emotions , 2010 . : New York, USA State University of New York at Buffalo , PhD Thesis . The Effects of Motivation and Training Format on the Ability to Detect Hidden Emotions T Kanade , Y Tian , J Cohn . Comprehensive Database for Facial Expression Analysis . 2000 . Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG′00) . Grenoble, France 484 - 490 . . Comprehensive Database for Facial Expression Analysis SM Lord . Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges , 2010 . : PA, USA DIANE Publishing Co. , . Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior Detection Program Underway, but Opportunities Exist to Strengthen Validation and Address Operational Challenges D Matsumoto , H Hwang . Evidence for training the ability to read microexpressions of emotion . Motiv Emot , 2011 . 35 ( 2 ): 181 - 191 . DOI: 10.1007/s11031-011-9212-2 http://doi.org/10.1007/s11031-011-9212-2 . Evidence for training the ability to read microexpressions of emotion D Matsumoto , J LeRoux , C Wilson-Cohn , J Raroque , K Kooken , P Ekman , N Yrizarry , S Lovewinger , H Uchida , A Yee , 等 . A new test to measure emotion recognition ability: Matsumoto and Ekman’s Japanese and Caucasian Brief Affect Recognition Test (JACBART) . J Nonverb Behav , 2000 . 24 ( 3 ): 179 - 209 . DOI: 10.1023/A:1006668120583 http://doi.org/10.1023/A:1006668120583 . A new test to measure emotion recognition ability: Matsumoto and Ekman’s Japanese and Caucasian Brief Affect Recognition Test (JACBART) FT McAndrew . A cross-cultural study of recognition thresholds for facial expressions of emotion . J Cross-Cult Psychol , 1986 . 17 ( 2 ): 211 - 224 . DOI: 10.1177/0022002186017002005 http://doi.org/10.1177/0022002186017002005 . A cross-cultural study of recognition thresholds for facial expressions of emotion J Mervis . DOD funds new views on conflict with its first Minerva grants . Science , 2009 . 323 ( 5914 ): 576 - 577 . 19179505 DOI: 10.1126/science.323.5914.576 http://doi.org/10.1126/science.323.5914.576 . DOD funds new views on conflict with its first Minerva grants T Metzinger . Exposing lies . Sci Am Mind , 2006 . 17 ( 5 ): 32 - 37 . DOI: 10.1038/scientificamericanmind1006-32 http://doi.org/10.1038/scientificamericanmind1006-32 . Exposing lies M O′Sullivan , M Frank , C Hurley , J Tiwana . Police lie detection accuracy: the effect of lie scenario . Law Hum Behav , 2009 . 33 ( 6 ): 530 - 538 . DOI: 10.1007/s10979-008-9166-4 http://doi.org/10.1007/s10979-008-9166-4 . Police lie detection accuracy: the effect of lie scenario S Polikovsky , Y Kameda , Y Ohta . Facial Micro-expressions Recognition Using High Speed Camera and 3D-Gradient Descriptor . 2010 . 3rd International Conference on Crime Detection and Prevention (ICDP 2009) . London, UK 1 - 6 . DOI: 10.1049/ic.2009.0244 http://doi.org/10.1049/ic.2009.0244 . Facial Micro-expressions Recognition Using High Speed Camera and 3D-Gradient Descriptor S Porter , L ten Brinke . Reading between the lies . Psychol Sci , 2008 . 19 ( 5 ): 508 - 514 . 18466413 DOI: 10.1111/j.1467-9280.2008.02116.x http://doi.org/10.1111/j.1467-9280.2008.02116.x . Reading between the lies S Porter , L ten Brinke . The truth about lies: what works in detecting high stakes deception? . Leg Criminol Psychol , 2010 . 15 ( 1 ): 57 - 75 . DOI: 10.1348/135532509X433151 http://doi.org/10.1348/135532509X433151 . The truth about lies: what works in detecting high stakes deception? S Porter , M Juodis , L ten Brinke , R Klein , K Wilson . Evaluation of the effectiveness of a brief deception detection training program . J Forensic Psychiatry Psychol , 2009 . 21 ( 1 ): 66 - 76 . DOI: 10.1080/14789940903174246 http://doi.org/10.1080/14789940903174246 . Evaluation of the effectiveness of a brief deception detection training program S Schubert . A look tells all . Sci Am Mind , 2006 . 17 ( 5 ): 26 - 31 . DOI: 10.1038/scientificamericanmind1006-26 http://doi.org/10.1038/scientificamericanmind1006-26 . A look tells all A Vrij , PA Granhag , S Porter . Pitfalls and opportunities in nonverbal and verbal lie detection . Psychol Sci Public Interest , 2010 . 11 ( 3 ): 89 - 121 . DOI: 10.1177/1529100610390861 http://doi.org/10.1177/1529100610390861 . Pitfalls and opportunities in nonverbal and verbal lie detection B Wang , X Fu . Gender difference in the effect of daytime sleep on declarative memory for pictures . J Zhejiang Univ-Sci B , 2009 . 10 ( 7 ): 536 - 546 . 19585672 DOI: 10.1631/jzus.B0820384 http://doi.org/10.1631/jzus.B0820384 . Gender difference in the effect of daytime sleep on declarative memory for pictures G Warren , E Schertler , P Bull . Detecting deception from emotional and unemotional cues . J Nonverb Behav , 2009 . 33 ( 1 ): 59 - 69 . DOI: 10.1007/s10919-008-0057-7 http://doi.org/10.1007/s10919-008-0057-7 . Detecting deception from emotional and unemotional cues S Weinberger . Airport security: intent to deceive? . Nature , 2010 . 465 ( 7297 ): 412 - 415 . 20505706 DOI: 10.1038/465412a http://doi.org/10.1038/465412a . Airport security: intent to deceive? Q Wu , X Shen , X Fu . Micro-expression and its applications . Adv Psychol Sci , 2010 . 18 ( 9 ): 1359 - 1368 . (in Chinese) . Micro-expression and its applications The authors wish to express sincere appreciation to Dr. Bo WANG (Department of Psychology, Central University of Finance and Economics, Beijing, China), Ke-ren LI SHEN, and Christopher KAUSCH (Jacobs University Bremen, Germany) for their assistance with English language editing. 10.1631/jzus.B1100063 Project partially supported by the National Basic Research Program (973) of China (No. 2011CB302201) and the National Natural Science Foundation of China (No. 61075042) 2011-03-02 2012-02-06 2011-08-29 2022-01-05 Copyright © Zhejiang University and Springer-Verlag Berlin Heidelberg 2012 2012 Journal of Zhejiang University-SCIENCE B (Biomedicine and Biotechnology) 03 13