The basic concept of facial coding is that a trained observer can detect fleeting facial movements that indicate the true emotions that the subject is experiencing, even if the subject is trying to conceal those emotions. I’ve written about the process of interpreting these facial expressions in the past, notably in Facial Coding and Emotionomics. Yesterday, USA Today has picked up on the topic with a major story about the possible applications of different facial coding techniques in choosing investments: (more…)
Articles about 'Facial Coding'
Tue 26 Feb 2008
Digg itFri 14 Dec 2007
Neuromarketing technology is relatively new on the scene, and has been employed primarily by deep-pockets corporate customers. Application to politics has been mostly general and academic; my 2006 piece, The Neuroscience of Political Marketing, discussed research by Emory’s Drew Westen that showed that committed party voters did not process information in a rational or analytical manner. (Not really breaking news…) Now, the massive budgets of the 2008 U.S. presidential election are bringing out the neuromarketers in droves. (more…)
Mon 22 Oct 2007
At a conference presentation last week (see Neuromarketing in Montreal), I made the point that the most important frontier for neuromarketers may be product design. Why struggle to make ads more appealing when you could be making the product itself more appealing by tapping into the consumer’s true feelings and reactions? According to a WIRED.com report by Bryan Gardiner, it looks like frog design’s Harmut Esslinger and other mainstream designers might concur. (more…)
Wed 26 Sep 2007
A few weeks ago, WIRED published an interesting story on the massive amount of testing that has gone into producing Halo 3. The biggest part of this has been usability testing to ensure that the game is continuously playable. By eliminating places where most players get hopelessly lost, or killed many times in a row, they reduce frustration with the game and keep players engaged. Some of the data is from observing players through one-way mirrors, some from data collection (mapping player routes, aggregating data from many players into scattergrams, etc.), and some from self-reporting by the game players themselves: (more…)
Wed 15 Aug 2007
When we wrote our recent review of Emotionomics: Winning Hearts and Minds by Dan Hill, our interest in facial coding was sparked. Or, perhaps, re-sparked; when we read Malcolm Gladwell’s Blink, we found his discussion of facial coding to be quite intriguing. Simply put, the concept of facial coding is that we reveal our emotions by our facial expressions. Particularly in some social settings, we may seek to conceal these, but the underlying emotion we experience still registers on our face, even as a brief micro-expression. So, when your mother-in-law passes you an unpalatable slab of mystery meat, you may smile and say, “Thanks, it looks delicious!” Facial coding experts, though, say you would probably register a brief disgust reaction even as you take pains to avoid being rude.
If this is true, it would be handy information for marketers and market researchers. Surveys and focus groups often fail to accurately predict the success of an ad or a product in the marketplace. Being able to gauge the true reaction of individuals would be extremely useful, to say the least. That’s what Dan Hill says his firm, Sensory Logic, actually does - analyze fleeting facial expressions while subjects view products, print ads, television commercials, and so on, in order to determine their real reaction.
But does facial coding actually work? Let’s start by looking at the history of the topic. Believe it or not, the concept that we reveal our true emotions in our facial expressions was put forth by none other than Charles Darwin. The famed evolutionist wrote The Expression of the Emotions in Man and Animals in 1872, and suggested that mammals revealed their emotions in their faces. The topic didn’t get a lot of attention until the 1960s, when Paul Ekman, now professor emeritus of psychology at the University of California, San Francisco, began travelling to other nations to see if at least some facial expressions were universal across geography and cultures. He found that individuals from distant cultures did correctly interpret expressions on photos from other groups.
Eventually, Ekman and Wallace Friesen, also of UCSF, created the Facial Action Coding System (FACS). Published in 1978, FACS was based on combinations of 43 unique facial muscle positions which yielded thousands of expressions. Ekman also noticed that sometimes emotions were fleeting, resulting in “microexpressions.” These were more readily observed when a slow-motion tape of a subject’s face was played back, but a trained individual could also observe them in real time. Since that time, facial coding has hardly taken off as an academic discipline. It seems that more interest has come from law enforcement and, more recently, anti-terrorism agencies. Interrogators have been trained to identify and interpret the expressions of subjects being interviewed, principally to determine if the subject is lying.
Over the years, Ekman and fellow researchers have further extended their work on facial coding, and even produced tools to help others develop facial coding skills (see links below).
Neuromarketing readers are probably most interested in the marketing applications of this technique, and it seems the main proponent of marketing use is Dan Hill, author of the aforementioned Emotionomics. Hill’s firm, Sensory Logic, Inc., uses facial coding and related techniques to analyze customer reactions to advertisements, products, etc. The book describes many real-life example of how the firm has used facial coding to choose between ad versions, determine why ad campaigns failed to perform as expected, evaluate reactions to proposed product features, etc.
Despite the apparently extensive body of work performed by Sensory Logic, facial coding hasn’t yet become widely accepted. Like fMRI scans and other neuromarketing technologies, facial coding seems to be still on the fringe of commonly used and accepted marketing practices. If Hill’s new book becomes popular, perhaps that will change.
Facial Coding Resources
Scientific American Mind: A Look Tells All [ SPECIAL REPORT: THE BODY SPEAKS ]
A person’s face will always reveal his true feelings–if, like Paul Ekman, you are quick enough to recognize microexpressions
The Naked Face
Subtitled “Can you read people’s thoughts just by looking at them?”, this Malcom Gladwell article appeared in The New Yorker
Sensory Logic
Firm uses facial coding techniques in market research.
Emotionomics
Our review of Emotionomics: Winning Hearts and Minds by Dan Hill.
Paul Ekman
Website of Paul Ekman, professor emeritus at UCSF and leader in facial coding research.
Facial Coding Instruction
PC software that claims to teach one to recognize microexpressions and facial muscle patterns indicating emotions.
DataFace Website
Site has resources related to understanding facial coding and the Facial Action Coding System (FACS).
Sun 12 Aug 2007
Emotionomics
Posted by Roger Dooley under Neuromarketing , Neuroscience and Marketing BooksNo Comments
Emotionomics: Winning Hearts and Minds
Hill spends an early chapter on how facial coding works that includes illustrations of the “seven core emotions” revealed by facial expressions: surprise, fear, anger, disgust, sadness, contempt, and happiness. He also notes that happiness is demonstrated by true smiles and social smiles - the latter involve only the mouth and may indicate that the subject is lying. Hill’s approach evaluates advertising in two dimensions: Impact and Appeal. Impact is the intensity of the emotion(s) exhibited, while Appeal is a measure of the positive or negative aspect of the emotion. In addition to the Impact/Appeal divide, Hill also uses a time dimension in evaluating advertisements like television commercials where emotions can change as the ad progresses. In some cases, eye tracking data may be used to determine what portion of an ad is causing the emotional response.
Hill describes what he calls the Emotionomics Matrix, a sort of wheel that includes what he considers the most important human motivations: Defend at the center, with Acquire, Learn, and Bond around the perimeter. Emotions such as anger, happiness, and disgust are placed at various points in and around the wheel. Hill refers to this through much of the book, which may be a bit off-putting for readers who don’t necessarily buy into that simple breakdown of human motivation and emotion. Still, it provides a framework for the analysis that makes up much of the book.
The best parts of Emotionomics are the real-world examples of facial coding applications. One such example is from the automotive industry, where an automaker had Hill’s firm evaluate a major national print ad campaign that included an apology for previous quality problems. Overall, the campaign was viewed negatively by about 80% of the subjects. Even three quarters of current owners of the product reacted negatively – perhaps not surprising, since the ad more or less suggests that the current owners were duped into buying a piece of junk. Another ad from a pharmaceutical company under fire for product safety was evaluated; one variation of the ad was defensive on the safety topic, while the other was more upbeat. While the verbal responses from subjects viewing the two ads didn’t vary much, facial coding analysis showed a much more negative response to the ad, particularly among a group of subjects supportive of the brand. The dramatic difference in the facial coding response convinced the firm to shelf the defensive ad and run only the upbeat version.
In another example, an unnamed appliance maker showed subjects a new feature being considered for addition to an appliance. The subjects responded enthusiastically to the survey questions: twice as many saw more of an upside than saw a downside to the feature, and only 5% were concerned about the feature being something else to “go wrong” later. Facial coding analysis, though, determined that 79% of the subjects had a negative emotional reaction to the feature, making it a doubtful addition indeed.
Facial coding is another window into the black box of the human mind. (Many readers were introduced to the idea by Malcolm Gladwell in his best-selling Blink; for a lengthy and interesting article by Gladwell on the topic, check out The Naked Face.) Neuroeconomics and neuromarketing researchers use complex and costly fMRI machines as another window. Other technologies, such as EEGs, offer yet another perspective. It’s likely that each approach will yield different insights. One advantage of the facial coding approach is that it doesn’t require complex, noisy machinery or that the subject be immobile, wear a helmet or sensors, etc. Theoretically, at least, this should result in more natural reactions. Oddly, despite the length of time that Ekman and a few others have been conducting research and that Hill’s firm has been using it, facial coding has not caught on in a big way, even as an academic research topic. Various law enforcement agencies seem to be the most enthusiastic adopters of facial coding; they teach the technique to interviewers to help them identify deception and better understand the individual being questioned.
There are a plethora of examples, we’d like to see more statistically significant A/B testing. While the anecdotal data from these many efforts is compelling, it would be more so if we could directly correlate the difference in observed expressions with varied performance in the marketplace. Of course, most of Hill’s customers want to test their marketing before they spend the money, and most types of marketing don’t lend themselves to precise A/B testing.
Emotionomics is lavishly illustrated. Some of the graphics are mostly eye candy supporting some phrase in the text, but there are plenty of pie charts and bar graphs for those who like to see their numbers in graphic rather than tabular form. The profusion of illustrations keeps the pages turning quickly. We found the closing section on Leadership and Employee Management a bit superfluous, but readers who have built up their enthusiasm for Hill’s approach through the rest of the book may find them interesting. Emotionomics is longer on pop psychology than neuroscience, though that doesn’t necessarily invalidate the conclusions reached from analyzing advertisements.
Overall, we can hardly argue with the basic premise of Emotionomics: people react emotionally to advertising and products, and trying to get them to accurately describe their feelings, explain their behavior, or predict future actions, with forms, surveys, and focus groups is a fool’s errand. The large number of examples illustrating the divide between stated and actual response to an ad or product, and the convincing explanations of why some products and ad campaigns worked and some didn’t, make Emotionomics a must-read for any marketer wanting a different set of insights into customer behavior.
Subscribe
