Catchy Headlines, Bogus Data

HeadlineAs perfect proof of its point, a link to an article titled A Catchy Headline The Biggest Draw For News Article Readers induced me to click and read it. The article noted that a new study by Harris Interactive showed that catchy headlines topped interesting visuals as a reason to read an article. This is from the Harris release:

A catchy headline is the top influencer on Americans’ likelihood to read an online or print article in full (54%), though the inclusion of interesting pictures (44%) and interesting data or research (43%) are also strong lures… News junkies are less likely to be lured into reading an article by the presence of an interesting picture (32% junkies, 46% moderate, 41% not interested)… Females (58%) are more likely than males (50%) to be drawn in by a catchy headline, while males (47%) are more likely than females (40%) to be attracted by interesting data or research.

This sounds like great news for headline writers. And the male vs. female divide is great, actionable information for content publishers aiming at a particular demographic, right? Well, maybe not.

If Only We Knew…

Harris is a huge research firm. They employ over 700 people, many of whom no doubt hold impressive credentials in statistics and other relevant disciplines. I have no doubt that the above data, gathered from over 2,300 survey participants, was collected and analyzed using the best practices for such research.

There’s only one problem. This lovely, detailed data is bogus, despite the large sample size and low calculated margin of error.

What’s wrong with this data? It’s simple. Asking people why they do things is terribly unreliable. Our behavior as humans is influenced by many, many factors, most of which aren’t conscious or rational.

Rationalize This

Even when do things for reasons we aren’t aware of or can’t articulate, we are quick to explain why we did them in rational, logical terms. Indeed, neuroscientist Michael Gazzaniga says we all have something called “the interpreter” that incorporates our actions into a logical and consistent framework. Split-brain research as shown that people have a remarkable ability to fabricate explanations for their actions that seem logical but have no basis in reality. (See How “The Interpreter” Screws Up Market Research for more on this topic.)

No Way, Not Me!

Asking someone whether a picture would induce them to read an article is a perfect example – for many of us, our logical mind will say, “Of course I wouldn’t be so shallow as to read an article based on a picture. It’s the quality of the news. And, yes, a well-written headline can draw me in.”

Get The Real Story

If you want to get the real story on the behavior of your customers, readers, etc., don’t rely on self-reported data. While such data can be fine for simple facts, like, “Did you eat breakfast today?” it will rarely answer questions like, “Why do you prefer Grey Goose vodka?”

While neuromarketing studies are one way to get at real feelings, one simple solution is to measure actual behavior. Don’t ask people if they are more likely to click on a link accompanied by a picture – test it! Today, it’s trivially simple to deliver different content versions to web visitors and measure what really does work. Instead of asking whether customers would buy the product if it was 10% cheaper, try out the lower price on a sample of real customers and measure the boost in response.

In short: worry less about what your customers SAY, and more about what they DO.

Have you encountered research data that was statistically accurate but provided misleading cues about your customers’ behavior or intentions? Share it in a comment!

email

This post was written by:

— who has written 985 posts on Neuromarketing.

Roger Dooley writes and speaks about marketing, and in particular the use of neuroscience and behavioral research to make advertising, marketing, and products better. He is the primary author at Neuromarketing, and founder of Dooley Direct LLC, a marketing consultancy. Follow him on Twitter.

Contact the author

Brainfluence: 100 Ways to Persuade and Convince Consumers with Neuromarketing Get 100 amazing brain-based marketing strategies! Brainfluence is recommended for any size business, even startups and nonprofits!
Guy KawasakiRead this book to learn even more ways to change people's hearts, minds, and actions.   — Guy Kawasaki, author of Enchantment and former chief evangelist of Apple
Brainfluence Info

{

29 responses to "Catchy Headlines, Bogus Data" — Your Turn

}

Gregory Ciotti 2. October 2012 at 11:09 am

Great points Roger.

I will mention that Eyetrack III (http://www.poynter.org/uncategorized/24963/eyetrack-iii-what-news-websites-look-like-through-readers-eyes/) has pointed out that headlines *do* seem to at least draw a more immediate (and longer lasting) gaze from readers on news sites.

Reply

Roger Dooley
Twitter: rogerdooley
2. October 2012 at 11:40 am

Good point, Gregory. I could have been clearer in the original post – I have nothing against headlines, and consider a well-written headline to be a great inducement to click and read. But, I don’t think asking people “why did you click?” is going to give accurate results in every case. Eye tracking, as you mention, is one way to measure actual behavior vs. relying on self-reporting.

Roger

Reply

Bram de Haas
Twitter: bramdehaas
2. October 2012 at 12:07 pm

Great article, it’s so easy to base conclusions on data presented to you but so often the real story is quite diffferent then the obvious conclusion that is ready to be drawn. From my time as a newspaper journalist student I know that both visuals and headlines are incredibly important and when you have the options you should always be using both to attract more clickthroughs.

Reply

Roger Dooley
Twitter: rogerdooley
2. October 2012 at 7:09 pm

I agree about visuals AND headlines as being important, Bram. And it’s entirely possible that headlines ARE the most important factor, but not because readers say so.

Roger

Reply

Gibron Williams
Twitter: oevae
2. October 2012 at 12:08 pm

Test your customer response in all seasons (spring, summer, winter, fall) and paying close attention to where the product or service lies (hype, advantage, choice, cost, replacement) within its useful market life. Offering customers the iPhone G4 two weeks after the release of the iPhone G5, may be beneficial to some and pointless to others. At Oevae we help our clients research the market, as we’ve found many small / midsize companies have less time to evaluate / test markets than they do managing their products and services. So as Roger makes his point very clear, testing is the real identifier for what your customers do and want.

Head Honcho

Reply

Gary 2. October 2012 at 12:16 pm

Roger, testing whether someone clicks on a link is certainly simple enough, but how do you connect it with segmentation data like gender and “news junkie”?

Reply

Roger Dooley
Twitter: rogerdooley
2. October 2012 at 7:12 pm

Gary, that is a complicated question. Or at least it requires a complex answer! In some cases, you may not know. If you are dealing with logged in users, you may have a wealth of data about them. Or, even without a login, you can track user behavior on your site. Other resources may also be able to deliver data about a user based on third party cookies.

Roger

Reply

Stephen Palmer 2. October 2012 at 1:43 pm

Roger, good catch, you are right on the money.

This is also why it’s a flawed methodology to ask customers how they found a business, giving options (online, referral, yellow pages, radio ad, etc.).

People simply don’t remember accurately. They can say they heard about a company from radio ads, and the company doesn’t even do radio ads.

I see web forms like this a lot, and not only does it create unnecessary friction for website users, but the data can’t even be trusted.

Reply

David Murray 2. October 2012 at 4:10 pm

Another good article and like the other writers, I agree that asking people why they do something never gives good results. I think AB split testing is the way to go.

That’s one of the problems with traditional consumer research. Remember New Coke? The Coca Cola company spent years and millions of dollars testing and asking people what they wanted. They then gave them something they said they wanted and all hell broke loose. The vast majority of our decisions are unconscious left brain ones and when asked to give a logical explanation, our right brain takes over. Of course it doesn’t know anything so it tries to come up with a logical explanation.

Reply

Harmony Major
Twitter: PrezNC
2. October 2012 at 7:01 pm

Absolutely spot-on.

As delicious as your articles are, I was fully prepared to disagree as I clicked over — full-on skeptic mode — because I thought the article was going to state that authors using the catchiest headlines present bogus data (with them). But here, again, you’ve nailed it.

I wrote an article years ago about product research, wherein I stated,

“Don’t ask subscribers — we lie.”

Not intentionally — well, some do sometimes… in order to sway results, i.e., pricing in their favor — but usually, simply because we don’t know.

While not taking away from the value and power of informed deduction, (necessary, for instance, when evaluating split-test results in marketing), I really appreciate the validity of your opinion here — and the recommendation to pay closest attention to what one’s audience DOES… as that often differs from what they speak.

Reply

Roger Dooley
Twitter: rogerdooley
2. October 2012 at 7:13 pm

At least my headline got YOUR attention, Harmony!

Roger

Reply

Walter W. 3. October 2012 at 2:41 pm
Roger Dooley
Twitter: rogerdooley
3. October 2012 at 8:35 pm

Old news, but still relevant, Walter: Are Brain Scan Findings Fishy?.

Reply

Vadim 3. October 2012 at 6:17 pm

Great read. Our motto has always been, “only pay attention to what you can measure.” This lead us to make more accurate decisions about how to redesign our website. One great “secret” tool we found is crazyegg.com. If you have a chance, take a look at it.

Reply

Robert Clarke
Twitter: opedmarketing
4. October 2012 at 10:01 am

After reading John Caples “Test Adverting Methods”, I wrote a blog on this very subject:

http://op-ed.ca/2012/09/24/the-most-important-overlooked-and-easiest-way-to-get-more-traffic-to-your-blog/

Some human behaviour doesn’t change, Mr. Caples wrote his book 50 years ago :)

Reply

Roger Dooley
Twitter: rogerdooley
4. October 2012 at 10:17 am

Hard to argue with Caples!

Roger

Reply

Suzanne
Twitter: prosperouscoach
7. October 2012 at 5:02 pm

I am a big believer in visual marketing and the affect images have on the subconscious mind when it comes to a prospect taking action. I don’t have precise proof as I’ve only tested my theory on my own landing pages. One thing is for sure, colorful, visual call to actions and creative design work to encourage someone to buy a product. I totally agree that asking somewhat what they might do and what they actually do are two different things.

Reply

Michele Bergh
Twitter: beinspired1
8. October 2012 at 2:13 pm

Great article and so true. That’s why, large companies, when doing research around their target market, often use techniques that speak in terms of metaphors rather than just having them answer questions. You can get at a much deeper “true” meaning that way.

Reply

Harmony Major
Twitter: PrezNC
8. October 2012 at 5:37 pm

The idea of “metaphorically asking” is so interesting. :)

Reply

Ameenah
Twitter: TheGameScouts
8. October 2012 at 6:24 pm

Thank you for clarifying that the quality of information when quoting statistical is undeniably important. Self-report is easily the most superficial type of survey. You’re very right to say that it makes a lot more sense to pay more attention to stats collected by comparing actual activity (like clicks or hits.) Great read!

Reply

jen 9. October 2012 at 3:19 am

Great article. Totally spot on. The power of a great headline can not be emphasized enough. Every advertising agency, newspaper and marketing department knows this unequivocally.

Reply

Ana Hoffman
Twitter: AnaTrafficCafe
9. October 2012 at 10:16 pm

I can definitely see where you are going with this, Roger.

If I had to answer whether I was more moved by a headline or an image, I am not sure I’d have a good answer.

I suppose hitting all the points would be a good solution.

Reply

Ben Drake
Twitter: BenDrake_
10. October 2012 at 11:06 pm

Asking users to explain an action void of conscious reasoning or decision making results in nothing but b.s. “Well I hadn’t really thought about why I clicked on this article, but the half naked woman in the picture might have had something to do with it” isn’t going to be a very popular answer on your poll. No matter how true it is. Must have been that catchy title…

The only way to find out what people will do is to watch. Figuring out why has been the goal of everyone, from psychologists to marketers to angry parents. Those who do will be those who influence.

Reply

Roger Dooley
Twitter: rogerdooley
11. October 2012 at 4:48 pm

Exactly, Ben. People are even less likely to answer honestly if the response might make them look bad. In truth, of course, they often just don’t know.

Roger

Reply

Pep 12. October 2012 at 2:19 am

Yeah, right. But you completely used a “catchy headline” to make me read this post. And I didn’t find anything about catchy headlines…

Reply

Ben Drake
Twitter: BenDrake_
12. October 2012 at 10:58 am

That’s because the post wasn’t about headlines. It was about the use of bad data to justify a branding technique of using catchy headlines rather than catchy graphics…

Reply

Eddie Gear 12. October 2012 at 11:26 pm

Very True Roger, These day, people just about write anything just because they can get to people to read their articles. This effects the reputation of the bloggers and the businesses.

Reply

Sarah 15. October 2012 at 3:44 pm

I was so excited when a big report on why Coloradans donate came out – until I learned about their methodology. It’s exactly as you describe – they used a phone survey!

http://www.coloradononprofits.org/wp-content/uploads/understandinggiving.pdf

Reply

Caimin
Twitter: GeniusStartup
25. October 2012 at 9:49 am

Great.

Now I have to resurrect my swipe file of waltzing cat pictures.

Darn you, Roger!

Reply

Leave a Reply