Stephen J. Dubner, Freakonomics co-author, explains why using data to predict future trends is trickier than you may think.
When higher ed leaders discuss big data these days, as well as its potential to help predict future trends and, therefore, courses of action, stories about turkey breasts, hand washing and monkey sex don’t also make the rounds; but according to the Freakonomics author, they should.
“What people say they feel or say they do versus what they actually do are often two completely different things,” explained Stephen J. Dubner, journalist, author and this year’s Infocomm 2016 Las Vegas keynoter. “And this is what makes the use of data tricky whenever industry looks to it for answers in behavior.”
For colleges and universities looking to big data to extrapolate courses of action regarding everything from admissions to learning outcomes, never has the time to invest in its study (data science) or in very skilled data analysts been more critical. Within three very specific anecdotal stories, Dubner reminds everyone that there are four reasons why looking to data for all the answers will be one of the hardest tasks education has ever tackled.
So before higher ed admin and leadership invest time and money in data analysis, be sure the data your institution has collected is first and foremost, accurate.
1. There Will Be Unforeseen Correlations
Trailing what seemed to be a somewhat banal statement that “economists can’t predict everything,” Dubner gave the glaring example of the 2012 recession, saying that “zero percent of economists predicted the Great Recession, otherwise known as their job.”
“The reason for this,” he continued, “is because as we advance in this age and increasingly rely on data to help predict the future, it often can’t. And one of the reasons it often can’t is because there are too many unforeseen variables.”
Dubner gave the example of artificial insemination…for turkeys.
Probably unbeknownst to many, turkeys raised for slaughter no longer procreate. Why? Because they are physically unable.
“As the health movement swept across the country and more people turned to white meat consumption, farmers began raising turkeys that had especially large breasts to produce the demanded white meat. As a consequence, these large-breasted turkeys have chests so large they are unable to procreate, and farmers had to invest in using labs to artificially inseminate female turkeys.”
(Next page: People often lie)
2. People Don’t Always Do What They Say They Do
According to what Dubner has found in his research, data often just shows the best version of people, not how people actually behave.
An interesting example is in hand washing. Dubner related the story of a study conducted in a pediatric hospital in Australia where 73 percent of all doctors self-reported that they do, of course, wash their hands after using the bathroom.
However, when the hospital administration asked nurses to spy on doctors (listening by the door to see if running water sounded after flushing), the nurses reported that only nine percent of doctors wash their hands.
3. Changing Behavior is Extremely Difficult
Hearing about the study mentioned in Australia, Dubner said that Cedar Sinai Medical Center in Los Angeles became concerned that their own doctors may not be washing their hands, and when they also asked nurses to spy on doctors, the nurses reported that only 60 percent washed their hands.
Wanting 100 percent hand-hygiene compliance, the hospital formed a committee charged with reaching this goal.
“They came to the conclusion that it wasn’t a lack of knowledge about hygiene as they were, after all, doctors,” explained Dubner, “so they tried incentives to get the doctors to change. The first incentive was just a memo sent out on behalf of the committee asking doctors to wash their hands. Of course, nothing changed.”
Realizing that, as a rule, positive incentives work better than negative ones, the committee then formed a “posse patrol” that would hide in patients’ rooms during a doctor’s rounds, and if the doctor washed his or her hands after using the bathroom, the posse would spring out, congratulate the doctor, and then present them with a $10 Starbucks gift card.
“I have to admit, as I was studying this experiment, I thought to myself ‘That’s a dumb idea because these doctors make a lot of money. What would they care about a small gift card?’ But you know, that was my mistake, because if there’s one thing you should never underestimate, it’s the power of ‘free.’ I’m sure many of you at Infocomm would literally wait in a long line to get a pair of sweat pants for your dog if it was free.”
Dubner continued the story, saying that though the doctors did respond well to the gift cards, telling the committee they liked the idea, the needle still didn’t move and the rate of hand-hygiene compliance didn’t change.
“What the committee didn’t anticipate was that by gamifying incentives—offering a small reward for small steps to progress—the doctors themselves gamified the idea. So what probably happened is that after they received the card, they knew the posse would no longer be on the floor or rotation, so they began not washing their hands after that. Or perhaps they figured out the posse’s schedule and only washed their hands when they knew the posse was on their round,” he speculated.
The incentive that finally did work? A horrifying visual aid.
Dubner related that it was finally an oft-quiet committee member who suggested that the committee members themselves, composed of hospital leadership, put their hands in a petri-dish to see if the committee washed their own hands. The anonymous results revealed that the committee was just as bad as the doctors.
“The committee, embarrassed at their own hand-hygiene compliance rate, then took a photo of one of the unwashed, bacteria-covered petri dish hands and used that image as a screensaver on every hospital computer as a reminder that everyone should wash their hands. And this is the incentive that worked! The hospital reached 100 percent compliance,” said Dubner.
“It just goes to show you,” he noted, “that incentivizing, predicting what will work and the process of trial & error is a tough road that takes time and patience.
(Next page: The basics of human behavior often elude us)
4. Remember that Behavior is Often Simplistic
When wondering why people behave the way they do for data analysis, it’s probably smart to remember that, at our core, we are just slightly evolved animals.
Dubner’s final tale discussed a study conducted at Yale University by an economist to see how people use currency; specifically, why they save and why they spend.
Training capuchin monkeys to use metal washers as currency over the course of months, the study aimed to see if monkeys used the currency to buy certain foods they liked over others, and whether doubling the price of these much-loved foods would sway the monkeys from purchasing them.
What the researchers didn’t predict was the monkeys use of currency for other primitive desires.
“One day, one of the monkeys was seen handing over a washer to another monkey, and the researcher thought ‘Oh! Is this altruism? Have I just discovered that this is a basic behavior?’ But then the monkey that received the washer began picking the other monkey’s fur…and then have sex,” explained Dubner.
“What the researcher had witnessed was not altruism,” he continued, “but monkey prostitution.”
Dubner concluded his keynote by saying that while using data to predict the future and enable actionable steps sounds like a great idea in theory, it’s always wise to remember that when it comes to behavior, very little is easily predictable.