I love how much my dogs think they'll like Indian
You may have seen some headlines recently stating that 80% of Americans support mandatory labeling of foods containing DNA. The Washington Post has an article up about a recent survey that produced these results, and the internet has been laughing uproariously ever since.
Unfortunately for those who love laughing at people who are stupid, this is actually a classic example of bad study design that is somewhat manipulative. You would think that a survey would be pretty simple to lay out. Ask the question, collect the results. Try to get a large sample size and aim for diversity within your sample.
The place where this falls apart is in how questions are asked. In this example, the question wasn’t a simple one off, it was part of a list laid out as follows:
Do you support or oppose the following government policies?
A tax on sugared sodas.
A ban on the sale of marijuana
A ban on the sale of food products made with trans fat
A ban on the sale of raw, unpasteurized milk.
Calorie limits for school lunches
Mandatory calorie labels on restaurant menus
Mandatory labels on foods containing DNA.
Mandatory labels on foods produced with genetic engineering
A requirement that school lunches must contain two servings of fruits and vegetables
Mandatory country of origin labels for meat.
This is a classic case of priming. Priming is the unconscious way that a stimulus can affect later behavior, in this case the other statements that surround the question about DNA influencing whether people believe the statement is reasonable. The outlandish statement is buried in between a number of actually controversial issues, which means that most people reading it will not pay close attention and assume that it is an issue they are familiar with, most likely GMO labeling. Even the creator of the survey himself doesn’t believe these results show clearly that Americans are scientifically illiterate:
“I do not interpret answers to these sorts of questions as necessarily reflecting some sort of deeply held beliefs, but rather they often represent quick, gut reactions. As the ballot initiatives in California, Colorado, Oregon, and Washington have shown, mandatory GMO labels initially poll at very high levels (at levels similar to what we found in the recent survey), but in all four of those states, labeling failed to garner 50% of the vote. Clearly, many people’s views about mandatory GMO labeling are not fixed constructs, but are (at least at this point) somewhat malleable and are open to education and persuasion.” –Jayson Lusk
Ben Lillie explains in detail how the participants are most likely to assume that the question doesn’t mean DNA, but rather modified DNA or altered DNA. It’s actually incredibly difficult to build a study effectively and clearly, which means statistics should always be taken with a grain of salt.