Evolved or Created? It depends on how you ask...

[Quintessence]

When it comes to learning, asking questions is an important component of the process. Asking questions is so important that how questions are asked can impact what you end up learning about something. When researching things like social trends and public opinion, being mindful of how questions are asked is critical to gathering good data. If you don't ask the right questions, you'll end up with data that doesn't accurately represent the population you are surveying. When it comes to asking about religious topics, how things are framed can become especially important.

There's an interesting case in point for this when it comes to asking about acceptance of biological evolution. All to often the discussion of the topic is oversimplified into black-and-white "creationist" and "evolutionist" categories. "Theistic evolution," however, is also a thing. When people are asked about creation and evolution in a way that enables them to immediately express their thoughts about the role of god(s) in this process, responses shift pretty dramatically:

From: Exploring Different Ways of Asking About Evolution

This difference is more pronounced among certain demographics than others, and you can read about that here (How highly religious Americans view evolution depends on how they’re asked about it). The short of it is if we start by asking "did humans evolve over time (evolutionism)" versus "humans have exited in their present form since the beginning of time (creationism)," we will overestimate the number of creationists and underestimate the number of theistic evolutionists.

Have you noticed you might respond differently when asked about creation and evolution in different ways? What other possible improvements could be made to asking about this topic for research purposes?

 

[Subduction Zone]

Pollsters have known for quite some time that how one asks a question can affect the polled results. I do like the three part question since it allows people to respond as to what they believe with less fear of going against their religious beliefs. I have also referred Christians to Biologos, a Christian science based site that accepts evolution, though sadly too often they are taken to be "not real Christians" .

 

[Altfish]

It doesn't affect the correct answer though.

 

[sun rise]

How one asks a question also parallels how bias is shown in news reporting. Comparing the headlines between Fox and MSNBC, for example, shows that how a story is reported reflects the bias of the outlet.

 

[Polymath257]

So which one represents the 'real' views of people? There is a tendency to say that the one-question form does, but do we know that? how would we test it?

I find this as being more information about how the beliefs of most people are context-dependent and not very solid.

 

[Jose Fly]

So which one represents the 'real' views of people? There is a tendency to say that the one-question form does, but do we know that? how would we test it?

I find this as being more information about how the beliefs of most people are context-dependent and not very solid.

I think for most people it's not really much of an issue, and likely isn't something they've spent a lot of time thinking about. So perhaps that's the reason there's a bit of ambiguity and variability in the survey results.

 

[Brickjectivity]

There's an interesting case in point for this when it comes to asking about acceptance of biological evolution. All to often the discussion of the topic is oversimplified into black-and-white "creationist" and "evolutionist" categories

I think you don't have a hope of answering this question, Q.

If collectors are asking questions they will sometimes suggest answers. What would you do if you could either get 5 data points or 2 depending on whether you could nudge someone into answering? Just a tiny nudge to help them decide faster. If data collection involves phone calls, the representatives will lie both to you and to the people. If it involves grabbing passing strangers, the requirements will be fudged sometimes to get more people in. The statistical significance of things will always be thrown off. You have to ask from top to bottom "Who will want to alter or lie about the results?"

Surveys inform respondents, especially multiple choice ones. They don't just ask questions. People are always curious about the surveys, and there is a fun factor. "What if I do this?" They will first of all try to guess at who wants the data and for what purpose, but the surveyor almost never wants them to know this unless they seek skewed data.

The people who manage data collection always are under pressure to get results, and they pass that pressure 'Down' to the peons who pass it on down to their peons. Once I was pulled aside by some college age surveyors, and they fudged my age by 2 years in order to allow me into their results. I've also done phone calls for surveys and noted other reps claiming more authority on the phone than they actually had (merely by inflection and leaving things out or by causing people to infer authority), so that people wouldn't hang up as often. There's a huge difference between someone who wants to help and someone who is worried about not helping. None of this was according to procedure, so procedure despite monitoring, careful management, training, selection and all kinds of safeguards got flubbed marginally and why? Its because procedures sooner or later are inconvenient.

If its a paper survey lies will come from the respondents rather than from the data collectors you hope. Who will gather and protect the data? Will it be temps, and then will the data will be passed on to the management for approval? Even if data transmission is a secure process there will be ways that management can influence the data simply by their desire for it to be collected in a timely fashion.

Have you noticed you might respond differently when asked about creation and evolution in different ways? What other possible improvements could be made to asking about this topic for research purposes?

I will try to keep this short, so try not to get bored.

These surveys do not take into account my point of view, and that is a problem for you. I automatically feel like I am not being represented by them; but I also don't like answering 'Other view' or 'No answer' which makes me feel unknown and unimportant. What I want is to add to your survey, but I know that is not going to happen. If you asked me these questions in 1987 (when I was 10) I would have answered as a young earth creationist. If you asked in 1992 I would have answered as someone who believed that Genesis was scientific but represented days as longer periods of time. I would by then be credulous about the lies and obfuscation of ICR but still swayed partially and trying to fit the Bible into biological Science. Then if you asked me in 2009 I would have answered simply that I accepted evolution but that my views on God were changing. They still are.

Overall these two surveys err in calling people to choose a position without acknowledging that their point of view can change. This alienates those on the edge. They try to gather a spectrum of views of people whose views are unchanging. They do not ask "Has your view on this ever changed?" "Could it ever change?" "What was your most recent view that is different from what you think now?"

 

[Quintessence]

Pollsters have known for quite some time that how one asks a question can affect the polled results. I do like the three part question since it allows people to respond as to what they believe with less fear of going against their religious beliefs.

Yup, that's precisely what it helps with. It also helps avoid oversimplifying the complexity of the issue. You're also right that the impact of question phrasing has been known for quite some time, but it does well to remind ourselves about this from time to time.

What I also found interesting is that asking the question differently impacts some demographics more than others. For example, for those who identify as 'unaffiliated' or 'white mainline Protestant' religiously, how the question is asked doesn't matter. But for 'white evangelical Protestant,' 'black Protestant' and 'Catholic' groups, it does make a difference.

It doesn't affect the correct answer though.

I'm not sure why you raise this point. This is true of asking about anyone's opinions on any topic. This is not a thread to debate what the "correct" answer to the question is.

How one asks a question also parallels how bias is shown in news reporting. Comparing the headlines between Fox and MSNBC, for example, shows that how a story is reported reflects the bias of the outlet.

In most cases it is even worse for news reporting because they do not do their polls in a scientific manner. They don't attempt to get representative samples.

So which one represents the 'real' views of people? There is a tendency to say that the one-question form does, but do we know that? how would we test it?

I find this as being more information about how the beliefs of most people are context-dependent and not very solid.

In a fashion, both ways of asking the question reflects the 'real' views of people. If asked to oversimplify, someone's 'real' views may go one way but go another when an additional component of complexity (say, god having a role in our example) is added into consideration. I could see this as being taken as evidence that beliefs are often malleable, but I also think that complexity is another big factor here. Our thoughts and feelings about things probably run deeper than what can be asked about in a sentence or answered with stock responses. Gods know I feel that way constantly when I've taken surveys.

I think for most people it's not really much of an issue, and likely isn't something they've spent a lot of time thinking about. So perhaps that's the reason there's a bit of ambiguity and variability in the survey results.

Maybe. Stories about origins are culturally pervasive though, so I'd wager most folks in our culture have spent a fair amount of time considering the topic. Those who haven't would be exceptions to the general rule.

 

[Quintessence]

Overall these two surveys err in calling people to choose a position without acknowledging that their point of view can change. This alienates those on the edge. They try to gather a spectrum of views of people whose views are unchanging. They do not ask "Has your view on this ever changed?" "Could it ever change?" "What was your most recent view that is different from what you think now?"

Sometimes there will be questions about that - it depends on what the scope of the study is. The organization that did this research has looked into the changing religious landscape on various occasions, but I don't think that was really part of this particular line of inquiry. It was more of a "gee, let's try these two different methods so we can better frame our questions when we ask about this for future research" type of study. A methods study, in other words.

 

[Jose Fly]

Maybe. Stories about origins are culturally pervasive though, so I'd wager most folks in our culture have spent a fair amount of time considering the topic. Those who haven't would be exceptions to the general rule.

Well, that's kinda the point. They've "considered it" from a cultural/religious perspective, but not from a scientific one. I'd bet that 95-99% of the people they've asked have little to no idea what's in the fossil record or anything about comparative genomics between humans and other primates.

IOW, the survey can tell us what people believe, but it isn't telling us what they know.

 

[Jumi]

This questionnaire in the US? I think if asked in Europe, theistic evolutionists would probably score even higher and there wouldn't be that many US style creationists...

 

[Quintessence]

Well, that's kinda the point. They've "considered it" from a cultural/religious perspective, but not from a scientific one. I'd bet that 95-99% of the people they've asked have little to no idea what's in the fossil record or anything about comparative genomics between humans and other primates.

I'm not sure what your background is, but in my country - the city and state where I grew up and the places I've lived - cultural accounts of origin stories span the breadth of literature, folklore, religion, and sciences. In my experience, folks are exposed to all of these during their public education on a reoccurring basis. I was pretty sure this was a baseline standard for public education across my country, but I could be wrong about that.

That said, it almost sounds like you're asking about college-level comprehension of a topic. I'm not sure that's a fair standard to ask of people. I'm aware fields have been shifting a lot, but I'm pretty sure comparative genomics is something usually covered in graduate school and barely touched upon in college (and then, only if you major in life science that requires that in the curriculum).... forget about having it in public education.

 

[Jose Fly]

I'm not sure what your background is, but in my country - the city and state where I grew up and the places I've lived - cultural accounts of origin stories span the breadth of literature, folklore, religion, and sciences. In my experience, folks are exposed to all of these during their public education on a reoccurring basis. I was pretty sure this was a baseline standard for public education across my country, but I could be wrong about that.

I'm in the US, and when I was in high school in the 1980's in Ohio, evolution was barely covered even in what was then the equivalent of AP Biology. The school apparently decided it was safer to just avoid the subject altogether, rather than stir up controversy by teaching it. OTOH, we were regularly exposed to, in various ways, Christianity and its associated beliefs.

That said, it almost sounds like you're asking about college-level comprehension of a topic.

No, I don't think a basic overview of fossils and how humans share specific types of genetic sequences with other primates is too much for high school.

I'm not sure that's a fair standard to ask of people.

If so, that bolsters my point. If the vast majority of the people surveyed are ignorant of the basics of the fossil record and genetics, then asking them about how humans came to be is basically asking them what their beliefs are. And for many folks, I would guess that's equivalent to asking them what their religion teaches on the subject.

I'm aware fields have been shifting a lot, but I'm pretty sure comparative genomics is something usually covered in graduate school and barely touched upon in college (and then, only if you major in life science that requires that in the curriculum).... forget about having it in public education.

I've explained the basics of shared genetic sequences between humans and other primates to non-science people in about 30 minutes. It's pretty easy actually, especially when I have something to draw on.

 

[ImmortalFlame]

I think this thread makes excellent points about an often under-discussed aspect of the creation/evolution debate. Generally, I hold the position that the distinction between theistic evolution and non-theistic evolution shouldn't much factor into the debate, since the debate should be between the position that claims evolution doesn't occur and the position that it does. However, I have noticed an increasing number of threads that seem to indicate some confusion around the idea of creationism/intelligent design and theism in general, and tend to construe the debate along the lines of non-theistic evolution versus theism (including theistic evolution). In most debates I would still contend that this distinction isn't hugely impactful, much in the same way that a debate between those who believe in a flat and round earth isn't necessarily about a theistic position versus a non-theistic position, but the general confusion between creationism and theism is still presenting an issue in some discussions. For my part, at least, I try to make it clear that I see no strict divide between theistic and non-theistic evolution in most debates, although I'm sure a debate between the two positions would actually be somewhat more interesting and constructive than a lot of the debates we tend to see.