1. Strategies for Developing High-Quality
Survey Instruments
John F. McCauley
Associate Professor, University of Maryland
Pew Research Center
November 30, 2017
4. Overt Measures; Stated Support for VE:
1. “Violence against civilians to defend religion can
be justified.”
(2. Violence in the name of Islam can be justified)
(3. Violence can be justified to solve problems)
5. Covert Measures; True Support for VE:
“The government is considering a national
polio vaccination. Would you say that you
are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
“The government is considering a national
polio vaccination, which Al Qaeda IM’s
leaders likely oppose. Would you say that
you are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
Control Treatment
12. Challenges:
1. Basic Comprehension Levels
2. Multiple Local Languages
3. Consistency bias, order effects
4. Assuming the General from the Specific
5. Enumerator Fallibility
14. Covert Measures; True Support for VE:
“The government is considering a national
polio vaccination. Would you say that you
are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
“The government is considering a national
polio vaccination, which Al Qaeda IM’s
leaders likely oppose. Would you say that
you are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
Control Treatment
15. Alternative:
“The government is considering shutting
down schools in some areas of the North.
Would you say that you are in favor or
against this plan?”
In favor
Against
Don’t know/no answer
“The government is considering shutting
down schools in some areas of the North,
which Al Qaeda IM’s leaders likely
support. Would you say that you are in
favor or against this plan?”
In favor
Against
Don’t know/no answer
Control Treatment
16. Alternative:
“The government is considering shutting
down schools in some areas of the North.
Would you say that you are in favor or
against this plan?”
In favor
Against
Don’t know/no answer
“The government is considering shutting
down schools in some areas of the North,
which Al Qaeda IM’s leaders likely
support. Would you say that you are in
favor or against this plan?”
In favor
Against
Don’t know/no answer
Control Treatment
In favor of policy =
In favor of terrorist group
18. Strategies:
1. Short, simple questions
2. Include overt and experimental measures
3. Include multiple measures and vary placement
19. Strategies:
1. Short, simple questions
2. Include overt and experimental measures
3. Include multiple measures and vary placement
4. Vary treatment (endorsement)
20. Covert Measures; True Support for VE:
“The government is considering a national
polio vaccination. Would you say that you
are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
“The government is considering a national
polio vaccination, which Al Qaeda IM’s
leaders likely oppose. Would you say that
you are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
Control Treatment
21. Alternative 2:
“The government is considering a national
polio vaccination. Would you say that you
are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
“The government is considering a national
polio vaccination, which Boko Haram’s
leaders likely oppose. Would you say that
you are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
Control Treatment
22. “The government is considering a national
polio vaccination. Would you say that you
are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
“The government is considering a national
polio vaccination, which Boko Haram’s
leaders likely oppose. Would you say that
you are in favor or against this initiative?”
In favor
Against
Don’t know/no answer
Control Treatment
Controls for specific
endorser effect
Alternative 2: