Interim Survey Data Analysis: Survey Details

Sections: Introduction -> Survey Details -> Sample Information -> Survey Items -> Hypotheses and Analyses of Models -> Conclusion

 

This research is co-funded by the Economic and Social Research Council and YouGov, with one of the benefits of the latter being access to the company’s large UK panel of online survey respondents. As such, an original survey was designed to cover the areas specified by the theories outlined in the literature review. The survey was split into two waves to reduce the number of questions that respondents were required to answer in a single sitting, with the hope that their attention would be sustained and the quality of their responses maintained. Further, and more importantly, splitting the survey into two waves allowed the questions on perception to be temporally separated from the questions on activities. This reduced the possibility of respondents’ perceptions of politics being influenced by their answers to questions about their political activities. Thus, the first wave of the survey included sections on political activities, group membership and voluntary activity, cultural preferences and activities, and social networks. The second wave then contained sections on a range of political opinions, perceptions of societal hierarchies and their explanations, and background information such as parental occupation, educational level, and religious beliefs.

The various sections of the survey were tested in thirteen cognitive interviews conducted with administrative staff in the Department of Government at the University of Essex, and with staff from a range of departments at YouGov, none of whom design or field surveys as part of their work. Further, the survey was circulated for comments to the team specialising in political polling at YouGov. Following the cognitive interviews and review by specialists the survey was redrafted and each wave was piloted with 100 respondents from the YouGov panel to check that there were no remaining substantive or technical issues. Following final minor amendments the first wave of the survey was fielded between Monday 17 March 2014 and Tuesday 01April 2014. The second wave of the survey was then fielded between Monday 07 April 2014 and Thursday 17 April 2014.

Table 1 presents the number of panel members who started and completed each wave of the survey. As can be seen, the proportion of those starting the first wave who went on to complete both waves is 79.57%, which is a respectable retention rate. A further fourteen respondents displayed consistent signs of satisficing and were removed to leave a final sample of 1501, or 78.83% of those who started the first wave.

 

Table 1 – Starters and Completers of Each Wave of the Survey

2014-10-14 Interim Survey Data Analysis Table 01

 

An extensive analysis comparing the starters and completers in each wave, and between waves, has been undertaken and revealed no substantively important differences between those who dropped out of the survey at any stage and those who completed both waves.

Although the end date of the first wave was only one week from the start date of the second wave the great majority of respondents had at least two weeks between completing waves, as can be seen in Table 2.

 

 Table 2 – Number of Respondents Completing

Each Wave of the Survey within One Week

2014-10-14 Interim Survey Data Analysis Table 02

The median completion time for the first wave was just over 18 minutes whilst for the second wave it was just under 18 minutes. A median completion time of 36 minutes across both waves is a good amount of respondent time to secure in an online self-completion survey. Unfortunately, the mean completion time cannot be accurately calculated because respondents could leave the surveys and return to them at a later time, which inflates the mean to unrealistic levels. The notable number of such respondents can be seen in Chart 1 and Chart 2, which show the distribution of completion times for each wave of the survey.

 

Chart 1 – Number of Respondents in One Minute

Completion Time Categories for the First Wave

2014-10-14 Interim Survey Data Analysis Chart 01

 Table 2 – Number of Respondents in One Minute

Completion Time Categories for the Second Wave

2014-10-14 Interim Survey Data Analysis Chart 02

For the sake of ease the completion times have been grouped into one minute categories, and the charts reveal that both distributions exhibit positive skewness (manifested in the long tail of columns representing the higher completion times). With the exception of the 60 minutes or more categories, we cannot know to what extent this skewness is the result of respondents leaving the survey and returning to it at a later time. Importantly, 68.42% (1027) respondents completed the first wave in between 10 and 24 minutes, whilst an almost identical 68.22% (1024) completed the second wave between those times. Since the great majority of respondents not falling into these categories took longer to complete the survey we can, on the basis of time at least, assume a reasonable level of response quality.

Sections: Introduction -> Survey Details -> Sample Information -> Survey Items -> Hypotheses and Analyses of Models -> Conclusion

Back to Academic page

 

Advertisements

5 thoughts on “Interim Survey Data Analysis: Survey Details

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s