6 Research Design
Choosing the right research design is one of the most important decisions you make in your thesis. Your design must fit your research question. This chapter outlines the main options available to you.
6.1 Quantitative vs. qualitative research
| Quantitative (Confirmatory) | Qualitative | |
|---|---|---|
| Goal | Test hypotheses about relationships between variables | Explore meanings, experiences, patterns |
| Data | Numerical (survey responses, coded content) | Text, images, audio (interviews, documents) |
| Analysis | Statistical tests (regression, ANOVA, etc.) | Thematic analysis, grounded theory, discourse analysis |
| Output | Generalizable findings across populations | Rich, contextualized insights about individuals |
| Thesis structure | Hypotheses + conceptual model | Sub-questions |
Both approaches are valid in Communication Science. The choice depends on your research question and the state of knowledge in your area. I generally supervise quantitative research, but am open to qualitative or mixed-method approaches as well.
6.2 Survey research
A survey measures variables via a questionnaire administered to a sample of participants. It is the most common design in Communication Science master’s theses.
When to use it: When you want to assess the prevalence of variables and/or examine associations and relationships between them in a larger sample.
Key considerations:
- Sample size and recruitment (you need to conduct a power analysis beforehand, see e.g., this blogpost)
- Validated scales vs. self-constructed items
- Platform (Qualtrics is the standard at VU)
- Online convenience sampling vs. probability sampling
Self-reported media use correlates only r ≈ .38 with device-logged behavior — below the convergent validity threshold (Parry et al., 2021). If actual use (not perceived use) is central to your research question, consider whether self-report is a sufficient proxy. Always label self-reported use explicitly as such and qualify your inferences accordingly.
Recent research estimates that 30–90% of responses in crowdsourcing platforms (e.g., Prolific, MTurk) may be inauthentic, with AI-generated responses reaching up to 45% in some studies (Panizza et al., 2026). Even 3–7% polluted data can distort results enough to invalidate conclusions. When recruiting online, enable paradata tracking (response time, keystroke patterns) in Qualtrics, include attention checks, and discuss your data quality approach with me before launching your survey.
6.3 Experimental research
An experiment manipulates one or more independent variables and measures effects on dependent variables, with random assignment of participants to conditions.
When to use it: When you want to establish causal effects.
Key considerations:
- Sample size and recruitment (you likewise need to conduct a power analysis beforehand, see e.g., this blogpost)
- Online convenience sampling vs. probability sampling
- Stimulus design and setting
- Use of deception (should participants know about the purpose of the study?)
Design notation: Describe your design clearly, e.g.:
“A 2 (color of logo: red vs. blue) × 2 (medium: TV vs. newspaper) between-subjects experiment with purchase intention as the dependent variable.”
Always specify:
- Whether it is between-subjects, within-subjects, or mixed
- Whether randomization was used (it should be)
- Whether you conducted a manipulation check
6.4 Content analysis
Content analysis systematically codes and analyzes a corpus of media content (text, images, video, social media posts).
Types:
- Manual content analysis: human coders apply a codebook; requires intercoder reliability testing
- Automated content analysis: computational methods (e.g., dictionary-based, supervised machine learning, LLMs) via R, AmCat, or Coosto
Key considerations:
- How was the corpus assembled? (sampling strategy)
- What is the unit of analysis?
- How were variables operationalized and coded?
- What are the reliability and validity of your coding?
Example corpus description:
“We collected all articles published by the five most-read Dutch online news outlets between January 1 and March 31, 2024, using the Coosto API. After removing duplicates and non-news items (e.g., job listings), the final corpus consisted of 12,847 articles. The unit of analysis was the individual article.”
6.5 Qualitative research
Qualitative designs explore social phenomena in depth without the goal of statistical generalization.
Common approaches in Communication Science:
- Interviews (semi-structured, in-depth)
- Focus groups
- Ethnographic observation
Analysis frameworks:
- Thematic analysis — identifying recurring patterns across data
- Grounded theory — building theory inductively from the data
- Discourse analysis — analyzing how language constructs meaning
In qualitative theses, the method chapter describes the methodology (e.g., grounded theory), data collection (who, how many, how recruited), and data analysis procedures.
6.6 Experience sampling (ESM)
An experience sampling study asks participants to complete brief surveys repeatedly over several days or weeks — typically triggered by a timer or an event (e.g., each time they use social media). This captures behavior and experiences as they happen, rather than relying on retrospective recall.
When to use it: When your research question is about how experiences or behaviors fluctuate within people across situations, rather than average differences between people.
Key considerations:
- Requires a sampling app (e.g., m-Path, movisensXS) and careful pilot testing
- Participants complete many measurements — keep each survey very short (2–5 minutes)
- Analysis uses multilevel modeling (observations nested within persons)
- Ethics and burden: participants must be debriefed and compensated appropriately
Example: In a two-week ESM study (Masur, 2019), n = 164 participants completed a brief survey each time they engaged in digital communication, capturing 1,104 situational events. The analysis found that 72% of the variance in self-disclosure lay within persons across situations — a finding a one-time survey would have missed entirely.
6.7 The analysis plan
Your Research Proposal must include an analysis plan that specifies (for each hypothesis or sub-question) exactly which statistical test on which variables will be used to answer it. For example:
“H1 will be tested with a simple linear regression with social media exposure as the predictor and vaccine hesitancy as the outcome variable.”
This plan prevents ad-hoc analysis decisions later. In the final thesis, the analysis plan is not a separate chapter. Instead, it is integrated into the Results section.
I strongly suggest to preregister your analysis plan on the Open Science Framework (OSF) before data collection. See Open Science for details. We will discuss this right before data collection.
6.8 References
Masur, P. K. (2019). Situational Privacy and Self-Disclosure: Communication Processes in Online Environments. Springer International Publishing. https://doi.org/10.1007/978-3-319-78884-5
Panizza, F., Kyrychenko, Y., & Roozenbeek, J. (2026). Survey-taking AI tools surpass human abilities. Here’s what we can do about it. Nature. https://doi.org/10.1038/d41586-026-00386-2
Parry, D. A., Davidson, B. I., Sewall, C. J. R., Fisher, J. T., Mieczkowski, H., & Quintana, D. S. (2021). A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nature Human Behaviour, 5, 1535–1547. https://doi.org/10.1038/s41562-021-01117-5