STEP 2: Study design#
Conceptualization. In an empirical research project, the study design encompasses conceptualizing and planning the methodology for data collection and analysis, including the development of a study protocol, how to operationalize the research question and variables, as well as planning the next steps regarding data collection, analysis, and interpretation (Steps 5-8). In essence, the study design step lays out the foundation for the entire project and provides a roadmap for all subsequent steps. It is essential for the study design to be well-conceived, well-executed, and well-documented to ensure the quality, integrity, and generalizability of the findings of the research.
Documentation and flexibility. Documenting the decision-making process throughout the research project and creating a detailed data collection protocol are crucial for enhancing reproducibility, enabling other researchers to understand and replicate the study with greater ease. It is essential to maintain flexibility in this step, allowing for adjustments as long as the project is still being planned. After all major methodological and analytical decisions have been made, it’s advantageous to consider adopting an open science approach to be transparent (e.g., a preregistration or Registered Report documents one’s original ideas with a timestamp; Haroz, 2022; Laine, 2017).
Sample size and power. Another important aspect of the study design step is determining the appropriate sample size, target population (e.g., neurotypical individuals or patients), as well as the sampling strategy (e.g., stratified or convenience sampling; Stratton, 2021). To ensure that the study has sufficient statistical power to detect meaningful differences or associations, justification of one’s sample size is helpful at this stage, e.g., via a power analysis (➜ GPower, ➜ Justification Shinyapp or ➜ g_ci_spm; Cohen, 1962; Jones et al., 2003; Kemal, 2020; see glossary) or funding constraints.
Ethical considerations and approvals. In this step and for most psychological empirical research projects, approval by the local ethics committee or an institutional review board should be applied for. Considering ethics in experimental design involves taking steps to protect the rights and welfare of participants, weighing costs and benefits while minimizing risks, ensuring the privacy of participants and the confidentiality of their data, but also obtaining approval for sharing data as openly as possible. It also involves considering the impact of the findings on society and potential biases that may exist in the study.
Data structure and management. Determining the data structure and arrangement enhances the accessibility and organization of the collected data. For example, complex neuroimaging, EEG, or eye tracking data can be organized using the ➜ Brain Imaging Data Structure (BIDS; Gorgolewski et al., 2017; see also Step 5). First thoughts regarding a data management plan should already be done now.
Collaboration and expertise. Drawing on the experience from supervisor(s), mentor(s), and/or collaborator(s) is key in this step, as they might have specific expertise or experience with certain aspects of the planned project. In this and the previous step, the role of communities and collaboration with others for the sharing of knowledge and expertise becomes especially clear (e.g., in ‘Big Team Science’ projects; see Step 1 and Baumgartner et al., 2023; Forscher et al., 2023; Hall et al., 2018 for primers and tutorials).
Authorship. Finally, criteria, tasks, and rules for (co-)authorship should be discussed already at an early stage of the project, and re-discussed over its course if changes arise (➜ CRediT statement; Brand et al., 2015; Tay, 2021; see glossary and Step 8).
Key questions in this step#
What are thr hypotheses and how can they be tested?
Which independent variables (IVs) are manipulated?
Which dependent variables (DVs) need to be measured?
Is approval by an ethics/institutional review board (IRB) needed?
How large should the sample be?