STEP 4: Piloting

A pilot study, also known as exploratory trial, is a preliminary small-scale study conducted to assess potential problems, duration, and other factors before a full investigation. This is often a reflective and iterative process (Thabane et al., 2010). By setting criteria based on important feasibility objectives and research goals, these pilot studies enable researchers to determine the feasibility of a more extensive, time-consuming, and expensive main study and to test whether the operationalization (Step 2) makes sense (see ARIADNE for resources related to piloting, e.g. ➜ data simulation). First, regarding feasibility, it is common and recommended to always test a few “pilot” participants with your whole set-up before starting Step 5 (the data collection), to test if participants understand the instructions of the new experiment and all procedures work as planned. The design of the main study can then be modified for improvements based on the findings of this pilot study. Of note, another complementary method for better determining a study’s feasibility is to simulate data, which allows researchers to test multiple hypotheses and prepare for prospective outcomes before carrying out the primary investigation. Second, on an operationalization level, these preliminary data should be used to check whether all dependent variables can be extracted from the raw files. It is also important to note here that data from pilot studies or participants should be kept separate from the data of the main study. Crucially, it is considered controversial to use pilot data to calculate preliminary estimates of the effect size and variability of the outcome measures to estimate the required sample size for the main study (Albers et al., 2018; Sakaluk, 2016). In conclusion, piloting and data simulation are essential steps for study planning and design, enabling researchers to evaluate viability, foster greater transparency, and enhance the overall quality of their research.