Meta-analysis is a widely used method to synthesize quantitative research. Traditionally, the main purpose of meta-analyses has been to estimate the combined effect size of multiple replications. However, there is more to meta-analysis than merely obtaining an overall effect size estimate. For instance, one can assess whether study-level characteristics have an influence on the effect of interest (e.g., age or ethnicity of respondents, types of measures used for the dependent and independent variables).
This course offers an introduction to the basic principles of meta-analysis.
The course consists of four sessions and assignments. The assignments apply what we have discussed in the lecture to your own research field. The assignments add up to a final assignment in which you conduct a small meta-analysis in your own research field. The deliverable is a short report (max. 2,000 words) describing the method, presenting the results, and interpreting them (i.e., a draft of a publishable meta-analysis).
The main objectives of this course are:
- To gain knowledge on how to search the literature for relevant studies.
- To know how to obtain the right information from these studies.
- To learn how to perform a meta-analysis.
- To be able to perform additional analyses, such as subgroup analysis, moderator analysis, and publication bias analysis
- To be able to properly interpret results of meta-analysis and the related analyses.
- After completing the course, you will be able to conduct and write up a publishable meta-analysis.
- Although not a main goal of this course, during the course you will most certainly become more critical in reviewing other studies for their methodology and proper reporting thereof.
No prior knowledge about, or experience with meta-analyses is needed for this course. However, you should have basic methodological knowledge, allowing you to evaluate a research design and be able to interpret results of a study in terms of its effect size (e.g., correlation, regression coefficient, Cohen’s d).
During the course, we use Meta-Essentials (a meta-analysis tool freely available from www.meta-essentials.com) for demonstrations and assignments. No prior experience is required to use this tool. You are free to use any other tool if you are already familiar with it (like the meta or metafor packages for R).
In the first session, we will start with an introduction in what meta-analysis actually is (and what it is not). Subsequently, we begin at the very beginning of research synthesis: how to search for relevant studies, minimize (publication) bias, and retrieve the necessary information from studies.
The second session is about how to derive a combined effect size and the different methodological choices you have to make along the way (weighting of studies, what to do if studies report multiple relevant effect sizes, etc.).
During the third session, we dive into the topic of analyzing differences between effect sizes. We discuss various techniques to explain differences between studies (subgroup analysis and moderator analysis).
In the fourth session, we discuss meta-analytic structural equation modelling (MASEM), the potential influence of publication bias on your meta-analysis, and how to report your meta-analysis in a research paper.
Henk van Rhee is a PhD candidate at the Department of Technology and Operations Management of the Rotterdam School of Management (RSM), Erasmus University Rotterdam (EUR). His research focusses on work environments; moreover he is one of the developers of Meta-Essentials, a free and easy-to-use tool for meta-analysis. Visit www.meta-essentials.com for more information.