Research synthesis and meta-analysisMethodology courses and philosophy of science
Number of sessions: 4
Hours per session: 4
Entry level: Introduction
- free for PhD candidates of the Graduate School
- €440,- for non-members
- consult our enrolment policy for more information.
Meta-analysis is a widely used method to synthesise quantitative research. Traditionally, the main purpose of meta-analyses has been to estimate the combined effect size of multiple replications.
However, there is more to meta-analysis than merely obtaining an overall effect size estimate. For instance, one can assess whether study-level characteristics have an influence on the effect of interest (such as, age of respondents, country in which the study is performed, types of measures used for the dependent and independent measures).
The course consists of four weekly sessions of four hours. The sessions consist of a mix of lectures and in-class exercises. For the in-class exercises, example datasets are provided when necessary. Participants are encouraged to bring their own datasets when possible.
The in-class assignments add up to a final assignment in which you will conduct a small meta-analysis in your own research field. For the final assignment you cannot use the example datasets provided. You will have to use a set of studies from your own research field.
The deliverable is a short report (max. 2,000 words) describing the method, presenting the results and interpreting them. In other words, a draft of a publishable meta-analysis.
Although not a main goal, after this course you will be more critical in reviewing other studies for their methodology and proper reporting thereof.
- Know how to search the literature for relevant studies
- Be able to obtain the right information from these studies
- Be able to perform a meta-analysis and interpret the results and to perform additional analyses
- (subgroup analysis, moderator analysis, and publication bias analysis)
- Be able to conduct and write up a publishable meta-analysis
No prior knowledge about or experience with meta-analyses is required for this course. However, you should have basic methodological knowledge, allowing you to evaluate a research design and be able to interpret results of a study in terms of its effect size (such as, correlation, regression coefficient, Cohen’s d).
During the course, Meta-Essentials (a meta-analysis tool freely available from www.meta-essentials.com) is used for demonstrations and the exercises. No prior experience is required to use this tool. However, participants are free to use any other tool if they are already familiar with it (like the meta or metaphor packages for R).
Search for relevant studies and retrieve the necessary information
In the first session, we will start at the very beginning of research synthesis: how to search for relevant studies, minimise publication bias, and retrieve the necessary information from studies.
Derive a combined effect size; the different methodological choices
The second session is about how to derive a combined effect size and the different methodological choices you have to make (weighting of studies, what to do if studies report multiple relevant effect sizes, etc.).
Analysing differences between effect sizes: various techniques to explain heterogeneity
During the third session, we dive into the topic of analysing differences between effect sizes. We discuss various techniques to explain heterogeneity between studies (subgroup-analysis and moderator analysis).
Your meta-analysis in a research paper
In the fourth session, we discuss how to analyse potential publication bias and how to report a meta-analysis in a research paper.
About the instructor
Henk van Rhee is a PhD candidate at the Department of Technology and Operations Management of the Rotterdam School of Management (RSM), Erasmus University Rotterdam (EUR). His research focusses on work environments; moreover he is one of the developers of Meta-Essentials, a free and easy-to-use tool for meta-analysis. Visit www.meta-essentials.com for more information.