1. Systematic Reviews
and
Meta-analysis
Dr Nazli Khatib (MD, PhD)
Professor (Physiology)
International Initiative Impact (3ie) Evaluation Awardee
JNMC
DMIMS(DU)
1

2. Objectives of the talk
• To provide the needs, characteristics and constitution of
systematic review
• To provide an overview of the process of planning,
undertaking and writing a systematic review
• To provide an overview of meta-analysis
2

3. Levels of evidences in health care
3

4. The need for SR ??
• Healthcare providers, consumers & policy makers are
inundated with loads of information.
• No time, skills & resources to find, appraise and interpret this
evidence and to incorporate it into healthcare decisions.
• SR respond to this challenge by identifying, appraising and
synthesizing research-based evidence and presenting it in an
accessible format.
4

5. Systematic review
5

6. So, a systematic review.....
• ...attempts to pool all pragmatic evidences in order to answer
a specific research question.
• ...uses precise, systematic methods that are selected with a
view to minimizing bias, thus providing reliable findings from
which conclusions can be drawn and decisions made (Antman 1992,
Oxman 1993).
6

7. Key characteristics of an SR
• An explicit methodology
• A systematic search that identifies all studies
• An assessment of the validity of the findings of the included
studies
• A systematic presentation, and synthesis of the characteristics
and findings of the included studies.
7

8. Need for an SR....
SR can be conducted in an effort to:
• Resolve conflicting evidence
• Help people make practical decisions about health care.
• Address questions where clinical practice is uncertain
• Establish feasibility of an intervention
• Summarize and help people to understand the evidence.
• Confirm the appropriateness of current practice or highlight a
need for future research.
• Avoiding redundant unnecessary trials
8

9. SR with Meta-analysis
• Meta-analysis is the use of statistical methods to summarize
the results of independent studies.
• Not every SR leads to a meta-analysis. But it is better if a
meta-analysis is based on an underlying systematic review.
9

10. Before you start with a SR...
• Need to make sure no other systematic review identical to
yours has recently been conducted
• Need to make sure there is a need for review
10

11. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
11

12. SR titles
12
Scenario Structure Example
Basic structure. [Intervention]
for [health
problem].
Antibiotics for acute
bronchitis.
Comparing two
active
interventions.
[Intervention A]
versus
[intervention B]
for [health
problem].
Immediate versus delayed
treatment for cervical
intraepithelial neoplasia.
Type of people
being studied or
location of
intervention
mentioned
explicitly.
[Intervention]
for [health
problem] in
[participant
group/location].
Inhaled nitric oxide for
respiratory failure in
preterm infants

13. 1. Defining the review question
• A clearly defined, focused review begins with a well framed
question.
• A well-formulated research question increase the efficiency of the
review by limiting the time and cost of identifying and obtaining
relevant literature.
• The review question should be in PICO format.
• SR can focus on broad questions, or be more narrowly defined.
13

14. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
14

15. 2. Searching for studies
• Electronic databases
• “Grey literature” ( thesis, Internal reports, pharmaceutical industry
files)
• Checking reference lists
• Unpublished sources known to experts in the specialty (seek
by personal communication and look in trial registers)
• Hand searching
• Has to be as comprehensive as possible
15

16. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
16

17. 3. Selecting studies
(Inclusion and exclusion of studies)
• Scan titles & abstracts to retrieve possibly relevant studies
• Correspond with investigators, where appropriate, to clarify
study
• Decide if studies meet inclusion criteria and record reasons
for exclusion
• Done by two independent review authors
17

18. Selecting studies: Prisma chart

19. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
19

20. 4. Extracting data
• Data needed from each study
 Details of the trial
 Numerical data of the outcomes in each group to enter
into meta-analysis
20

21. 21

22. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
22

23. 5. Assessing the risk of bias
• If poor quality trials are the building blocks of a systematic
review, the review may follow high quality methods, but the
quality of evidence may still be poor
• Use of poor quality methods may change the results of a trial
• Inclusion of poor quality studies in meta-analysis is likely to
exaggerate treatment effect
• The evaluation of the validity of the included studies is
therefore an essential.
23

24. Risk of bias
24

25. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
25

26. 6. Analyse data - Meta-analysis
Principles
• Patients in one trial are NEVER directly compared with those
in another trial
• Each trial is analysed separately
• Summary statistics are collected /calculated for each trial
• Each study is weighted
• These summary statistics are added together to give the
‘meta-analysis’
26

27. Why perform a meta-analysis?
• Increase power
• Improve precision of estimate
• Quantify treatment effects and their uncertainty
• Assess consistency of results
• Answer questions not posed by individual studies
• Settle controversies from conflicting studies
• Generate new hypotheses
27

28. When can you do a meta-analysis?
• When more than one study has estimated an effect
• When characteristics of the studies are sufficiently similar so
it makes sense to combine them
• When the data are available in the form we need to combine
them (e.g. outcome measured in similar ways)
28

29. When not to do a meta-analysis?
• ‘Mixing apples with oranges’
 Not useful for learning about apples
 Useful for learning about fruit
 Meta-analysis may be meaningless and genuine effects may be
obscured if studies are too clinically diverse
• Meta-analysis in presence of serious
 publication and/or reporting biases may produce an
inappropriate summary
29

30. Steps in doing a meta-analysis
1. Define comparisons for your review
2. Decide on outcomes for each comparison
3. Data retrieval
4. Entering and pooling data
5. Exploring heterogeneity
6. Interpreting results
30

31. Defining comparisons and outcome
31

32. Steps in doing a meta-analysis
1. Define comparisons for your review
2. Decide on outcomes for each comparison
3. Data retrieval
4. Entering and pooling data
5. Exploring heterogeneity
6. Interpreting results
32

33. Retrieving, entering & pooling data
33

34. Data analysis: Forest plot
Forest plot of comparison: Ghrelin v/s Control; Outcome: Mortality
34

35. Steps in SR
1. Define the review question
2. Search for studies
3. Select studies
4. Extract data
5. Assess risk of bias in included studies
6. Analyse data and undertake meta-analyses
7. Address reporting biases
8. Present results and ‘Summary of findings’ tables
35

36. 8. Summary of findings table (SoF)
• There are 3 key parts of a SoF table:
 Information about the review
 Summary of the statistical results
 Grade of the quality of evidence.
36

37. Composition of a SR
37

38. Grants for SR
• ICMR
• 3ie’s Systematic Review Call 5 is funding 8 systematic reviews.
• Department For International Development, UK Aid are
commissioning 10 new SR on topics such as education, health,
infrastructure and humanitarian assistance.
• NIHR Cochrane Programme Grant Scheme
• NIHR Cochrane Incentive Awards
• USAID funded reviews
38

39. Grants for SR
• 3ie, along with the Canadian International Development
Agency (CIDA), the United States Agency for International
Development (USAID), and the Millennium Challenge
Corporation (MCC) is inviting proposals for SR in international
development. This call will fund up to 15 review questions in
the areas of agriculture, infrastructure, business
development, governance and education interventions.
39

40. Summary
• SR is “the application of scientific strategies that limit bias by
the systematic assembly, critical appraisal and synthesis of all
relevant studies on a specific topic.” (Cook 1995)
• Help people to resolve conflicting evidence & make practical
decisions about health care.
• Steps in SR: Def RQ Search studies Select studies
Extract data  Assess risk of bias Analyse data Address
reporting biases Present results and ‘SoF’ tables Interpret
results and draw conclusions 40

41. Summary...
• Meta-analysis is the use of statistical methods to summarize
the results of independent studies.
• Steps in meta-analysis: Define comparisons & outcomes
Data retrieval Entering and pooling data Exploring
heterogeneity Interpreting results
41

42. To increase the power & precision
42

43. Generate new hypotheses
43

44. 44