Resources for Systematic Reviews & Evidence-based Public Health Practice

What is Evidence-Based Public Health?

  • The process of integrating science-based interventions with community preferences to improve the health of populations. (Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: An evolving concept. Am J Prev Med. 2004 Dec;27(5):417-21.)
    • Evidence-based public health is defined as the development, implementation, and evaluation of effective programs and policies in public health through application of principles of scientific reasoning, including systematic uses of data and information systems, and appropriate use of behavioral science theory and program planning models. (Brownson, Ross C., Elizabeth A. Baker, Terry L. Leet, and Kathleen N. Gillespie, Editors. Evidence-Based Public Health. New York: Oxford University Press, 2003.)

How to Develop a Systematic Review

Centre for Reviews & Dissemination

Cochrane Collaboration

The Production of a Systematic Review (Campbell Collaboration)
Producing a Systematic Review (Campbell Collaboration)
The Campbell Collaboration focuses on education, crime and justice, and social work. They currently have two points of access to resources on producing Campbell Collaboration systematic reviews.

Methods Guide for Comparative Effectiveness Reviews: Identifying, Selecting, and Refining Topics (US Agency for Healthcare Research and Quality [AHRQ])

PRISMA: Transparent Reporting for Systematic Reviews and Meta-Analyses
PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Use it to ensure you have included the necessary items important for a high-quality systematic review or meta-analysis. The site includes a 27-item checklist and flow diagram. This flow diagram (or flow chart) has been incorporated into the MS Excel excel workbook developed by librarians at the UT School of Public Health to use for tracking online database searches and search results.

Evidence-Based Practice Meta Search Site

TRIP (Turning Research Into Practice) Database
TRIP searches multiple EBM databases in one fell swoop. Databases listed in the following section that are included in TRIP are designated as such. Do you need to search the others if you search TRIP? The primary database will always have the most current information and some people feel more comfortable searching in the native interface. Having said that, try searching TRIP to see where the majority of the items come from, then go to the site and do one more search just to be sure. TRIP includes:

Additional Sources to Search for Systematic Reviews

UTSPH Students, Staff, and Faculty

Need help developing a search strategy? The Library has a series of presentations that can help you develop your search.

Suggested databases include:

Free Resources to search for systematic reviews

Cochrane Reviews (Cochrane Collaboration; abstracts free to public; included in TRIP)

Effective Health Care Program (US Agency for Healthcare Research and Quality [AHRQ]; included in TRIP)
The EPCs review all relevant scientific literature on clinical, behavioral, and organization and financing topics to produce evidence reports and technology assessments. These reports are used for informing and developing coverage decisions, quality measures, educational materials and tools, guidelines, and research agendas.

Bandolier (included in TRIP)
The impetus behind Bandolier was to find information about evidence of effectiveness (or lack of it), and put the results forward as simple bullet points of those things that worked and those that did not: a bandolier with bullets. Information comes from systematic reviews, meta-analyses, randomized trials, and from high quality observational studies.

EPPI-Centre (Social Science Research Centre, University of London)

Campbell Collaboration Register of Interventions and Policy Evaluations
“The Campbell Collaboration was founded on the principle that systematic reviews on the effects of interventions will inform and help improve policy and services. C2 offers excellent editorial and methodological support to review authors throughout the process of producing a systematic review. ” Reviews are in the areas of criminal justice, education, and social welfare.

The Community Guide

PubMed link for the public (National Library of Medicine)

  • PubMed is free to the public. Searches will retrieve an abstract and links to the online FT when available.
  • Use the PubMed search filter that has been developed to help you find SRs.

Sites that Summarize Systematic Reviews

Cochrane PICO by the Cochrane Editorial Unit

Cochrane Primary Health Care Field
ADA Center for Evidence-Based Dentistry

Centre for Reviews and Dissemination Databases (Centre for Reviews and Dissemination, University of York, UK)

The Community Guide

Online Evidence-Based Emergency Medicine
“This section is a collection of Annals EBEM reviews and abstracts summarizing systematic reviews (SRA) of the best evidence relevant to clinical questions that arise during emergency medical patient care.”

Note: The following sites have been suggested for inclusion in the list of sites that provide high quality summaries. They need to be investigated but they are included here in case you want to check them out.

  • Evidence-Based Child Health (EBCH) Summaries by the Cochrane Child Health Field, EBCH Editorial Office
  • Evidence Transfer Program: Review Summaries by the Cochrane Nursing Care Field
  • Evidence Update by the Effective Health Care Research Programme Consortium
  • health-evidence.ca by health-evidence.ca
  • Health Knowledge Network Evidence Bulletins by the Cochrane Consumers and Communication Review Group
  • Plain Language Summaries by the Cochrane CAM Field
  • Policy Liaison Initiative summaries by the Australasian Cochrane Centre
  • Rx for Change by CADTH
  • Rapid Evidence Summaries on the early management of burns in the emergency setting by the Cochrane Wounds Group and the Cochrane Injuries Group
  • SUPPORT Summaries by the The SUPPORT Collaboration

Standards for Reporting Studies

Equator Network
This site is designed to provide you the standards that should be used when publishing studies. You can use the resources to guide you as you evaluate study publications, including systematic reviews and meta-analyses, or when writing up your own findings.

  • Library for health research reporting
    Includes PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), STROBE (STrengthening the Reporting of OBservational studies in Epidemiology), CONSORT (CONsolidated Standards of Reporting Trials), STARD (STAndards for the Reporting of Diagnostic accuracy studies)as well as many, MANY others.

Assessment of Studies

GRADE: Grading of Recommendations Assessment, Development and Evaluation (GRADE Working Group)
“The GRADE working group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of present grading systems in health care. Our aim is to develop a common, sensible approach to grading quality of evidence and strength of recommendation.”

Downs & Black checklist: The feasibility of creating a checklist for the assessment of the methodological quality of both randomized and non-randomized studies of health care interventions. J Epidemiol Community Health 1998; 52:377-84.

Development of Quality Criteria To Evaluate Nontherapeutic Studies of Incidence, Prevalence, or Risk Factors of Chronic Diseases: Pilot Study of New Checklists (US Agency for Healthcare Research and Quality [AHRQ])

Criteria for Distinguishing Effectiveness from Efficacy Trials in Systematic Reviews (US Agency for Healthcare Research and Quality [AHRQ])
“To propose and test a simple instrument based on seven criteria of study design to distinguish effectiveness (pragmatic) from efficacy (explanatory) trials while conducting systematic reviews.”

The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses
“Nonrandomised studies, including case-control and cohort studies, can be challenging to implement and conduct. Assessment of the quality of such studies is essential for a proper understanding of nonrandomised studies. The Newcastle-Ottawa Scale (NOS) is an ongoing collaboration between the Universities of Newcastle, Australia and Ottawa, Canada.” A copy of the scale in both Word and PDF may be found on the page. Unless otherwise noted by the authors, this scale has not been empirically tested.

Critical Appraisal Skills Program Appraisal Checklists (CASP-UK)

Assessments of Effects

  • Technology Assessments (Agency for Healthcare Research and Quality)
    “AHRQ’s technology assessment program uses state-of-the-art methodologies for assessing the clinical utility of medical interventions. Technology assessments are based on a systematic review of the literature, along with appropriate qualitative and quantitative methods of synthesizing data from multiple studies.”
  • Centre for Reviews and Dissemination Databases(Centre for Reviews and Dissemination, University of York, UK)
    • Health Technology Assessment Database
      “The HTA database brings together details of completed and ongoing health technology assessments from around the world. The abstracts in the database are descriptive rather than analytical and do not form critical appraisals of the reports.” Limit your search to the HTA database by checking the box next to it.
  • Natural Standard (Natural Standard)
    “Natural Standard is an international research collaboration that aggregates and synthesizes data on complementary and alternative therapies.” More about this site…
  • Testing Treatments: Better Research for Better Healthcare
    “Aimed at both patients and professionals, Testing Treatments builds a lively and thought provoking argument for better, more reliable, more relevant research, with unbiased or fair trials, and explains how patients can work with doctors to achieve this vital goal.”

Economic Assessments

Critical Appraisal Skills Program Appraisal Checklists (CASP-UK)

NHS Economic Evaluation Database (Centre for Reviews & Dissemination)
Note: This is included in the Cochrane Library. “The database aims to assist decision-makers by systematically identifying and describing economic evaluations, appraising their quality and highlighting their relative strengths and weaknesses.”

Center for the Evaluation of Value and Risk in Health (Tufts Medical Center, Institute for Clinical Research and Health Policy Studies)
The CEA Registry is a comprehensive database of over 1,700 cost-utility analyses on a wide variety of diseases and treatments.

Evidence-Based Guidelines

Clinical Practice Guidelines (US Agency for Healthcare Research and Quality [AHRQ])

National Institute for Health and Clinical Excellence Clinical Guidelines (NICE, UK)

Best Practices

Model Practices Database (National Association of County and City Health Officials (NACCHO))

Promising Practices Network (RAND Corporation)
“PPN s target audience includes policymakers, service providers, and other decision makers at all levels who care about improving outcomes for children and families.” More about this site..

Evidence Base (National Institute for Health and Clinical Evidence (NICE))
…”these papers help support evidence-based decision making by public health practitioners”.

Health-Evidence.ca
“The ultimate goal of the research team is to facilitate the adoption and implementation of effective policies/programs/interventions at the local and regional public health decision-making levels across Canada”

TRIP (Turning Research Into Practice) Database
TRIP searches multiple EBM databases in one fell swoop.

Theory at a Glance: A Guide for Health Promotion Practice
“This monograph describes influential theories of health-related behaviors, the processes of shaping behaviors, and the effects of community and environmental factors on behavior. It makes health behavior theory accessible and provides tools to solve problems and assess the effectiveness of health promotion programs. NIH Publication No. 05-3896″

Search for Clinical Trials

On January 11, 2011, the BMJ published an editorial and an analysis written by members of the Cochrane Acute Respiratory Infections Group. The Cochrane members discussed issues concerning the importance of drawing on data from trials, not just those reported on in the research literature. The authors had earlier written a systematic review that reported favorably on Tamiflu based on a meta-analysis that included 10 trials, only 2 of which were reported on in the peer-reviewed literature. They were criticized for their analysis as they had not reviewed the actual data, so they contacted those involved with the trials to get the original data. However, they were unable to do so. Consequently, when they updated their review, they opted to not include the original meta-analysis. The article then goes on to discuss the importance of reviewing all trial data, not just those trials that are written up in the research literature.

This article is a must-read if you are considering a systematic review which involves clinical trials. If you decide you want to use original data from clinical trial, how do you find existing and past clinical trials? Take a look at these registries.

EBPH Meta-Sites

These sites provide additional links to EBPH resources. Many of the items listed on this page can be found on these pages as well.

Free Online Research Journal Collections

There are many sites that provide access to free research journals. As you review the sites, be sure to pay close to attention as to what is free. Some publishers restrict access by date– the most current issue is subscription-based only, for example. Some offer certain sections for free but other sections require a subscription. There may be overlap between these collections and some of the journals that are free may require you to register before viewing.