How to Conduct Searches for Prognostic Systematic Reviews (Without Tearing Your Hair Out)

Writing a search strategy for a prognostic systematic review can be…. Woof. Let’s just say it’s not a simple affair.

First off, there are 4 different types of prognosis research1:

  1. Overall prognosis2
  2. Prognostic factors3,4
  3. Prognostic models5
  4. Treatment selection factors/models 6

Then, as if that didn’t already have you scratching your head trying to slot your question, there are different search approaches recommended depending on what type of prognostic research you’re looking for, because it can be hard as shit to track that stuff down.

Though I’ve been involved in a few prognostic reviews to date, I’ve been finding that they’re spaced juuuuust far enough apart that I start forgetting the methods literature and find myself neck deep in papers again trying to make sense of it all. So this is a very selfish post trying to spell everything out while it’s fresh so that I don’t have to keep doing literature deep dives (and here’s praying that future me will update this as new literature comes out. Please oh please future me don’t be lazy).

Prognostic Search Filter Performance Summary

First and foremost, I should probably say that whenever possible, I try to avoid applying a prognostic filter. That said, there are sometimes cases where it would really, really help rein in a project.

So this is my very unofficial summary of papers I’ve found helpful on that account. For a full list of prognostic search filter papers, I’d definitely recommend the ISSG Search Filters Resource: https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home/prognosis

I’ve provided the full Ovid Medline search filters at the bottom of this post as well, because yes I’m that lazy that I don’t want to dig through these articles every time.

ArticleOverall PrognosisPrognostic FactorsPrognostic ModelsTreatment Selection Factors/ Models
Boulos et al (2021)7N/AThe Irvin filter8 is probably the best bet, though a lot of other search elements are important for prognostic reviews, including citation chain searching.N/AN/A
Kavanagh et al (2021)9McMaster Hedges Sensitive Prognosis filter10 appears to be the best (despite the write up in the paper).N/AMcMaster Hedges Sensitive Clinical Prediction Guide filter10 appears to be the best (despite the write up in the paper).N/A
Geersing et al (2012)11Don’t use a filterFilters are probably not going to work. Try instead to search the population and the predictor of interest.Sweet. Try the Ingui + Geersing11 filter because that seems to work pretty well.Don’t use a filter

Reference/Citation Searching

Finally, I can’t stress enough how much people are finding citation chain searching helpful for prognostic reviews. It just keeps coming up as an important aspect of the approach. What does citation chain searching or snowball searching mean?

Let’s say you’ve done all your screening and you have 10 included studies.

Backwards citation chain searching: You look at each of those 10 included studies and skim their reference lists to see if any other relevant papers are mentioned there. If any look tempting you’d screen them for inclusion the same as you did for all the other results (in duplicate).

Forwards citation chain searching: You take those same 10 included studies, but you use a database like Scopus to see who has since cited that paper. Again, any promising leads would be screened in duplicate for eligibility.

Full Prognostic Search Filters

Irvin filter8 (as reported in Boulos et al 2021)

  1. Cohort Studies/
  2. incidence.tw.
  3. Mortality/
  4. Follow-Up Studies/
  5. prognos*.tw.
  6. predict*.tw.
  7. course.tw.
  8. Survival Analysis/
  9. or/1-8

McMaster Maximum Sensitivity Prognosis10

  1. incidence.sh.
  2. exp mortality/
  3. follow-up studies.sh.
  4. prognos*.tw.
  5. predict*.tw.
  6. course*.tw.
  7. or/1-7

McMaster Maximum Sensitivity Clinical Prediction Guides10

  1. predict*.mp.
  2. scor*.tw.
  3. observ*.mp.
  4. or/1-3

Ingui + Geersing filter11

  1. Validat$.mp. or Predict$.ti. or Rule$.mp
  2. (Predict$ and (Outcome$ or Risk$ or Model$)).mp.
  3. ((History or Variable$ or Criteria or Scor$ or Characteristic$ or Finding$ or Factor$) and (Predict$ or Model$ or Decision$ or Identif$ or Prognos$)).mp.
  4. Decision$.mp. and ((Model$ or Clinical$).mp. or Logistic Models/)
  5. Prognostic and (History or Variable$ or Criteria or Scor$ or Characteristic$ or Finding$ or Factor$ or Model$)).mp
  6. Stratification.mp. or “ROC Curve”/ or Discrimination.mp. or Discriminate.mp. or cstatistic.mp. or “c statistic”.mp. or “Area under the curve”.mp. or AUC.mp. or Calibration.mp. or Indices.mp. or Algorithm.mp. or Multivariable.mp.
  7. or/1-6

(A special thank you to Cooray et al 202012 for laying this out so clearly and saving all of us the heartache of trying to convert from the PubMed-based (but also weirdly still Ovid?) search described in Geersing)

Many thanks to my colleagues Leah Boulos, MLIS, and Mélanie Brunet, PhD, for providing feedback on this blog post.

References

  1. Moons, K. G. et al. Implementing systematic reviews of prognosis studies in Cochrane. Cochrane database Syst. Rev. 10, ED000129 (2018).
  2. Hemingway, H. et al. Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes. BMJ 346, 1–11 (2013).
  3. Riley, R. D. et al. A guide to systematic review and meta-analysis of prognostic factor studies. BMJ 364, k4597 (2019).
  4. Riley, R. D. et al. Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research. PLOS Med. 10, e1001380 (2013).
  5. Steyerberg, E. W. et al. Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research. PLOS Med. 10, e1001381 (2013).
  6. Hingorani, A. D. et al. Prognosis research strategy (PROGRESS) 4: Stratified medicine research. BMJ 346, (2013).
  7. Boulos, L., Ogilvie, R. & Hayden, J. A. Search methods for prognostic factor systematic reviews: A methodologic investigation. J. Med. Libr. Assoc. 109, 23–32 (2021).
  8. Irvin, E. & Hayden, J. Developing and testing an optimal search strategy for identifying studies of prognosis [abstract]. XIV Cochrane Colloquium; 2006 October 23-26; Dublin, Ireland. 153 http://www.mrw.interscience.wiley.com/cochrane/clcmr/articles/CMR-10107/frame.html (2006).
  9. Kavanagh, P. L. et al. Optimizing a literature surveillance strategy to retrieve sound overall prognosis and risk assessment model papers. J. Am. Med. Inform. Assoc. 28, 766–771 (2021).
  10. McMaster Health Research Information Unit. Health Information Research Unit – HIRU ~ Search Strategies for MEDLINE in Ovid Syntax and the PubMed translation. http://hiru.mcmaster.ca/hiru/HIRU_Hedges_MEDLINE_Strategies.aspx (2015).
  11. Geersing, G.-J. et al. Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews. PLoS One 7, e32844 (2012).
  12. Cooray, S. D. et al. The unrealised potential for predicting pregnancy complications in women with gestational diabetes: A systematic review and critical appraisal. Int. J. Environ. Res. Public Health17, (2020).