Funnel plot: Difference between revisions

Revision as of 16:37, 3 January 2011

Latest revision as of 19:47, 24 May 2025

(47 intermediate revisions by 38 users not shown)

Line 1:

{{Short description|Type of graph used in research}}

[[Image:Funnelplot.png|thumb|right|An example of a funnel plot.]]

[[File:Funnelplot.png|thumb|right|An example funnel plot showing no publication bias. Each dot represents a study (e.g. measuring the effect of a certain drug); the ''y''-axis represents study [[Precision (statistics)|precision]] (e.g. the inverse standard error or number of experimental subjects) and the ''x''-axis shows the study's result (e.g. the drug's measured average effect).]]

A '''funnel plot''' is a useful graph designed to check the existence of [[publication bias]] in [[systematic review]]s and [[meta-analysis|meta-analyses]].

A '''funnel plot''' is a graph designed to check for the existence of [[publication bias]]; funnel plots are commonly used in [[systematic review]]s and [[meta-analysis|meta-analyses]]. In the absence of publication bias, it assumes that studies with high precision will be plotted near the average, and studies with low precision will be spread evenly on both sides of the average, creating a roughly [[funnel]]-shaped distribution. Deviation from this shape can indicate publication bias.

== Quotation ==

Funnel plots, introduced by Light and Pillemer in 1984<ref>{{Cite book

| authorauthor1 = R. J. Light,
| author2 = D. B. Pillemer

| title = Summing up: The Science of Reviewing Research

| publisher = [[Harvard University Press]]

| year = 1984

| ___location = Cambridge, Massachusetts.

| isbn = 978-0-674-85431-4

}}</ref>

| url-access = registration

and discussed in detail by Egger and colleagues,<ref>{{Cite journal

| url = https://archive.org/details/summingupscience00ligh

| author = [[Matthias Egger]], [[G. Davey Smith]], [[M. Schneider]] & [[C. Minder]]

}}</ref>

and discussed in detail by [[Matthias Egger]] and colleagues,<ref>{{Cite journal

| author = [[Matthias Egger]], [[George Davey Smith|G. Davey Smith]], [[M. Schneider]] & [[C. Minder]]

| title = Bias in meta-analysis detected by a simple, graphical test

| journal = [[BMJ]]

| volume = 315

| issue = 7109

| pages = 629–624629–634

| year date=September 1997

| month = September

| pmid = 9310563

| url= | pmc = 2127453

| url = http://www.bmj.com/cgi/content/full/315/7109/629

| doi=10.1136/bmj.315.7109.629

| pmc = 2127453

}}</ref><ref name="SterneJ2001Funnel">{{Cite journal

| author author1= [[Jonathan A. C. Sterne]] & [[|author2=Matthias Egger]] | title = Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis

| title = Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis

| journal = [[Journal of Clinical Epidemiology]]

| volume = 54

| issue = 10

| pages = 1046–10451046–55

| year date=October 2001

| month = October

| pmid = 11576817

| doi = 10.1016/S0895-4356(01)00377-8

}}</ref>

are useful adjuncts to meta-analyses. A funnel plot is a [[scatterplot]] of treatment effect against a measure of study sizeprecision. It is used primarily as a visual aid tofor detecting bias or systematic [[Studystudy heterogeneity|systematic heterogeneity]]. A [[Symmetry|symmetric]] inverted funnel shape arises from a ‘well-behaved’ data set, in which publication bias is unlikely. An asymmetric funnel indicates a relationship between treatment effect estimate and study sizeprecision. This suggests the possibility of either [[publication bias]] or a systematic difference between smallerstudies of higher and largerlower studiesprecision (typically ‘small study effects’). Asymmetry can also arise from use of an inappropriate effect measure. Whatever the cause, an asymmetric funnel plot leads to doubts over the appropriateness of a simple meta-analysis and suggests that there needs to be investigation of possible causes.

A variety of choices of measures of ‘study size’precision’ is available, including total sample size, [[Standard error (statistics)|standard error]] of the treatment effect, and inverse [[variance]] of the treatment effect ([[Weight function|weight]]). Sterne and Egger have compared these with others, and conclude that the standard error is to be recommended.<ref name="SterneJ2001Funnel"/>

When the standard error is used, straight lines may be drawn to define a region within which 95% of points might lie in the absence of both [[Heterogeneousstudy heterogeneity|heterogeneity]] and publication bias.<ref name="SterneJ2001Funnel"/>

In common with [[confidence interval]] plots, funnel plots are conventionally drawn with the treatment effect measure on the [[Cartesian coordinate system|horizontal axis]], so that study sizeprecision appears on the vertical axis, breaking with the general rule. Since funnel plots are principally visual aids for detecting asymmetry along the treatment effect axis, this makes them considerably easier to interpret.

== Criticism ==

The funnel plot is not without problems.

If high -precision studies really are different thanfrom low -precision studies with respect to [[effect size]] (e.g., due to different populations examined) a funnel plot may give a wrong impression of publication bias.<ref>{{Cite journal

| author = [[Joseph Lau]], [[John P. A. Ioannidis]], [[Norma Terrin]], [[Christopher H. Schmid]] & [[Ingram Olkin]]

| title = The case of the misleading funnel plot

| journal = [[BMJ]]

| volume = 333

| issue = 7568

| pages = 597–590597–600

| year date=September 2006

| month = September

| doi = 10.1136/bmj.333.7568.597

| pmid = 16974018

Line 57 ⟶ 58:

}}</ref>

The appearance of the funnel plot can change quite dramatically depending on the scale on the y-axis &mdash; whether it is the inverse square error or the trial size.<ref>{{Cite journal

| author author1= [[Jin-Ling Tang]], [[|author2=Joseph LY Liu]] | title = Misleading funnel plot for detection of bias in meta-analysis

| title = Misleading funnel plot for detection of bias in meta-analysis

| journal = [[Journal of Clinical Epidemiology]]

| volume = 53

| issue = 5

| month date=May May2000

| year = 2000

| pages = 477&ndash;484

| doi = 10.1016/S0895-4356(99)00204-8

| pmid=10812319

}}</ref>

}}</ref> Researchers have a poor ability to visually discern publication bias from funnel plots.<ref>{{Cite journal|last1=Terrin|first1=N.|last2=Schmid|first2=C. H.|last3=Lau|first3=J.|year=2005|title=In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias|url=https://www.jclinepi.com/article/S0895-4356(05)00082-X/abstract|journal=Journal of Clinical Epidemiology|language=English|volume=58|issue=9|pages=894–901|doi=10.1016/j.jclinepi.2005.01.006|issn=0895-4356|pmid=16085192|url-access=subscription}}</ref>

== See also ==

Line 76:

{{Reflist}}

=== Further reading ===

* {{citation | last1=Sterne | first1=J. A. C. | last2=Sutton | first2=A. J. | last3=Ioannidis | first3=J. P. A. | last4=Terrin | first4=N. | display-authors=3 | title=Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials | journal=BMJ | volume=343 | pages=d4002 | year=2011 | doi=10.1136/bmj.d4002 | pmid=21784880| doi-access=free }}

* Adapted from ''[[Cochrane Collaboration|Cochrane]] Handbook for [[Systematic review|Systematic Reviews]] of Interventions''

* {{citation | last1=Higgins | first1=J.P.T. | last2=Thomas | first2=J. | last3=Chandler | first3=J. | last4=Cumpston | first4=M. | last5=Li | first5=T. | last6=Page | first6=M.J. | last7=Welch | first7=V.A. | title=Cochrane handbook for systematic reviews of interventions | publisher=Wiley Blackwell | edition=2nd | year=2019 | url=http://www.handbook.cochrane.org/ | isbn=9781119536611}}

{{Statistics}}

[[Category:Research methods]]

[[Category:Statistical charts and diagrams]]

[[Category:Meta-analysis]]

[[Category:Systematic review]]

[[de:Funnel plot]]