Content deleted Content added
m Open access bot: doi updated in citation with #oabot. |
|||
| (16 intermediate revisions by 6 users not shown) | |||
Line 1:
{{
▲{{more citations needed|date=March 2023}}{{Futures studies}}The '''Delphi method''' or '''Delphi technique''' ({{IPAc-en|ˈ|d|ɛ|l|f|aɪ}} {{respell|DEL|fy}}; also known as '''Estimate-Talk-Estimate''' or '''ETE''') is a structured communication technique or method, originally developed as a systematic, interactive [[forecasting]] method that relies on a panel of experts.<ref>{{cite journal | vauthors = Dalkey N, Helmer O | year = 1963 | title = An Experimental Application of the Delphi Method to the use of experts | journal = Management Science | volume = 9 | issue = 3| pages = 458–467 | doi=10.1287/mnsc.9.3.458| hdl = 2027/inu.30000029301680 | hdl-access = free }}</ref><ref>{{cite report | vauthors = Brown BB | title = Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts. | publisher = Rand Corp | ___location = Santa Monica CA | date = September 1968 | id = P-3925 | url = https://apps.dtic.mil/sti/citations/AD0675981 }}</ref><ref>{{cite report | vauthors = Sackman H | title = Delphi assessment: Expert opinion, forecasting, and group process. | publisher = The Rand Corporation | ___location = Santa Monica CA | id = R-1283-PR | date = 1974 | url =https://apps.dtic.mil/sti/citations/AD0786878 }}</ref><ref>{{cite report | vauthors = Brown T | title = An Experiment in Probabilistic Forecasting | publisher = The Rand Corporation | ___location = Santa Monica CA | id = R-944-ARPA | date = 1972 | url = https://apps.dtic.mil/sti/citations/AD0777061 }}</ref><ref>{{cite book |title=The Delphi Method: Techniques and Applications |url=http://is.njit.edu/pubs/delphibook/ |year=1975 | veditors = Linstone HA, Turoff M |archive-url=https://web.archive.org/web/20080520015240/http://is.njit.edu/pubs/delphibook/ |___location=Reading, Mass. |publisher=Addison-Wesley |isbn=978-0-201-04294-8 |archive-date=2008-05-20 |url-status=dead}}</ref> Delphi has been widely used for business forecasting and has certain advantages over another structured forecasting approach, [[prediction markets]].<ref name="Green_2008" />
Delphi can also be used to help reach expert consensus and develop professional guidelines.<ref name="Taylor2020"/> It is used for such purposes in many health-related fields, including clinical medicine, public health, and research.<ref name="Taylor2020"/><ref name=Moher2010/>
Delphi is based on the principle that forecasts (or decisions) from a structured group of individuals are more accurate than those from unstructured groups.<ref name="rw2001">{{cite book | vauthors = Rowe G, Wright G | author-link2 = George Wright (psychologist) | date = 2001 | chapter = Expert Opinions in Forecasting: The Role of the Delphi Technique | veditors = Armstrong | title = Principles of Forecasting: A Handbook of Researchers and Practitioners | series = International Series in Operations Research & Management Science | volume = 30 | pages = 125–144 | ___location = Boston | publisher = Kluwer Academic Publishers | doi = 10.1007/978-0-306-47630-3_7 | isbn = 978-0-7923-7401-5 | chapter-url = https://www3.nd.edu/~busiforc/handouts/Other%20Articles/expertopinions.pdf }}</ref> The experts answer questionnaires in two or more rounds. After each round, a [[facilitator]] or change agent<ref>{{Cite journal | vauthors = McLaughlin MW |date=1990 |title=The Rand Change Agent Study Revisited: Macro Perspectives and Micro Realities |jstor =1176973 |journal=Educational Researcher |volume=19 |issue=9 |pages=11–16 |doi=10.3102/0013189X019009011 |issn=0013-189X}}</ref> provides an anonymised summary of the experts' forecasts from the previous round as well as the reasons they provided for their judgments. Thus, experts are encouraged to revise their earlier answers in light of the replies of other members of their panel. It is believed that during this process the range of the answers will decrease and the group will converge towards the "correct" answer. Finally, the process is stopped after a predefined stopping criterion (e.g., number of rounds, achievement of consensus, stability of results), and the [[mean]] or [[median]] scores of the final rounds determine the results.<ref name="rw1999">{{cite journal | vauthors = Rowe G, Wright G | title = The Delphi technique as a forecasting tool: issues and analysis. | journal = International Journal of Forecasting | date = October 1999 | volume = 15 | issue = 4 | pages = 353–375 | doi = 10.1016/S0169-2070(99)00018-7 | s2cid = 10745965 }}</ref>
Special attention has to be paid to the formulation of the Delphi theses and the definition and selection of the experts in order to avoid methodological weaknesses that severely threaten the validity and reliability of the results.<ref>{{cite journal | vauthors = Markmann C, Spickermann A, von der Gracht HA, Brem A | title = Improving the question formulation in Delphi-like surveys: Analysis of the effects of abstract language and amount of information on response behavior. | journal = Futures & Foresight Science | date = March 2021 | volume = 3 | issue = 1 |
Ensuring that the participants have requisite expertise and that more domineering participants do not overwhelm weaker-willed participants, as the first group tends to be less inclined to change their minds and the second group is more motivated to fit in, can be a barrier to reaching true consensus. <ref>{{cite journal | vauthors = Shang, Zshida| title = Use of Delphi in health sciences research: A narrative review. | journal = Medicine (Baltimore)| date = Feb 2023 | volume = 102 | issue = 7 | pages = e32829 | doi = 10.1097/MD.0000000000032829 | pmid = 36800594| pmc = 9936053 }}</ref>
==History==
Line 17 ⟶ 18:
Experts were asked to give their opinion on the probability, frequency, and intensity of possible enemy attacks. Other experts could anonymously give feedback. This process was repeated several times until a consensus emerged.
In 2021, a cross-disciplinary study by Beiderbeck et al. focused on new directions and advancements of the Delphi method, including [[Real-time Delphi]] formats. The authors provide a methodological toolbox for designing Delphi surveys including among others sentiment analyses of the field of psychology.<ref name="Beiderbeck_2021">{{cite journal | vauthors = Beiderbeck D, Frevel N, von der Gracht HA, Schmidt SL, Schweitzer VM | title = Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements | journal = MethodsX | volume = 8 | issue = |
==Key characteristics==
Line 65 ⟶ 66:
=== Use in health settings ===
The Delphi technique is widely used to help reach expert consensus in health-related settings.<ref name="Taylor2020"/> For example, it is frequently employed in the development of [[medical guideline]]s and [[Protocol (science)|protocol]]s.<ref name="Taylor2020">{{cite journal |vauthors=Taylor E |title=We Agree, Don't We? The Delphi Method for Health Environments Research |journal=HERD |volume=13 |issue=1 |pages=11–23 |date=2020 |pmid=31887097 |doi=10.1177/1937586719887709 |s2cid=209519275 |url=https://journals.sagepub.com/doi/pdf/10.1177/1937586719887709|url-access=subscription }}</ref>
====
Some examples of its application in [[public health]] contexts include [[metabolic dysfunction–associated steatotic liver disease|non-alcoholic fatty liver disease]],<ref>{{cite journal | vauthors = Lazarus JV, Mark HE, Anstee QM, Arab JP, Batterham RL, Castera L, Cortez-Pinto H, Crespo J, Cusi K, Dirac MA, Francque S, George J, Hagström H, Huang TT, Ismail MH, Kautz A, Sarin SK, Loomba R, Miller V, Newsome PN, Ninburg M, Ocama P, Ratziu V, Rinella M, Romero D, Romero-Gómez M, Schattenberg JM, Tsochatzis EA, Valenti L, Wong VW, Yilmaz Y, Younossi ZM, Zelber-Sagi S | display-authors = 6 | title = Advancing the global public health agenda for NAFLD: a consensus statement | journal = Nature Reviews. Gastroenterology & Hepatology | volume = 19 | issue = 1 | pages = 60–78 | date = January 2022 | pmid = 34707258 | doi = 10.1038/s41575-021-00523-4 | s2cid = 239891445 | doi-access = free | hdl = 11424/243850 | hdl-access = free }}</ref> iodine deficiency disorders,<ref>{{cite journal | vauthors = Schaffner M, Rochau U, Stojkov I, Qerimi Rushaj V, Völzke H, Marckmann G, Lazarus JH, Oberaigner W, Siebert U | display-authors = 6 | title = Barriers Against Prevention Programs for Iodine Deficiency Disorders in Europe: A Delphi Study | journal = Thyroid | volume = 31 | issue = 4 | pages = 649–657 | date = April 2021 | pmid = 32912084 | doi = 10.1089/thy.2020.0065 | s2cid = 221622474 }}</ref> building responsive health systems for communities affected by migration,<ref>{{cite journal | vauthors = Pottie K, Hui C, Rahman P, Ingleby D, Akl EA, Russell G, Ling L, Wickramage K, Mosca D, Brindis CD | display-authors = 6 | title = Building Responsive Health Systems to Help Communities Affected by Migration: An International Delphi Consensus | journal = International Journal of Environmental Research and Public Health | volume = 14 | issue = 2 | pages = 144 | date = February 2017 | pmid = 28165380 | pmc = 5334698 | doi = 10.3390/ijerph14020144 | doi-access = free }}</ref> the role of health systems in advancing well-being for those living with HIV,<ref>{{cite journal | vauthors = Lazarus JV, Safreed-Harmon K, Kamarulzaman A, Anderson J, Leite RB, Behrens G, Bekker LG, Bhagani S, Brown D, Brown G, Buchbinder S, Caceres C, Cahn PE, Carrieri P, Caswell G, Cooke GS, Monforte AD, Dedes N, Del Amo J, Elliott R, El-Sadr WM, Fuster-Ruiz de Apodaca MJ, Guaraldi G, Hallett T, Harding R, Hellard M, Jaffar S, Kall M, Klein M, Lewin SR, Mayer K, Pérez-Molina JA, Moraa D, Naniche D, Nash D, Noori T, Pozniak A, Rajasuriar R, Reiss P, Rizk N, Rockstroh J, Romero D, Sabin C, Serwadda D, Waters L | display-authors = 6 | title = Consensus statement on the role of health systems in advancing the long-term well-being of people living with HIV | journal = Nature Communications | volume = 12 | issue = 1 |
====
Use of the Delphi method in the development of guidelines for the reporting of health research<ref name=Moher2010>{{cite journal | vauthors = Moher D, Schulz KF, Simera I, Altman DG | title = Guidance for developers of health research reporting guidelines | journal = PLOS Medicine | volume = 7 | issue = 2 | pages = e1000217 | date = February 2010 | pmid = 20169112 | pmc = 2821895 | doi = 10.1371/journal.pmed.1000217 | doi-access = free }}</ref> is recommended, especially for experienced developers.<ref name="EQUATOR">{{cite web |title=Developing your reporting guideline |url=https://www.equator-network.org/toolkits/developing-a-reporting-guideline/developing-your-reporting-guideline/ |website=www.equator-network.org |publisher=[[EQUATOR Network]] |archive-url=https://web.archive.org/web/20220929172737/https://www.equator-network.org/toolkits/developing-a-reporting-guideline/developing-your-reporting-guideline/ |archive-date=29 September 2022 |language=en |url-status=live}}</ref> Since this advice was made in 2010, two systematic reviews have found that fewer than 30% of published reporting guidelines incorporated Delphi methods into the development process.<ref>{{cite journal | vauthors = Wang X, Chen Y, Yang N, Deng W, Wang Q, Li N, Yao L, Wei D, Chen G, Yang K | display-authors = 6 | title = Methodology and reporting quality of reporting guidelines: systematic review | journal = BMC Medical Research Methodology | volume = 15 | issue = 74 |
=== Online Delphi systems ===
Line 85 ⟶ 86:
The Argument Delphi, developed by Osmo Kuusi, focuses on ongoing discussion and finding relevant arguments rather than focusing on the output. The Disaggregative Policy Delphi, developed by Petri Tapio, uses cluster analysis as a systematic tool to construct various scenarios of the future in the latest Delphi round.<ref>{{cite journal | vauthors = Tapio P | year = 2003 | title = Disaggregative Policy Delphi: Using cluster analysis as a tool for systematic scenario formation | journal = Technological Forecasting and Social Change | volume = 70 | issue = 1| pages = 83–101 | doi = 10.1016/S0040-1625(01)00177-9 | s2cid = 53516828 }}</ref> The respondent's view on the probable and the preferable future are dealt with as separate cases. The computerization of Argument Delphi is relatively difficult because of several problems like argument resolution, argument aggregation and argument evaluation. The computerization of Argument Delphi, developed by [[Sadi Evren Seker]], proposes solutions to such problems.<ref>{{cite journal | vauthors = Seker SE | year = 2015 | title = Computerized Argument Delphi Technique | journal = IEEE Access | volume = 3 | issue = 2| pages = 368–380 | doi = 10.1109/ACCESS.2015.2424703 | doi-access = free | bibcode = 2015IEEEA...3..368S }}</ref>
A fast-track Delphi was developed to provide consensual expert opinion on the state of scientific knowledge in public health crises.<ref name=fast-track/> It can provide results within three weeks, while the conventional Delphi can take several months
==Accuracy==
Line 92 ⟶ 93:
Overall the track record of the Delphi method is mixed.<ref name=":1">Khodyakov, D., Grant, S., Kroger, J., Bauman, M. (2023). ''RAND methodological guidance for conducting and critically appraising Delphi panels.'' RAND Corporation. www.rand.org/t/TLA3082-1 https://doi.org/10.7249/tla3082-1</ref> There have been many cases when the method produced poor results. Still, some authors attribute this to poor application of the method and not to the weaknesses of the method itself. The ''RAND Methodological Guidance for Conducting and Critically Appraising Delphi Panels'' is a manual for doing Delphi research which provides guidance for doing research and offers a appraisal tool.<ref name=":1" /> This manual gives guidance on best practices that will help to avoid, or mitigate, potential drawbacks of Delphi Method Research; it also helps to understand the confidence that can be given to study results.
It must also be realized that in areas such as science and technology forecasting, the degree of uncertainty is so great that exact and always correct predictions are impossible, so a high degree of error is to be expected. An important challenge for the method is ensuring sufficiently knowledgeable panelists. If panelists are misinformed about a topic, the use of Delphi may only add confidence to their ignorance.<ref name = "Green_2008">{{cite journal | vauthors = Green KC, Armstrong JS, Graefe A | title = Methods to elicit forecasts from groups: Delphi and prediction markets compared. | journal = Foresight: The International Journal of Applied Forecasting | date = June 2008 | volume = 8 | pages = 17–20 | doi = 10.2139/ssrn.1153124 | url = https://repository.upenn.edu/cgi/viewcontent.cgi?article=1168&context=marketing_papers | doi-access = free }}</ref>
One of the initial problems of the method was its inability to make complex forecasts with multiple factors. Potential future outcomes were usually considered as if they had no effect on each other. Later on, several extensions to the Delphi method were developed to address this problem, such as [[cross impact analysis]], that takes into consideration the possibility that the occurrence of one event may change probabilities of other events covered in the survey. Still the Delphi method can be used most successfully in forecasting single scalar indicators.
| |||