Content deleted Content added
Andi Fugard (talk | contribs) No edit summary |
→Evaluating collective impact: fix broken link and add archive url for Guide to Evaluating Collective Impact |
||
| (2 intermediate revisions by 2 users not shown) | |||
Line 10:
A wide range of different titles are applied to program evaluators, perhaps haphazardly at times, but there are some established usages: those who regularly use program evaluation skills and techniques on the job are known as '''program analysts'''<!-- only this title is bolded, having been made a target of a #REDIRECT page from [[Program Analyst (administration)]], as a primary search term to land here, parallel to Policy Analyst in the related profession; please do NOT bold others without talk page discussion -->; those whose positions combine [[administrative assistant]] or [[secretary]] duties with program evaluation are known as program assistants, program clerks (United Kingdom), program support specialists, or program associates; those whose positions add lower-level [[project management]] duties are known as Program Coordinators.
The process of evaluation is considered to be a relatively recent phenomenon. However, planned social evaluation has been documented as dating as far back as 2200 BC.<ref>{{cite book|author=Shadish, W. R.|author2=Cook, T. D.|author3=Leviton, L. C.|title=Foundations of program evaluation: Theories of practice.|year=1991|publisher=Sage.|___location=Newbury Park, CA}}</ref> Evaluation became particularly relevant in the United States in the 1960s during the period of the [[Great Society]] social programs associated with the [[John F. Kennedy|Kennedy]] and [[Lyndon Johnson|Johnson]] administrations.<ref>{{cite web|url=http://www.dol.gov/oasam/programs/history/dolchp06.htm|archive-url=https://web.archive.org/web/20080413170714/http://www.dol.gov/oasam/programs/history/dolchp06.htm|url-status=dead|archive-date=April 13, 2008|title=U.S. Department of Labor – Brief History of DOL – Eras of the New Frontier and the Great Society, 1961-1969|publisher=dol.gov}}</ref><ref>[https://www.archives.gov/research/guide-fed-records/groups/051.html National Archives, Records of the Office of Management and Budget (1995) ''51.8.8 Records of the Office of Program Evaluation'']</ref>
Program evaluations can involve both [[quantitative method|quantitative]] and [[qualitative method]]s of [[social research]]. People who do program evaluation come from many different backgrounds, such as [[sociology]], [[psychology]], [[economics]], [[social work]], as well as [[political science]] subfields such as [[public policy]] and [[public administration]] who have studied a similar methodology known as [[policy analysis]]. Some universities also have specific training programs, especially at the [[Postgraduate education|postgraduate]] level in program evaluation, for those who studied an [[Undergraduate education|undergraduate]] subject area lacking in program evaluation skills.<ref>American Evaluation Association's [https://www.eval.org/Education-Programs/University-Programs list of university programs].</ref>
Line 36:
This can be difficult considering that child abuse is not a public behavior, also keeping in mind that estimates of the rates on private behavior are usually not possible because of factors like unreported cases. In this case evaluators would have to use data from several sources and apply different approaches in order to estimate incidence rates. There are two more questions that need to be answered:<ref name="Barbazette">{{cite report|last=Barbazette |first=J. |year=2006 |title=What is needs assessment? |url=http://media.wiley.com/product_data/excerpt/57/07879752/0787975257.pdf }}</ref>
Evaluators need to also answer the ’how’ and ‘what’ questions<ref name=Barbazette/> The ‘how’ question requires that evaluators determine how the need will be addressed. Having identified the need and having familiarized oneself with the community evaluators should conduct a performance analysis to identify whether the proposed plan in the program will actually be able to eliminate the need. The ‘what’ question requires that evaluators conduct a [[task analysis]] to find out what the best way to perform would be. For example, whether the job performance standards are set by an organization or whether some governmental rules need to be considered when undertaking the task.<ref name=Barbazette/>
Third, define and identify the target of interventions and accurately describe the nature of the service needs of that population<ref name=Rossi/>
Line 49:
#:Evaluators need to compare current situation to the desired or necessary situation. The difference or the gap between the two situations will help identify the need, purpose and aims of the program.
#Identify priorities and importance
#:In the first step above, evaluators would have identified a number of interventions that could potentially address the need e.g. training and development, [[organization development]] etc. These must now be examined in view of their significance to the program's goals and constraints. This must be done by considering the following factors: cost effectiveness (consider the budget of the program, assess cost/benefit ratio), executive pressure (whether top management expects a solution) and population (whether many key people are involved).
# Identify causes of performance problems and/or opportunities
#:When the needs have been prioritized the next step is to identify specific problem areas within the need to be addressed. And to also assess the skills of the people that will be carrying out the interventions.
Line 141:
* '''Early phase:''' CI participants are exploring possible strategies and developing plans for action. Characterized by uncertainty.
''Recommended evaluation approach:'' Developmental evaluation to help CI partners understand the context of the initiative and its development:<ref>{{cite web
''Recommended evaluation approach:'' Developmental evaluation to help CI partners understand the context of the initiative and its development:<ref>{{cite web|last1=Preskill|first1=Hallie|last2=Parkhurst|first2=Marcie|last3=Juster|first3=Jennifer Splansky|title=Guide to Evaluating Collective Impact|url=http://www.fsg.org/Portals/0/Uploads/Documents/PDF/Evaluating_Collective_Impact_Assessing_Progress_2.pdf|website=www.fsg.org|publisher=FSG, Inc.|access-date=2014-09-19}}{{Dead link|date=November 2018 |bot=InternetArchiveBot |fix-attempted=yes }}</ref> "Developmental evaluation involves real time feedback about what is emerging in complex dynamic systems as innovators seek to bring about systems change."<ref>{{cite web|last1=Patton|first1=Michael|title=Evaluation Approaches and Techniques|url=http://tei.gwu.edu/evaluation-approaches-and-techniques#developmental-evaluation|website=The Evaluators' Institute|publisher=George Washington University|access-date=2014-09-19}}</ref>▼
| last1 = Preskill
| first1 = Hallie
| last2 = Parkhurst
| first2 = Marcie
| last3 = Juster
| first3 = Jennifer Splansky
| title = Guide to Evaluating Collective Impact
| url = https://www.fsg.org/resource/guide-evaluating-collective-impact/#resource-downloads
| date = 2014-05-05
| access-date = 2025-08-19
| website = www.fsg.org
| publisher = FSG, Inc.
| url-status = live
| archive-url = https://web.archive.org/web/20250502000322/https://www.fsg.org/resource/guide-evaluating-collective-impact/#resource-downloads
| archive-date = 2025-05-02
}}
▲
* '''Middle phase:''' CI partners implement agreed upon strategies. Some outcomes become easier to anticipate.
Line 166 ⟶ 183:
===Budget constraints===
Frequently, programs are faced with budget constraints because most original projects do not include a budget to conduct an evaluation (Bamberger et al., 2004). Therefore, this automatically results in evaluations being allocated smaller budgets that are inadequate for a rigorous evaluation. Due to the budget constraints it might be difficult to effectively apply the most appropriate methodological instruments. These constraints may consequently affect the time available in which to do the evaluation (Bamberger et al., 2004).<ref name="Bamberger" /> Budget constraints may be addressed by simplifying the evaluation design, revising the sample size, exploring economical data collection methods (such as using volunteers to collect data, shortening surveys, or using focus groups and key informants) or looking for reliable [[secondary data]] (Bamberger et al., 2004).<ref name="Bamberger" />
===Time constraints===
| |||