Learning and Evaluation/Archive/Connect/Questions: Difference between revisions

Revision as of 17:15, 23 December 2013

Latest revision as of 21:31, 12 December 2023

(30 intermediate revisions by 14 users not shown)

Line 1:

<div style="margin:2px 0 0;background:#2E2C2C;border-radius:1em 1em 0 0;padding: 1em 2em 0;font-family:'Hoefler Text',Constantia,Gabriola,Georgia,Cambria,'Palatino Linotype','Baskerville Old Face',Garamond,'Times New Roman',serif;font-size:1.2em;color:#F7F6F1;opacity:85%;">

<languages/>

{{TNT|Evaluation/Header

|page summary=This is the parlor discussion page.

|links=

|more content=<div style="color: #2E2C2C; text-align:center">{{Programs:Evaluation_portal/Parlor/Questions/Question-form}}</div>

}}

{{TOCright|width=40%}}

{{User:MiszaBot/config

|minthreadsleft = 2

|minthreadstoarchive = 1

|algo = old(30d)

|archive = Programs:Evaluation portal/Parlor/Questions/Archives/%(year)d-%(month)02d

}}

<div style="float:right;margin-left:0.5em;width:47%;text-align:right;">

== Have you created or collected surveys for your program implementations? ==

<!--

-->[[Special:MyLanguage/Grants:Evaluation/Library|<!--

--><span<!--

--> style="font-size: {{{top font-size}}}; color: #F7F6F1; border-bottom: 1px dotted #F7F6F1;"<!--

--> <!--

-->><span title="

Links to case studies and evaluation resources">

[[Programs:Evaluation_portal/Library/Overview#Program_Leader|Program leaders]] have been inquiring about survey design since the creation of the [[Programs:Evaluation_portal/Library/Overview#Program_Evaluation_and_Design_Team|Program Evaluation and Design team]] here at the Wikimedia Foundation.

Library</span><!--

We’re excited to let you know that we are proceeding with exploring how we can support your efforts to survey participants in your programs. In order to proceed, we need your help! '''We would like you to share any surveys that you have sent and collected with our team.'''

--></span><!--

-->]]<!--

-->&nbsp;'''·''' <!--

-->[[Special:MyLanguage/Grants:Evaluation/Tools|<!--

--><span<!--

--> style="font-size: {{{top font-size}}}; color: #F7F6F1; border-bottom: 1px dotted #F7F6F1;"<!--

--> <!--

-->><span title="

Tools for evaluation">

This will allow us to learn more about what types of information you are collecting through surveying, and the types of surveys you are producing. This will also '''allow us to work with you to build tools for better evaluation and surveying'''!

[[File:Students filling out surveys.JPG|thumbnail|Surveys are important tools in evaluation!]]

#You can email me your surveys in email or as attachments ({{nospam|sarah|wikimedia.org}})

#You can email me a link to your survey ({{nospam|sarah|wikimedia.org}})

#You can post a link to you survey on wiki in response to this post.

#It can be in any language

Tools</span><!--

We’ve already identified a number of survey priorities, and while survey development will be ongoing, we want to be sure to include the interests of everyone in our planning process. As we identify, prioritize, and begin to take steps to meet the survey needs of you and other [[Programs:Evaluation_portal/Library/Overview#Program_Leader|program leaders]], '''the more you are able to share the better we can learn from what you’ve done, and the better we can respond to everyone’s needs'''. If you have developed or used surveys to help tell the story of your program, please send them our way so we are sure to review and consider everyone's interests.

--></span><!--

-->]]<!--

-->&nbsp;'''·''' <!--

-->[[Special:MyLanguage/Grants:Evaluation/Parlor|<!--

--><span<!--

--> style="font-size: {{{top font-size}}}; color: #F7F6F1; border-bottom: 1px dotted #F7F6F1;"<!--

--> <!--

-->><span title="

Introduce yourself and interact with people in evaluation">

We look forward to your sharing and thank you for your time and amazing work. -- [[User:SarahStierch|SarahStierch]] ([[User talk:SarahStierch|talk]]) 23:57, 11 December 2013 (UTC)

Parlor</span><!--

:Hi, the main feedback form for training events/editathons that we use is this: [[wmuk:File:Event_feedback_form_template.pdf]]. We have it set up in Survey Monkey so that data can be input and we are able to run summary reports to analyse the results. The form was seen previously by the PE&D team and the feedback was very useful! [[User:Daria Cybulska (WMUK)|Daria Cybulska (WMUK)]] ([[User talk:Daria Cybulska (WMUK)|talk]]) 09:03, 12 December 2013 (UTC)

--></span><!--

:: A copy of the only survey TWG has done so far can be found [https://docs.google.com/a/fanhistory.com/forms/d/1kvLJYT8tKFmOoK-7mb8yWDc9yojT8b2nEElgwepCQhc/viewform here]. The results have been used in two pieces of research we have done to identify costs related to our work- --[[User:LauraHale|LauraHale]] ([[User talk:LauraHale|talk]]) 18:30, 15 December 2013 (UTC)

-->]]<!--

:::Great, thanks [[User:LauraHale]] and [[User:Daria Cybulska (WMUK)]]! [[User:SarahStierch|SarahStierch]] ([[User talk:SarahStierch|talk]]) 19:26, 16 December 2013 (UTC)

-->&nbsp;'''·''' <!--

===CIS-A2K Evaluation Forms===

-->[[Special:MyLanguage/Grants:Evaluation/News|<!--

Hi, we at [[India Access To Knowledge|CIS-A2K]] have recently recruited a person into the team. As part of this I have used [https://docs.google.com/forms/d/1HB_47S2M0tnMWN15zl5P1MlRlF9plH39JIHnm3y0Nk4/viewform this evaluation form] to seek feedback from the community members. Will share some more soon :) --[[User:Visdaviva|Visdaviva]] ([[User talk:Visdaviva|talk]]) 06:54, 21 December 2013 (UTC)

--><span<!--

:Wikimedia [[India Access To Knowledge/Events/Train the Trainer Program|Train the Trainer Program]] [https://docs.google.com/forms/d/1iLXAzDAbWWbZQn2_TbOb3XZEzkVSN-g6EdEl8pPYMM8/viewform participant feedback form] —unsigned by [[User:Visdaviva|Visdaviva]] ([[User talk:Visdaviva|talk]])

--> style="font-size: {{{top font-size}}}; color: #F7F6F1; border-bottom: 1px dotted #F7F6F1;"<!--

::Great! Thanks [[Usre:Visdaviva]]! Two questions: Would you like feedback for improvement (if needed) from our team? Second, if so, do you have a deadline for one or both of the surveys that you would need that feedback by? Thanks again for sharing! [[User:SarahStierch|SarahStierch]] ([[User talk:SarahStierch|talk]]) 17:15, 23 December 2013 (UTC)

--> <!--

-->><span title="

Current events, blogs and news">

=== Dutch Evaluation Forms ===

News</span><!--

To respond to the question about surveys:

--></span><!--

We are quite busy with streamlining our entire workflow from idea to evaluation for events. It all starts with a checklist, more of a grocery list with the forms that are needed for an activity. These depend on:

-->]]<!--

* Is there financial support needed for the activity?

-->&nbsp;'''·''' <!--

** If yes: you need a google spreadsheet for the costs

-->[[Special:MyLanguage/Grants:Evaluation/Boiler room|<!--

* Is support from the office needed in organising the activity?

--><span<!--

** If yes: We need to reserve and monitor the hours of staff involved

--> style="font-size: {{{top font-size}}}; color: #26A9E0; border-bottom: 1px dotted #26A9E0;"<!--

* Will there be a training for new editors?

--> <!--

** If yes: you need two surveys and [[wmnl:Standaardopzet_projectpagina|two project pages]] (work in progress). [https://docs.google.com/a/wikimedia.nl/forms/d/1U0lWN39STgGSCjprMnwz9LbBqBs4tnVgMWVHM5NSHS4/viewform The first survey] should be held during event and [https://docs.google.com/a/wikimedia.nl/forms/d/1Ph5bkmjl3jaZbQxEPoUpeBE4qLtQLI9x4aFWkBmwASg/viewform the second survey] after a couple of months. The project pages are on our chapter wiki and on the Dutch Wikipedia. The project page on our chapter wiki consists of all the basic information (type of activity, date, ___location, etc.). The project page on Wikipedia consists of the Wikipedia related information (articles to write, articles that need improvement, who's working on the article, etc.)

-->><span title="

* Do people have to register to attend?

** If yes: you need a registration form (currently a google form, this will be a form linked to our CRM in the near future)

Once someone has filled in the grocery list we help them setting up the documents. The forms for the costs, surveys and registration just have to duplicated by a staff member and shared with the volunteers that help with the activity. The reasons that a staff member duplicates these forms are:

# Surveys can not be made public available (downloadable) like spreadsheets and text documents.

# This way we - the staff members- always can get to the information and fill in the information fields that we don't expect volunteers to do (for example: every event will get a unique identifier that needs to filled in on all documents).

Portal planning and maintenance area">

All the forms and documents mentioned and linked above are in Dutch, please let me know if you want anything translated. [[User:Ter-burg|Ter-burg]] ([[User talk:Ter-burg|talk]]) 13:30, 12 December 2013 (UTC)

::Thanks [[User:Ter-burg]], we will let you know if we have any questions. Are you seeking any feedback for your survey questions? [[User:SarahStierch|SarahStierch]] ([[User talk:SarahStierch|talk]]) 19:27, 16 December 2013 (UTC)

::: Yes, [[User:SarahStierch|Sarah]]. If you have any tips for our surveys that would be great. I'm especially interested if there is specific data that would make our events internationally comparable. I've made a lot of progress since this post. We now have a "[[wmnl:Handleiding_projecten|manual]]" on events that leads a volunteer through all necessary steps of setting up an event with a lot of templates and standard forms. These surveys are part of this process.

Leave feedback</span><!--

--></span><!--

-->]]<!--

-->

[[File:Three white gear shapes.svg|75px|right|link=Special:MyLanguage/Grants:Evaluation]]

{{clear}}

[[File:Internet-group-chat.svg|20px|link=Special:MyLanguage/Grants:Evaluation/Parlor/Questions]]

<!--

-->[[Special:MyLanguage/Grants:Evaluation/Parlor/Questions|<!--

--><span<!--

--> style="font-size: {{{top font-size}}}; color: #26A9E0; border-bottom: 1px dotted #26A9E0;"<!--

--> <!--

-->><span title="

Ask questions and discuss program evaluation">

:::Oh, and I do have a question. Can you translate the following questions from survey one for me? Thank you.

::::#Was vooraf duidelijk wat het evenement in zou houden? informatie via website, facebook, email

Ask questions and discuss program evaluation</span><!--

::::: Translation: ''Did you know up front what the event would be like?'' Sometimes visitors come to our events with different expectations. With this question we hope to determine if we've managed their expectations in a good way.

--></span><!--

::::#"MEE BEZIG gebruikt bij artikelen" (from Wat heeft u tijdens de edit-a-thon kunnen doen?)

-->]]<!--

:::::Translation: ''I've used the template that indicates I'm still working on the article.'' We have a template that (new) users can add to a page to indicatie that they're working on it at that exact moment to hold off more experienced and faster editors that might delete their edits before they even have the chance to finish their work. We ask all the participants at an edit-a-thin to use this template, but sometimes they forget to use it.

-->

::::#"Een afbeelding geplaatst" (from Wat heeft u tijdens de edit-a-thon kunnen doen?)

</div>

:::::Translation: ''I've added an image to an article''

:::[[User:SarahStierch|SarahStierch]] ([[User talk:SarahStierch|talk]]) 01:03, 17 December 2013 (UTC)

<div style="float:left;margin-left:0.5em;width:49%;">

:::: [[User:Ter-burg|Ter-burg]] ([[User talk:Ter-burg|talk]]) 10:05, 17 December 2013 (UTC)

This is the parlor discussion page.

[[File:Two white gear shapes.svg|55px|left|link=Special:MyLanguage/Grants:Evaluation]]

</div>

<div style="color: #2E2C2C; text-align:center">{{TNT|Grants:Evaluation/Parlor/Questions/Question-form| uselang={{uselang}} }}</div>

{{clear}}

</div>{{Historical|category=[[Category:Evaluation archive]]}}

{{TOCright|width=auto}}

{{archive box auto|base=Grants:Evaluation/Parlor/Questions/Archives/|auto=dated}}<!--

-->{{User:MiszaBot/config

|minthreadsleft = 2

|minthreadstoarchive = 1

|algo = old(30d)

|archive = Grants:Evaluation/Parlor/Questions/Archives/%(year)d

}}

== Rationale for the privacy form ==

I can not understand the rationale for having participants in a course/event/edit-a-thon to [[ProgramsGrants:Evaluation_portalEvaluation/Shop/Wikimetrics/Forms|fill a form to opt-in in Wikimetrics]]. In principle I could get any set of random user names from the history of any page, create a CSV file, upload it on Wikimetrics and analyse the users' activity. Without even meeting anybody. So why for events do we need an opt-in form? Again, in principle I could do this kind of analysis downloading the complete dump of a given edition of Wikipedia and write some tools to calculate the some things that Wikimetrics does. All that without asking anybody.

Given this premise, with Wikimedia Italia we organized recently a course for librarians (see the [[:w:it:Progetto:GLAM/Biblioteche/Progetti/Wikipedia_e_bibliotecari:_un_incontro_possibil|project page]]) where every participant created a personal account. We then asked everybody to put their usernames in that page so that we could monitor their progress with Wikimetrics. We did not know about this opt-in form until today and so they have not signed it, even if they have been explicitly warned during the course. What can we do? Thank you. -- [[User:CristianCantoro|CristianCantoro]] ([[User talk:CristianCantoro|talk]]) 11:37, 10 December 2013 (UTC)

:Thank you [[User:CristianCantoro|Cristian]] for your comment and chance for clarification. You are correct that, in many cases, participant user names are a matter of public record, in that case it may not be necessary, but a matter of operating in a completely open manner in which participants actually approve of their being tracked and monitored via Wikimetrics which make it rather simple to do so. Importantly, Wikimetrics increases accessibility to metric data in a way that user names are transferred to Wikimedia Foundation (internationally to the United States) and to best cover everyones privacy interests and protections we need to allow them to opt in, as well as out, of this monitoring when we know we are collecting usernames explicitly for this purpose. The particular language and guidance for disclosing the who, what, where, and why that have been thoughtfully crafted by the WMF legal team to be sure privacy interests are protected and transparency is upheld. This is a shift in practice that we are working to assist program leaders in coordinating so that the future collection of usernames will best protect all parties interested. The usernames you already have public record of are still useable, we just recommend that you also allow people to opt out should they become aware of this usage of their usernames and wish to be excluded from monitoring (you may want to post the language and possibly instructions for "opting out" on the page they added their names to). I hope this helps, please let me know if I can answer any further. [[User:JAnstee (WMF)|JAnstee (WMF)]] ([[User talk:JAnstee (WMF)|talk]]) 21:58, 10 December 2013 (UTC)

Line 93 ⟶ 123:

::::Can you perhaps expand on the point you're implying by adding this citation needed tag, Nemo? It's not very helpful on its own. [[User:Jtmorgan|Jtmorgan]] ([[User talk:Jtmorgan|talk]]) 21:43, 18 December 2013 (UTC)

==Wikimania evaluation ==

I am curious, how is Program Evaluation planning to instrument Wikimania and evaluate Wikimania this year? Also, will Program Evaluation provide a guide to Wikimania participants about how to maximize the "impact" of their participation? Some informal discussion about how to get value out of the Wikimania hackathon has already happened on the tech mailing list. Thanks, --<font style="white-space:nowrap;text-shadow:#008C3A 0.1em 0.1em 1.5em,#01796F -0.1em -0.1em 1.5em;color:#000000">[[User:Pine|<font color="#01796F"><b>Pine</b></font>]][[User talk:Pine|<font color="#01796F">^✉</font>]]</font> 19:16, 23 July 2014 (UTC)

:Hello Pine. Somehow missed this question. We have been working with the organizers of this year's Wikimania to develop an exit survey similar to that which was done following the Wikimedia Conference in Berlin (View their very recent reporting and link to survey data [[Wikimedia_Conference_2014/Feedback_evaluation|here]]). Similarly, the Wikimania survey targets goals of learning, networking and collaboration and will include measurement of the conference content overall; the organization and logistics of the conference; evaluation of the themes and their related content presented in terms of usefulness; participant networking and collaboration; overall learning outcomes; participant satisfaction and basic background demographics. The items are currently under final review and comment with a number of current, past, and future Wikimania planning stakeholders. This survey will also link to an optional set of items to assess survey-based outcomes for hackathon participants as well. While also still in the planning stage with the planning team, those items will likely include measurement of perceptions about the structure of hackathon days and projects, opportunities for mentoring, guidance, and collaboration and the usefulness and outcomes and/or potential outcomes of those opportunities. In addition to this exit survey, Ed Saperia is arranging for participation counts to be taken at each of the sessions and pushing to have people share lessons learned and hot conversations via twitter to the conference hashtag for potential content analysis. [[User:JAnstee (WMF)|JAnstee (WMF)]] ([[User talk:JAnstee (WMF)|talk]]) 19:24, 28 July 2014 (UTC)

::Thanks [[User:JAnstee (WMF)]]. This is good to hear. Will you also be evaluating the online impact of Wikimania particularly as it relates to active editor statistics and editor diversity? Also, would you be willing to adapt or create a survey to evaluate the Research Hackathon next week? If you can draft a survey I can review it with the other event organizers. It may be possible for us to simply adapt the dev hackathon survey for the Research Hackathon. Thanks, --<font style="white-space:nowrap;text-shadow:#008C3A 0.1em 0.1em 1.5em,#01796F -0.1em -0.1em 1.5em;color:#000000">[[User:Pine|<font color="#01796F"><b>Pine</b></font>]][[User talk:Pine|<font color="#01796F">^✉</font>]]</font> 07:32, 31 July 2014 (UTC)

:::That is a good follow-up point. As we do not directly evaluate, but work to support self-evaluation of programs, I cannot speak to those goals and related evaluation strategies except for the portion of the conference for which organizers have shared goals for direct online impact, the hackathon. For the hackathon days, organizers plan to track the product of hackathon efforts in terms of the bugs fixed, templates created/added, analyses run, citations added, and the like, however this tracking is the responsibility of the organizers. As for tracking online activity of all Wikimania conference participants, that ''would'' be possible with usernames, and Wikimetrics at appropriate follow-up points, however, since the primary goals of Wikimania have not been stated in terms of intermediate behavior outcomes specific to online activity, there is currently no plan for that at this time, only immediate outcomes and short-term learning, networking, and motivation outcomes as targeted by the existing survey draft items. As for that draft, I would be happy to also send your way for input as well (I will share that draft to your inbox). [[User:JAnstee (WMF)|JAnstee (WMF)]] ([[User talk:JAnstee (WMF)|talk]]) 17:34, 31 July 2014 (UTC)