Learning patterns/Analysing effects of offline meetups: Difference between revisions
Content deleted Content added
ASociologist (talk | contribs) Creating a new Learning Pattern :Learning_patterns/Analysing effects of offline meetups (edited with FormWizard) |
ASociologist (talk | contribs) updated |
||
| (5 intermediate revisions by the same user not shown) | |||
Line 17:
<!--write a description of a common problem that this pattern is supposed to help solve, or a question that it answers.-->
Offline face-to-face meetings are an important part of the Wikimedia community. However, there are only few systematic assessments of their effects of such meeting on online behaviour [[Learning_patterns/Collecting_data_on_offline_meetups |as it can be difficult to obtain the meeting data]]. Once collected, statistical methods can be used to assess whether the offline meetups have an effect on the online behaviour of Wikipedians. In this learning pattern, I want to highlight some methods how the meeting data can be analysed, and discuss problems and limitations which must be kept in mind when working with meetup data.
==What is the solution?==
<!--write a description of how this pattern solves the problem or answers the question.-->
This learning pattern will provide guidelines on how to analyse effects of offline meetups (on online behaviour).
===Effect on what?===
Like in any other research project, the first step is to identify what you are actually interested in and which research question you want to answer. Do you want to know...
* whether users make more edits after attending a meetup?
* whether users make higher quality edits after attening a meetup?
* whether users remain active for longer after attending an edithaton?
* whether users taking part in introductory edithatons feel more welcome in the community?
* whether users know the rules of Wikipedia better after attending an edithaton?
* whether users are more likely to become administrator after having met other administrators face-to-face?
* whether users start (or stop) using the ''talk page'' to interact with those that they have met?
Many questions can be asked about the effects of offline meetups on the online behaviour of Wikipedians. To identify the effects, different data and strategies of analysis might be needed. This learning pattern will outline approaches and relevant thigns to consider irrespective of the specific research question. Nevertheless, before starting any analysis, it is important to make your questions and aims clear.
===Getting the data===
To answer your research question, you not only need [[Learning_patterns/Collecting_data_on_offline_meetups | data on the offline meetups]] but you probably need to merge it with data capturing the online behaviour of Wikipedians. This online data might come from the [https://dumps.wikimedia.org/enwiki/latest/ Wikipedia data dump] (see [[Learning_patterns/Working_with_the_Wikipedia_data_dump_for_research | a learning pattern on using it for research]]), from collected [[Learning_patterns/Collecting_data_on_requests_for_adminship | requests for adminship]], from additionally conducted (offline) interviews or surveys, or other sources - depending on the research question. When only working with meetup data, one can analyse their spatial and temporal distribution, the networks that have developed, etc., but only when merging them with other data sources, one is able to assess whether therse offline meetups have an effect on other behaviours and attitudes.
===Identifying effects===
Once the research question is set and the data is obtained, the next task is to assess whether there is an effect of meetup participation on behaviour and attitudes. Making claims about the effects of meetups requires a comparison with users that have not attended any. Depending on the research question and the computational possibilities, it might make sense to work with the full list of users and to statistically control for tenure and activity in analyses (so that one can reliably say that it is the attendance at a meetup that makes people contribute more and not for example only the number of previous edits). Another option would be to use a matching approach. With a matching approach, one would match each user that attended a meetup with a user that did not attend one but is comparable in other regards. For example, imagine user A and B which registered on the same date and had a comparable level of activity in their first month. In their second month, user A attended a meetup while user B did not. User A and B could now be matched and compared (this is the very basic idea of covariate matching). Additionally, one can make use of the longitudinal nature of the data and compare before- and after levels of activity, for example using a difference-in-differences design ([https://wikiworkshop.org/2022/papers/WikiWorkshop2022_paper_16.pdf see this paper using diff-in-diff and covariate matching]).
Different statistical models and comparisons can be useful, this depends on the research question and the specific data structure. However, in any case, it is important to keep in mind that meetup attendees are a self-selective group and comparing their behaviour to comparable others is most important to identify actual meetup effects.
<!--Examples of situations where you would apply this pattern, or real-world situations where it has been used (links and images are encouraged!-->
* Use this pattern when you want to better understand offline meetups of a Wikipedia community.
* I have used this approach in [[Grants:Project/nschwitter/The_Role_of_Offline_Ties_of_Wikipedians | my project]].
==Endorsements==
<!-- Want to endorse this as a good pattern? Add your name by clicking the button in the infobox, or edit this section directly. -->
[[User:ASociologist|ASociologist]] ([[User talk:ASociologist|talk]]) 15:01, 13 June 2022 (UTC)
==See also==
Line 34 ⟶ 56:
===Related patterns===
<!--A bulleted list of related patterns. See Grants:Learning_patterns#Learning_patterns for ideas-->
* [[Learning_patterns/Collecting_data_on_offline_meetups | Collecting data on offline meetups]]
* [[Learning_patterns/Working_with_the_Wikipedia_data_dump_for_research | Working with the Wikipedia data dump for research]]
===External links===
<!--A bulleted list of off-wiki resources related to this pattern (books, websites etc)-->
* [https://wikiworkshop.org/2022/papers/WikiWorkshop2022_paper_16.pdf Paper using German meetup data]
===References===
| |||