WikiLiteracy: Enhancing students’ digital literacy with Wikipedia

In January 2019 the University of Derby delivered its first module entirely dedicated to and structured around editing and writing articles for Wikipedia. The course focused on using Wikipedia as a means to improve students’ skills in writing for public consumption, in addition to enhancing their digital and collaborative skills. Students contributed to 118 articles across a range of topics, which were viewed over 11.2 million times, providing them with a public platform no university assignment could match, and introduced them to the challenges of interaction and engagement in a global editing community. Students’ confidence in their digital capabilities was assessed at the start and end of the module and showed a clear increase in confidence across all categories.


Introduction
Many students come to university with, at best, basic digital literacy skills. They frequently lack the critical thinking skills necessary to assess and evaluate the material they are expected to use in higher education, the research skills to find that material, and the synthesis skills to collate and combine that material into something new. This in itself is not necessarily an issuestudents come to university precisely to learn and develop, and of course one of the primary functions of an academic librarian in higher education is to support students in this process.
What should be of concern to educators is that is that many students are still leaving university feeling a lackin a recent Jisc (2019) survey (Digital Experience Insights Survey 2019) only 42% of students felt that their course prepared them for the digital workplace. There is clearly still much work to be done on further embedding digital technologies and digital skills within classroom-based teaching. In the same Jisc survey, students reported that when digital technologies are used as part of their course, 75% feel more independent, 76% fit learning into life more easily, 69% understand things better, and 68% enjoy learning more, and 44% reported that they would like more use of technology within their course.  Another issue of concern is how frequently students turn to websites such as Google and Wikipedia rather than using the curated digital resources available via their university library. A 2016 study from Australia showed that some 87.5% of students were regularly using Wikipedia as part of their university studies and that their use and perception of its usefulness actually increased with the level of study (Selwyn & Gorard, 2016).
The author began to wonder whether this initial instinctive turn towards Wikipedia could actually be harnessed and, if properly directed, provide an opportunity to address both of these areas of concern simultaneously. Having previously organised edit-a-thons at the University, the author http://dx.doi.org/10. 11645/13.2.2669 was familiar with the basics of article editing and creation and had come to realise that the skills required to edit and write Wikipedia articles -the need to research an unfamiliar topic, assess sources of information for accuracy and credibility, write in a particular style and tone, cite and reference sources used -were all very much akin to those expected of students at university, but not always explicitly taught in any extended fashion within the curriculum.
Whilst Wikipedia itself has been in existence since 2001, these educational benefits of Wikipedia have only really come to academic attention within the last ten years. These benefits range from its role as a platform for providing student writers for public audiences (Purdy, 2009); as a means for collaborative writing and editing (Mak & Coniam, 2008;Zheng, Niiya & Warschauer, 2015) and providing students with an awareness of the global information community (Konieczny, 2012); a means of addressing systemic bias in information sources (Nuri, 2017;White, 2018); and as a tool for developing digital literacy (Konieczny, 2016;Patch, 2010).
Surveys of librarians (Luyt, Ally, Low Nur & Ismail Norah, 2010;Zlatos, 2014) and teaching staff (Bayliss, 2013;Chen, 2010) show that there is widespread agreement that students ought to be taught how to evaluate information, and particularly web-based information, responsibly. However, it seems few address Wikipedia specifically in library and research instruction other than to warn against its use, without any real elaboration on why (Colón-Aguirre & Fleming-May, 2012;Rempel & Cossarini, 2013).
Educators have begun incorporating Wikipedia formally into their teaching, either in the lecture room or classroom (Konieczny, 2014;Evenstein Sigalov & Nachmias, 2017) or in librarian instruction (Calhoun, 2014), and Wikipedia itself has launched its own educational program to support and encourage these initiatives.

Wikipedia Education Program
Wiki Education (WikiEdu) was founded in 2013 as a spin-off of the Wikimedia Foundation, the non-profit organization behind Wikipedia. Its most established program is the Wikipedia Education Program, which has supported over 43,000 students in higher education institutions across the US and Canada since 2010, adding over 44 million words to Wikipedia (WikiEdu, 2018).
The number of UK institutions involved is much smaller, largely because WikiEdu focuses solely on the US and Canada, with UK educational activities being supported by the smaller UK chapter of Wikimedia. Around 15 universities have been actively delivering modules in the 2018/19 academic year, including Edinburgh, Queens University Belfast, Imperial College London, UCL and Stirling. The UK chapter has also released a report which maps engagement with Wikimedia project against various existing digital literacy frameworks, in a similar way to that which the University of Derby did with its own Information Literacy Framework (Bruszik, 2018).
Many of these programmes used the Wikimedia Outreach Dashboard, a platform specifically designed to assist in the management of programs and events such as educational courses or edit-a-thons. It functions in a manner somewhat akin to an academic VLE (virtual learning environment), with a timeline, the ability to upload documents and links and embed Wikipedia's http://dx.doi.org/10.11645/13.2.2669 own training modules. These modules are short and interactive, designed to guide new editors through the basics of Wikipedia editing, as well as introducing concepts such as article evaluation, suitable sources, citations and plagiarism. The Dashboard also allows educators to monitor students' editing activity, giving them the ability to see what articles the students are working on, the scope of the changes made, the option to compare before-and-after versions of the articles, and the impact of the edits made via page view statistics. This Dashboard was used at the University of Derby to co-ordinate the module discussed in this report.

Background to project
In the summer of 2018, the author delivered a workshop to teaching staff at the University's internal Learning and Teaching Conference demonstrating various ways in which Wikipedia could be used as a learning and teaching tool, in place of or alongside more traditional 'library skills' activities. Examples included evaluating sources used in Wikipedia articles for accuracy and academic credibility; researching topics using open access sources and then comparing the same research performed using library subscription sources; comparing Wikipedia articles with traditional encyclopaedia articles, textbooks or journal articles; using Creative Commons or Wikimedia Commons to find images; and discussing issues of neutrality and systemic bias in Wikipedia articles.
The intention was to open a dialogue on the use of Wikipedia in education, to encourage academics to consider incorporating elements of Wikipedia into their teaching or assignments, using some of the approaches discussed, and to highlight how the library could support them with this.
In order to lend credibility to the concept and to demonstrate how versatile a tool Wikipedia could be, these examples were mapped against the University's internal Information Literacy Framework (University of Derby, 2015; Table 1), itself based on the SCONUL Seven Pillars of Information Literacy (Bent & Stubbings, 2011

Scope
Assess current knowledge and identify potential sources of information Use Wikipedia article references to identify sources used.

Plan
Construct strategies for locating information and data Research topic using open access sources from Wikipediathen compare same research using library subscription sources.

Gather
Locate and access the required information and data Use library subscription sources to provide references for articles with unverified claims.

Evaluate
Review the research process and compare/evaluate information and data This demonstrated how easily Wikipedia could be integrated into teaching and assessment as a tool to teach core information and digital literacy concepts, in contract to the traditional view within academic of Wikipedia as a (not especially reliable) information source. This is a distinction that Wikimedia has frequently made: that Wikipedia should be not be viewed as an information source in itself, but as a summary of information sources or a tertiary source (Wikimedia UK, 2019).
The workshop was designed to be hands-on and gave academic staff an opportunity to try some of these activities themselves, as well as introducing the basics of editing Wikipedia. Feedback was positive, with several academics commenting that they had not considered using Wikipedia in this way before.
As a direct result of this workshop the programme leader for the undergraduate Publishing programme approached the author, intrigued by the potential of Wikipedia to help shape literacy skills by actively creating content within a classroom environment. A new module within the Publishing programme, 'Content Development', was currently under development, though it had as yet no module leader attached, nor yet any content created for it. The author was asked to take on the role of module leader in designing and delivering the module, entirely structured around the use of Wikipedia.

Module design 2.1 Aims
The new 'Content Development' module is a second year core undergraduate module on the BA Writing and Publishing programme. This programme aims to explore the entire spectrum of the creative process from conception and writing through to content development, editing, publishing and sales. In the first year, students took modules in the history and culture of publishing, markets and media, editorial skills, reading for writing development, and general grammar and literacy.
The intent of the module was to provide the students with the opportunity to create live published content (as opposed to practice assignment pieces) for an external 'client', writing in line with the client's 'manual of style', topic and content requirements. Wikipedia seemed ideal to stand in as this 'client', with an almost unlimited range of topics to write about, whilst also providing a live audience in the world's largest readership and giving students an opportunity to work within a global editing community, interacting with other editors and collaborating on article projects.
The module's learning outcomes were: 1. exercise professional and appropriate situational judgement when working collaboratively within teams and with third parties.
2. demonstrate personal skills and abilities such as negotiation, communication, project management, and problem-solving.
3. compose, edit, adapt and repurpose text and content for a variety of audiences in a professional context and in accordance with legal, moral and ethical considerations.
4. prepare content to professional standard for digital output.
As shown, the primary intention of the module was to provide students with the opportunity to develop written digital content for their 'client' (Wikipedia). With such an emphasis on writing, research, evaluation and review of articles in a digital environment, it also afforded an ideal opportunity to explore digital literacy skills in more depth than is usually offered within the curriculum. In-class teaching time for library sessions can be difficult to come by, often limited due to time pressures, and sessions frequently therefore cover only the basics of research, online resources and referencing. While the library lays on a wide variety of skills enhancement workshops, these take place outside of core teaching hours and on a voluntary basis, which inevitably limits reach.
There are of course many different ways of developing digital literacy skills with students both within and outside of the curriculum, but Wikipedia in particular seemed a perfect vehicle for this module. It simultaneously provided the end goal (the articles created for the client), the learning apparatus (the Dashboard), the learning materials (the training modules and community of editors) and the structure (Wikipedia's own guidelines and rules).

Module structure
The original intention was to structure the module around the University's Information Literacy Framework (University of Derby, 2015), as detailed in the staff workshop previously mentioned and illustrated in Table 1. However, at the same time the module was in development, the University became a partner with Jisc in the development of the new 'Building Digital Capabilities' Discovery Tool (Figure 1; Jisc, 2018) and launched its own internal 'Digital Derby' drive to assess and develop the digital capability of staff and students. This new framework seemed a more natural fit to the aims of the module, with its enhanced focus on creation, collaboration and participation, whereas the Information Literacy Framework was more heavily focused on discovery and use of information.
The module structure (Table 2) was therefore largely inspired by the six elements of digital capability identified by Jisc as part of the 'Building digital capabilities' framework (Jisc, 2018):  ICT proficiency (functional skills)  information, data and media literacies (critical use)  digital creation, problem solving and innovation (creative production)  digital communication, collaboration and participation (participation)  digital learning and development (development)  digital identity and wellbeing (self-actualising) © Jisc CC BY-NC-NC  The first two weeks focused largely on the practicalities of getting started with Wikipedia editing, with the students setting up accounts and learning how to make small simple copy-edits and add references. The training modules provided by Wikipedia as part of the Outreach Dashboard were of great help here, as the students could work through these at their own pace before class, identifying any issues or areas of confusion that could then be discussed and worked through in class.
Subsequent weeks then addressed different elements of digital literacy: critical evaluation of articles and sources; research and referencing; plagiarism and copyright; media literacy; cybersafety and online communities; feedback and peer reviewing; and systemic bias. These are all elements as relevant to students' on-going digital literacy capabilities as to future publishing careers, and throughout the module material was related both to Wikipedia and the wider publishing context. The module was assessed via two formative assignments: the first an individual assignment comprising 40% of the grade, the second a group assignment comprising 60% of the grade. The first assignment required students to substantially copy-edit an article of their choice: these edits could involve adding references, info boxes, new sections etc, but must add at least 1500 words of additional content to the article. They also had to write a 500-word report detailing the choice of edits made and the approach used.
The second group assignment required the students to create a portfolio of edits. The portfolio had to be agreed in advance with the module leader, to ensure the student groups were all doing comparable amounts of work. These portfolios could contain two brand new articles, for example, or translation of existing articles into English, or a mixed portfolio of substantial copyedits, such as improving grammar and readability, improving structure, adding in new content or missing references. The students also had to individually submit a weekly blog via Pebblepad (an e-portfolio platform), detailing and explaining the work done that week.

Custom marking rubric
A custom marking rubric was created (Table 3), to reflect the unique approach to the assignments required for this module and to align with the module's learning outcomes. We originally investigated using the sample rubric created by Wikipedia (Wiki Education Foundation, 2017), but felt that this would not suit the dual purpose of marking from both a Wikipedia editing and Publishing perspective.
Our custom rubric focused on three primary criteria: the first criterion (35% of the grade) focusing on the students' demonstrated engagement with the Wikipedia community and their awareness of the roles and responsibility of a Wikipedia editor/author; the second (50%) focusing on the knowledge demonstrated of Wikipedia's structure, rules and regulations relating to editing behaviour, ethics, standards and output; and the third (15%) focusing on the standard of the written output: language, structure, grammar, spelling etc.

Criterion 2 (50%)
Demonstrates very insightful knowledge of the online sources and community, using ideas introduced in the course. A very high level of practical thought and awareness of the medium.
Incorporates a significant level of knowledge of the community and online sources, using ideas introduced in the course. A high level of practical thought and awareness of the medium.
A good level of knowledge of the community and online sources, but with a few contradictions or misunderstandings. A good level of practical thought and awareness.
Fair, yet clear, evidence of knowledge of the online sources and community. However, some ideas need to be developed and refined further. A satisfactory level practical thought.
There is too little evidence of knowledge of the online sources and community, and some ideas are incoherent. A weak level of practical thought has been applied.
Little engagement with, or real awareness of, the online sources and community. A poor level of practical thought has been applied.
There are few instances of understanding the topic. May also contain serious errors. A very poor level of awareness and very little sign of practical thought being applied.

Criterion 3 (15%)
Appropriate language used throughout; very precisely written, with no, or only trivial, mistakes in spelling, grammar and syntax.
Appropriate language used; well written and precise, with only occasional errors in spelling, grammar and syntax.
Some mistakes in spelling, grammar, syntax, and weaknesses in terms of awkward expression, but a good standard, nevertheless.
Some awkwardness in style and/or presentation and a number of errors in phrasing, grammar, spelling and syntax, but satisfactory overall.
Significant errors in writing and presentation, at times making the meaning of the work unclear.
Contains frequent and significant errors in phrasing, spelling, grammar and syntax, many of which make the meaning of the writing unclear.
Serious errors in phrasing, spelling, grammar, and syntax that make the meaning of the writing very unclear, with a standard of presentation unacceptable. The rubric aimed to make clear to the students that their actual output was only part of their assignmentswhat we were looking for was the demonstrated level of engagement with their 'client', their compliance with the 'client's own requirements for style, content, structure, approach, and their approach to co-editing within a dynamic and changing environment. These are elements which it is anticipated they will be required to deal with in potential publishing careers -editing unfamiliar material, having their own content edited, altered or even deleted, having to liaise and work with unfamiliar or even anonymous co-editors, responding to potentially harsh criticism, conforming their own person style to meet that of the client.

Delivery
The module was taught via weekly 4-hour classes. Given the central focus of the module was hands-on Wikipedia editing, the classes took an active learning approach, with a minimum of traditional lecture-style delivery. This was combined with a 'flipped classroom' approach, with the students being supplied with required reading (often Wikipedia guides) ahead of time and required to undertake the week's relevant Wikipedia training module, embedded within the Outreach Dashboard.
Each class began with an introduction to the week's focus, frequently involved class-based discussion, either verbally or using polling software such as PollEverywhere. This latter was particularly used in the early weeks when students were less familiar or comfortable with the module leader, and therefore less likely to volunteer information verbally. This discussion then led into hands-on activities designed to lead the students into applying the theory just discussed with practical applications. These activities will be detailed in the section 3.1.
This would take up the first half of the class, with the second half devoted to editing Wikipedia, applying the principles and skills focused on in the first half and drawing on the guidance and practical examples provided in that week's training module. These training modules were especially useful in bridging the gap between theory and practice, in taking the concepts discussed and specifically applying them to Wikipedia. This approach also provided an immediate means of assessing learning and understanding, as the author could monitor all students' edits via the Dashboard and see how they applied the material from the first half of the class.
Each week students also followed up on the edits made in previous weeks, to see how many had been edited further by other editors, expanded or potentially removed. This was a valuable insight into Wikipedia's own quality control processes, and where changes had been made, Students were provided with pre-selected lists of articles to choose from, directed to randomly generate an article using the 'Random article' link in Wikipedia, or specifically assigned a particular article, depending on the nature of that week's focus. For example, in the 'Offline Research' class, students were specifically assigned an article relating to local (Derbyshire) history and directed to use the Library's print Local Collection in order to research their topic. This proved to be one of the most challenging tasks set throughout the whole module, since students could not use the internet to research at all and had no control over their choice of topic. However, since unfamiliar research topics and offline research are likely to confront them in future publishing careers, it was a valuable exercise.
All students registered with the module's 'campaign page' on the Wikipedia Outreach Dashboard, ensuring that any and all edits they made were logged. This meant that the module leader could see who had and had not done the pre-required 'flipped' preparation, what edits were being made in class and by whom. This was very useful in providing the ability to monitor levels of engagement, and which students were potentially struggling with the content.

Activities
Each week's class involved active learning activities in addition to the hands-on Wikipedia editing. These aimed at giving the students' the opportunity to take that week's learning and apply it in a practical way, aligned to the six Jisc Digital Capabilities (Figure 1). For example, when focusing on copyright and plagiarism, the students were provided with an abbreviated version of Secker and Morrison's Copyright Card Game (2015), which tested their knowledge of copyright law in an engaging way.
In the week devoted to sources and citations, the students were tasked with using the 'Citation Hunt' tool (Citation Hunt, 2016) to find statements within Wikipedia articles that lacked verification and doing research to find an appropriate source of information to verify the statement. If they could find no such supporting material, they were encouraged to edit the article to remove the statement. As with all edits, the author could see via the Dashboard how many references were added and the source of the reference to ensure a quality check.
In the weeks devoted to image/media literacy, the class focused not just on sourcing legal images online but also how to take good quality photographs of their own for use online and how to licence these for use via Creative Commons. A learning technologist was invited to attend and gave a special 'photography masterclass' on how to take good photographs on mobile devices and how to enhance these images using free photo-editing apps.
An eye-opening activity for many students was the week devoted to Cyber Safety. Students were asked to Google themselves using publicly available information about themselves: their name, their email address, their social media handles. One student found their name and picture being used on a site with someone else's email address, which quite alarmed them, and they immediately took steps to rectify. Another was surprised at just how information about themselves they had been able to find via a friend's unlocked Facebook account and texted their friend there and then in the class! This generated a lot of discussion about digital footprints and how important it was to take control of one's internet footprint, not only to control the negative but to enhance the positive. Several students commented on how they had not thought about creating a positive digital footprint and how their Wikipedia editing activity might contribute to that.
One of the best received activities was the speed-dating peer review activity. All the articles the students had been editing for their first assignment were printed out and arranged in a circle around a large table, with enough copies of each article for every student to make notes on. Students were instructed not to tell each other who had been editing which article. They were given 5 minutes to review and assess the article, highlighting strengths and weaknesses, elements they would alter or remove, areas where they felt the article could benefit from additional content. After the allocated time, they would be alerted to move their copy of the article to the bottom of the pile and move to the next seat. Eventually they would have completed the circle and returned to their original seat, with every article having been reviewed by every student.
The students were then paired up and each given two of the piles of reviewed articles, with care taken that no group received their own articles. They were asked to collate this feedback together to produce no more than a page of comments and suggestions. These were then returned to the module leader, who paired the students up again in different combinations, providing them with the page of feedback for their own articles, and they were asked to deliver this to one another. Students were then permitted to keep the page of feedback to reflect on. This was one of the areas with the biggest increase in confidence in the Digital Literacy Self-Assessment (see section 4.3) and was also mentioned several times in the weekly blogs required for the second assignment, with students' keeping both their peers' and the author's feedback in mind when working on the assignment.

Output
By the end of the module the students had made a total of 1,090 edits over 124 articles, creating six brand new articles, and adding over 51,700 words and 361 missing references to Wikipedia. The articles worked on have since been viewed a combined 18.6 million times (at time of writing). Edited articles ranged from young adult fiction to female motorcyclists, folk archetypes to corporate entities, mythological creatures to toast ("Bridge of Clay", 2019; "Marjorie Cottle", 2019; "Villain", 2019; "The Tussauds Group", 2019; " Amphisbaena", 2019;"Toast", 2019). New articles created include young adult book series, Norwegian museums and 4 th century BC philosophy ("The Shock of the Fall"; 2019; "National Museum of Art, Architecture and Design", 2019); "Western philosophy: Classical period", 2019).
Students were directed to Wikipedia-generated lists of missing articles to give them inspiration for subjects they might choose. However, they were not formally required to choose from these lists, and many students instead chose subjects of personal interest. Quite unintentionally, this provided a perfect example to draw upon when addressing issues of systemic bias in Wikipedia, as it demonstrated how editors' own backgrounds and interests influence their choice of topics and how this issue writ large can result in a lack of diversity and truly representational global coverage in Wikipedia.

Feedback
Because the approach using Wikipedia was entirely new, ongoing feedback was sought from the students, rather than waiting until the end of the module for the formal module evaluation questionnaire feedback to be collated. Each week at the end of the class the students were asked to provide on-the-spot feedback, via post-it-notes. They were asked to anonymously write down one thing they liked, one thing they didn't like and one thing they learned and stick these on the door as they left.
Feedback was consistently positive, with students especially singling the hands-on activitybased approach as a highlight: I liked gaining a little more confidence I enjoyed how hands-on this lecture was I liked how hands on we are with Wikipedia I liked that we got to do a lot of work, which was good I liked how practical it was There was nothing I didn't like The majority of the elements identified as 'dislikes' were beyond the control of the module leader, often relating to furniture or the temperature of the room. Students did frequently single out their frustration with their own abilities as a dislike, commenting for example, 'I don't like how stressful it was to find references for some things', or 'didn't like not being able to find a citation'. Some 'dislikes' commented on their own dissatisfaction with how poorly written or structured some Wikipedia articles were, evidence that the students' critical evaluation skills were developing.
Unfortunately, when commenting on what they had learned, students often highlighted facts they had learned from Wikipedia articles, rather than focusing on the skills they were applying in the class. In future, this element of the feedback may be reworded to be more explicit about what learning was being referred to! The formal module feedback delivered via the end of module evaluation questionnaires was also positive, with all mean values falling within the University's quality guidelines. The module teaching was singled out as the highest rated element, something immensely gratifying to the author! The quality of the digital learning experience and the learning activities were also highly rated.

Digital Literacy confidence
Since one of the primary functions of taking this approach with Wikipedia was to use it as an experimental vehicle to improve the students' digital literacy capabilities, there needed to be some form of base line in order to assess any potential improvements. Therefore, in the first class of the module, the students were asked to fill in a digital literacy self-assessment questionnaire (Table 4), developed by the author to assess their level of confidence across a range of digital capabilities. The students were asked to rate their confidence level on a scale of one to four, with one being 'very unconfident' to four being 'very confident', with the ratings then converted into a numeric value in order to arrive at an average score. Q16. How confident do you feel that your digital skills make you ready for the workplace?
On average the students rated their digital skills as 'somewhat unconfident', with the highest levels of confidence relating to copyright and plagiarism, (which was surprising, given the general complexity of copyright law), and the lowest relating to metadata and digital media (unsurprising, given students' lack of awareness of the purpose of metadata outside of social media usage (Mitchell, 2010).
In the final session the students were again asked to assess their confidence using the same self-assessment questionnaire. This demonstrated that the students' confidence ratings had increased in every single category, with the highest increases in the areas of peer review, ability to identify bias, writing content for public consumption, and, unsurprisingly, general Wikipedia skills. Students' overall rating was now between 'confident' and 'very confident'.
Although not a formal assessment of capability since it was assessing confidence and not testing ability, it clearly demonstrated that over the course of a 12-week module, using Wikipedia to evaluate, edit, write and research articles proved to be an effective means of enhancing students' confidence in their digital skills.

Conclusion
Whilst this approach was specifically designed with Publishing students in mind, the potential applications are significantly wider. Individual elements and activities can be used in isolation: for example, the author intends to utilise some of the activities used in this module within library instruction sessions to teach referencing and critical evaluation. After attending the internal Learning and Teaching conference presentation mentioned at the start of this article, Business and Forensics programmes at the University of Derby have also incorporated individual elements of the Wikipedia approach into their teaching by getting students to evaluate and compare references in a Wikipedia article to a comparable peer-reviewed article.