Improving information literacy and academic skills tuition through flipped online delivery

The COVID-19 pandemic forced UK universities to move the majority or all of tuition online. The Library Academic Support Team at Leeds Beckett University used that shift as an opportunity to improve information literacy (IL) and academic skills tuition across the institution. Instruction and support were redesigned on a flipped basis to ensure that online delivery improved on face-toface delivery rather than simply replicating it. This project report reviews that work with usage statistics, user feedback, practicalities of service provision and discussion of impact. The report extends existing literature with a model of significant institution-level changes to IL and academic skills instruction which could be applied elsewhere. It concludes that the shift to flipped online learning was a qualified success, with the revised approach proving notably more popular and inclusive, also providing other benefits such as more focused in-class discussion.


Introduction
This project report reviews how the Library Academic Support Team (LAST) of Leeds Beckett University used the shift to online-only tuition necessitated by the COVID-19 pandemic as an opportunity to improve information literacy (IL) and academic skills tuition across a higher education institution. In the 2020-21 academic year traditional synchronous lectures and workshops were replaced by a suite of flipped learning options, with the only remaining "live" aspects of the team's tuition being online workshops, appointments and group Question and Answer (Q&A) sessions. These Q&A sessions followed up suites of asynchronous exercises and guidance, were often tailored to the needs of specific disciplines, and supported provision of more focused in-class discussion. This report outlines the organisational context, current literature and intervention made, before providing service usage statistics, user feedback and review of impact. It demonstrates a transferrable means by which scaffolded flipped learning can be used to improve or at least maintain effective tuition in IL and academic skills through online-only delivery, extending existing literature with a large-scale in-pandemic example. http://dx.doi.org/10.11645/16.1.3108

Project context 2.1 Local environment
Leeds Beckett University is a post-1992 institution in the UK, with approximately 25,000 students on a range of full-time, part-time and distance learning degrees and research programmes. The LAST provides academic skills tuition across the University, from foundation level to PhD and staff training, on topics such as finding and using information, critical reading, academic writing, referencing and mathematics. Instruction is provided in alignment with the University's Education Strategy (Leeds Beckett University, 2016) and delivered on a tripartite basis to support wider learning development: in-curriculum tuition, open workshops, and one-toone appointments. This instruction is aligned with website resources and a range of independent study modules addressing specific levels and topics. The overall structure is designed to encourage long-term independent learning while providing learners with practical short-term support where required.
In the academic year preceding the intervention covered by this report (2019-20), circa 15 Full-Time Equivalent librarians and academic skills tutors within the LAST delivered 560 synchronous in-curriculum classes and 575 one-to-one appointments, with further 410 students attending open workshops. Although some sessions were provided virtually prior to the pandemic, the majority were delivered face-to-face. Despite the mid-year disruption caused by the outbreak of the COVID-19 pandemic, user demand for support was evident. With direction from University leadership to prepare for a year of online delivery of the team's tuition, even as some course-level tuition resumed face-to-face, the challenge for 2020-21 was how to develop the service. Subsequent planning was guided by two principles: firstly, to extend and improve the tuition provided where possible, not merely replicate the "old world" online; and secondly, to maintain the emphasis on students as independent learners, while providing additional layers of support as required.

Wider environment
Early 2020-21 requests from academic staff were predominantly for traditional, didactic lectures delivered online for their students, whether simple library inductions or more complex information analysis sessions. However, follow-up liaison with these colleagues and review of relevant literature soon highlighted the potential benefits of flipped learning in the changed operational scenario. Given the operational context, there was particular interest in the potential to support engagement, grow understanding pre-class, focus discussion, and adapt the model for long-term and one-shot delivery without necessarily committing additional staff time (Loo et al., 2016).
The literature also showed that any negative impact of a change to flipped learning would be minimal. Studies have found that flipped IL instruction results in positive student feedback and either slightly improves academic outcomes or does not significantly impact on them (Arnold-Garza, 2014;Brooks, 2014;McCue, 2016;Dommett;2018;Poole, 2021). While the challenges of assessing the impact of IL instruction have rightly been noted (Markless & Streatfield, 2017), it is equally true that effective, well-studied historic tuition models would fall under a broad definition of flipped learning (Abeysekera & Dawson, 2016). Without face-to-face encounters to embed initial progress, some concern remained that solely online tuition might limit the effectiveness of flipped learning. However, distance learning courses already transcend such challenges, face-to-face follow-up was not a permissible option due to the pandemic, and studies such as that of Weightman et al. (2016) and Greer et al. (2016)  designed IL training to be effective in a range of delivery methods. As far back as 2015, Nguyen argued that since correctly designed online learning does not appreciably disadvantage learners, attention should shift to the practicalities of effective delivery.
The LAST was mindful of the impact of the pandemic on current and incoming students, for example, from time out of study, among multiple other factors. Morgan memorably noted that 'our students will never be normal again' (2021), if indeed they ever were, and highlighted that further work was required on understanding their needssuch as, she noted, Leeds Beckett University's Pre-Arrival Questionnaire. The pilot study of the 2020-21 intake highlighted that 52.9% of incoming students surveyed had concerns about online study, 54.5% of those engaged in study in 2019-20 had not been required to study in lockdown, and that 73% of respondents were concerned about coping with future studies. While most students nationally were relatively satisfied with their institution's response to the pandemic, they did also have greater concerns and uncertainty regarding the year ahead (Hewitt, 2020). Providing increased certainty about provision of IL and academic skills support early on could therefore also support the general student experience.
Given the capacity for the pandemic to exacerbate existing educational and societal inequalities (British Academy, 2021), flipped learning also had further attractions. With more generic content able to be prepared as recordings, in-year staff time could thus be freed up for flexible deployment at the point of need, whether that be follow-up tuition of specific cohorts or the provision of additional support through open workshops or one-to-one tutorials. Digital poverty among some students had been more of a concern early in the pandemic, but the University's various schemes for ensuring access to appropriate technology made this less of an operational issue for the next academic year, rendering it feasible to address remaining instances on a case-by-case basis alongside course teams. Effectively, flipped online instruction would enable the continued delivery of a valued service, while potentially improving tuition and enhancing service agility in challenging operational circumstances.

Intervention
The use of flipped online learning began with the LAST's first contact with new students for 2020-21. Previously delivered as a face-to-face library, Information Technology (IT) and academic skills induction to specific cohorts at the request of academic staff, this now took the form of a 1-hour induction module in the University Virtual Learning Environment (VLE), following on from a voluntary pre-enrolment module on university study. The module was introduced by a short pre-recorded video and primarily took the form of written guidance. To increase engagement, this guidance was split into short sections aligned to relevant practical tasks, such as essential IT setup, and provided in a variety of formats, including quizzes and interactive learning objects. Academic and support staff directed students to work through the module at their own pace, revisiting it as required through the year. Where requested, it was also followed up by a live online Q&A class with a member of the LAST at a relevant point later in the semester.
The LAST's in-curriculum tuition was also moved to flipped delivery. Most instruction was provided in advance of classes as suites of short video tutorials, exercises and other learning objects, designed to take approximately 1 hour to work through asynchronously, whether at undergraduate or postgraduate level. The learning objects were housed in the team's workspace in the university VLE, then copied into specific academic modules as required, embedding the tuition alongside the student's standard module content. Standard materials and learning activities were built for different academic levels, and then further lightly customised where required for specific disciplines and assignments. The use of short, interconnected videos rather than single session-length videos enabled reuse of recordings across topics and sessions as required. It also allowed academic staff to deploy sub-sections of the material to address specific learning challenges and gave students more capacity to work through tuition stage by stage. The only remaining live aspect of the tuition was, where requested, an online synchronous follow-up Q&A session to discuss the pre-provided materials.
The LAST used Panopto to record the video tutorials, having some previous experience of using it to support learners. Other materials were provided as Word documents, Powerpoint slides and written guidance within the VLE. The online synchronous sessions were delivered in whichever online classroom a particular cohort was using, often Blackboard Collaborate or Microsoft Teams, for ease of student access and also to further convey that the LAST's instruction was a standard part of in-curriculum tuition, not a bolt-on. Occasional additional training was required for individuals in the use of a particular online classroom platform, but otherwise the use of standard systems limited training needs to the issuing of team-wide briefing documents to ensure consistency of practice. Ultimately, as all of the technologies involved were relatively simple systems, the methodology could be replicated at other institutions using alternative systems, depending on local need and practice.
The open workshop programme moved exclusively online, to Blackboard Collaborate, having previously been predominantly face-to-face. However, session style and content changed less, following a relatively recent redesign to support more independent construction of learning. Classes and discussion remained in synchronous form online, but with introductory activities provided in advance to attendees and session recordings distributed afterwards for revisiting as required. One-to-one appointments also remained unchanged, but for moving exclusively online.
To ensure consistency, LAST colleagues worked collaboratively on materials for all levels, submitting them for peer review and regularly meeting formally and informally to reflect on progress and pool knowledgethe usefulness of such engagement in producing effective materials having been noted even pre-pandemic (Youde, 2019). While production of the adapted content for online and flipped delivery was a major undertaking, LAST learning materials are subject to annual review and revision even under normal circumstances.

Service statistics and initial analysis
The combined impact of the pandemic and new online delivery formats inevitably resulted in changes to data collection and analysis, making year-on-year comparisons imprecise. However, an overall upwards trend in demand for LAST tuition is clear, with data indicating increased student engagement with online materials.
In 2019-20 the LAST delivered 208 synchronous face-to-face inductions. In 2020-21 the online induction module that replaced this saw 60,998 page views, with 2,477 first year students enrolled, each spending an average of 1 hour in the module. Students gave the module an average feedback score of 4.5/5, with 485 (19.5%) giving extensive feedback on its usefulness. For the open workshop programme in general, over 2,700 individual workshop bookings were made. The caveat here is that a booking doesn't always translate to actual attendance, as students receive recordings of any session they book onto, so are encouraged to book even if they cannot attend a specific live class. However, actual attendance did also grow, from 450 students in 2018-19 and 411 in 2019-20 to 1,217 in 2020-21. There was a particular increase in taught postgraduate students booking workshops: from 54 in 2019-20 to over 1,000 in 2020-21. The shift to exclusively online provision also led to increased workshop bookings by students who were enrolled as distance learners prior to COVID-19, from 35 to 263.
In-curriculum and open classes were well-received, with positive feedback from academic staff and students rating LAST in-curriculum sessions and open workshops at an average of 4.61/5 for usefulness in 2020-21. Viewing figures for session recordings were also consistently higher than numbers booked onto sessions, implying extensive revisiting or sharing of material. For example, 187 students booked onto literature review writing workshops across the year, but those recordings were viewed 247 times.
Numbers of one-to-one appointments decreased from 575 in 2019-20 to 411 in 2020-21. However, this was to be expected, after a spike in demand across all levels early in the pandemic when the University's physical campuses were closed, and due to the increased engagement with other forms of LAST tuition. Numbers of students booking but not attending appointments dropped from 30 to 10, perhaps reflecting the relative convenience of attending online appointments compared to a physical campus, or the fact that students could be seen quicker owing to the team no longer being restrained by physical location.

Further analysis
The intervention achieved its aim of extending the support provided by the LAST, with increased engagement figures and positive feedback indicating that the tuition provided was valued by staff and students alike. However, the precise impact of the flipped approach on attendance, engagement, learning and effectiveness of tuition is less clear. This is due to the inadvisability of direct comparisons with pre-pandemic learning (Austen, 2020); the wider impact of COVID-19 on the student experience and learner behaviour; and the traditional difficulties of isolating and assessing the impact of skills tuition. Nonetheless, several notable factors are apparent.
Academic colleagues, students and the LAST instructors all commented on the effectiveness of the synchronous online Q&A sessions which followed in-curriculum asynchronous classes. Student questions were now more focused, after greater opportunity to digest material and trial any practical skills in advance. There was also greater capacity to answer questions in-depth than would be the case in traditional lectures or workshops, as well as the capability to ask questions by online chat extending the range of active participants in sessions. Academic colleagues have also continued to welcome the impact of LAST in-curriculum tuition on student performance. In some instances, the flipped approach was overtly praised as more effective, and, despite some initial concerns, in no instances was it deemed less effective by academics post-delivery.
While agreeing with this analysis, the LAST instructors noted the different practical challenges of the revised form of teaching. Unlike in traditional lectures, the agenda for live sessions was largely devolved to learners, albeit with capacity to pivot to core points where they did not emerge through student questions. This required some adaptation, particularly regarding how to proceed in instances where students had not engaged with pre-session materials or had limited questions, where repeating pre-session materials would actively undermine the worth of presession engagement. Thankfully such instances were rare, helped by effective promotion of pre-session engagement by course teams. Indeed, session effectiveness now being more dependent on student engagement actively helped embed the message of individuals as responsible for their own learning.
Students praised the greater inclusivity of the revised system, in session feedback and via departmental forums and the Students' Union. They noted that the improved capacity to revisit materials until fully comprehended was particularly helpful for students whose first language was not English, or those with hearing impairments, or simply anyone initially struggling to understand a certain point, particularly as all recorded materials included closed captions. While student preferences don't necessarily equate to effective tuition, similar comments were also received from academic staff, and the approach represents an adaptation of the existing sectorwide practice of sharing instructional content in advance of classes. The decrease in one-to-one appointments could also be taken as indicative of increased numbers accessing the required guidance without needing individual support, although other factors could also have influenced appointment demand.
While additional work was required to record sessions and build learning objects in the summer of 2020, staff time was freed up throughout 2020-21, as had been hoped. This resulted in enhanced service agility; as with less requirement to provide live content, staff could be flexibly deployed to address specific circumstances as they arose, for example, in the months in which demand for appointments grew. Unexpectedly, this was outside of term-time, perhaps reflecting increased year-round learning while traditional vacation employment and leisure activities were unavailable. Similarly, some tuition was adapted as the year progressed, with more flipped inclass content introduced for open workshop sessions after it emerged that some students were often unable to complete pre-class activities due to late booking.
From analysis of individual in-curriculum sessions, it became apparent that timing and promotion of material could significantly support the sessions' success. Where in-curriculum tuition of clear relevance to a specific assignment was effectively promoted to students by course staff, with a related synchronous Q&A session provided near an assignment deadline, student engagement was high. Where students were provided with large quantities of instructional material by course staff wishing to provide them with a general portfolio of longterm guidance, engagement was lower. This learning communication challenge is not unique to flipped or online learning, but its importance arguably rises when extensive suites of pre-prepared content are available for ready dissemination. Well-focused provision of less content can be more effective.
More generally, the challenge of isolating the impact of flipped online learning is the key caveat on this project report. More students may have engaged with additional academic skills tuition due to the challenges of studying in a pandemic even if a flipped approach had not been adopted. Further research could usefully explore such points, as well as specific reasons for engagement (for example, whether extension, remedial or desire for more instructor contact time), the impact of the blended learning scenarios of 2021-22, level-and course-specific impact, and the relationship of the instruction to other factors, such as course culture and available departmental support. It would also be useful to explore the intervention's impact on students' longer term learning development, particularly given their positive initial feedback. Nonetheless, there is still adequate evidence that the revised strategy supported greater numbers of students more inclusively and more flexibly than beforewhether by enabling learners to revisit recordings more readily or by assisting increased numbers of distance and postgraduate learners to engage. The revised approach also resulted in consistently positive feedback while freeing staff time to support the provision of a more agile and effective overall service to learners.

2021-22 changes
The operational context changed again for academic year 2021-22. The Department of Education expected a return to on-campus tuition unless the pandemic situation changed significantly. Requests from academic staff for face-to-face tuition for their students, and National Student Survey feedback on the merits of the on-campus experience showed some customer expectations, but it was also apparent that many courses would continue with significant online tuition, demonstrating a need for blended learning and service flexibility.
While supporting the University's general return to campus, the LAST were able to demonstrate the impact of its flipped approach to IL and academic skills tuition and retain such practice in 2021-22. This was even true of in-curriculum sessions. Whether the synchronous Q&A aspect was provided in-person or online, initial tuition was still provided in advance in the VLE. Similarly, initial induction remained a VLE module and the open workshop programme remained predominantly online, although with capacity for more blended learning if useful as the year progressed. Successful aspects of the 2020-21 approach were also extended where feasible, continuing the principle of improving tuition despite changing times. New induction modules were produced to help with the transition to the second year of undergraduate courses, as well as for taught postgraduates and apprentices, with new content supporting student wellbeing provided by colleagues from Student Services. An element of flipped learning was also brought to one-to-one appointments, for example, with Nursing students now provided with pre-meeting content on common enquiry topics, sometimes negating the need for an appointment.
Assessing the long-term impact of the transition to a flipped learning approach will be challenging, given uncertainty about the impact of the pandemic on student behaviour and learning, and the return to providing some tuition face-to-face. However, the popularity of such tuition in 2020-21 is already clear, and 2021-22 service statistics and the additional research noted above will further extend and contextualise what is already known of its impact alongside the multiple other factors at work. http://dx.doi.org/10.11645/16.1.3108

Conclusions
The Library Academic Support Team of Leeds Beckett University used the shift to online tuition necessitated by the COVID-19 pandemic as an opportunity to redesign IL and wider academic skills instruction on a predominantly flipped basis for academic year 2020-21, with induction and in-curriculum classes exclusively so. Traditional lectures and workshops were retired, replaced by suites of flipped learning resources, with the only remaining synchronous incurriculum tuition follow-up Q&A sessions.
Following these changes, the numbers of in-curriculum classes booked by academic staff for their students and the numbers attending open workshop events both rose significantly, with the revised formats receiving consistently positive feedback. In areas where less in-curriculum sessions were booked, more students attended open workshops, demonstrating their desire for some form of academic skills tuition. Both students and academics welcomed the improved inclusivity, enhanced capacity to revisit materials, more focused discussion of class content and greater support for long-term learning development and progression. Much tuition now being pre-recorded also increased service agility by freeing up in-term staff time to respond to the challenges of specific cohorts as they arose.
While possessing some local features, this general methodology could also be applied in other institutions and contexts. If doing so, it is recommended to retain the following aspects of the methodology in particular: the use of multiple short pre-recorded videos and learning objects to allow easier repurposing as required; the induction module, as a more effective means of providing initial support; and the consistent institution-wide approach. Above all, user engagement supported the project's success, liaising with academic staff to ensure the timely deployment of appropriate materials, and also that it was conveyed to students that the effectiveness of the learning was dependent upon their active participation.
While the exact extent to which the flipped learning strategy contributed to the increased engagement cannot be isolated, given the wider impact of the pandemic on the student experience and learner behaviour, there was sufficient positive evidence to justify its retention and expansion. In future, continuing service appraisal will further extend knowledge of this format of IL and academic skills tuition, including level and course-specific research, and in the blended learning scenarios of 2021-22 and beyond.