A review of information literacy programmes in higher education: The effects of face-to-face, online and blended formats on student perception

This review will aim to establish if there is strong evidence to suggest a student preference for delivery format within information literacy teaching. This research supports and builds on research previously undertaken by Cardiff University (Weightman et al., 2017). Weightman et al (2017) addressed the effect of face-to-face or online learning specifically on learning outcomes. This review specifically focuses on the effects of these methods, and blended formats, on student preference. This research informs teaching practice specifically within Cardiff University’s library service but also teaching practice generally. A comprehensive systematic literature search was undertaken in four databases: Library, Information Science and Technology Abstracts (LISTA), British Education Index, ERIC and Scopus. Seven new papers were identified to update the previous discussions on student preference of information literacy teaching format (Weightman et al., 2017). Critical appraisal was undertaken of these newly identified papers. Weightman et al.’s (2017) systematic review suggested that there was no student preference in relation to delivery format. Of the seven new papers identified in this review, two (DaCosta, 2007; Gorman & Staley, 2018) show a slight preference for format; one for online and one for face-to-face although there are limitations to the studies. Of the five remaining studies (Craig & Friehs, 2013; Kelly, 2017; Lag, 2016; Lapidus et al., 2012; Matlin & Lantzy, 2017) all showed a comparable experience between formats, although limitations of these studies are also acknowledged. The update search and appraisal of the literature concurs with previous conclusions (Weightman et al., 2017) that experiences are comparable and student preference is generally neutral in relation to delivery format. Student learning outcomes and student preference are comparable regardless of format (Weightman et al., 2017).


Introduction
In 2017 a systematic review was published investigating the effects of face-to-face and online formats on student skills and views in information literacy (IL) programmes (Weightman et al., 2017). The review aimed to determine the effect of teaching method on student learning outcomes. The research was undertaken to provide evidence to inform Cardiff University Library Service's decision to move to an online library induction as opposed to the more traditional faceto-face format. The results of the review demonstrated that training is equally effective across a range of delivery methods. Cardiff University therefore instated an online library induction. This led to further considerations: are there other influences which could help us to determine the format of our IL teaching? Would wider exploration of the literature on student preferences for a particular format reach the same conclusion as this earlier research into learning outcomes? These questions are particularly relevant given that the timing and location of IL workshops in UK higher education can often be determined by schools or faculties. This can therefore reduce autonomy over the delivery of IL teaching. In addition, staff resources and time in the curriculum are under pressure and it is important that we determine the most effective pedagogical means of delivery. If we had evidence of student preference, supplementing the evidence relating to learning outcomes, it could give librarians the confidence to deliver more online or blended learning. This is turn could potentially provide a greater level of autonomy over the design of IL workshops when this is not always possible in a strictly face-to-face format. Although consideration to the wider pedagogy of the subject specific course should also be taken into account. Weightman et al. (2017) noted that, while undertaking their review relating to learning outcomes, many of the included studies also discussed student perceptions of the delivery format. These findings relating to student preference are discussed briefly within the review however the search strategy undertaken for the review did not specifically seek to find literature which addressed student perceptions. Therefore, this systematic literature review aims to build on this research by undertaking a wider search of the literature to specifically identify further papers which address student perceptions of the delivery format of information literacy teaching: face-to-face, online or blended. It aims to determine if there is evidence within the literature to suggest a student preference for format within information literacy teaching which in turn could help to give librarians the confidence to deliver IL in an online or blended format, especially when face-to-face teaching with the timetable is limited.

Methods
A search strategy was undertaken in four relevant databases: • Library, Information Science and Technology Abstracts (LISTA) • British Education Index • ERIC • Scopus These databases were chosen as they were deemed to be the most relevant subject specific databases which would contain the literature required to answer the research question. These databases provided coverage of a range of specialities which relate to the research area including library and information science, education, and social science literature.

Inclusion and exclusion criteria
The criteria for selection of the papers is defined in Table 2.

Study selection
Abstract sifting, critical appraisal and study selection was undertaken by a single reviewer. To undertake a best practice review two independent reviewers should sift and critically appraise the papers to decrease the potential for bias. This was unfortunately not possible in undertaking this review however a single reviewer reviewed all papers, therefore providing some level of consistency.

Quality assessment
All papers were critically appraised using a checklist adapted from Morrison et al. (1999), replicating the process undertaken in the original systematic review looking specifically at learning outcomes (Weightman et al., 2017). The checklist includes many standard, best practice questions. It was specifically selected for use in this review as it relates to interventions in an educational teaching setting. Critical appraisal documentation for the newly identified seven studies are available upon request.

PRISMA diagram
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) provide a minimum set of checklists and flow diagrams which should be used to show transparency when undertaking high level reviews (PRISMA, 2015). A PRISMA diagram was compiled depicting the flow of information through the different phases of the review (Figure 1).

Figure 1: PRISMA diagram
No. of records identified by searching databases

N=1243
No. of records identified by update databases

N=59
No. of records found through forward/back searching/ handsearching/alerts N=519 No. of records after removing duplicates

N=1081
No. of records of abstracts screened N=1081 No. of records excluded based on abstracts N=1024 No. of full text articles obtained N=57

No. of records included in review N=24
No. of records excluded based on full text N=33

Study quality
Included studies were identified from those identified in Weightman et al. (2017) and from the search strategy outlined in Table 1. Seven new studies matching the inclusion and exclusion criteria were identified through the search and will be considered alongside the 17 papers previously identified (Weightman et al., 2017). The 17 papers were included alongside the additional seven new papers to provide a full overview of the relevant literature, presented in a single paper.
Of the 24 included papers, one is a randomised controlled trial (Goates et al., 2017) whereas the other studies are not randomised but do include a control group.
Of the seven newly identified studies within this review, three are controlled before/after studies (CBA) meaning they undertake a survey of the students pre and post intervention (Craig & Friehs, 2013;DaCosta, 2007;Kelly, 2017) and 54 are controlled with the survey only undertaken after the intervention (CA) (Gorman & Staley, 2018;Lag, 2016;Lapidus et al., 2012;Matlin & Lantzy, 2017).

Participants:
Nine sections, 234 students of an undergraduate biology lab class

Measuring: student confidence
Measured tool: pre/post survey Neutral "65% felt they learnt better from a librarian than from online instruction […] student confidence increased most with the interactive tutorial" Differing levels of previous library instruction amongst the participants. Only 66% of students submitted survey responses as one live faceto-face group was interrupted and students left before the survey was administered.

Results
Weightman et al.'s (2017) systematic review considers students' perceptions of the delivery method of IL training as defined in 22 studies; three of these studies only gather student views from the intervention group and two are researching student perceptions of library inductions. These have therefore been excluded from this study (see inclusion/exclusion criteria Table 2). Of the remaining 17 studies, 11 identify no preference relating to the delivery format of the information literacy teaching (Beile & Boote, 2004;Holman, 2000;Koenig & Novotny, 2001;Kraemer et al., 2007;Lantzy, 2016;Nichols Hess, 2014;Nichols et al., 2003;Schilling, 2012;Silk et al., 2015;Vander Meer & Rike, 1996;Wilcox Brooks, 2014). Of the five studies which suggest a student preference there are variations in what this preference is. One study (Alexander & Smith, 2001) finds that an online format is favourable. Three studies (Churkovich & Oughtred, 2002;Shaffer, 2011;Wilhite, 2004) identify that face-to-face delivery is preferable or of higher satisfaction. One study (Goates et al., 2017) suggests a preference for the flipped format. One further study's (Silver & Nickel, 2007) results are unclear. Overall the review (Weightman et al., 2017) finds that although there are advantages and disadvantages to the delivery methods, there is no clear preference. Weightman et al's systematic review evidences robust methodology including extensive database searching, reference list follow-up, citation tracking, hand-searching of 2016 editions of eight journals and two independent reviewers. It should be noted for transparency that one of the named authors on the systematic review (Weightman et al., 2017) is also the author of this systematic literature review update paper.
Of the seven studies identified within the methodology of this review, two studies identified that students declared a preference for a specific format. DaCosta (2007) compared online and face-to-face. This study shows a preference for learning via a virtual learning environment (VLE), specifically Blackboard, rather than face-to-face. This study aims to determine student confidence following completion of the teaching. Respondents in the intervention group demonstrated a rise in confidence: when asked how they felt about further Blackboard learning in the future the 60% who had first stated they would 'give it a try' prior to the intervention then stated 'bring it on.' An equivalent to this question was not posed to the control group so it is not known how the control group would have responded to this. Overall, when considering both the control and intervention groups' responses, the confidence level was greater among the intervention group (69%) than it was in the control group (58%). This shows a very slight preference towards online learning via a VLE platform, however the sample size of the participants involved in this study was not provided. Gorman and Staley's (2018) study concludes a student preference, but provides conflicting results; the students in this study prefer the online format overall but in contrast favour face-toface for improved confidence. In relation to student preference, 7.1% prefer in person whereas the majority (92.9%) prefer the online format. This data is however based on limited respondents; only the intervention group were asked this question and of this group only respondents who had received previous face-to-face library instruction could respond. This left 14 respondents, of which one stated they prefer face-to-face. The students in this study also self-selected. As stated, student participants in this study suggested they favour face-to-face in relation to an improved confidence level. Of the intervention group surveyed 44.4% state that their skills had improved, 50% state their skills were about the same and 5.6% state their skills were worse, whereas of the control group 87.5% state that their skills had improved, 12.5% state their skills were about the same and none report their skills were worse. This suggests a much higher level of improved confidence amongst the control group receiving face-to-face instruction. The sample size of the participants was however small (38 participants). Craig and Friehs (2013) compare a video based online tutorial, a HTML based online tutorial and face-to-face. This study evidences a preference for a video based online tutorial and faceto-face when compared to a static HTML online tutorial. All students regardless of intervention suggest an improvement in confidence. The students who are exposed to the static HTML tutorial report an increase of 25.1% in confidence whereas those assigned to the online video tutorial intervention report a 45.5% increase in confidence and those students within the control group (face-to-face) report a 41.5% increase in confidence. The online video format and face-toface interventions report a similar level of confidence increase however it should be noted that the students undertaking the static HTML intervention started with a higher confidence rating and therefore this could explain the difference in results. This group also self-reported that they had a greater level of prior experience with searching databases. Overall, student confidence increases the most for the online video format but only marginally in comparison to the face-toface control group. However due to an error in the face-to-face control group, not all participants were surveyed.
The remaining four studies report neutral results with students not showing a preference for any of the formats to which they were exposed. Kelly (2017) measures student confidence and research self-efficacy comparing face-to-face and blended formats. In order to ensure that all participants in the blended intervention group were exposed to the videos the lecturer played them in a lecture environment. This means that the students in the intervention group were not able to engage with the videos as would be the case in a more traditional blended approach. They were unable to watch at their own pace, return to the videos etc. Results from this study show reported confidence was not significant between those exposed to face-to-face or blended formats. Perceived self-efficacy levels increase in both interventions.
Lag (2016) also uses a blended format in the form of flipped classroom in comparison with traditional face-to-face. The results of this were deemed to be neutral although student characteristics amongst the groups are not described; no student preference is identified. Student participants are surveyed and asked to rate the usefulness of the session. The mean for the control and intervention group are similar; 3.95/4 and 3.92/4 respectively. Student participants are also asked to rate the quality of the session and this once again returned similar results; 3.89/4 (face-to-face) and 3.79/4 (blended). Usefulness and quality of sessions is rated the same.
Similarly comparing face-to-face and blended interventions, Lapidus et al.'s (2012) study also concludes there is no student preference for format. Student participants are asked to rate the effectiveness of the course structure on their learning using a Likert scale. Of the control faceto-face group 53% agree, 29% somewhat agree, 6% are undecided, 6% somewhat disagree and 6% disagree. Of the blended intervention group 67% agree, 19% somewhat agree, 5% are undecided, 0% somewhat disagree and 9% disagree. The student perception of the blended format in particular varies from very positive to very negative. The authors believe this reflects differing attitudes towards online methods of course delivery. Although this variation is evident and the blended format does show a slightly stronger rating in relation to students' agreement with the statement (67% in comparison to 53%), the results are not strong enough to suggest a preference for format. The response rates for the survey in this study are also very limited; 17 out of 146 participants responded from the face-to-face control group and 21 out of the 160 participants responded from the blended intervention group.
Matlin and Lantzy's (2017) study compares face-to-face and online formats. Students from two cohorts are asked to rate their confidence following the interventions. In the first cohort the mean score for the face-to-face control group is 3.51 and the mean score for the online intervention group is 3.40. In the second cohort the mean score for the face-to-face control group is 3.63 and the mean score for the online intervention group is 3.50. The average scores for the face-to-face control group are marginally higher, but not enough to be deemed significant. The study also measures clarity of the session content and students' comfort in asking for help. Alongside the confidence ratings, these all show a comparable experience for both the control and the intervention groups.

Discussion
Overall research suggests no overall preference for delivery method of IL teaching. Alongside this there is also no overall difference in learning/skills outcomes (Weightman et al., 2017). However, some papers did identify a preference for format. Of those papers which show a student preference for a delivery format, it's worth considering if there are any notable similarities which can be identified in the setting, cohort or methodology of the study which could suggest why they may have had identified a preference.
Seven papers identified a preference: one favoured online (Alexander and Smith 2001), three favoured face-to-face (Churkovich & Oughtred, 2002;Shaffer, 2011;Wilhite, 2004), one favoured learning via VLE which for the purposes of this discussion we will consider as favouring online (DaCosta, 2007), one favoured flipped classroom (Goates et al., 2017) and one showed conflicting results between favouring face-to-face and online (Gorman & Staley, 2018) and as such we will exclude this paper from the discussion.

Setting
Of these studies, four are undertaken in the US (Alexander & Smith, 2001;Goates et al., 2017;Shaffer, 2011;Wilhite, 2004), one in the UK (DaCosta, 2007) and one in Australia (Churkovich & Oughtred, 2002). The fact that the majority of these studies are undertaken in the US is not likely to be significant given that the majority of all of the literature included within this review was undertaken in the US (21 of the 24 relevant studies). Of the four studies which are undertaken in the US, one favours online (Alexander & Smith, 2001), one favours flipped (Goates et al., 2017) and two favour face-to-face (Shaffer, 2011;Wilhite, 2004), therefore showing no consistency in country of origin of teaching.
Of the studies which show a preference for delivery format there is no clear consistency in the type of university they are attending. Those studies which show students prefer learning online are students in Western Kentucky (Alexander & Smith, 2001) and De Montfort University (DaCosta, 2007), those which show students prefer learning face-to-face are Deakin University (Churkovich & Oughtred, 2002), University of New York at Oswego (Shaffer, 2011) and University of Oklahoma (Wilhite, 2004), and the students who prefer learning IL in a blended format are attending Brigham Young University (Goates et al., 2017).

Cohort
The cohort of students in each study could be significant. IL teaching is usually embedded within a wider curriculum, for example IL teaching within a medical degree. In this instance, IL teaching delivered via a blended, flipped classroom format could be demonstrated as preferable to students because they are already familiar with problem based learning (PBL) and the expectation of being required to undertake pre-learning before their tutorials.
Within the studies identified, the students who show preference for online are undergraduates on a media course (Alexander & Smith, 2001) and undergraduates in the faculty of health and life sciences (DaCosta, 2007). Perhaps this is significant. It could be anticipated that media students will be more comfortable with use of online resources. Those who favour face-to-face are undergraduate sociology students (Churkovich & Oughtred, 2002), postgraduates studying a research methods course (Shaffer, 2011) and business undergraduates (Wilhite, 2004). Perhaps the influence of postgraduate and undergraduate is significant but not enough data is available here to draw this conclusion.
Cohort size could be a determinant however this varies across the studies. Of those who favour online the cohort size is 88 (Alexander & Smith, 2001) but unfortunately is not identified by DaCosta (2007), those who favour face-to-face are in a cohort size of 174 (Churkovich & Oughtred, 2002), 59 (Shaffer, 2011) and 44 (Wilhite, 2004), and those who favour flipped are in a cohort of 122 (Goates et al., 2017).
Overall, there do not appear to be any easily identifiable trends which identify whether the cohort could have affected the outcome of the study.

Methodology
Only one of the papers identifies a preference for the flipped classroom approach, and this paper is published in 2017 (Goates et al.). In fact, of all the papers included within the review only three consider all three teaching methods; online, face-to-face and flipped (Beile & Boote, 2004;Churkovich & Oughtred, 2002;Kraemer et al., 2007). The fact that only three papers are identified which consider all three teaching styles immediately reduces the likelihood of flipped classroom being the favoured delivery method. It seems feasible that if this review were to be updated in the next few years, as flipped classroom becomes more prevalent, the outcome could justifiably be swayed towards this.

Discussion conclusions
Of the papers identifying a student preference for delivery method of IL teaching, there does not appear to be any obvious influencing factor. This finding could be deemed significant in itself. This could justifiably lead to suggesting that the results of this review, that overall students do not demonstrate a preference, are robust as they are not influenced or determined by the setting of the study or which student cohort is involved. It remains to be seen over time whether the methodology of the study would have had a direct effect.

Limitations
One of the most significant limitations of this study is the ability to identify all relevant literature. It should be noted that some of the papers included from the 2017 systematic review (Weightman et al.) did not surface in this systematic literature search (Table 1). Much of the literature on student perceptions does not appear to be the main or single focus of the article and therefore is commented on within the body of the text rather than title and abstract. This inevitably means that relevant papers might not have been identified as the databases searched did not have full text search functionality.
Of those studies which could be identified through the database searches, some are excluded as they do not include a control group. The exclusion criteria (Table 2) states that the studies need to compare at least two interventions. Although this helps to improve study quality, it does mean that several papers which evaluate a single intervention are excluded when the results may have been of interest. There may be some value in considering these if the participants' exposure to previous information literacy has been defined. Unfortunately, from the majority of these instances commentary is made regarding student perceptions of online or blended learning, but it isn't clear whether the participants have previously been exposed to face-to-face information literacy workshops and if so, what the content of these was.
As previously highlighted this study is also undertaken by a single reviewer and therefore could have introduced study selection bias.

Conclusions and implications for practice
There is strong evidence to suggest that overall students do not demonstrate a preference for delivery method: face-to-face, online or blended. Of course, learning preference and effective learning are not necessarily inclusive: learners are not always good at identifying how they learn best. However alongside this review suggesting there is no preference towards delivery method, there is also no overall 'statistical significant difference between formats in skills outcomes for students' (Weightman et al., 2017).
As IL teaching is usually embedded within a wider curriculum it is not always possible for librarians to maintain overall autonomy over the delivery format of the teaching, for example it is sometimes dictated from the overarching school/faculty or dictated by availability of time in the timetable. In light of the current body of research, if the librarian is restricted by these factors, they can be confident that they could alter the current format with there being no negative impact on learning outcomes and student satisfaction would not be affected. Blended learning or online learning could therefore be developed to enable more autonomy over IL teaching, especially when only one-shot teaching is an option within timetable restrictions. When delivering teaching via blended or online formats, we can be confident that overall students do not demonstrate a preference. Consideration should instead be given to time available to develop these resources, access to a learning technologist or the pedagogy of the course rather than focusing on the effect of the delivery method on student skills and preference given that this has been proven to be minimal.
It should be noted that there is now a wealth of research which has been undertaken considering the influence of delivery methods on student preference and learning/skills outcomes in IL. Further research in this specific area is probably not necessary however, if further research in this area is to be done, it would be beneficial to see studies comparing all three formats and using a robust methodology (e.g. randomised controlled trials). Information professionals should perhaps also look to the body of literature relating to student preferences for delivery format in all teaching, rather than placing focus solely on IL.