‘Babe... you're a bit of a know it all’: Student love and breakup letters to a library research skills tutorial

This study employed a novel user experience method, the love/breakup letter, to evaluate the usefulness of an online information literacy (IL) tutorial for students writing assignments with research requirements. Thematic coding of the letters showed that this method elicited revealing responses from students about their confidence in doing library research, preferred learning styles, and intentions to use such a tool again. In contrast with highly-favourable prelaunch user testing results, the majority of students wrote breakup letters to the tutorial and several bypassed it altogether despite their explicit assignment instructions. The findings have implications for librarians exploring new research methods and investigating practical applications of online IL teaching tools designed for self-directed learning.


Introduction
Dear Library Research Skills Tutorial, You're just not that important to me. We're in different places. We would have been good together a few years ago, when I was young and didn't know what I was doing. But I'm older now, more mature, and I don't need you. Best of luck (Student #14) Online learning objects, now ubiquitous in academic libraries, offer the potential to support selfdirected learning and supplement or even replace face-to-face instruction (Saunders, 2018;Thornes, 2012;Watts, 2018). To that end, in February 2018, Concordia University Library, a library in an urban comprehensive university in Montréal, Canada, launched the Library Research Skills Tutorial. Comprised of thematic guides, videos and learning activities, the seven-module tutorial was envisioned as a self-directed learning tool that would effectively help students find, cite and evaluate appropriate information sources for their research assignments, especially when library workshops or librarian consultations were not feasible. Informal prelaunch user testing of the tutorial garnered positive feedback, with participants attesting in surveys that the sections they reviewed were clear, useful, well designed and easy to navigate. Yet these tests took place inside the library, with recruited participants. They thus revealed little about the intended applications of the tutorial as a tool which students might find relevant when pursuing habitual academic activities. This study therefore first aimed to investigate whether the http://dx.doi.org/10. 11645/15.3.2887 Library Research Skills Tutorial would be useful when recommended to students with actual research assignments in specific courses.
In addition, rather than focus on discrete measures of effectiveness or potential areas for design improvement, the authors hoped to capture students' instinctive reactions to the tutorial within their own circumstances and context. Consequently, the love/breakup letter method was employed as an alternative to the more common surveys, focus groups and usability studies. Though this novel user experience (UX) method has been gaining momentum in the library world as a new way of exploring students' affective reactions to tools and services, it is still not prevalent within the academic library literature. This study therefore also sought to investigate whether love/breakup letters could be effectively employed and analyzed as part of a qualitative library study.

Literature review 2.1 Tutorial best practices and evaluation methods
Librarians seeking guidance on the design or redesign of information literacy (IL) tutorials will find a rich pool of material to consult. Literature describing best practices and discussing evaluation and assessment is abundant, as evidenced by the publication of no less than three review articles in the last ten years (Blummer & Kritskaya, 2009;Hartog, 2018;Watts, 2018).
Yet consensus on what works best has by no means been reached. Researchers have frequently reported conflicting comments from users in their evaluations of tutorial characteristics such as length, pace, level of detail, interactivity, and level of difficulty (Bury & Oud, 2005;Held & Gil-Trejo, 2016;Weiner et al., 2012). Examples of ongoing debates include those over the usefulness of accommodating different learning styles (Rapchak, 2017;Watts, 2018), the effectiveness of split-screen guidance, as well as the true merits and meanings of interactivity (Hartog, 2018;Watts, 2018). Inconclusive findings such as these serve as reminders not only that tutorial designers face difficult challenges (Held & Gil-Trejo, 2016) but also that students' immediate context and motivations will often be at least as relevant as the instructional design of any one teaching tool (Mery et al., 2014;Watts, 2018).

Content and scope
Studies have repeatedly examined tutorial characteristics such as navigability, ease of use, clarity, level of detail, comprehensiveness, accessibility, findability and specific weaknesses in need of improvement (Bowles-Terry et al., 2010;Leslie 2017;Scales et al., 2014;Weiner et al., 2012). Such evaluations have regularly led to tutorial enhancements or redesign (Befus & Byrne, 2011;Bury & Oud, 2005;Held & Gil-Trejo, 2016;Phillips & Kearly, 2003). Yet researchers have less frequently explicitly reported investigating factors related to students' own context and setting. The importance of such factors, however, has not escaped attention. As early as 2001, Veldof and Beavers, citing Wiegand, declared their aspiration to 'locate the library in the life of the user rather than the user in the life of the library'' (p. 4), by for instance encouraging students to freely describe their overall attitudes toward library research and their personal expectations of the tutorial as applied to hypothetical course assignments. Scales et al. (2014) invited students to reflect on how they might apply the skills they learned from an online tutorial in an IL course to other classes, though answers to these questions were not described or analyzed in the article. Held and Gil-Trejo (2016) invited volunteer students to reflect on how they might or might not make use of the tutorials they were testing in their own academic journeys, but no specific courses or assignments were invoked.

Participant recruitment and setting
When evaluating freestanding online tutorials, researchers frequently report finding students via flyers or university portals (Mery et al., 2014;Turner et al., 2015) and offering them incentives such as cash or gift cards to take part in evaluation exercises set outside of the classroom (Bender & Rosen, 1998;Bowles-Terry et al., 2010;Bury & Oud, 2005;Held & Gil-Trejo, 2016;Mestre, 2012;Veldoff & Beavers, 2001). On the other hand, when tutorials are embedded within IL or first-year courses, registered students usually serve as the study participants, often as a mandatory part of the course (Noe & Bishop, 2005;Scales et al., 2014;Tooman & Sibthorpe, 2012).
In few of these cases are participants testing an online IL tutorial as a self-directed research tool for class assignments which were not already specifically designed with that tool in mind. This study therefore sought to fill that gap by asking: 1. Would undergraduate students consider the Library Research Skills Tutorial useful if recommended to them as a self-directed learning tool when completing a research assignment?

User experience and the love/breakup letter method
Unlike more traditional attitudinal research tools such as surveys, focus groups and interviews, love/breakup letters offer students a narrative-based approach to creatively describe their visceral reactions or strong emotions towards tools and technologies (Gerber, 2012;Walker et al., 2018). They are part of the user experience (UX) research and design toolbox, which employs a variety of behavioural and attitudinal techniques based on ethnographic approaches (Stiles, 2017). UX aims to holistically examine users' perceptions about the effectiveness of a tool or service, their affective responses to it, and the quality of their relationship with it (Kuniavsky as cited in Walton, 2015). In love/breakup letter exercises, participants are typically asked to 'personify a product and write a personal message to it,' with results often described as 'unexpectedly deep and revealing' (Martin & Hanington, 2012, p. 114).
Originally conceived by the Smart Design consultancy firm in 2009 (Martin & Hanington, 2012) love/breakup letter instruments have since been employed and analyzed in educational technology, human-computer interaction and product-design research studies (Gerber, 2012;Vasques et al., 2017;Walker et al., 2018). Libraries have also started to use this methodology in order to gauge reactions to new products such as discovery tools (Parker, 2017), and to solicit feedback about specific digital services (Gregory, 2018) as well as overall library services (Brook et al., 2019;Green, 2017;Irving, 2020;Stiles, 2017;Texas Tech University, 2018). However, at the time of writing, no library-based research study employing this methodology had been published in an academic journal, nor had any love/break up letters been analyzed using a basic qualitative research approach. http://dx.doi.org/10. 11645/15.3.2887 This study therefore sought to answer: 2. Will the love/breakup letter method work as part of a qualitative study in a library setting? Will students know how to participate, and will it provide helpful feedback for analysis?

Research design
To answer these questions, the researchers followed a basic qualitative research approach (Plano Clark & Creswell, 2015), using thematic coding to analyze letters written to the Library Research Skills Tutorial by undergraduate students. A two-question survey preceding the letter also assessed participants' self-reported IL exposure and skills.

Participants
Ethical approval for this study was obtained from the Concordia University Human Research Ethics Committee (Certificate No. 30009916).
Using non-probability purposive sampling (Plano Clark & Creswell, 2015) such as referrals from other liaison librarians, personal experience with faculty members and perusal of the 2018-2019 University class schedules, approximately 15 potential courses were identified for the study. Only courses incorporating at least one assignment with a library research component were considered. Four professors agreed to participate and include their classes in the study, representing social science courses at the 200 (2), 300 (1), and 400 (1) levels. Student participation in the study was optional and written informed consent was obtained from all participants prior to the letter writing exercise. Fifty-two (52) students took part, with varying participation rates in each class: 2 / 75 (200-level), 21 / 50 (200-level), 21 / 41 (300-level), 8 / 9 (400-level).

Data collection
The researchers worked with the participating professors to incorporate a paragraph about the tutorial and the evaluation exercise into their assignment descriptions. The paragraph specified that consultation of the tutorial was required for the research assignment but that the evaluation, which would take place on the day of assignment submission during class time, was optional. A few weeks before the assignments were due, the researchers asked the professors to remind students about both the tutorial and the evaluation. In order to encourage spontaneity during the activity, the letter-writing method was not revealed ahead of time.
On the day of the visit to the class, the researchers read a prepared script, asking students to imagine the tutorial as a person and to address a love, breakup, or indifferent letter to it. In case students were apprehensive about the activity or lacked inspiration, the researchers provided further illustration, letting students know that they could for instance tell the tutorial what aspects of it they would never want to change (love), what about the relationship didn't work or was annoying (breakup), or why they chose to not even go on a first date (indifferent). The students were told that they would be given a maximum of 20 minutes to write the letter. According to Martin and Hanington, 'longer time frames [would] make participants overthink its contents' (2012, p. 114). The instrument included a two-question survey, instructions on how to complete the activity, and lined sheets to write the letter. The procedure for the first class visited, however, was slightly different and proved much less successful. In this 200-level class, the research assignment was optional and could be substituted by an in-class test. As a majority http://dx.doi.org/10. 11645/15.3.2887 had chosen the test over the research assignment, the letter-writing activity did not take place in the class and students were instead invited to contact the researchers to volunteer for the study. Only two of the 75 students chose to participate, and the activity took place in a library conference room, with the researchers following the same script as described above.

Data availability statement
The dataset generated and analysed during the current study is available in the Concordia University Dataverse at https://doi.org/10.5683/SP2/QOAVHE (Dennie & Breier, 2018).

Data analysis
The letters were first divided into overall letter types, such as love or breakup letter, based on the responses provided by the students. Then, the letters were transcribed in Dedoose, a webbased qualitative data analysis software (https://www.dedoose.com/). The researchers performed independent parallel coding to analyze the text of the letters using a general inductive approach (Thomas, 2006). They compared, edited, and merged their codes into a single final set of codes and themes. The library's Teaching and Learning Librarian also reviewed the transcribed letters and emergent themes as a peer debriefing process (Janesick, 2015).
Eight letters were excluded from the analysis. Seven of these were addressed to either the Library, a subject guide, or a database instead of referring to the tutorial, and one consisted of lyrics from a 1980s pop song. Of note, five of the eight excluded letters originated from the 400level class. The content of these letters revealed that most of the students in this class were not aware of the tutorial prior to the evaluation exercise. One student wrote, 'I did not really know about you. Perhaps, if one of my professors had been more explicit about finding you & using you as a tool, I would have.' (Student #48). Unlike in the other courses, it appears that the professor neglected to discuss the tutorial and research project in this class despite the researchers' requests, leaving students to learn about it in their research assignment description.

Letter types
Though the initial intention was to divide the responses into love, breakup or indifferent letters, during analysis small variations on these types emerged. Letters expressing feelings of frustration or lack of interest, such as 'I just don't understand you, you're just too complicated' (Student #10) and 'I am bored when I am with you' (Student #15) were categorized as breakup letters. Letters including terms of affection or appreciation such as 'you are the best tool' (Student #1) and 'once I got to understand you, there was nothing like it' (Student #2) were labeled as love letters. Letters expressing lack of interest, with passages such as 'Sorry for bailing on our first date. I just already knew how to conduct library research,' (Student #20), were renamed from 'indifferent' to 'never used' as this term more accurately described the conscious bypassing of the tutorial despite assignment instructions to consult it. Finally, four letters told stories that did not clearly fit into any of the three predefined types. Two letters included statements of both appreciation and dislike for the tutorial, with no clear indication of whether the writers were in love or wanted to break up; these were categorized as ambivalent. Two intriguing letters began by gratefully acknowledging the help the tutorial provided, but ended by intimating that they no longer needed the tool, with passages such as 'I've grown http://dx.doi.org/10. 11645/15.3.2887 since then and want to fly on my own now' (Student #29). In keeping with the dating theme, the researchers labeled these letters as one-night stands, an unexpected variant on the breakup. The majority of responses (52%) constituted breakup letters. Only 23% of students wrote love letters, followed closely by 16% never used letters (see Figure 1).

Figure 1: Letter types
It is of note that students in the introductory, 200-level classes exhibited the least appreciation for the tutorial (see Figure 2), with 87% of their letters falling into breakup and never used categories.

Participation
In order to determine whether students had understood how to participate in the letter writing activity, their responses were analyzed to see whether they were structured as letters. For example, did the responses open with 'Dear…' or did they address the tutorial directly by referring to it as though it were a person?
The analysis revealed that 73% of the total responses were written as personalized letters. Those breaking up seemed the most enthusiastic about the format, with 83% formatting their responses as letters to the tutorial. Of the love-like responses, 70% were written as letters. Only 43% of the never used responses adopted the letter format. Some participants remarked on their positive experience while handing in their letters, offering comments such as: 'I actually enjoyed writing this', 'It's a creative way of expressing yourself, '

Survey questions
Two survey questions about confidence in performing library research and past library workshop attendance preceded the letters. Students' confidence in performing library research was greater for those having taken a library workshop in the past (Mdn = 5) than for those who had not (Mdn = 4) (see Figure 3), with a Mann-Whitney test indicating that this difference was statistically significant, U(Nworkshop=18, Nno-workshop=22,) = 82.5, p < .005.

Figure 3: Students' confidence in performing library research according to past workshop attendance
No statistical significance between letter types or class level and past workshop attendance or confidence in library research was revealed. The researchers noted, however, that although nearly all (6 of 7) never used letter writers also indicated never having attended a workshop in the past five years, approximately 50% of writers of all remaining letter types indicated having attended a library workshop.

Letter themes
The authors identified five underlying themes during letter coding, which they grouped into two overarching conceptual categories. The first category, 'It's you', referred to passages addressing the tutorial's characteristics, with themes directly describing design or content (see Table 1). The second category, 'It's not you, it's me', consisted of excerpts referring to students' own context: their intentions to use again, their attitudes towards library research, and their learning/searching styles or preferences (see Table 2).  'It's you' passages mostly emerged from love letters and were either generic in nature or nonactionable. These included phrases such as 'You helped me find everything I needed to write my paper' (Student #37), coded as 'usefulness', or 'Everything was well presented' (Student #40), coded as 'good design'. Other excerpts related to the level of detail in the tutorial and revealed contradictory assertions. For example, Student #16 wrote 'when it comes to finding information on specific topics, I find it tends to be limited', whereas Student #23 wrote 'I think for someone looking for specific info it might be a bit overwhelming and hard to situate themselves.' The more numerous 'It's not you, it's me' excerpts proved more fruitful for analysis. A common theme in this category was 'intention to use again', emanating mostly from breakup or never used letters. Half of the comments here alluded to the potential value of the tutorial and offered some promise or intention of future use, such as 'I hope we can still be friends and that it's okay if I come to you in a time of need' (Student #26). Of the other half of passages indicating no intention to use again, many offered best wishes for success with others, such as 'Hopefully you find someone else that can benefit from you' (Student #21 The most prevalent theme in the student context category dealt with attitudes towards library research, with more than half of the excerpts coded as 'confidence'. Students wrote passages indicating that they already knew how to do library research, or had learned it previously, as exemplified by Student #33: 'All this had information that I generally knew'. These statements of confidence were often attenuated, even by students in the introductory 200-level classes, with suggestions that the tutorial would be more appropriate for those with less experience, such as 'someone doing a research paper for the first time' (Student #4).
The final theme identified in this category related to students' learning and searching style preferences. Most excerpts were from breakup letters and specified a preference for other more familiar research sources such as Google, YouTube, and even a library subject guide. Student #22 summed it up as such: 'I have found other, better, more user friendly resources to have my academic paper writing time with.'

Discussion
In answer to the first question this study investigated, undergraduate students did not consider the Library Research Skills Tutorial to be useful when recommended as an independent learning tool for help with writing research assignments. This finding is significant for academic librarians spending time and energy on the development of high-quality tutorials with the intent of having them serve as self-directed or assigned learning tools.
The high proportion of breakup letters written by students sharply contrasts with the overwhelmingly positive feedback received during informal prelaunch user testing. This rejection seems closely tied to the fact that participants, instead of being recruited to evaluate the tutorial for its own sake or for an imagined future use, were directed to use the tool for an actual assignment in their course. The literature has shown that when doing library research with the goal of completing an assignment, students generally look for tools that accomplish tasks rather than teach concepts (Bowles-Terry, 2010;Nichols Hess & Hristova 2016;Rothera, 2015;Veldof & Beavers, 2001). Farkas (2014) asserts that 'students are not usually looking to learn how to do research, but to do something specific. Students want quick answers to their questions, not a tutorial' (emphasis added, slide 43). Though the Library Research Skills Tutorial was designed to include short stand-alone segments and videos aimed at answering quick questions, when taken as a whole it definitely constitutes a learning, rather than a doing, tool. And if students often use 'highly strategic, goal-focused, linear navigational strategies' (Rothera, 2015, p. 47) when researching online, lack of enthusiasm by participants in this study for exploring a sevenmodule library research tutorial should not be surprising. The highly-generic comments about design and content found in the majority of love letters suggest that in addition to the participants who admitted bypassing the tutorial altogether, many engaged with it only minimally.
Students wrote that the tutorial, though possibly appropriate for other people or under other circumstances, was not well suited for their own needs. Other studies show that even online tutorials with highly-positive student reviews will not necessarily be considered personally relevant by those same students, nor adopted by them as ongoing objects of attachment. When Donaldson (2000) evaluated an online tutorial designed for a first-year IL class, a strong majority rated the tutorial's content, delivery method, and presentation as excellent or good, but rated its value to their own program of study as only fair. Tooman and Sibthorpe (2012) found that though students were highly satisfied with their tutorial experience, 60% said that they would not need the tool again, despite the fact that most required multiple attempts to succeed in the accompanying competence quizzes.
The expressions of confidence found in breakup and never used letters closely align with responses documented by Colosimo and Kasuto (2012) in their video tutorial testing. They also echo comments recorded in the online tutorial evaluations by Phillips and Kearly (2003): 'I already know this stuff; don't make me do this' (p. 354) and by Veldof and Beavers (2001): 'this might be good for beginners, but I think it is annoying' (p. 9). Though this study did not test confidence against actual evidence of research skill, the literature convincingly argues that the Dunning-Kruger Effect, or the tendency of individuals less competent in certain skill areas to overestimate their abilities, also applies to undergraduates' self-assessment of library research proficiency (Gross & Latham, 2012;Mahmood, 2016). And since confident students 'are unlikely to seek opportunities to build skills they think they already have' (Gross & Latham, 2009, p. 337) confidence also likely contributed to the lack of engagement with the tutorial by participants in this study. That students in the introductory 200-level class proportionally wrote the most breakup and never used letters, and that their confidence levels showed no statisticallysignificant difference from those of 300-level students, supports these contentions.
In terms of this study's second research question, results indicate that the love/breakup letter method can indeed work as part of a qualitative study in a library setting. The letters provided unique feedback, offering valuable insights into student attitudes toward library teaching tools and their usefulness for class assignments. The findings, and the authors' observations, also support the idea that letter writing-particularly of the breakup variety-can be a satisfying and cathartic experience for participants (Gerber, 2012). It is important to note that love/breakup letters were initially intended to measure product loyalty (Martin & Hanington, 2012) rather than the effectiveness of library teaching tools. Yet the method proved flexible enough to allow for the emergence of unexpected letter types and conceptual categories which seemed particularly suited to the evaluation of IL tools. The never-used letter type and the 'It's not you, it's me' category, for instance, offered useful insight into students' subjective understandings of their own abilities to do library research unassisted. And the one-night stand letter type seemed uniquely relevant to the context of teaching objects, which seek to engender independence on the part of their users at least as much as attachment.

Limitations and further research
The love/breakup letter methodology was not without its limitations. Future researchers considering the method for purposes of refining tutorial design should take note of the high proportion of generic, non-actionable comments. Similarly, Priestner (2018) warned researchers of the potential use of less informative euphemisms by 'creative writers' (p. 47). Further, sometimes the creative language in the letters actually obscured possible intent. Lines such as 'I feel that you don't understand me anymore (…) I have to change myself to suit you' (Student # 17) added creativity to the exercise, but could also tempt researchers into overanalyzing what are simply personal relationship tropes. Finally, though the method proved adaptable in the evaluation of an IL tool, the authors posit that letter writing exercises addressed to intentionally habit-forming library services or tools such as bibliographic management software or study spaces could produce especially fruitful results.
In terms of the study's design, a specific limitation is that evidence of students' actual research trajectory or success was not examined. Further research might combine an affective instrument such as the one used here with analysis of research assignments and bibliographies, or of a detailed research diary (Tolland et al., 2019).

Conclusion
This study sought to determine whether students would find a freestanding online research skills tutorial useful if recommended to them as an aide while working on a research assignment, and whether qualitative analysis of love/break up letters to that tutorial would provide unique and helpful results.
Most students did not find the Library Research Skills tutorial useful when recommended to them for their research assignments. Comments in the letters support the notion that learning objects should aim to quickly answer specific questions rather than offer IL lessons. They also indicate that regardless of quality, online teaching tools will not prove useful to students who simply want to find resources nor to those who feel they do not need instruction. These findings should have implications for IL professionals assessing their approach to tutorial conceptualization or revision.
Writing personalized letters to the tutorial allowed participants to express their feelings about a library tool in a creative and satisfying way, and qualitative analysis of these letters provided the researchers with a unique perspective on students' customary research habits. Indeed, the love/breakup letter method offers promising opportunities for librarians seeking to understand the place of the library and its resources in the everyday lives of students.