A needs analysis for information literacy provision for research: a case study in University College Dublin

The purpose of this research was to establish the baseline information literacy of incoming postgraduate research students, which in turn could inform the development of information literacy provision to support research. Evidence Based Librarianship and Information Practice (EBLIP) underpinned the methodological framework. An online survey questionnaire, information behaviour observation and a focus group formed the triangulation of methods used in data collection. Findings identified a wide variation in information literacy within and across disciplines; deficiencies in the ability to trace current and ongoing research; difficulties in the conceptualization of research questions and literal rather than lateral thought. However, it must be noted that the non probability nature of the purposive sampling for the survey questionnaire results in data which cannot be extrapolated to other populations. As this study was used to satisfy the partial requirements of an MLIS degree, the constraints of the thesis necessitated the truncation of the EBLIP process, so that the implementation steps were not included. Nevertheless, this study’s contribution to the field of enquiry lies not only in its feasibility as a practical application, but it also in the contribution it makes in an area where a research deficit has been identified (Corrall 2007; Research Information Network 2008).


Introduction
Fundamental and rapid changes in the information environment have facilitated greater accessibility to a wide range of material online, with a multiplicity of access points. However, this has resulted in the electronic information environment appearing complicated to the user, and it is perceived that the skills and competencies of research students in the identification, management and use of these resources have not kept pace with the rapidity of change (Research Information Network 2008). It has also been noted that although information literacy is well represented in institutional strategies at undergraduate level, it is less evident in graduate attributes and research strategy documents (Corrall 2007). The significance of this omission is conspicuous given the high priority attached to postgraduate skills training by the U.K. Research Councils (Corrall 2007). Also noticeable by its absence is research on the topic (Corrall 2007;Research Information Network, 2008). The Research Information Network points to the lack of evidence in the assessment of researchers' training needs, recommending that library and other training providers should adopt "more systematic and innovative approaches to identifying and assessing the needs of researchers to enhance their information-related skills and competencies" (Research Information Network 2008, p. 9).
This research deficit has been felt in practice in University College Dublin. In line with current government policy to develop Ireland as a knowledge society, graduate /PhD training has been restructured, and now includes the acquisition of generic or transferable skills in addition to the core requirement of thesis submission. In common with its progenitor, the Joint Statement of the UK Research Council's Training Requirements for Research Students (UK Grad Programme [online]), information literacy is implicit rather than explicit in the Irish Universities Association's Skills Statement (Irish Universities Association 2008). This structured programme of research and personal development is based on individual needs that have been identified and documented in personal Research and Professional Development Plans (RPDPs), which, when submitted to the research supervisor, form the basis of an agreed development strategy. However the adoption of a RPDP for research students also presents a challenge to the library and its development of information literacy provision. As the RPDP is agreed between the research student and the research supervisor, neither the incoming student's information literacy competencies or needs are actually known to library staff. It is also possible that self assessment, an integral part of the RPDP could lead to over-rated skills evaluation. The variation in implementation of the RPDP model also presents a challenge. This has been recognised in the adoption of this model in the United Kingdom (UK Grad Programme 2004) The aim of this research is to contribute to breaching the identified gap by exploring the information literacy needs of the research student at the outset of their research programme. An appraisal and review of the literature substantiated the evidence for this need.

Literature review
A review of the literature also revealed a lack of research in the identification of levels of information literacy in the research student population. The literature review used to support the report, Mind the Skills Gap: Information-handling Training for Researchers corroborated this and pointed to the fact that the "searches provided few results focused solely on postgraduate and postdoctoral training" and that "there are a few examples of groups conducting investigations into issues such as information literacy in higher education" (Research Information Network 2008, p.31). This was also evidenced in the lack of published case studies available for comparative purposes with this research, although one such, undertaken in the University of Leeds in -2006(Newton 2007) correlated quite closely with this study, and allowed for some comparison. In common with other social sciences, the gap between research and practice is a preoccupation in the field of Library and Information Science (Booth 2003), and it has been argued that practitioners do not make enough use of research to improve services or practice (Cullen 1998). However, it has also been suggested that the greatest obstacle to finding research is that librarians do not publish their research (Crumley and Koufougiannakis 2002). Cognizance of this need to bridge the research-practice divide was taken in structuring this study's methodology.

Methodology
The Evidence Based Librarianship and Information Practice (EBLIP) model provided the framework capable of bridging this research-practice gap. This model of knowledge management promotes the use of research in making decisions that benefits individuals (Booth 2003), and its tenets informed the research design, as it "promotes the integration of user reported, practitioner-observed and researchderived evidence as an explicit basis for decision making" (Booth 2006, p.65). EBLIP's initial stage is the conversion of information needs from practice into focused, structured questions (Booth 2003), which in this case were formulated as follows: • What are the information literacy competencies of incoming research students?
• Are there different requirements for different disciplines?
• Are requirements predicated by student profile?
• Do current information literacy programmes meet their requirements?
• Is the delivery of the current programme appropriate to its need?
• How can the current research inform future development?
The constraints imposed by thesis requirements, necessitated the truncation of the EBLIP process, and the implementation-related tasks, i.e. application of the results and evaluation fell outside the scope of this study.

Information literacy assessment
Although the lack of research focusing on information literacy for the graduate student has been noted, two information literacy assessment tools compatible with this study were identified. The first was a checklist developed by Loughborough University for use as an informal method of self reflection before attendance at face to face sessions (Stubbings and Franklin 2005). The second was the diagnostic questionnaire of the London Metropolitan University's Applied Information Research (AIR) programme (Andretta 2005), which, although designed to establish information literacy competencies at the point of entry to a taught master's programme, was nevertheless considered pertinent. The conjunction of these tools, one self assessment, the other a diagnostic test formed the basis of a survey questionnaire, which was launched at the beginning of the academic year, and which provided quantitative empirical evidence in this study. Qualitative methods were also used. Information behaviour was observed in information literacy workshops organised for research students in the second half of the first semester, and a focus group in the latter half of the second semester was used to gather user reported evidence.

Data collection -survey questionnaire
While it has been suggested that surveys are of limited value in identifying training needs (Research Information Network 2008), the use of a survey method in this research is defended on the basis that it was a component of a triangulation of data collection methods, and that the use of a web format ensured its efficiency as a means of data collection from a wide audience. The questionnaire design incorporated five components: • Personal profile -school/discipline: degree programme: educational history: age: gender • Library induction activities • Information literacy self assessment.
• Diagnostic tool to determine expertise in specified areas of information literacy.
The sample was purposive, targeting UCD's entire research postgraduate intake in autumn 2007. It is acknowledged that the non probability nature of this sampling results in data which cannot be extrapolated to other populations. Thus, the empirical data cannot be ascribed to the research postgraduate population at large. Nevertheless this study provided valuable insight into the information literacy and needs of a group of postgraduate students at the outset of their research, which possible future studies may support or refute.
The Bristol Online Survey (BOS) was used to administer the survey. Given the exploratory nature of the research, a descriptive analysis of the data was felt to be the most appropriate. As the data analysis function of BOS is underdeveloped, the data was exported to Microsoft Excel to facilitate manipulation into tabular and graph formats for clarity in the analysis and interpretation.

Data collection -information behaviour observation field study
The importance of pitching information literacy provision at the right level of complexity for its audience has long been accepted. The inappropriateness of using information literacy instruction methods based on undergraduate behaviours to meet the needs of researchers has been noted (Booth 2006). The recursive nature of research must also be considered. Aiming to establish the concerns of research students Bruce (2001) studied literature reviews, a genre of academic writing common to almost all disciplines, and recommended that as a basis for information literacy education students should be encouraged to adopt a psychological rather than topical view of relevance, and a subjective rather than objective view of information. This advocacy links to research in the area of information seeking behaviour. Kuhlthau's Information Search Process (ISP) was considered the theoretical framework most relevant to this study, as the genesis of her work was in the observation of a common pattern in students' behaviour in the initial stages of research (Kuhlthau 1993). Her premise is that while the objective of library and information services and systems is to increase access to resources and information, levels of access vary and range from the operational to the intellectual, and it is the latter which addresses interpretation of information and ideas within sources (Kuhlthau 2005). In its identification of a "zone of intervention", when a user could benefit from guidance and assistance, Kuhlthau links library and information services to the information search process. Thus, this model was selected to underpin an information behaviour observation field study, which satisfied the practitioner observation component of the EBLIP framework. For this study, non participant observation in the classroom was the chosen methodology, its primary advantage being that the subjects studied are oblivious to the research, and that information behaviour is not altered in any way. Simple note taking was the data logging mechanism. This field work concentrated on the following three variables: • What are the information literacy competencies of incoming research students?
• Are there different requirements for different disciplines? • Are requirements predicated by student profile (e.g. international students, mature students, research Masters, or PhD entry level?)

Data collection -focus group
The third strand of the EBLIP framework is user reported evidence, and a Focus Group was used to realise this. Its purpose was to elicit research students' response to the information literacy programme offered in that academic year, and to gauge to what extent it met the needs of the targeted audience.
All researchers who had attended any of the workshop sessions throughout the year were invited to attend via the library's researchers' e-mailing list. Flipcharts were used to document the emerging findings and a colleague helped with its facilitation. There were six participants.

Ethics
The research satisfied UCD's criteria for exemption from ethical approval (University College Dublin Human Research Ethics Committee 2008), as the survey tool was anonymous, its data stored electronically, and password protected.

Data analysis 4.1 Survey response
The survey questionnaire elicited 153 responses, one of which was incomplete and was discounted. This was an overall response rate of 22%, with a rate of 23% from the PhD cohort, and 13% from the Research Masters' students. The low number of participants in some categories, especially the 50-59 and 60+ age groups, results in data which may not be statistically significant.

Personal profile
Of the participants, 78% were enrolled as PhD students; 14% as Research Masters, and the remaining category, "Other" as predominantly Taught Masters students. The majority of participants (88%) were full time, with 93% of those registered for PhD studies full time; whereas 23% of the Research Masters' cohort was registered as part-time students. The survey population comprised 84 females (55%) and 68 males (45%). The age profile is shown in The non Irish component of the population was 26%, and English was not the first language for 22%. Of the participants, 72% were Irish graduates, with 52% UCD graduates. Previous postgraduate study had been undertaken by 57%, 65% of whom had done so in Ireland, and 43% in UCD. Of those who had already completed a postgraduate programme, 73% had done so within the previous five years (2003)(2004)(2005)(2006)(2007). The rate of response from each of the Colleges is illustrated below in Table 2. With the exception of the School of Sociology, each of the thirty five schools was represented in the sample, with the greatest number of respondents (18) from the School of Agriculture, Food Science and Veterinary Medicine.

Figure 2: Participants by College
Not unexpectedly, there were high rates of computer access off campus, with 85% having access to a computer at home, and 88% owning laptops. Off campus Internet access was also quite high at 85%, with 45% having WIFI off campus.

Library induction
Although the library induction programme at the start of Semester 1 is available to all students new to UCD, its focus is primarily on the undergraduate students. Prior to their current postgraduate study, 62% of the respondents who had received library instruction previously, 38% had done so within UCD.
Presentations by librarians and physical library tours were the most common means, although some, (16%) had taken part in interactive workshops. While library instruction had been integrated and timetabled into courses for 17%, it was credit bearing for only 7%. Online tutorials had been used by 10%.

Self-assessment checklist
Spread over five questions, each with a number of sub sections, the self assessment comprised forty two questions in total, ranging from quite basic functions such as the use of the library catalogue, to the more complex such as tracing completed and ongoing research. Information retrieval, selection, management and ethical use were all included. The self assessment criterion was the user's perception of his/her own abilities on a scale from very confident to no familiarity. In the data analysis, the "very confident" and "confident" responses were merged, as each demonstrated perceived competency, "fairly" confident suggested some measure of doubt, "not confident" and "no familiarity" are self explanatory. The data for these last three categories was analysed separately.

Self-assessment findings
The findings are summarised and discussed below under the following headings: • Use of library sources • Searching strategies • Use of Internet sources • Ethics

Use of library sources
Not unexpectedly, high confidence levels were registered in the use of the library catalogue to find books, and also in its use to reserve and renew material. However, there was a diminution in confidence in its use to find journals (both print and electronic), with diffidence spread across the disciplines. While broad overall confidence was registered in locating journal articles, almost a quarter (24%) of the Research Masters students lacked confidence in this area. Of those who had previously undertaken postgraduate study, 12% expressed a lack of confidence. The selection of appropriate databases elicited lower confidence rates than for the location of journal articles, with 24% lacking confidence or familiarity. There was a notable lack of confidence in the use of citation indexes to track articles, which was spread across thirty one of the thirty four schools represented in the study.

Figure 3: Use of Citation Indexes
Some lack of confidence was also registered in finding reference materials, with 14% of the PhD and 29% of the Research Masters groups falling in this category. Lower confidence levels were expressed in the sourcing of report literature, with 40% of the PhD group and 57% of Research Masters lacking confidence. However, discipline was a factor in this finding, as 75% of those who expressed confidence in this area were in the science, medicine and technology group of subjects. Students with previous postgraduate experience registered higher confidence levels in the sourcing of government and official publications than those who had not. Locating theses as a component of ongoing and completed research registered a lack of confidence, particularly in the Research Masters group (Figure 2), while unsurprisingly, those who had previous postgraduate experience expressed greater confidence.

Figure 4: Finding Theses
A difference in confidence levels between the groups was also manifested in finding conference papers, with 56% of PhD students either confident, or "fairly confident", compared to 29% of Research Masters.
These findings suggest that there is a lack of confidence in the use of many library sources at the outset of research study. They also highlight the necessity for the inclusion of an introduction to information sources as a component of information literacy provision for the research student.
Fundamental to the use of library sources is an understanding of the information environment, and information literacy sessions should make the connections between the research cycle and the information environment, so that the students are equipped to negotiate and exploit these sources.

Searching strategies
Successful information retrieval is essential in the identification of completed and ongoing research. The findings of this study highlight areas where information literacy provision should be implemented.
The use of multiple keywords to focus a search resulted in high confidence levels overall, with 15% reporting lack of confidence, and only 3.3% indicating no familiarity. Confidence rates in saving/exporting and e-mailing references was 48% overall, with a further 26% of students being fairly confident. Browsing subject-based lists of e-journals also yielded high levels of confidence.
However, the use of controlled vocabulary elicited much lower overall confidence levels (at 30%), with 48% expressing lack of confidence or no familiarity, which was seen across all variables. Similarly the use of truncation and wildcards as a component of strategic searching indicated low levels of confidence, with 66% of the PhD group and 81% of Research Masters reporting lack of confidence. Again, this lack of confidence was evident across all variables. It must be acknowledged, however, that the use of library terms, such as 'controlled vocabulary', 'truncation' and 'wildcards' may have been a contributing factor to these results. This illustrates the importance for those responsible for information literacy education to explain terminology as appropriate, but most importantly to avoid the use of jargon.

Use of Internet sources
Not unsurprisingly, the overall rates of confidence for searching the Internet were high. High levels of confidence in the use of search engines were also expressed, although the use of advanced search options displayed slightly less confidence, with 12% of the research student body lacking confidence in the use of, or familiarity with this facility. Confidence levels were not as high for searching subject gateways, with 49% expressing no familiarity, and 20% lacking confidence in their use. Although confidence levels were quite high for the evaluation of websites, 23% of the study's population either lacked confidence, or were unfamiliar with this practice. However, in citing websites in academic work, 52% of the Research Masters either lacked confidence or had no experience, compared to 28% of the PhD group. The lack of confidence or familiarity in the under 25 age group in the use of discussion lists (80%) and setting up alerts (62%) was surprising, as both are components of social software, and are also included in virtual learning environments, with which this age group would be familiar.
These findings suggest that while the majority of students feel confident in using the Internet, its value for academic purposes could be further enhanced through information literacy sessions covering advanced search strategies, evaluation processes, citation requirements and social networking tools.

Ethical use of information
In the self assessment of the ethical use of information, levels of confidence were high in quoting an author in academic work, but were less assured in paraphrasing, and of the Research Masters group, 24% either lacked confidence in, or were unfamiliar with this academic skill. This group was also less confident than its PhD counterpart in understanding and avoiding plagiarism, although it is interesting that a small percentage of PhD students (7%), who had previously completed postgraduate study also lacked confidence in avoiding plagiarism. High levels of confidence were registered for citing a book correctly, and also for citing journal articles, but lower confidence was found in citing a web page. The creation of a bibliography or list of references elicited high levels of confidence, although these findings also show that not all are using reference management tools, such as EndNote for this purpose. These findings make clear that training in the use of a reference management tool such as EndNote needs to be include in the provision of information literacy, as there was a lack of confidence in its use throughout the entire survey population.

Diagnostic questionnaire
This section comprised a number of multiple choice questions covering searching skills, evaluation skills and referencing skills. Although a high level of confidence was expressed in searching the Internet in the self assessment section, this diagnostic questionnaire revealed that there was some lack of knowledge in how the Internet worked, with 35% not knowing the function of stop words, 40% unsure of how phrase searching operates, and 31% not understanding the Boolean use of "OR".
High correct scores were achieved in searching the catalogue, although low level of knowledge of local catalogue services was anticipated as the students had not been exposed to local induction so early in the academic session. Knowledge of interlibrary loan services was lacking, but this service was also unlikely to be required quite so early in the academic cycle. There was also a lack of awareness on the presence and value of subject portals, while high expectations of access to electronic journals were recorded, as 47% believed they could download all electronic journal articles off campus. The evaluation skills series of questions yielded high correct scores, and plagiarism was well understood.
In referencing skills, however, the citation for a chapter of a book presented some difficulty, as 47% failed to identify the correct author to check the catalogue, and 34% were unable to differentiate a journal citation from that of a monograph.
In the final section of the survey questionnaire the participants' preferred method of information literacy provision was sought. Six expressed options were proffered, with the additional option of "other" for alternatives not listed. Figure 5 illustrates the preferred options:

Figure 5: Preferred Method of IL Provision
Half the participants indicated their preference for more web based information. Interactive workshops covering specific resources and skills (which was the current format) were selected by 52% of participants and 31% wanted information literacy to be integrated in course work. The need to collaborate with the schools was cogently argued in the open question, as the following quotation illustrates: It seems that the library provides lots of presentations but they aren't explained enough as to what they're for and of what benefit they will be. There should be specific research presentations for each school, based on the needs established by the school and then they should be advertised/ promoted by the school.

Information behaviour -field studies
Observation of information behaviour was undertaken in a series of credit bearing induction programme information literacy workshops organised by the Library for research students in the Colleges of Arts and Celtic Studies and Engineering, Mathematical and Physical Sciences. Library staff membership afforded legitimate attendance from the participants' view, which facilitated unobtrusive observation.
This micro study provided evidence that on entry to postgraduate research study, within a formal academic information seeking environment, the concept of building a search strategy progressively was not well developed. While many expressed familiarity with specific databases, it was obvious from observation that most had difficulty constructing search strategies. Questions asked throughout the sessions indicated that a step by step approach to path-finding was essential. The majority displayed literal rather than lateral thought, and showed no expertise in the use of synonyms or of breaking a research topic into concepts. These areas of difficulty are summarized as follows: • Identifying and conceptualising search terms -almost all were keying a whole sentence into search boxes. • Use of synonyms.
These shortcomings were observed across all disciplines and across varying demographic profiles. Workshops where the mix of disciplines was quite broad presented difficulties, as students did not always recognise or identify common elements across different sources. Unfamiliarity with library terms, such as 'portals', 'gateways', 'metadata', and 'search engines' was also evident. The need for assistance in establishing the criteria for database selection was also observed.

Focus group
While participants were complimentary of the information literacy programme offered in the 2007/2008 academic session, they were open and frank in discussion and offered constructive criticism.
The apparent assumption on the part of library staff that intellectual ability equated with either IT ability or information literacy was questioned. It was emphasized that the postgraduate research cohort was not a homogeneous group and this created problems in the workshops. For example, participants lacking confidence or with lower levels of expertise valued the prescriptive exercises within workshops, whereas those students at a more advanced level preferred to use their own examples. The reluctance to ask questions in open sessions was also raised, and it was pointed out that research students even within colleges may not know each other well, and might not feel comfortable exposing their lack of knowledge in front of their peers.
In discussing relevance, it was suggested that a problem based approach centred on the student's own research would be really useful. It was also suggested that workshop periods be extended from 90 minutes to two hours to help the participants achieve the workshop's learning outcomes by contextualizing these in their own research, the tendency to pack too much information into a session was considered counter-productive, and it was suggested that less coverage would result in greater confidence, which could be expanded through personal research activity or by delivering more advanced sessions.
A discipline specific approach was favoured over a generic one, and it was felt that it would be useful if the individual Schools' Research modules could promote the complementary library courses on offer.
The confusing nature of the electronic environment was stressed, and it was suggested that library staff underestimate this complexity for library users, with the result that students identify one or two information sources early in research and become reliant on these.

Recommendations for practice
This research has shown that information literacy providers should consider the following strategies in order to successfully engage with graduate research students: • Engage collaboratively with all training providers.
• Ensure all disciplinary variation is well understood, i.e. in the research process, the research output, and in information behaviour. • Adopt a theory of adult learning, acknowledging a priori knowledge and experience, but do not assume knowledge or skill. • Use the literature review process as a framework for information literacy programmes, ensuring ownership of the programme becomes the student's. • Where possible, allow the students to use their own research for interactive work -"real tasks with real consequences". • Include the concepts of the ISP model in the programme.
• Take into account gender differences.
• Use a blend of workshops and 1:1

Conclusions
The triangulation of methods used in this research yielded a considerable amount of data which linked research students' personal profiles to a baseline information literacy appraisal, achieved largely through the conjunction of self assessment and diagnostic testing. The data analysis generated clear indicators of areas that information literacy providers must address, such as tackling the widespread lack of confidence in the use of advanced search techniques and in particular variation determined by gender and age, encouraging lateral rather than literal thinking and the formulation of more sophisticated search strategies. While we acknowledge that students' perception of their information literacy does not equate with competence, the confidence factor is nevertheless important, as it is suggested that low self efficacy may be a significant factor (Kurbanoglu 2006), which may account for the over reliance on familiar information sources identified early in the research process, as found in this case study.
The need to decouple ICT expertise from information literacy competences was also expressed in this study's findings. Although ICT proficiency does not imply information literacy, it is reasonable to suggest that it is now impossible to attain any standard of information literacy without some level of ICT skills. This is confirmed by the study where low levels of ICT skill hindered the development of information literacy.
The relationship between the library's information literacy programme and the individual schools' research skills modules was raised in this study, suggesting a need for a greater committed collaboration between library and faculty, which too was recommended in the recent RIN report on graduate skills (2008).
Perhaps the most significant finding of this exploratory study was the realization that successful information literacy support for research relies not just on the identification of researchers' information literacy deficits, but on the mutual understanding by researcher and educator of the research process. Fundamental to this is the need on the part of LIS professionals to dispel the assumption that because students have reached PhD level they are competent in information seeking. This raises the issue of fundamental differences between information literacy for taught programmes and for research. As this study has shown PhD programmes are associated with demographic diversity which reflects a wide variation in terms of information literacy and competences. In line with Bent et al's work (2007) this study recognizes the need to establish the students' a priori knowledge and experience, the relevance of the information literacy programme to their work, and the importance of facilitation in the development and delivery of information literacy provision for the research student. The utilization of the literature review as a framework for the development of information literacy provision should be also be considered as Bruce (2001) has argued that the process of reviewing the literature helps the students to start formulating the research questions and enables them to articulate or refine search strategies that suit the context of their research. In addition Kracker (2002) has suggested that the incorporation of Kuhlthau's ISP model into information literacy provision for research students would help the student understand that uncertainty lies within the process itself and not within the individual, thus alleviating feelings of inadequacy, incompetence and embarrassment, which were articulated in this study's Focus Group, and were manifested in the reluctance to ask questions publicly in workshops.
Gender was identified as a significant variable in this case study, with males consistently more confident than females. Cognizance of this factor should be incorporated into information literacy provision.
The findings of this study suggest that online information and tutorials are valued by the students who perceive them as complementing the face to face sessions.