Development and validation of an information literacy self-efficacy scale for medical students

The aim of this research is to develop and validate a scale for the evaluation of medical students’ information literacy self-efficacy beliefs, as this plays a crucial role in the development of lifelong learning objectives. Curriculum developers and medical educators need to have a good understanding of information literacy in order to decide when specific support and training should be integrated in the curricula. The use of a trustworthy, user-friendly tool in a large population able to detect different aspects of students’ information literacy self-efficacy beliefs could help to evaluate an entire curriculum. A 5-factor model was developed and validated within a 6-year medical curriculum (n=1252). Internal consistency of the subscales was high (α: 0.845-0.930). In conclusion, the Information Literacy Self-Efficacy Scale for Medicine (ILSES-M) could be an added value for evaluating medical students’ information literacy self-efficacy beliefs. Furthermore, it could form the basis for curriculum development as well as a guideline for critical curriculum reflection.


Contextual background 1.1 Information literacy self-efficacy
Information literacy (IL) is a well-studied topic and has been defined (ACRL, 1989;ACRL, 2015) and studied in different ways.Society is expected to produce confident, independent and selfregulated learners ready to deal with the continuous digitisation of information and to use new technologies.Furthermore, students need to handle the overwhelming and continuous flow of new scientific information.
Information literacy training is a core component of the academic curriculum and is believed to enhance student learning (Kilic-Cakmak, 2010;Virkus, 2003).Within the IL research of medical curricula, it is suggested that IL training should be incorporated early in the curriculum and should be developed throughout the complete curriculum, as only then students can develop and retain the necessary skills (Dale & Campbell, 2012;De Meulemeester, De Maeseneer, De Maeyer & Peleman, forthcoming;Kinsley, Galbraith, Herring, Stowers, Stewart & Kingsley, 2011).
Together with IL, self-efficacy is one of the core components in the process of training lifelong learners (Candy, Crebert & O'Leary, 1994;Kurbanoglu, 2003).Self-efficacy (SE) has been defined by Bandura (1986, p. 33) as 'people's judgments of their capabilities to organize and execute courses of action required to attain designated types of performances'.The level of SE will determine how resilient and persistent a person will be, which plays an influencing role on the way people will attain the necessary skills (Bandura, 1977;Pajares, 1996).It is also important that the individual participates as this can have a positive impact on the level of SE and the assessment scores (Burgoon, Meece & Granger, 2012).Kurbanoglu (2003) introduced the concept of 'perceived-self-efficacy for information literacy' and a new research topic within the broader domain of IL was launched.Kurbanoglu and her colleagues (Kurbanoglu, Akkoyunlu & Umay, 2006) were convinced that 'attaining of high sense of self-efficacy beliefs is as important as possessing information literacy skills'.Students need to be confident in using their IL skills and be willing to develop and apply the skills.Students with a lower SE will avoid active learning and are therefore less inclined to develop their IL skills and thus become lifelong learners.On the other hand, students with a higher SE will have many more goals and will be motivated to finish their tasks (Kurbanoglu, 2003;Kurbanoglu, 2010).
Within an academic context, information literacy self-efficacy (ILSE) has been associated with study success (Bayram & Comek, 2009), with SE beliefs playing an important role in levels of student motivation (Pinto & Sales, 2010).According to Ross, Perkins and Bodey (2016), students' ILSE seem to be related to a desire to learn new things for their own pleasure and satisfaction.Students showing higher levels of SE will be proactive and will be more willing to try new behaviour.These are key motives for enhancing ILSE skills and thus, for the development of lifelong learning.Within a problem-based environment such as a medical curriculum, these skills are indispensable.Medical students are required to be self-regulated learners: coaching students can help them to develop SE, motivation and performance (Carr & Johnson, 2013;Langendyk, 2006).
To integrate the variety of IL in a proper and qualitative way, it is therefore important that curriculum developers or medical teachers have some understanding of students' ILSE, trying to evaluate and take into account how students feel, and analysing the possible impact this may have on their self-confidence in learning.

Why use a scale?
DeVellis (2016, p. 11) defined scales and their purpose thus: Measurement instruments that are collections of items combined into a composite score and intended to reveal levels of theoretical variables not readily observable by direct means are often referred to as a scale.We develop scales when we want to measure phenomena that we believe to exist because of our theoretical understanding of the world but that we cannot assess directly.
For the evaluation of the development of ILSE within medical students, a stable survey method for longitudinal research was needed.Research reported that SE needs to be measured directly (rather than indirectly) by the use of a self-reporting scale (Cassidy & Eachus, 2003) as it reports a degree of certainty a person has towards a task.To survey the ILSE of medical students, a validated scale should therefore be developed.The start of the development of a scale is based on a process of defining which 'concepts' one would like to analyse.
In this study, questions -which are further called 'items' or 'indicators'related to medical student's ILSE development were asked.
Looking for a coherence of the survey responses (in relation to the items) and checking whether all theoretical concepts are part of only one scale (one dimension) or rather different scales ('subscales' or different dimensions) is done by performing a factor analysis.The 'factor' is thus a set of answers (items) having similar response patterns (correlation between the items) and refer to a certain concept.The correlation between the items influences how the items within the factors contribute.

Information Literacy Self-Efficacy Scale
The most common scale to assess ILSE is the Information Literacy Self-Efficacy Scale (ILSES), designed and validated by Kurbanoglu, Akkoyunlu and Umay (2006).For this research, we focused on the validated scale consisting of 7 factors with 28 items.The original factors are 'labelled' as follows: A= 'Defining the need for information', B= 'Initiating the search strategy', C= 'Locating and accessing the resources', D= 'Assessing and comprehending the information', E= 'Interpreting, synthesizing, and using the information', F= 'Communicating the information' and G= 'Evaluating the product and process'.
Previous literature research (De Meulemeester, Buysse & Peleman, 2018) on ILSE within a medical curriculum revealed the need for a survey evaluating these beliefs within the context of a medical curriculum, in a large-scale population and for a longer period.Specifically within a medical context, to the best of our knowledge, no validated instrument exists for the measurement of medical-specific information literacy self-efficacy.
The specific context in which the questions are asked, however, is considered extremely important (Bandura, 1986;Bong & Skaalvik, 2003;Cassidy & Eachus, 2003;Lin, Tan & Tsai, 2013;Marsh, Walker & Debus, 1991;Pajares, 1997).For the purpose of this research, with the approval of the original author, Professor Serap Kurbanoglu, an adapted version of ILSES has been designed.In addition to the inclusion of all original questions from the 28-item scale, ten specific medical IL items (Table 1) have been added to evaluate medical-oriented ILSE within students (De Meulemeester et al., 2018).The adapted version of the Information Literacy Self-Efficacy Scale within a medical curriculum is abbreviated to ILSES-M.Other researchers who used and validated the original ILSES 28-item scale obtained different (Batarelo Kokić & Novosel, 2014) or similar subscales and/or item selections.Usluel (2007) studied the ILSE of student teachers in Turkish academic departments for teacher-training in primary education.The component analysis resulted in a 20-item scale with 4 subscales: A) 'self-efficacy in the analysis and evaluation of information', B) 'self-efficacy in using ICT to access information', C) 'self-efficacy in citing information resources' and D) 'self-efficacy in using the library'.The second subscale is more oriented towards ICT skills and will therefore not be comparable with other subscales from the ILSES-M.
In the study by Batarelo Kokić and Novosel (2014), a Croatian version of the original ILSES has been validated, but with regard to the level of complexity of IL skills, differences in placements of items were noted in comparison to the original Turkish ILSES.According to Batarelo Kokić and Novosel (2014), this could be explained by the differences in the outcomes of the IL courses.
Other researchers have used other methods to evaluate the level of SE perception.Özbıçakçı et al. (2015) rated the level of difficulty by calculating the total score of the Likert ratings, with a minimum score of 28 and a maximum score of 196 points (28-item scale).The researchers interpreted a low IL perception as a score between 1 and 65 points, moderate with a score of 66 to 130 points and a high perception with a score of 131 to 196 points.Within this research, an overall sum score for the obtained level of ILSE is not found to be applicable.As subscales have different numbers of items, it was decided to work with mean scores per subscale.This enabled us to compare scores between subscales and to set a cut-off score.However, setting up a cut-off score is not easy.The different skills students need to become information literate people change throughout their academic career, and are influenced by the rapid pace of technological innovation.

Aim of the study
The aim of this study is to evaluate whether the ILSES-M can be validated as a valuable scale, so that it can be used for further research on medical students' ILSE within a complete curriculum.The research questions underpinning our study are:  Can the original ILSES items still be used in another context? Will the more specific additional medical questions be an added value and an even fit in a separate subscale? Will the different items of the original ILSES be incorporated in the same subscales as in the original study or other research?
This study focuses therefore on the validation of the adapted version of the ILSES for medical students.

Participants and study design
This research has been set up in an academic environment within a cohort of medical students.This context is a problem-based learning environment where students are required to be engaged in self-regulated learning (Carr & Johnson, 2013;Langendyk, 2006) and where IL as such is an important element of evidence-based education.Development and validation of the ILSES-M is part of a larger longitudinal study (2011)(2012)(2013)(2014)(2015)(2016) in the Faculty of Medicine and Health Sciences at Ghent University (Belgium).Ethical approval was obtained by the Ethical Committee of Ghent University Hospital (protocol number PA 2011/017).All participating medical students signed the university's Informed Consent form.
Assessment of the ILSES-M was conducted in English.Students of an entire medical curriculum (Year 1 -Year 6), in the academic year 2013-2014, completed the 38-item ILSES-M on a scale from 0 ('I do not feel confident at all') to 100 ('I feel 100% confident').There was no time limit for answering the questionnaire.Students from Year 1 to Year 5 filled in the questionnaire at the start of the academic year (first two weeks); students of Year 6 completed the questionnaire at the end of the academic year, after their final clinical examination.

Statistical analysis
All analyses were performed with SPSS v. 24 (IBM Corp. Released 2013).Descriptive statistics were used to establish frequencies, mean and standard deviation (SD).

Development of the research instrument
Data from the original scale were not available any longer and in the current study, ten new items were added to this scale.This meant that it was not possible to look for a confirmation of this original scale using confirmatory factor analysisi.e.whether we were able to find the same components in our population.However, it was necessary to explore which items (questions) fit togetherthat is, form a subscaleand thus study a certain aspect of information literacy (example questions related to 'Bibliography').The technique chosen to look for items fitting together was an exploratory factor analysis (EFA), more specifically Principal Axis Factoring analysis (PAF), being the most widely used EFA method.Data from 1285 students (97.4%) was available to perform the PAF.
Factorability of the ILSES-M was examined with different kinds of measures.In a first phase, it was important to look at the descriptive statistics.Hence, should every student give the same response, it would not be possible to perform further analysis as there would be no variability.In our study, there was variability: in other words, characteristics that can take different values in different situations.
In the next step, the coherence between these variables has to be explored by means of 'correlations'.There should be some correlation between items; if all correlations are lower than 0.30, almost no coherence between items exist, so performing factor analysis would have no sense.In our study, coherence between items existed.
Next, one could look at the partial correlations for common parts between items.Partial correlation should be rather low, which means they have to have a great part in common.This was the case in our study.The Bartlett test of sphericity looks at whether the correlation matrix is not an identity matrix: i.e. a matrix in which all variables do not correlate with each other.In our study this is not the case (2(703)=33426.71;p<0.001).
The Kaiser-Meyer-Olkin measure (KMO) compares the sum of the partial correlations with the sum of the correlates and tests what we have observed manually before.The KMO measure lies between 0 and 1; a value closer to 1 means that there are indeed some factors (subscales) present.In our study there was a highly reliable KMO measure of sampling adequacy (0.953) which means we had an indication to perform factor analysis.
However, the KMO only gives an indication, therefore it is also important to examine at the individual item level whether all items had some 'load' (contribution) on a factor.The Measure of Sampling Adequacy (MSA) forms the base of the KMO and shows how high one item correlates with other items in the correlation matrix.An absolute lower limit is 0.50 which was the case for all items included in our analysis (Mortelmans & Dehertogh, 2008).
Because of high reliable results of the discussed analyses, it was decided to include all items in the further factor analysis.

Determination of the subscales
Extraction of factors is based on the Kaiser-Gutmann rule.This states that components with eigenvalues greater than 1 (the amount of explained variance between variable and component) should be retained and also on the scree-plot (Floyd & Widaman, 1995).According to Costello and Osborne (2005, p. 3), 'the scree test involves examining the graph of the eigenvalues … and looking for the natural bend or break point in the data where the curve flattens out.The number of datapoints above the "break" (i.e.not including the point at which the break occurs) is usually the number of factors to retain'.
In our study, five factors were extracted.In the screeplot, a last datapoint was also seen at six factors, meaning five factors could be extracted.Furthermore, it was chosen to rotate the factors, as a non-rotated solution is based on posing maximal variance on the first and following factors in which they have to be forced orthogonal.
Rotation of the model makes it easier to interpret.One technique to rotate is the Varimaxrotation.To have good factors (subscales), it is important that all items being part of that factor should only contribute (load) to that factor: i.e. have a load of >0.40.It is therefore important to look for items loading on more than one factor or items not loading high enough.Those items should be stepwise removed from the factor analysis (Mortelmans & Dehertogh, 2008).
The Principal Axis Factoring with Varimax rotation yielded 35 items and 5 factors accounting for 58.34% of the total variance.
Descriptive statistics and the items are presented in Table 2. Three items did not meet the criteria required to load (>0.40) on one of the subscales, which means they did not make a significant contribution to a subscale: two original-scale questions 'Limit search strategies by subject, language and date', 'Write a research paper', and one of the ILSES-M unique questions 'Find cited references' (Table 2).(Bland & Altman, 2002).Internal consistency of the ILSES-M subscales was examined and indicated high reliability with Cronbach's alpha above 0.70 in all subscales (Subscale 1: 0.930; subscale 2: 0.911; subscale 3: 0.858; subscale 4: 0.872 and subscale 5: 0.845).Eliminating certain items did not increase Cronbach's alpha, showing that our subscales had good internal consistency.

Labelling of the factors
In order to use and analyse the scale for further research, every factor (subscale) was labelled according to theoretical concepts they cover: subscale 1 (11 items) was labelled as 'Evaluating and Processing Information' (Table 3); subscale 2 (10 items) 'Medical Information Literacy Skills' (Table 4); subscale 3 (6 items) 'Searching and Finding Information' (Table 5); subscale 4 (4 items) 'Using the Library' (Table 6); and subscale 5 (4 items) 'Bibliography' (Table 7).The five subscales cover a mix of general as well as more specific skills a medical student needs in order to become a professional who is literate in medical information.It is thought that because of this method of 'general' labelling, all 5 subscales can withstand changes in time.Compared to the original scale, only two items, 'Limit search strategies by subject, language and date' and 'Write a research paper', could not be sustained, which suggests that the original scale is already time-independent.It was not surprising that the first item did not stand the test of time, since medical students are not taught to limit their search strategy on these specific aspects.Teachers even recommend not using language and time limits.The item on 'Writing a research paper' was possibly rejected as students may regard this as a more professional skill linked to the work of a researcher.
Other researchers use a different way of labelling and focus more on the complexity of skills by labelling the subscales as 'Basic information literacy skills', 'Intermediate information literacy skills' and 'Advanced information literacy skills' (Batarelo Kokić & Novosel, 2014; Kurbanoglu et al., 2006).This type of labelling was not eligible for the ILSES-M, since learning is a continuous process.Students starting their university career can feel self-confident about a skill.Once they understand the complexity of IL skillsparticularly those that are domain-specificthey feel less confident.This could obviously have an impact on the way students perceive their ability to obtain further ILSE.Self-efficacy goes hand in hand with the process of learning, and the active use of skills.At different moments during the learning process, students could experience a decline in their skills.
Within the presentation of the 5 subscales we will compare with the items of the original scale and with the subscales/items reported in the research of Usluel (2007), as similar item and/or subscale selections were found.This shows that the items related to the more general IL skills load in the same way even if the scale is used in a different setting.

Subscale 1: 'Evaluating and Processing Information'
The first subscale (Table 3) includes 11 items, all from the original scale (ILSES) and consists of generally important aspects in the practice of evaluating and processing retrieved information.ILSES: Information Literacy Self-Efficacy Scale; ILSES-M: Information Literacy Self-Efficacy Scale for Medicine, Usluel (2007): Subscale 'Self-efficacy in analysing and evaluating information'.

Subscale 2: 'Medical Information Literacy Skills'
The second subscale, as described in Table 4, covers 10 items, one item of the original ILSES and 9 new added items.This original item, 'Initiate search strategies by using keywords and Boolean logic', fits very nicely under this new label, as medical education instructors focus very heavily on search strategies with the use of keywords, which is not limited to the use of Medical Subject Headings (MeSH) alone.The new questions seem to be an added value for evaluating medical students' information literacy self-efficacy.Over the course of the study years, an increase in medical information literacy skills could be detected (Figure 1).These positive results might be the result of specific efforts made by the curriculum committee to improve this skill.From the first year, medical students at Ghent University have to attend a specific medical information literacy course.These skills have to be practised and are integrated in other courses in all subsequent years.

Subscale 3: 'Searching and Finding Information'
The third subscale (Table 5) consists of 6 original scale items, identifying more general skills in searching and finding the right information.

Subscale 4: 'Using the Library'
The fourth subscale (Table 6), is focused on the use of the library and loaded only the 4 specific original items in relation to the use of the physical library.In medical sciences, however, a huge digital evolution is occurring.Many medical libraries no longer have physical collections: accessing a digital collection of scientific literature requires skills that differ from the library skills that were defined when the original scale was developed.Results in this study show that medical students do not feel very confident in 'Using the Library'.First year medical students have the highest scorescompared to medical students from other study yearson this subscale.Those students completed the questionnaire immediately after starting at university, which might explain the high scores.During their secondary education, they had become familiar with working in (non-digitised) general libraries, but had not yet experienced what it means to search for digital medical literature.ILSES: Information Literacy Self-Efficacy Scale; ILSES-M: Information Literacy Self-Efficacy Scale for Medicine, Usluel (2007) -Subscale 'Self-efficacy in using library'.Usluel's (2007) subscale 'self-efficacy in using library' is identical to the ILSES-M subscales 'Using the Library'.The same items were selected and internal consistency is similar (Table 6).

Subscale 5: 'Bibliography'
The last subscale (Table 7), consists of 4 items, also based on items from the original ILSES and focused on how to create and organise a bibliography.One item, 'Reference the sources I use in a reference style used in medicine', loaded logically on the more medical subscale ('Medical Information Literacy Skills') rather than on subscale 'Bibliography'.Results show that students score higher later in the curriculum, i.e. they feel more confident as their curriculum proceeds.One possible explanation is that students only become self-assured when they master these skills; for 'Bibliography' this is often only the case when they definitely need them to write their Master's thesis.ILSES: Information Literacy Self-Efficacy Scale; ILSES-M: Information Literacy Self-Efficacy Scale for Medicine, Usluel (2007) -Subscale 'Self-efficacy in citing information resources'.
In the same line as for subscale 4, the subscale of Usluel ( 2007) 'self-efficacy in citing information resources' is identical to the ILSES-M subscale 'Bibliography'.The same items were selected and internal consistency is similar (Table 7).
A possible explanation for the similar results with the research of Usluel (2007) independent of the different cohorts of userscould be that the items within these subscales are less contextrelated.Additionally, both studies took place in different years (before 2007 and 2013), which might add to the time-independency of the scale.Several factors might explain the observed differences: extension of the original scale, another setting and type of cohort or even students influenced by a technologically changing world.

Evaluation of students' ILSE development
Finally, to evaluate the evolution of the ILSE for every student, composite scores based on the mean were calculated for the overall scale as well as for each subscale for every study year (Table 8).The test used to look for these differences was the parametric One-way-ANOVA with Cronbach's alpha set at 0.05.
Overall mean ILSES-M scores are significant (p<0.001).Information literacy self-efficacy beliefs evolve over time.Mean scores range from a minimum of 28.4 in the first Bachelor year for the 'Medical Information Literacy Skills' to a maximum of 70.7 in the third Master year for 'Bibliography'.Higher scores indicate students have higher levels of ILSE.For the first three subscales, students increase their ILSE throughout the curriculum.In contrast, students feel less confident 'Using the Library', as most of the mean scores are mediocre.For the subscale of 'Bibliography', students' SE increases only much later in the curriculum (Year 4).When an overall difference was found between each group's scores in different years, post hoc tests were used to look for pairwise comparisons.As correction for multiple post-hoc tests, Tamhane T2 post-hoc tests were used.As documented in Table 9, for 'Medical Information Literacy Skills' (subscale 2) a significant difference between almost all study years could be found.As shown in Figure 1, there is progress in those specific skills.In contrast, for 'Using the Library' (subscale 4) almost no significant differences could be found between the different study years.

Conclusion
As indicated in previous research (Bazrafkan et al., 2017;Dale & Campbell, 2012;Eskola, 2007), IL courses should be integrated within a whole curriculum and not only at the start.Use of the ILSES-M scale can provide an added value in integrating the IL skills within the whole of a medical curriculum.Additionally, when integrating IL courses, medical educators should consider specific training formats that could have a real impact on students' SE beliefs (Aper, Reniers, Koole, Valcke & Derese, 2012).Practicing and getting the necessary feedback will enhance their SE beliefs (Ross et al., 2016).The results of this study show a clear positive evolution of the student's ILSE process.
For a medical curriculum, SE plays an important role.Being able to estimate themselves is of crucial importance in the students' future professional practices.However, our results suggest that the ILSE of medical students is not very high.The maximum mean percentage for all different subscales is only 70.7, implying that students could be more cautious when evaluating themselves because they are taught to do this and receive possible feedback on this process.However, further research is needed to align the results of this research by means of a qualitative study with student representatives for the different study years.
The original 28-item ILSES of Kurbanoglu et al. (2006) has been successfully expanded.The new ILSE items are an added value for a medical cohort, as different criteria were decisive for the selection: the items should evaluate SE in developing medical IL skills, be institutionalindependent and be general enough to be time-independent.It is very likely that in a few years "Using the library" will have no extra value in the evaluation of medical students' IL skills, and that other items (or another subscale) in relation to data management will be added.
The ILSES-M could help instructors to visualize at which level of the students' learning process they identify problems about their IL beliefs.Evaluating a complete curriculum could possibly indicate the specific groups of students for whom additional training or extra attention is needed, and/or key moments within the curriculum when such attention would be beneficial.Using the different subscales for evaluation might bring about more targeted adjustments within the courses.The results could form a reliable basis on which to adapt and outline the integration of IL courses within the curriculum.Questions such as 'How do students feel when they need to start working on their thesis?','Do they need to refresh their knowledge?' or 'Are they more insecure when starting a more advanced search strategy?',might be answered thanks to an evaluation via the ILSES-M.The scale could also help to evaluate new cohorts of first year students.It would be interesting to know if a new cohort feels more or less confident compared to previous years.The digital competences of youngsters could already have an impact on the way students start at university and how they perceive themselves as (medical) information (il)literate people.
Further research should look at the question whether the use of the ILSES-M has an effective impact on the further development and improvement of the medical curriculum.When curriculum changes take place, it is important to see whether those changes have an impact on the ILSE of students and whether the impact is short-term or long-term.Analysing crosssectional as well as longitudinal results will help the researchers to get a better idea about the details of the results: 'Is a certain cohort of students different?','Does gender have an impact on how the students' information literacy beliefs evolve?', 'Do curriculum changes have an impact on the ILSE of students?'The qualitative study may help to explain how students interpret a certain question, and how this interpretation can differ over time, or within a study year or cohort of students.
Even if the scale is testing self-efficacy rather than actual information literacy skills, it is believed that the collected results could give a clear idea about the evolution of perceived IL and the related need for support and training.
In conclusion, the Information Literacy Self-Efficacy Scale for evaluating medical students, ILSES-M, has been validated throughout a complete 6-year curriculum of medicine.The five subscales resulting from the validation indicated valuable Cronbach's alpha scores higher than 0.70.The ILSES-M can be used as a validated tool to evaluate medical students' ILSE at key moments in the curriculum and could be valuable in adapting and developing ILSE training within the medical curriculum.The ILSES-M can probably be used for analysing smaller or bigger groups, less and more experienced students.In addition, surveying with the ILSES-M requires little of the students' valuable time and can be organised on a digital learning platform.
Finally, the ILSES-M is written in English, which makes it institutional-independent and therefore easy to use for validation in other medical curricula.Further research should focus on the validation and use of the ILSES-M within other medical faculties and should look for the possible impact of cultural differences and diverse types of curricula.
Reference the source I use in a reference style used in medicine 1.4 Parallel research using the Information Literacy Self-Efficacy Scale Labels subscales: Factor 1= 'Evaluating and Processing Information'.Factor 2='Medical Information Literacy Skills'.Factor 3= 'Searching and Finding Information'.Factor 4 = 'Using the Library' and Factor 5= 'Bibliography'.

Figure 1 :
Figure 1: Boxplot mean scores on the ILSES-M subscales per study year

Table 2 :
Rotated Factor Matrix loadings (>0.4) for the items of the ILSES-MThe items are listed per factor and sorted starting with the highest loading within the factor.Once factors are extracted, it is crucial to look whether the extracted factors / subscales measure what they meant to measure by calculating a Cronbach's alpha

Table 8 :
Mean scores on the ILSES-M and subscales per study year

Table 9 :
Tamhane Post hoc tests for pairwise comparisons between study years per ILSES-M (sub)scales