IASL 2012 Conference, 11 to 15 November 2012
International Association of School Librarianship (IASL)
2003 Annual Conference, Durban, South Africa, 7-11 July 2003
OUR VIRTUAL CONFERENCE SESSION
Learning in the Information Age School:
DR ROSS TODD
This paper first articulates the core beliefs around which effective school library services need to be developed to ensure playing a central role and making a significant contribution to learning in information age schools. These core beliefs revolve around the concepts of difference, intervention and transformation. Based on these core beliefs, it explores the concept of evidence-based practice. The paper provides a brief overview of and rationale for evidence-based practice, identifies and discusses underpinning assumptions, and using findings from a recent research study in Australia, identifies some practical strategies and processes for undertaking effective evidence-based practice, and examines barriers to and challenges of evidence-based practice. In doing so, the paper explores a range of opportunities and options for maintaining effective library services in information age schools.
The provision of effective school library services and ensuring the vital future of school libraries rests on three key beliefs which I believe are the mandate and for the professional role of teacher-librarians. The first key belief is that the provision of information and information services makes a difference to the lives of people. If we do not believe that our information services can make a difference to people, then there is no point to their provision. It is as simple as that. Second, the key role of the school librarian centers on pedagogical intervention that directly impacts on and shapes the quality of student learning through their engagement with information. Learning in complex and diverse information environments does not happen by chance, and nor can it be left to chance. Explicit, systematic and planned pedagogical intervention must be the distinguishing and observable characteristic of the school library. Third, the role of pedagogical intervention is to bring on transformation. Learning takes place, and the lives of our students are transformed. The knowledge, skills, attitudes and values of learners are shaped and grow though their engagement with the school library and its pedagogical intervention. Learning outcomes matter. And in clearly articulating this transforming process through the school library -- that is, evidence-based practice -- the centrality and vitality of the school library are assured. If we do not believe that all students can learn, can develop new understandings through the school library and demonstrate outcomes, then we have to question why are we are teacher-librarians, and what are we doing. Each of these core beliefs are now elaborated.
1. Information makes a difference to people
The notion that information has the potential to change what people already know, and its place in the construction of understanding and knowledge is a fundamental tenant of education and the provision of libraries. The disciplinary areas of Librarianship and Information Science have a long tradition of exploring the relationship between people and information, and the "effects" conception of information is a well established viewpoint in these fields. This "effects" conception posits that people are not merely passive recipients of information, empty receptacles into which information can be poured; rather, people engage actively and highly selectively with information that surrounds them, and this engagement with information has some effect -- a person's existing knowledge is changed or transformed in some way. This "effects" orientation is faithful to the Greek and Latin roots of the word "information": in = within; formere = to shape or form; that is, information's effect is inward forming. As effect, information is concerned with human cognition, the relationship between information and the generator, the relationship between information and the user, the idea of desired information, the effectiveness of information transfer, and the effect of the information. It establishes that human knowing is a product of human action, human attitudes and values, and contextualized reality, that it is partial and temporary, ever-changing.
Bertram Brookes, regarded as one of the founders of information science in the 1970s, championed the importance of understanding the interactions of people and information, and saw this as the basis of person-centered professional library practice. Brookes considered that while the collection, organization and access to information sources were essential to professional practice, they were not the focus of professional practice; rather, the intent of such organization and access was the transformation of information into personal knowledge: "the interaction between the private, inaccessible thoughts and mental images of people, each unique, and the public documented artifacts" (1975). In essence, Brookes was interested in the effects of information as a human construct. He saw libraries, not in terms of their collections, access structures and staffing, but in terms of people's knowledge undergoing transformation through a dynamic interaction with information. Effect. Difference.
In essence, these are the building blocks of school librarianship. Conceptualizing information as it is internalized by people, and in terms of the differences / effects / impacts that information makes to people puts emphasis on the user / recipient of information, and shifts the professional responsibility from a concern about the transmitting and transferring of information (an access and exchange orientation) to a concern for understanding the human dimensions of the take up and use of information. It also puts emphasis on understanding the context of information needs, information seeking and use, and understanding how people move form their initial states of knowledge, to new understandings. In essence, information-as-effect expresses the fundamental belief of librarianship -- the formal provision of information through libraries and information agencies is about social good -- there is a flow on effect -- and that this effect is beneficial to people as individuals, and to society as a whole. At a fundamental level, then, school libraries exist to make a difference to the lives of young people. At a broader level, it is about a commitment to the social good -- not a rhetorical commitment that merely advocates the benefits of libraries, but a demonstrated commitment that shows the benefits and effects of libraries.
2. Instructional intervention is the key role of the teacher-librarian
There is wide acceptance that the teaching-learning role is a central dimension of the professional role of teacher-librarians. This is clearly established in many policy documents at international, national and professional association level. This role revolves around working closely with classroom teachers to design authentic learning experiences and assessments that integrate a range of information and communication abilities needed to meet curriculum objectives, and to provide learning opportunities that encourage students to become discriminating users of information and skilled creators of new knowledge. Underpinning this approach is the belief that people's engagement with information is something that does not happen by chance, and which cannot be left to chance. Information literacy, as the centre piece of the instructional role of the teacher-librarian, is about pedagogical intervention. It is about the systematic and explicit provision of a range of intellectual scaffolds for effective engagement and utilisation of information in all its forms (electronic, print, popular culture) and for constructing sense, understanding and new knowledge. How do students develop intellectual scaffolds? Prevalent among teachers, and sadly, some teacher-librarians who do not active engage in collaborative teaching-learning initiatives, is the view that there are developed by learners in a number of ways. First, mysteriously: the perception that someone else has taught them; second, vicariously: that students develop these somehow by sitting at a computer terminal; third, serendipitously: that students develop these by just doing assignments through haphazard information seeking over many years of schooling; fourth, slavery: students get someone else, such as parents and librarians to find and provide the resources for them. Instructional intervention is about moving beyond chance encounters with information to a more formal systematic and explicit approach through embedding learning scaffolds into the teaching and learning process. The research evidence to date suggests that deliberately planned pedagogical intervention impacts positively on mastery of information scaffolds, mastery of content, and attitudes to self, to learning, and to schooling in general.
The International Association of School Librarianship (IASL) Policy Statement on School Libraries asserts that "a planned program of teaching information skills in partnership with classroom teachers and other educators is an essential part of the school library program.... This cooperation with teachers may concern: development of the curriculum, the educational activities offered by the school to the child, as well as short and long term planning concerning the uses of materials, information technology and equipment, and development of information skills for the child's education". This essential role is also expressed in the International Federation of Library Associations and Institutions (IFLA) Manifesto for School Libraries (http://www.ifla.org/VII/s11/pubs/manifest.htm). It states that "the school library offers learning services, books and resources that enable all members of the school community to become critical thinkers and effective users of information in all formats and media", and that core school library services center on dimensions such as "supporting and enhancing educational goals as outlined in the school's mission and curriculum", "developing and sustaining in children the habit and enjoyment of reading and learning, and the use of libraries throughout their lives", and "working with students, teachers, administrators and parents to achieve the mission of the school". Very clearly, pedagogical intervention is at the core of being a teacher-librarian.
Further, an analysis of a cross section of role statements about school libraries in many professional associations around the world highlights two important points. First, it highlights the centrality of the instructional role. Second, it highlights why this role is deemed important. This importance centres on actions, changes and effects -- effects in relation to personal, social, intellectual and emotional needs and well being; effects that make a difference to the lives of people. It is about outcomes. Outcomes are the transforming effects of pedagogical intervention.
3. Learning outcomes, as the transforming effects of the teacher-librarians' pedagogical (and collaborative) intervention, are the raison d'être for school libraries.
If school libraries do not contribute to learning outcomes, and if teacher-librarians cannot articulate what these outcomes are, then school libraries are on shaky grounds. An outcomes focus of school libraries is clearly in line with syllabus developments across many countries, where emphasis is given to specifying learning outcomes, establishing measurable indicators for these outcomes, and providing feedback to the learning community of the achievement of these indicators. An outcomes focus is directed towards maximizing learning experiences of students, and where attention is given to identifying, understanding, and coming to terms with the real effects of information literacy interventions.
Lorenzen, Library Instruction Coordinator at Michigan State University defines outcomes-based education as a "method of teaching that focuses on what students can actually do after they are taught. All curriculum and teaching decisions are made based on how best to facilitate the desired outcome. This leads to a planning process in reverse of traditional educational planning. The desired outcome is selected first and the curriculum is created to support the intended outcome". Boschee and Baron define outcomes as" future oriented, publicly defined, learner-centered, focused on life skills and contexts; characterized by high expectations of and for all learners, and sources from which all other educational decisions flow" (Boschee & Baron, 1994). Towers posits that "education that is outcome-based is a learner-centered, results-oriented system founded on the belief that all individuals can learn" (Towers, 1996: 19). Spady and Marshall further define outcomes as "clear, observable demonstrations of student learning that occur after a significant set of learning experiences.... Typically, these demonstrations, or performances, reflect three things: (1) what the student knows; (2) what the student can actually do with what he or she knows; and (3) the student's confidence and motivation in carrying out the demonstration. A well-defined outcome will have clearly defined content or concepts and be demonstrated through a well-defined process beginning with a directive or request such as 'explain', 'organize', or 'produce'." (Spady & Marshall, 1996: 20,21).
Outcomes education is not new to teacher-librarians who have embraced a strong instructional focus to their roles. The focus on the development of students' information and critical literacies surrounding the use of print and electronic information sources is founded on fostering a range of information-related outcomes. Underpinning the very notion of information literacy in schools is the provision of explicit, systematic, integrated and contextualized learning experiences which seek to develop a range of information competencies, and inculcate a range of information values and attitudes. AASL's position statement on the role of the teacher-librarian in outcomes-based education establishes that the teacher-librarian "has an essential role in curriculum development. Outcomes-based education is a curriculum practice which establishes clearly defined learner outcomes based on the premise that all students can be successful learners. High expectation outcomes, which are essential for success after graduation, require carefully aligned curriculum, instructional strategies and performance-based assessment. In their unique roles as information specialist, teacher, and instructional consultant, library media specialists actively participate in both the planning and implementation of outcomes-based education".
The focus on difference, intervention and transformation raises what I believe to be one of the most critical questions facing school libraries today. The question is this: "What differences do my school library and its learning initiatives make to student learning outcomes? Or, expressing it another way: "What are the differences, the tangible learning outcomes and learning benefits of my school library"? Why is this such an important question? Take a look at this scenario, similar to one which was emailed to me recently. It was a cry for help, a cry for solution. A cry that perhaps is already too late.
I am a teacher-librarian at x. The teacher-librarians in this district are facing a grave situation. Recently our district superintendent presented his budget proposal to the school board. This proposal is a means to cut almost $4 million from the budget. One of his proposed cuts is to reduce the teacher-librarians in the district by half, and increasing library aide time to ensure facilities will still remain open, which he claims is a savings of almost half a million dollars.
This drastic cut to eliminate or reduce positions is on the basis that teacher-librarians did not have direct student contact. Our High School library has hundreds and hundreds of students a day and both of the teacher-librarians teach in classrooms regularly. If that does not constitute student contact, I don't know what does. We have voiced our disapproval, attended listening sessions and spoke out against the proposed cuts, recounting several studies that have shown a correlation between well staffed libraries and high student achievement in academics and reading.
I have written a letter to the newspaper editor and contacted the media, who responded immediately by sending a news team to our school to interview the director of human resources and me. The director has been quoted as saying," these propose cuts will not affect student learning and will not disrupt the learning process." I have heard from countless teacher-librarians and their similar struggles. Teacher-librarians are becoming targets for administrators as means of balancing the budget. How many of them will lose their jobs? How many school libraries will be forced to close? What measures do we need to undertake to show others outside of our profession the importance of school libraries staffed by teacher-librarians?
The answer, I believe, centres on the notion of evidence-based practice.
Evidence-based practice is where day-by-day professional work is directed towards demonstrating the tangible impact and outcomes of sound decision making and implementation of organizational goals and objectives. It is an evolving concept in many professions, and for many it represents a new paradigm for professional practice. It emerged in the early 1990s in the fields of Medicine and Health Care Services initially to teach medical students how to independently find, appraise and apply the best evidence, and to apply it to solving clinical problems (OıRouke, 1998, 1). Sackett defined evidence based medicine as the "conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. This practice means integrating individual clinical experience with the best available external clinical evidence from systematic research" (Sackett, 1996, 72-3). Implicit in this approach are important concepts such as "duty of care", "informed decision making" and "optimal outcomes", all seen as critical factors in making a difference to the well being and lives of people. At a fundamental level, the early evidence-based practice movement had as its goal the tangible capacity to make a difference to the lives of people, through carefully informed intervention to achieve optimal outcomes. Difference, intervention and transformation. Interest in evidence-based practice has grown exponentially since the early 1990s, and today it is acknowledged as an important approach to professional practice in many disciplines beyond the health care arena, to professional arenas such as education, social work and law.
Central to evidence-based practice is the combining of professional expertise, insight, experience and leadership with the ability to collect, interpret, and integrate valid, important and applicable user-observed and research-derived evidence to ensure significant outcomes (E-BEUK, 2002, 1). More recent explications of this concept establish evidence-based practice as an approach to professional work which argues that policy and practice "should be capable of being justified in terms of sound evidence about the likely effects" (E-BEUK, 2002, 1). Underpinning its role in education is the belief that student learning and student learning outcomes are "too important to allow [them] to be determined by unfounded opinion, whether of politicians, teachers, researchers or anyone else" (E-BEUK, 2002, 1). In other words, duty of care centres around being able to articulate clear learning outcomes, developing processes and strategies to enable these, and articulating the impacts.
In current usage, the concept of evidence-based practice thus has two important dimensions. First, it focuses on the conscientious, explicit and carefully chosen use of current best research evidence in making decisions about the performance of the day-by-day role. Second, evidence-based practice is where day-by-day professional work is directed towards demonstrating the tangible impact and outcomes of sound decision making and implementation of organizational goals and objectives. This latter dimension of evidence-based practice centers on local processes, local actions and local outcomes. A number of important notions are embedded in these dimensions. As a particular approach to practice, it moves beyond intelligent guesswork, clever hunches, and application of individual skills; beyond the anecdotal and tossing of coins so to speak, to establishing a sound research-based framework for decision making. However it is more than getting research into practice to guide day-to-day work. It is also about focusing on the delivery of services based on stated goals and objectives, and systematically demonstrating outcomes and endpoints in tangible ways, and critically reflecting on inputs and processes. It plays an important role in user-centered services to show that the rhetoric about those services is real, that expectations are met, and promised outcomes are actually delivered. In the context of school libraries and school goals and objectives, evidence-based practice means that the day-by-day work of teacher-librarians is directed towards demonstrating the tangible impact and outcomes of services and initiatives in relation to student learning outcomes.
There is much research evidence already established in the school librarianship field that, when coupled with the enormous professional experience and wisdom of teacher-librarians, can contribute to the sound development of meaningful learning experiences for students, and for the provision and management of information services. This is documented in numerous research reports and reviews (Callison, 2001; Todd, 1995, 2002; Haycock, 1994; Kuhlthau, 1993; Lance, 2000; Loertscher & Woolls, 2002; Oberg 2001), and leading journals of the field, such as School Libraries Worldwide, Knowledge Quest, and a range of country-specific journals such as Computing in New Zealand Schools, and Australian journals Scan and Access make available this research.
While the concept of school library outcomes, effectiveness and evaluation are not new, historically these has been directed to outputs in the form of statistical information related to resources, expenditure and facilities use, rather than in terms explicitly stated learning outcomes that identify and demonstrate the tangible power of the school library's contributions to the schools' learning goals. Recently I came across the following quotation: "Celebrate the understood, not the found". This statement expresses so eloquently and so simply the vision, mission, goals and actions of the school library. Historically, we have celebrated the found. We have documented, for instance, the number of classes in the library, the number of library items borrowed, the number of students using the library at lunch times, the number of items purchased annually, the number of web searches or hits, the number of resources purchased, even the number of books lost or monies collected in fines! These are measures of pathways to learning, not of learning itself. Celebrating the understood is what evidence-based practice is all about. It is understanding and showing how the school library helps students learn, and the learning outcomes that are enabled. Research tells us that learning outcomes can be charted in terms of: information processes and skills, mastery of networked information technology, reading, knowledge outcomes such as mastery of content, development of personal perspectives and viewpoints, independent learning strategies, changed attitudes and values, and gains in self concept and personal agency.
"Celebrating the understood" takes the information literacy agenda of school libraries to a new level. The goal of school libraries is not just the development of information literacy. Information literacy is not an end point. Information literacy instruction is part of making actionable all the information and knowledge that a school possesses or can access. But the critical question is: actionable for what, and for whom? To what end? WHY? Teacher-librarians need to be clear about what actually is their motivating force for their instructional roles in schools. The fundamental motive for the instructional role of the teacher-librarian has to go beyond enabling students to master a range of information handling skills. This is doing. And often it is perceived by teachers to be a library's doing, an add-on to an already crowded curriculum, and something not necessarily valued outside of the library. There is no question that this is important doing. However, it is my view that the development of an information literate student is integral to becoming and being. This begs the question: by developing information literacy skills, what do we want students to become? The destination is not an information literature student nor even an information literate school community, but rather, the development of knowledgeable and knowing people, students who are able to engage effectively with a rich and complex information world, and who are able to develop new understandings, insights and ideas, and indeed, a knowledgeable and knowing school community. This is what teachers would hope for. The development and use of human knowing, the construction of understanding and meaning is what learning is all about, and that defines the central role of the teacher-librarian.
Speaking from a constructivist perspective, Wilson (1996: 3) claims that learning which emphasizes "meaningful, authentic activities that help the learner to construct understandings and develop skills relevant to problem solving" is the central mission of the school. Hein (1991) emphasizes the idea "that learners construct knowledge for themselves; each learner individually (and socially) constructs meaning as he or she learns. Constructing meaning is learning. There is no other kind". These are powerful words. He goes on to say that "Learning is a personal and social construction of meaning out of the bewildering array of sensations which have no order or stature besides the explanations which we fabricate for them". The instructional initiatives of teacher-librarians centering on information literacy are about providing the best context and opportunities for people to make the most of their lives as sense-making, constructive, independent people. The provision of information does not necessarily mean that our learners become informed. Information is the input; through this input, existing knowledge is transformed, and new knowledge -- as understanding, meaning, new perspectives, interpretations, innovations -- is the outcome. Empowerment, connectivity, engagement, and interactivity define the actions and practices of the school library, and their outcome is knowledge construction: new meanings, new understandings, new perspectives.
This suggests a pedagogy that has knowledge construction and inquiry learning at its heart, where through access to multiple sources and formats of information, and information technology, learners acquire the intellectual scaffolds to engage with multiple perspectives, sources and formats of information to be able to construct their own understanding. In this context, the role of the teacher-librarian goes beyond developing a range of information literacy competencies, rather, it is significant responsibility of making actionable all the information and knowledge that a school possesses or can access so that students can construct their own understanding and develop their ideas in rich ways.
EVIDENCE-BASED PRACTICE IN ACTION: RESEARCH STUDY
As identified above, evidence-based practice centers on the key question: What differences do my library and its learning initiatives make to student learning? That is, what are the differences, the tangible learning benefits, defined and expressed in ways that lead the local school community to understand the important contribution of the library to learning outcomes, and to say: "we need more of this!"? It is about ensuring that daily efforts put some focus on not just gathering meaningful and systematic evidence on dimensions of teaching and learning that matter to the school and its support community, but critically reflecting on this evidence to shape ongoing instructional initiatives. These evidences will clearly convey that learning outcomes are continuing to improve, and inform the process of their continued improvement. This dimension of evidence-based practice focuses on local actions, local outcomes, and local evidence, set in the particular school context, curriculum, and learning goals.
What are teacher-librarians doing in relation to evidence-based practice? Do they engage in it? To explore some of these ideas, a research study was undertaken in 2002 in Australia. This study, sought to:
A survey instrument based on a Critical Incident approach was used to collect the data. The Critical Incident Technique, based on work of J. C. Flanagan (1954) centers on the collection of detailed reports of incidents / discreet experiences in which individuals do something in achieving an articulated purpose. The "critical" dimension is that the incident or experience must occur in a situation where the purpose or intention of the act seems clear to the observer and the consequences sufficiently definitive as to leave little doubt concerning its effects. Based on this technique, data are derived chiefly from in-depth analytical description of an "intact cultural scene", involving the gathering of facts before, during and after the event or experience. Typically this approach uses an open-ended questionnaire, gathering retrospective data, and where questions typically help respondents recall events or steps in the events without interfering with the quality of the recall. Some of the key features of the Critical Incident Technique, and important to this study were that: data are centered around real events; the tasks being analyzed are performed by real people; respondents are located in normal working environments; data are captured in normal task situations rather than in contrived laboratory or other experimental settings; an implicit or explicit storytelling process is used to deconstruct movement or steps through the incident; and prompted questions are used to ensure uniform data. Such a methodology has weaknesses, including reliance on memory, hence respondents were asked to focus on a "recent" event; and because the methodology tends to focus on recent events, the parts of the whole, then aspects of the richer experience may be overlooked.
The questionnaire collected data on school background, and to identify evidence-based practice, respondents were asked to describe one of the most recent curriculum units that she or he had planned and taught collaboratively with classroom teacher(s). The focus was to get an indication of what learning outcomes were achieved, and how respondents were able to identify these. Unit details included: Year/grade; Syllabus, Number and gender of students; their average age; brief description of students (eg. mixed ability, streamed, gifted and talented); Title of unit; Brief description of the unit (eg. time span of unit, number of sessions, lesson length); Syllabus outcomes addressed by unit (be specific); and Related information skills outcomes of unit. To document learning outcomes and their evidence, respondents were asked to identify:
The survey was distributed in the Australian journal Scan in May 2002. Scan is a quarterly journal that focuses on the interaction between information and effective student learning. Published by the Library and Information Literacy Unit of the Professional Support and Curriculum Directorate, NSW Department of Education and Training, Scan's articles and reviews explore the use of curriculum resources within the learning environment. Articles in Scan reflect best practice in teaching and learning, emphasizing an outcomes approach and the benefits of collaboration between teachers and teacher-librarians. Of interest to all educators, the journal is also a key component of the statewide support for teacher-librarians. It has a circulation of over 3000. 11 responses to this survey were received. These were very rich and detailed responses. A number of reasons could be posited for the low response number. The questionnaire required considerable thought and time to complete, and the busy daily agendas of many teacher-librarians may not have provided the time to complete it. Teacher-librarians may not have considered that the focus and/or outcomes of the study were important, and were unwilling accordingly to invest the time to complete it. It is possible too that teacher-librarians may not actually engage in evidence-based practice, and therefore have had little to contribute to the study. The responses came from 2 primary schools and 9 high schools, and all responses except one were from New South Wales. What follows is a brief summary of some of the findings.
Findings: Evidence-based practice strategies
Respondents could clearly articulate both curriculum and information literacy outcomes. In the brief descriptions of collaborative curriculum units that the respondents submitted, there was integration of curriculum objectives and information literacy objectives. The information literacy outcomes ranged across the broad spectrum of skills in relation to defining, locating, selecting, organizing, presenting and assessing information. These were articulated quite concretely. For example, outcomes were expressed in terms students being able to:
The strategies for documenting evidence fell into two broad categories: formal, structured records, and informal observational approaches.
The formal structured records were primarily the use of:
(a) Checklists. A range of simple checklist strategies, where both students and teacher-librarians provided checklist or ratings of perceived levels of skills and / or knowledge acquisition, mainly after the instructional period, and in four cases, both before and after so that comparisons of differences, changes in levels of knowledge and skills could be documented. These checklists were in relation to levels of mastery of information literacy competencies such as ability to identify main ideas, make notes, use different formats of information, understanding the differences in the different purposes of sources; competencies in relation to information technology, such as skills in searching, evaluating information on web sites, and using a range of presentation software such as Powerpoint and spreadsheets. Some respondents provided examples of these checklists.
In these cases, some attention was given to deriving general statements about outcomes achieved on the basis of these comparisons. This is a key process in evidence-based practice. This involved critically analyzing the accumulated data and, on the basis of indicators, deriving some general statements about student learning outcomes on the basis of the evidence provided. Some outcomes statements that respondents were able to determine included:
"More than 80% of the class showed improvement in their ability to effectively judge
the quality of web sites after the sequence of lessons to develop this awareness";
"Virtually all of the students recorded citations accurately in their essays following the input on bibliographic citations";
"When we analyzed the essays submitted at the end, and following through some of the web sites that the students had cited, we saw a dramatic decrease in the level of plagiarism. We had explicitly built this issue into our teaching, and discussed it with the students, both in terms of being responsible and ethical users of information, and teaching them some analytical strategies to express ideas in their own words. We were thrilled, and discussed these findings and processes in our recent staff meeting"
"We ran a quick survey at the beginning of the unit to see how students were thinking about the unit. They were not terribly motivated or interested, and said so in their surveys. In our teaching of the unit, we worked really hard to build interest and motivation, and when we ran the little survey at the end, we had almost all of the students indicating how much fun the unit was, and how much they learned. It was hard work creating motivational activities, but worth it. We not only felt we had achieved something, we had some proof"
(b) Rubric strategies. Three respondents indicated that they used rubric strategies where students' performance in final products were scaled according to a set of criteria that clearly defined what range of acceptable to unacceptable performances and/or information products look like. Some respondents identified a range of criteria relating to evaluating performances in products and presentations. For example, a semester paper was based on and scaled according to Gordon's rubric (2001) for evaluating the research process. This rubric focused on a number of dimensions: Planning (What message did I want my project to give? Did I succeed? What did I expect to learn? Did I accomplish my goals?); Meeting deadlines (How well did I manage my time?); Organisation (How well did I gather information? How well did I demonstrate the new ideas I have learned?); Working with the teacher-librarian (Did I ask questions and seek help when I needed it? Did I take initiative to set up meetings?); and Problem solving (What problems did I face and how did I solve them? What decisions did I make that shaped the project?) The students were asked to scale their performance in terms of: Excellent, Competent, Making some Progress, and Not yet competent, and were asked to write personal comments as well as the rating. In the feedback to the students, the teacher-librarian also provided ratings and comments. While the respondent provided no concluding statements about the learning outcomes using this process, an analysis of this rubric would enable some worthwhile patterns to be identified.
(c) Formal feedback strategies. One teacher-librarian used a simple feedback survey every term on what his library does "best" and "least" to help students with their school work. This is a general survey made available to the students which asks two questions: "During this term, how did the library best help you learn?" And "During this term, how could the library help you learn better"? The teacher-librarian reported that after one intensive collaborative with all the Year 8 teachers on more effectively using the internet for Science, the term survey clearly showed that the students believed that had quite dramatically improved their web searching skills, not just in terms of finding more pertinent resources, but also in terms of meeting assignment deadlines on time, and feeling more comfortable about using accurate web sites for their research. Each term, the teacher-librarian collated and presented the results of this survey -- both the positive and negative aspects -- at staff meetings. In his response to this research study, he said: "I do not let an opportunity go by when I let staff know about what the library contributes to learning. I always quote some of the things the students have said to illustrate my points. The school have got the idea that what I am on about in helping kids learn. The key thing in my view is to have something to say that goes beyond gut reaction. The student survey does just that. The teachers hear what the students say. I believe they listen a great deal to this". According to this teacher-librarian, the feedback is also used to make decisions on improving services, designing information literacy classes, and planning the whole library teamıs work agenda.
The informal observational approaches.
More predominant than planned strategies for recoding evidence, however was the use of informal observational and conversational strategies. All of the responses to the research survey indicated that observations of teacher-librarians, and in a few cases, observations of teachers were the basis for making statements about learning outcomes, either curriculum or information literacy. These were based on discussions and observations during the teaching time, and on review of student products. The approaches were "gut reactions", drawing on professional expertise and experience to identify outcomes. Respondents said:
"I rely on my long experience to work out what is happening with the students";
"I watch the students casually though fairly consistently while they work in the library";
"I get ideas from the kids of questions students ask when they are in the library";
"Often when I am chatting to a student doing a major assessment item, I will ask them about what they have learned in the library".
"I have discussions with the teachers about what is going on"
"I take note of student behaviours while they are in the library"
These more informal approaches to gathering evidence enabled the respondents to make some conclusions about learning outcomes. For example:
"The class teacher noted an improvement"
"Students completed learning journals"
"Students were certainly engaged in their learning"
"Students showed quite a lot of independence"
"Students worked well in groups"
"I saw increases in student motivation"
"Students displayed all or nearly all of the information skills"
"Students initiated email interaction and to me this showed engagement with the topic"
"I saw evidence of improved or extended technical vocabulary"
"The technology was used beyond my expectation"
What is particularly noticeable with the statements of outcomes based on casual observations and discussions is their lack of specificity and precision. Concrete outcomes were not clearly articulated.
Only in two cases were additional approaches to evidence-based practice provided. Here, the teacher-librarian examined the results of Year 7 English Language and Literacy Assessment tests, and sought to identify how one class group that had engaged in an intensive program of reading enrichment and literature discussions, compared to other students in the school. She noted that there appeared to be stronger test scores for this group of students. In the second case, the teacher-librarian compared borrowing records of students during a collaboratively implemented science unit in the lower high school. She found that those students with the highest number of items borrowed for the unit also achieved the highest scores on the test at the conclusion of the unit. While it is difficult to establish strongly stated conclusions, such patterns show promising school library-outcomes relationships worthy of richer documentation.
Findings: Value of evidence-based practice
Despite concerns and fears expressed about the intentions, processes and competencies in relation to undertaking evidence-based practice, respondents identified 6 key benefits of evidence-based practice.
(1) It provides evidence at the local school level that library initiatives make a visible contribution to learning, and that administrators, teachers and parents can see the real impacts:
"My boss actually talks about specific outcomes I have identified. He's proud of what we have achieved, and itıs not because I tell him how important our school library is, it is because I actually show him the evidence. He shares this with the parents in the school newsletter"
(2) It convinces administrators and community funders that the money invested in the school library is worth it:
"Money in my school seems to flow easiest to those happenings / teachers in the school where students achieve success, and it is clearly seen -- either in competitions, class projects, sporting events, exam results, student exhibitions, anything which show learning and success and which the school celebrates. I've learned over the last year or so that if I want to jump on the money bandwagon, I show the achievements of my library initiatives. This is usually outcomes related to information literacy lessons, or my literature enrichment activities".
(3) It demonstrates the teacher-librarianıs commitment to learning outcomes:
"When I tell the staff or parents about what the library is doing, I always try to tell
about what we have achieved for the students, not from the library's perspective, but
from the students. While I am not directly involved in the parent nights where teachers
meet with parents to discuss students' grades, I always set up a display for parents to
show our various projects and what the students learn through it. I get lots of positive
feedback that recognizes our involvement in students' learning"
"My colleagues around the school see and hear me involved in learning. I'm not seen as the circulation police or fines controller, or the shusher or the stamper, I'm seen and valued as a teacher".
(4) It helps teacher-librarians plan more effective instructional interventions and information services:
"the feedback from students, and results of analysis of what students have learned or
not learned helps me plan my teaching to be more effective, it identifies gaps in
students' information literacy skills so I can make it better for them. I get feedback
on what works and doesn't work. Sometimes you can put a lot of effort into something,
and then find out it didn't really achieve anything. That's hard to take, but makes
you really think about what you do, and what is important to do"
"The evidence helps me work out what is really important for me to do each day, rather than concentrating on functional or management things, which sometimes take on a magnitude of importance well beyond the time and energy given to them".
(5) It contributes to job satisfaction.
"When I can put my finger on what the students have achieved because of my work, I feel
terrific, and get more enthused about being a teacher-librarian. I feel as if I am making
a valuable contribution to the kids' learning, because I can see some actual results".
"I get a real buzz each day because I know I make a difference to these kids at school. I'd like to work more on finding out how it helps them become independent learners"
(6) It moves beyond anecdotal, guess work, hunches, advocacy, and the touting of research findings.
"I don't have to get on my library soap box and try and convince people about the value of the library. I make a habit of sharing with them details about every set of classroom units I do, and try and sum up how the students have benefited, using examples from their work. I don't think that advocacy without evidence goes far"
Evidence-based practice provides evidence at local school level that the school library program makes a difference to student learning outcomes, and contributes tangibly to the learning goals of the school. In moving beyond the anecdotal, it makes clear connections to local actions that are seen and valued by a school community. Local outcomes become tangible and visible. It de-emphasizes intuition, the anecdotal, and unsystematic processes and hasty decision making. It makes for easier decision making, because decisions are able to be justified on the basis of evidence. Evidence-based practice is effectiveness lead: it targets time, energies, scarce resources, and scarce staffing in improving and demonstrating effectiveness.
Findings: Evidence-based practice issues
Five key issues in relation to evidence-based practice were identified by the respondents. These were:
(1) Accountability fears. Two respondents thought that having to "prove your worth" through pressure to demonstrate learning outcomes and evidence of impact would be detrimental to the profession:
"It would encourage more anxiety and paranoia at a time when teacher-librarians' workloads are already full to overflowing"
"evidence-based practice might be used as a basis for getting rid of us. It's something we haven't done, or had had to do, and because we now are not able to produce anything that focuses on what learning outcomes we bring on, we may be assumed to be ineffective when accountability demands are made".
There are some clear messages here. It is perhaps folly to assume that because of professional qualifications and roles, teacher-librarians are somehow immune from any kind of accountability for processes and outcomes at a time when calls for educational accountability are increasing. The National Center on Educational Outcomes (http://www.coled.umn.edu/nceo/) defines accountability in terms of "an individual or group of individuals taking responsibility for the performance of students on achievement measures or other types of educational outcomes" NCEO, 2002, 1). Most systems of education have established goals for performance of students, and assess progress toward achieving those goals. A dual accountability is a major component of this: one that assigns responsibility to the student (student accountability) and the other that assigns responsibility to the educational system or individuals within that system (system accountability). Evidence-based practice is accountable practice. It is a systematic method to assure stakeholders -- all members of the school community, policymakers, funding authorities, and the public -- that schools and school libraries are producing desired results. It is based on professional consensus about what is important for students to know, and when and how to identify the extent to which they have mastered necessary skills and knowledge. It includes common elements such as goals, indicators or progress toward meeting those goals, measures, analysis of data, reporting procedures, and reflective actions, even consequences or sanctions. Accountability is a professional guarantee of outcomes; for teacher-librarians, it is the professional guarantee of information as effects, and provides a stronger basis for guaranteeing the longevity of the role.
Some teacher-librarians may feel that accountability questions professional authority, curtails professional freedom, is intrusive, even perhaps considered to be spying on doing the job. The appeal to professional authority or autonomy is often a mask for hiding what is really going on. Evidence-based practice does not reject the power of the expert opinion, nor does not devalue casual observation, professional insight and wisdom required to make often fast decisions in the workplace. Nor does it mean discarding the familiar, the current practice, or rejecting experience and instincts. Rather, it integrates both human insight, expertise and accumulated wisdom, and established research evidence in the conceptualization and delivery of service, and in systematically establishing and documenting the relationship between espoused and actual goals, and what this means for the provision of professional services. In linking actions, goals, outcomes and evidence, evidence base practice does not diminish professional authority; rather, it enhances it by taking uncertainty and guess work out of the role, its value, position, action and its public perception. It doesn't reject the power of the expert's opinion, rather, it evidences it. And this is a powerful dimension of professional credibility and authority. Evidence-based practice takes unsystematic observations, experiences and instincts one step further: recording observations systematically and in a reproducible and unbiased fashion, with the clearly planned approaches and frameworks to characterizing these observations. It is about establishing with more certainty the accuracy and the precision of the perceptions, observations, experiences and instincts.
Whether teacher-librarians view the notion of accountability positively or negatively, the current climate of education internationally is placing high stakes on learning outcomes and student performance, particularly through state-wide testing programs, with implications for school profile, quality of delivery, funding, and at times employment and sanctions, embedded in this. In the current scenario, the stakes appear to be high for teacher-librarians, particularly amid concerns centering on: the perceived lack of understanding of nature and dimensions of role of the school librarian, perceived lack of value, importance and appreciation, a negative perceptions of image, sometimes a perceived lack of support for role and the consequence of not being able to perform at the desired level, perceived low status, and ongoing funding and resourcing issues (Todd, 2001). This is an opportunistic time, not a threatening time. Some teacher-librarians may claim that they have been endeavoring to "prove their worth" for years without success for years, and that evidence-based practice is thus "another trick". However, on the other hand, is a waste of money and time to impose policies and practices without good evidence that they will lead to improvements over what previously existed; evidence of effective impact enables teacher-librarians to reclaim debate / control for professionals who know most about them -- teacher-librarians.
(2) Competency requirements. The second issue in relation to evidence-based practice identified in this study was in relation to the assumed competencies needed to undertake evidence-based practice. As respondents said:
"It seems as if I need to be a statistician to do this. I just do not have these skills, and I disliked research methods at university".
"We have to become researchers in order to undertake evidence-based practice, or at least have a mastery of statistics. Isn't that what the universities should be doing?"
There is no question that evidence-based practice does demand certain precision in identifying learning outcomes, establishing indicators of these, and skills in analyzing and synthesizing the evidence to establish specific achievements in learning outcomes. As identified above, there is an enormous body of work in the profession already that has given attention to information literacy outcomes, and developing indicators of these. These should guide the process. While I have personally long believed that all teacher-librarians should undertake training in research methods in their professional education, the intellectual skills required to undertake evidence-based practice are not formal quantitative and qualitative research methodologies and complex statistical analyses. Rather, they are the skills of examining student learning goals and needs, selecting appropriate learning outcomes, identifying desired indicators of these outcomes, establishing systematic approaches to locating and gathering the evidence of achieving learning outcomes, analyzing, organizing and synthesizing the outcomes, presenting and celebrating the outcomes in the school community, and reflecting on how this continues to inform the ongoing teaching and learning process. Evidence-based practice is about identifying, locating, selecting, exploring, formulating, focusing, organising, presenting, and assessing information. The information process that has guided the information literacy initiatives of school libraries and which has been the espoused educational platform for almost two decades is the very process of evidence-based practice. Evidence-based practice is thus a call for teacher-librarians to be exemplars of their rhetoric -- to practice what they preach in relation to information literacy.
The information literacy agenda does not ask that teacher-librarians become formal researchers in the academic sense, but does ask that teacher-librarians be researchers, like students, guided by the information process, and making use of a range of information scaffolds to achieve their goals. It does mean that reflective practices, guided by the available formal academic research, give some careful attention to learner assessment and instructional evaluation, to documenting, analyzing and synthesizing the outcomes of collaborative teaching-learning initiatives, and how these outcomes support and enhance the learning goals of the school. What is important is that evidence is gathered in a systematic way that highlights the learning gains, both in terms of a range of information and critical literacies, but also how developing these scaffolds enables more effective learning of curriculum content and how this contributes to the development of new knowledge. It can also highlight how the library plays a role in shaping attitudes and values, in contributing to the development of self-concept, and in contributing to a more effective learning environment. Apart from the tangible outcomes that demonstrate the central role of the school library in learning, evidence-based practice provides a wonderful opportunity for teacher-librarians to model the information process to their teaching colleagues. In addition, it provides a basis for targeting time, energies, and scarce resources; and for not doing things that do not work or do not matter. Effective practice that clearly produces intended results is responsive and accountable practice, and one that will provide considerable job satisfaction and build confidence in the central role that the library plays across the school.
(3) Time pressures. Three respondents raised the issue of the time commitment needed to undertake evidence-based practice. One respondent said: "I see the value of evidence-based practice, and have tried to implement measures. It takes time, and I feel the pressure when I have so many other things to do". This tension between belief versus action was also reflected in the comment: "I want to do it, but when do I find the time to do it?" and "I do not have enough time to do my current job as it is, let alone adding more, even though I would like to do this". One other respondent claimed: "In reality a lot of evidence is intuitive and the time element squeezes out the more formal measures". Compounding the time pressure was the situation of teacher-librarians scheduled to provide classroom teachers with release from face-to-face teaching: "I need to be free from providing release from teaching for classroom teachers so that I have time to undertake this. This is a barrier to making real collaboration happen and working together to identify the outcomes".
Teacher-librarians may feel the time pressure of evidence-based practice. It should not, however, be viewed as an add-on, another thing to do on top of busy schedules. Evidence-based practice is about best practice and reflective practice, where the process of planning, action, feedback and reflection contributes to the cyclic process of purposeful decision making and action, and renewal and development.
The argument that evidence-based practice is an add-on to the day-to-day work, particularly where there is limited staffing or where there is only one professional teacher-librarian in the school begs the question: what is the day-to-day work of the teacher-librarian? As established above, evidence-based practice assumes an acceptance of the notion that the provision of information makes a difference; that instructional intervention contributes to these outcomes, and that this instructional intervention is a central dimension of the role of the teacher-librarian. Rather than detracting from the day-by-day work of the teacher-librarian, evidence-based practice, with its clear focus on learning goals, processes and strategies to achieve those goals, and approaches to identifying nature and scope of the learning outcomes, is more likely to add clarity and focus to the day-to-day work. It gives emphasis to identifying effective actions, putting value on appropriate actions rather than actions for the sake of doing something. It is sharper practice -- more focused and productive practice. Explicit consideration of outcomes, and how these might be achieved and identified, makes evidence-based practice an effective guide to day-to-day practice.
(4) Evidence-based practice is contrary to lifelong learning. This issue was raised by one respondent. It was posited that "EBP is unrealistic, given the goal of lifelong learning that information literacy is all about. How can one realistically measure this outcome, especially when it may not be evident for many years?" This thoughtful comment reflects a misconception of what lifelong learning actually is. Lifelong learning is not some distant endpoint, it is a process made up of multiple moments in time, from now to then. If teaching for lifelong learning is part of the rhetoric of teacher-librarians and an integral dimension of school library programs and policies, then to dismiss charting its progress because it is some vague process makes a mockery of its importance. Providing learners with a clear understanding of how they in their formative years are actually learning in an information rich environment, particularly in terms of information literacy outcomes and indicators, providing them with feedback on their mastery, enabling them to refine their learning processes are essential components of lifelong learning, and thus fundamental to the work of teacher-librarians. If indeed the notion of lifelong learning is some elusive rhetoric, and we are unable to provide substance as to how we might enable our students to become lifelong learners, giving them explicit feedback and input along the way, then we are doing considerable disservice to our students. The rhetoric of lifelong learning must not become the scapegoat for not engaging in evidence-based practice.
(5) Lack of knowledge and skills to undertake evidence-based practice. This concern was expressed by all respondents:
"I lack the skills in devising accurate assessment tools";
"I need lots of practice with this to develop my skills";
"It would be nice to have access to some recent criterion-referenced or standardised tests to assess my students' standards and progress. This is really needed if we are to engage in evidence-based practice";
"I feel completely unqualified to accumulate sufficient or accurate evidence about what I do, or hope I am doing";
"I need to learn to write more performance descriptors";
"It would be really helpful to have some school-wide information literacy tests";
"There are limited training opportunities available to develop new skills, initiatives or approaches to implementing EBP".
These comments highlight real needs if the profession is to engage in evidence-based practice, and identify a range of specific themes around which ongoing professional development can be structured. There are implications for teacher librarianship education, particularly in developing both a rationale for, and skills in carrying out evidence based practice. There is a golden opportunity for professional associations to provide the appropriate professional development to its members. I have been so impressed with the work done in relation to this by SLAV, the School Library Association of Victoria, whose professional development program for 2003 titled "Engaging learning across the curriculum" gives considerable focus to evidence-based practice.
Evidence-based practice is about opportunities and options. Some teacher-librarians may say "why bother, it's futile", believing that such calls for evidence-based practice represent faddism or short-lived hype; that it may not do any good. This is a defeatist attitude. The more confrontational question is asking: "what are the potential implications and outcomes of not engaging in evidence-based practice?" One respondent make this thoughtful comment: "No change in the current situation for teacher-librarians will be forthcoming until they can successfully demonstrate and document evidence of their support, success and impact on children's literacy, with all its ramifications". If the answer to this question is a dismal perspective on the status quo, and if there is no personal motivation to engage in professional initiatives that might enable the profession to construct as preferred future, then the issue is a personal one that poses the question: "Is my role as a teacher-librarian a liability or a liberator of the profession"? If we are not prepared to commit ourselves to initiatives that have the potential to create a bright future for the profession, then we seriously need to consider why we are in it, and what in fact we might be better off doing. Retreating to a position of no hope is retreating to a short future for the profession.
At this time in our profession, it is not enough to just say that the library is important, nor is it enough to say that there is plenty of evidence out there -- why should I waste valuable library time getting mine? At the IASL conference in Auckland where I first presented the concept of evidence-based practice, I made the comment that many school administrators, school boards and parent communities are looking for tangible, documented evidence of the impact of their library on student learning, and use this as a basis for providing more library funding, technology, staffing. In a recent study published in School Library Journal (Lau, 2002:53) which explored Principal's perceptions of teacher-librarians, it was found that only 37% of principals said that the teacher-librarian made them familiar with current research of library programs and student achievement, and only 35% of principals were made familiar with current research on reading development. Principals and administrators want to know about student outcomes. The opportunity to identify local outcomes and local successes and to share these with school stakeholders is knocking. Such evidence can be the basis for richer, meaningful discussion between stakeholders -- students, parents, and community.
American Library Association and Association for Educational Communications and Technology. (1998). Information Power: Building Partnerships for Learning.
Boschee, F. & Baron, M. (1994). OBE: Some answers for the uninitiated. Clearing House, 67 (March/April), 193-96.
Callison, D. (2001). The Twentieth-Century school library media research record. In: A. Kent 7 C. Hall (Eds). Encyclopedia of Library and Information Science Vol 71 Supplement 34. New York: Marcel Dekker, 339-369.
E-BEUK: Evidence-Based Education, UK. (2002) Durham University's Curriculum, Evaluation and Management Centre. Available at: http://cem.dur.ac.uk/ebeuk/manifesto.htm
Flanagan, J. (1954) The Critical Incident Technique. Psychological Bulletin (July), 51(4), 327-358.
Gordon, C. (2000). Information Literacy in Action. Saxmundham, UK: John Catt Educational
Haycock, K. (1994) Research in school librarianship and the institutionalization of change. School Library Media Quarterly, 23, 227233.
Hein, G. (1991). Constructivist learning theory. Paper presented at the CECA (International Committee of Museum Educators) Conference in Jerusalem, Israel, 15-22 October 1991. Institute for Inquiry. Available at: http://www.exploratorium.edu/IFI/resources/constructivistlearning.html
Kuhlthau, C. (1993). Seeking meaning: a process approach to library and information services. Ablex.
Kuhlthau, C. (1999). Student learning in the library: what Library Power librarians say, School Libraries WorldWide 5(2), 80-96.
Lance, K. C. (2001). Proof of the power: Recent research on the impact of school library media programs on the academic achievement of U.S. public school students. ERIC Digest. EDO-IR-2001-05 October 2001. Syracuse, N.Y.: ERIC Clearinghouse on Information & Technology.
Lau, D. (2002). What does you boss think about you? School Library Journal, September, 52-55
Lorenzen, M. (1999). "Using Outcome-Based Education in the Planning and Teaching of New Information Technologies." In Information Technology Planning. Edited by Lori A. Goetsch. Binghamton, NY: Haworth Press, Inc, 141-152
Loertscher, D., & Woolls, B. (2002). Information literacy research: a review of the research: a guide for practitioners and researchers. 2nd ed. Hi Willow Research and Publishing, 2002.
Oberg, D. (2001). Research indicating school libraries improve student achievement. Access, 15(2), 11-14.
O'Rouke, A. (1998). An introduction to evidence-based practice. Available at: http://www.shef.ac.uk/uni/projects/wrp/sem3.html
Schwarz, G. and Cavener, L.A. (1994). Outcome-based education and curriculum change: Advocacy, practice, and critique. Journal of Curriculum and Supervision, 9(4), 326-38.
Spady, W. and Marshall, K. (1994). Light, not heat, on OBE. The American School Board Journal, 181 (November), 29-33.
Todd, R. (1995). Integrated information skills instruction: does it make a difference?. School Library Media Quarterly 23(2), 133-139.
Todd, R. (2002). Evidence-based practice I: The sustainable future for teacher-librarians. Scan, 21(1), 30-37.
Todd, R. (2002). Evidence based practice II: Getting into the action. Scan, 21(2), 2002, 34-41.
Todd, R. (2001). Transitions for preferred futures of school libraries: knowledge space, not information place; connections, not collections; actions, not positions; evidence, not advocacy. Keynote address: International Association of School Libraries (IASL) Conference, Auckland, New Zealand, 2001. Keynote paper, IASL conference 2001 virtual session: paper from Ross Todd.
Towers, J.M. (1996). An elementary school principal's experience with implementing an outcome- based curriculum. Catalyst for Change, 25 (Winter), 19-23.
Wilson, B. (1997). The postmodern paradigm. In C. R. Dills and A. Romiszowski (Eds.), Instructional development paradigms. Englewood Cliffs NJ: Educational Technology Publications. Also available at http://www.cudenver.edu/~bwilson/postmodern.html
Ross Todd has provided the following points for discussion during the virtual conference:
1. Your reaction to these ideas.
2. What impact does your school library have on student learning? What outcomes does it enable, particularly in terms of: information literacy outcomes, curriculum / knowledge outcomes, reading and literature outcomes, personal agency and life outcomes?
3. What strategies / initiatives / processes do you use to accumulate this evidence?
4. What are your most successful strategies to build influence in your school?
Dr Ross Todd
Read Ross Todd's
Information and Instructions
Read the comments
of other participants