How and Why Libraries Are Changing: What We Know and What We Need to Know
by Denise A. Troll
Copyright © 2002 by The Johns Hopkins University Press, Baltimore, MD 21218.
NOTICE: THIS WORK MAY BE PROTECTED BY COPYRIGHT
YOU ARE REQUIRED TO READ THE COPYRIGHT NOTICE AT THIS LINK BEFORE YOU READ THE FOLLOWING WORK, THAT IS AVAILABLE SOLELY FOR PRIVATE STUDY, SCHOLARSHIP OR RESEARCH PURSUANT TO 17 U.S.C. SECTION 107 AND 108. IN THE EVENT THAT THE LIBRARY DETERMINES THAT UNLAWFUL COPYING OF THIS WORK HAS OCCURRED, THE LIBRARY HAS THE RIGHT TO BLOCK THE I.P. ADDRESS AT WHICH THE UNLAWFUL COPYING APPEARED TO HAVE OCCURRED. THANK YOU FOR RESPECTING THE RIGHTS OF COPYRIGHT OWNERS.
Carnegie Mellon University Research Showcase @ CMU
University Libraries
January 2002
Carnegie Mellon University, troll@andrew.cmu.edu
Follow this and additional works at: http://repository.cmu.edu/lib_science
Part of the Other Social and Behavioral Sciences Commons
This Article is brought to you for free and open access by the University Libraries at Research Showcase @ CMU. It has been accepted for inclusion in University Libraries by an authorized administrator of Research Showcase @ CMU. For more information, please contact researchshowcase@andrew.cmu.edu.
Published In portal: Libraries and the Academy, 97- 121.
In November 2000, the Digital Library Federation (DLF) and the Council on Library and Information Resources (CLIR) commissioned a white paper to initiate discussion of how and why libraries are changing. Eight academic library directors met with representatives from the DLF and CLIR in March 2001 to discuss the issues. The outcome of this meeting was a proposal to conduct research that will begin to fill significant gaps in our understanding and to better position libraries to meet the needs and expectations of academic users and university and college administrators. The white paper and the forthcoming research should be of interest to anyone trying to cope with or make sense of transformational change in academic libraries. To reach a broader audience, the white paper has been revised and updated for publication in portal: Libraries and the Academy. The article examines the importance of understanding how and why libraries are changing, analyzes the limitations and difficulties of traditional library performance measures, and explores environmental factors that may help account for why library use is changing. It concludes with an overview of research designed to develop an understanding of how user behavior and preferences affect demand for and use of library collections, services, and facilities, and a call to contribute conscientiously to the legacy of academic libraries and librarianship.
The Problem
Academic libraries are changing in response to changes in the learning and research environment and changes in the behavior of library users. The changes are evolutionary. Libraries are adding new, digital resources and services while maintaining most of the old, traditional resources and services. Finding and funding the appropriate balance of digital and traditional initiatives challenges strategic and financial planners. Library administrators feel pressured to respond to the transforming needs and expectations of users, and in some cases, are pressured by university and college administrators to account for their expenditures and demonstrate the outcomes they achieve. Data are required to respond to these pressures, find the appropriate balance, and plan for the future. Libraries need data to justify their existence, secure resources, advocate their initiatives, and know what’s happening. We need data that illuminate not only how, but also why libraries are changing if we are to explain shifting patterns of library use, envision the trajectory of our evolutionary path, and win or bolster support for the changing directions of academic libraries.
What do we know about how and why libraries are changing? We know something about how libraries are changing, but with close scrutiny must admit that even this knowledge may be incomplete and uncertain. We know almost nothing about why libraries are changing because our traditional data collection practices tend to be myopic, counting selected activities within our purview and relying on anecdotal evidence about the larger context in which we operate as a basis for interpreting our data. If our discipline is library science, we’ve overlooked some significant independent variables that influence the dependent variables of library use. Let’s examine what we know about how libraries are changing. Traditional measures quantify a library’s raw materials or potential to meet user needs (inputs), and the actual use of library collections and services (outputs). Input and output statistics reveal changes in what libraries do over time; for example, they provide a longitudinal look at the number of books purchased and circulated per year. Traditional approaches to measuring inputs and outputs focus on physical library resources. Libraries are struggling with what to measure and how to measure inputs and outputs in the digital environment. We currently have no standard, comparable data to assess digital library trends within or across academic libraries. Similarly, usage data from commercial vendors of electronic resources cannot be compared easily because they measure or define the data differently. Even input and output data on traditional library resources reflect these problems. The data that libraries gather may not be consistent within or across institutions, so interpretation is difficult and the value of time-series trends and peer comparisons is dubious.
In the parlance of traditional library performance measures, the purpose of all inputs and outputs is to achieve outcomes. Outcomes are measures of the impact or effect that using library collections and services has on users. Good outcome measures are tied to specific library objectives and indicate whether these objectives have been achieved. [1] Outcomes assessments may indicate how well user needs are being met, the quality of library collections and services, the benefits or effectiveness of library expenditures, or whether the library is accomplishing its mission within the larger institution. Outcomes assessments can be difficult and expensive to conduct. For example, how do you articulate and assess a measure of the library’s impact on student learning and faculty research? Libraries have no standard definitions or instruments with which to make such assessments and no source of aggregate or contextual data to facilitate comparing and interpreting their performance. If university and college administrators do not require outcomes assessments, libraries may not pursue them. What we know about how and why library outcomes are changing is miniscule and speculative.
Traditional library performance measures do not cover the full scope of how libraries are changing or explain why these changes are occurring. For example, trend data indicate but do not explain why library use varies in relation to library size. Explanations of why libraries are changing require contextual information and interpretive techniques that we currently do not have. The tasks of interpreting and comparing data are confounded by different institutional goals and local library policies. Differences in institutional mission affect not only support for and (therefore) use of the campus library, but the library’s commitment to data collection and analysis. Confronted with these difficulties and yet clamoring for some vision of what is happening, the tendency is to aggregate existing heterogeneous data to reveal trends, then interpret the normalized data cautiously because they may be misleading. Even if these complicated problems were solved, traditional library measures would still provide an incomplete picture of the information landscape because they focus strictly on information services provided by libraries, ignoring information services provided by other entities on or off campus. Real library science requires examining activity beyond our walls and websites — activity that affects and helps explain behavior within our walls and websites.
The absence of standard definitions and procedures for gathering and interpreting reliable information that would enable us to document and to explain shifting patterns in library operations and use is adversely affecting strategic planning and the cases that library directors must make to win or bolster support for the library and its changing directions. Academic libraries cannot prepare effectively for the future or position themselves on campus until they understand their changing roles in the current learning and research environment, which is radically different from the environment a decade ago. Understanding and evaluating library usage patterns and developmental paths are prerequisites to formulating a critical and appropriate response to widespread, rapid changes in higher education. Arriving at this understanding absolutely requires a thorough examination of library assumptions and practices, and an exploration of changes in libraries and in the larger context in which libraries operate.
Academic libraries may be gathering data because they are easy to gather or because they have always been gathered, rather than gathering data that inform clearly articulated purposes or important decisions to be made. We need to gather meaningful, useful data. We do not have the resources or mandate to do otherwise. Yet what are we doing? Though ARL explicitly discourages interpreting quantitative rankings as indicative of performance or an assessment of quality, [2] we continue to use traditional measures of library inputs, outputs, and calculated ratios to rank and compare libraries. We do this along lines well entrenched in the profession, but invest little if any effort in surfacing the assumptions behind these ratios and comparisons or gathering the contextual information that would enable library administrators to interpret the data critically and apply them to their current strategic planning and case-building efforts. What does it mean, for example, to know that X number of books was added to the collection this year, Y materials were circulated, or Z reference questions were answered? Though ratios that relate traditional output measures to the size of the campus community facilitate comparison, what good is it to know the ratio of total volumes or library staff to the student population? Are we assuming that more is always better than less? Is it necessarily bad if market penetration of interlibrary loan or reference service reaches only a small percentage of students and faculty? Is there some magic formula for allocating the appropriate percentage of a library’s total budget to materials, staff, and operating expenses that will guarantee library outcomes in line with the university’s mission? Ratios and percentages may be interesting indicators of local trends or progress toward local goals, but what do they really mean for the future of libraries and librarianship? Web server statistics are another case in point. The data easily gathered by web servers may be interesting, but their meaning and application are elusive. Is the number of hits on a web page low because the page is unnecessary or because it is difficult to find? In what context are numbers about total database sessions, web page hits, and bytes transferred meaningful or useful?
New technologies have rendered traditional measures less effective in explaining what is happening in libraries. In response, many organizations are revising or amending their definitions and data points for monitoring libraries. For example, the Association of Research Libraries (ARL), Institute for Museum and Library Services (IMLS), National Information Standards Organization (NISO), National Commission on Libraries and Information Science (NCLIS), and EQUINOX project are all examining library performance measures, including new measures for the networked environment. [3] The Council on Library and Information Resources (CLIR) and International Coalition of Library Consortia (ICOLC) are also working on new measures for the networked environment, focusing on statistical measures for licensed resources provided by commercial vendors. [4]
Traditional library performance measures fail to explain fully what’s happening in libraries today because their scope is too narrow to encompass the field of change. For example, traditional measures do not capture sufficiently the readily apparent changes in the definition, preservation, and delivery of library collections. In the past a “collection” was what the library physically owned. Records in the library catalog referred to items in the collection. Libraries now license access to remote electronic collections that they do not own. The library catalog contains records with interactive URLs pointing to these licensed items. If a print subscription is cancelled, the library retains ownership of the (previous) physical volumes. If an electronic subscription is cancelled, the library often does not retain access to the (previous) digital volumes. In the past, multiple purchasing or subscribing libraries in effect archived and preserved print publications. In the digital arena of licensed access, few libraries continue to play this role and they do so only through negotiated agreement with the publisher. Traditional library performance measures do not reveal these significant changes or the real concerns or serious implications that arise from them.
Commercial publishers and information aggregators play a more active role in collecting, organizing, and preserving information in the digital environment than they did in the print environment. As the work of librarians in this arena declines and new technologies change relationships among libraries, publishers, authors, and artists, the role of libraries is shifting. Libraries are publishers when they digitize collections, host journals that are “born digital,” or assemble student or faculty works online. [5] Librarians are politicians when they lobby faculty authors not to sign away copyright to a print publisher, who then requires them or the library to pay for use of their own works. They are teachers when they help users develop information retrieval and evaluation skills in the digital environment and assume greater responsibility for the learning and research outcomes of their institution. They are researchers when they conduct user studies to assess user needs and expectations or the usability of their digital resources and websites. Librarians are expected to employ a greater variety of research methods in the digital environment than they did in the print environment (for example, user protocols, card-sorting studies, cognitive walk throughs, heuristic evaluations, and paper prototyping), and to collaborate with a wider range of people than in the past, including computer scientists, graphic designers, pedagogy experts, archivists, and museum curators. The core competencies required to perform these new tasks are different from those required of librarians in the traditional print environment. Traditional library performance measures do not—because they were not designed to—capture these changes or their implications.
New technologies are also changing the services that libraries provide; for example, online reference and instruction, desktop document delivery, self-checkout, and userinitiated library loan and direct borrowing. Librarians disagree about whether the shift to user-initiated services is a needed simplification of library operations or a cheapening or devaluing of library services analogous to fast food. [6] Usage statistics and cost analyses of these services are not readily available, but even a simple change in service can have significant impact on library operations. For example, the shift to providing email notices of overdue books and enabling online renewals resulted in a significant drop in revenue from fines in Carnegie Mellon University Libraries. According to a survey conducted by the Digital Library Federation, several libraries experienced a significant decline in revenue from fines and photocopying when electronic reserves were implemented. [7] Again, traditional measures do not—because they were not designed to—capture these changes or their implications.
Whether the fundamental mission of libraries has changed may be a matter of interpretation or local policy, but the environment and circumstances in which libraries pursue their mission is dramatically different from the environment and circumstances of the past. In contrast with library philosophy and practice centuries ago, libraries today often acquire materials and offer services “just in time,” rather than “just in case.” The cost of access to information appears to be more affordable than the cost of ownership. The escalating cost and volume of publications over time, widespread adoption of technology, and reduced barriers to Internet access may account for the trends of licensing access, increased use of interlibrary loan, and speculation that the digital divide is disappearing. [8] If the access model continues to offer more information at less cost to an increasing number of people, the worst-case scenario would reserve the ownership model for only high-cost, low-use materials. [9] Though this worst-case is unlikely to happen (at least in our lifetime), the access approach to acquiring library materials is risky business. If no one purchases or preserves the materials, interlibrary loan is not viable. If no one archives and moves digital collections to the inevitable new formats and platforms that the future will bring, access to them will be lost when the hardware and software become obsolete. [10] Libraries may have new measures to capture new inputs (for example, number of licensed databases, full text e-journal or e-book titles), but the measures do not capture the implications of the shift in library practice from purchased ownership to licensed access.
Traditional library measures indicate some differences among libraries and changes over time within libraries, but in the absence of additional standardized library measures, the consideration of contextual factors, and clearly articulated assumptions, these measures offer nothing that will help us recognize which differences or changes are significant in terms of fulfilling our mission and serving our constituencies in higher education. The situation is critical, but not necessarily dire. With additional knowledge, we can explain to university and college administrators how and why libraries are changing, demonstrate that our efforts contribute substantially and cost-effectively to the mission of the institution, and engage them in planning support for our future position on campus. To do this we need to discuss the issues, to examine our assumptions, to set strategic goals, to conduct research that will provide the information necessary to identify and to explain significant changes in library operations and use. Library directors must understand how and why libraries and library use are changing, must articulate and assess proficiencies for users and staff, and should plot a course into the future that is flexible enough to cope with the speed of change precipitated by information technologies and the Internet.
According to Raymond Kurzweil, the rate of technological progress currently doubles every decade, so a hundred years of progress at the current rate will happen in twenty-five years. Change itself will reach an exponential rate of growth by 2015. [11] We know that we cannot plan effectively for the future by projecting increased access to current technologies, but we dare not be discouraged. We must not allow the speed of change to inhibit or to paralyze our attempts to make sense of what’s happening in libraries and intervene to serve our constituencies. Though admittedly change sometimes occurs so quickly that by the time a research project is completed, the results are no longer applicable we must begin now to understand how and why libraries and library use are changing if we want to position libraries effectively in higher education a decade from now. [12]
Trends in Traditional Library Performance Measures
What do we know about how and why libraries are changing? All that we appear to know with confidence is what trends have emerged in traditional library inputs and outputs. The trend data are indicative but not explanatory of change. They are difficult to interpret because they lack context and are rife with hidden assumptions. Nevertheless, reasonable speculation abounds to account for the trends. The trends discussed in this paper are based on an examination of aggregate ARL data, reports from selected Oberlin Group libraries, and the recent survey of usage and usability practices at DLF member institutions. [13] The points address trends over the past five years and do not necessarily reflect every library’s experience. Library trends and the contexts for interpreting them are significantly different across institutions.
Trends in Traditional Input Measures
Budgets. With rare exception, over the past few years, libraries appear to have experienced slight increases in both their materials budget and educational and general operating (E&GO) budget. Some libraries are given additional funding when their institution creates new degree programs or research centers. Other libraries are expected simply to stretch their existing materials budget to cover the new information needs. What’s happening with staff salary lines is unclear, but even when salary lines are increased, they seldom keep pace with inflation. Current budgets appear to be insufficient to simultaneously:
• Keep pace with the skyrocketing cost of materials
• Maintain traditional resources and services
• Mount new digital resources and services
• Purchase and replace an increasing array of technologies that rapidly become obsolete
• Recruit or retrain staff with the requisite skills to use and maintain the technologies
• Accommodate the shifting need for space and supplies precipitated by increased automation and use of digital resources
According to the 1986 Standards for College Libraries published by the Association of College and Research Libraries (ACRL), “to sustain the range of library programs required by the institution” library budgets “shall be six percent of the total institutional budget for education and general purposes,” excluding capital costs and the costs of physical maintenance. The appropriation should be more than 6 percent if the library is responsible for acquiring, processing, and servicing audiovisual materials and microcomputer resources. [14] Another rule of thumb for a successful library operation is that the percentage of increase in library budgets each year should equal or exceed the percentage increase of tuition and fees. [15] Regardless of these guidelines, many libraries receive budget increases smaller than the suggested percentages, and there is no evidence that they are not providing the resources and services their institutions expect. Perhaps this is why arguments that the library is entitled to such budget increases fail with university and college administrators and why the January 2000 revision of the ACRL Standards does not make these claims. [16]
Endowments, monetary gifts, and different campus funding formula complicate efforts to understand the financial situation of academic libraries and the implications of their budgets. Aside from these considerations, library expenditures do not provide administrators with a complete picture of the cost of library resources and services in a networked environment. The cost of the campus-computing infrastructure must also be taken into account. Traditional library performance measures and unsubstantiated claims about the cost of successful library operations are inadequate to convince university and college administrators to increase the library’s budgets. Libraries need to define and determine how to measure what constitutes a successful library, and to figure out what success costs in a particular campus context and what percentage of that cost is to be provided by the institution, endowments, gifts, grants, etc.
Collections. To keep pace with increasing user demands for more desktop delivery of materials, libraries have been spending a growing percentage of their materials budget on licensing access to electronic resources. Over the past five years, some libraries have increased the number of serial and monograph titles purchased. Many libraries, however, have canceled serial subscriptions and purchased fewer monographs during this same period. Obviously collection growth is constrained by the financial situation of the library. The increased cost of materials, a more effective distribution of materials through collaborative purchasing (via consortia, organizational networks and inter-institutional agreements), or a shift in user expectations may explain the decline in acquisitions. “Just in time” information delivery may be becoming an acceptable replacement in some cases for the traditional “just in case” archival imperative. Some libraries may be operating with fewer physical volumes per student than in the past, but there is no contextual information in which to determine if this is good or bad for these library users. The belief is that users have access to more information overall. Martha Kyrillidou describes an “increasing and widening gap between the ‘access’ model libraries and the ‘ownership’ model libraries,” which may simply be a way for libraries to focus on what they do best “rather than trying to be everything to everybody.” [17]
Staff. Overall staff size appears to have been declining slowly over the past five years, but many library systems departments are hiring more people to maintain the equipment and software. Libraries are also hiring user interface designers to help develop their websites and researchers to conduct user studies, assess organizational effectiveness, and manage library statistics. Clerical positions are being eliminated throughout the library and positions are being created or reclassified at higher levels (with higher salaries) because more sophisticated technical or managerial skills are needed now than a decade ago. Typically, new or upgraded positions are accommodated by combining open positions because salary lines are insufficient to do otherwise. The need to retrain staff to keep pace with technological change adds the burden of finding increased funding for travel and training in an already strained budget. [18] Some libraries may be operating with fewer staff per student than in the past, but again there is no context in which to determine if this is good or bad. It may be fine in some library environments, but not others.
Equipment. Most libraries have replaced the card catalog with an automated system that runs on a powerful server. Microform equipment and photocopiers have been augmented with other computer hardware, software, systems, and peripherals, for example, to view and print full-text journal or newspaper articles. Many libraries have capital budgets insufficient to replace all of this equipment before it becomes obsolete. Equipment purchased with one-time funding from grants adds to the burden of replacement costs. Tracking all of the equipment and planning its replacement is a tedious task. Aside from these concerns, traditional measures do not help libraries with equipment planning. They provide no contextual information that would facilitate calculating how many computers the library should provide for public use. Relevant data to inform this decision would include the number of students who own a desktop or laptop computer, the number of computers available in public clusters or laboratories on campus, the number of electronic resources provided by the libraries, the purpose and configuration of public computers in the library, and the use of the library by the general “walk-in” public. Lacking this information, difficult and time-consuming queuing studies may be required to determine whether the number of public computers in the library is appropriate for the user population.
Space. Many libraries have been reducing or eliminating reader and staff spaces for years to accommodate growing physical collections. Many use or are considering offsite storage to solve their space problems (and wondering how to fund offsite storage from an already strained budget). Current library standards address different types or purposes of space and the square footage to be allocated to them. The standards provide guidelines for the dimensions of a designated reader, staff, or collection space, depending on its purpose, and guidelines for how to calculate the need for each type of space based on the type of library. For example, guidelines describe how to calculate the number of lounge chairs, individual study carrels, and group study tables a residential campus should have based on the campus population. The standards do take into account the space occupied by technology (computers, printers, scanners, fax machines, etc.), and the classroom space required for library instruction. [19] Often library spaces are smaller or fewer than the standards or guidelines propose. Whether this is good or bad depends on the local context. For example, at an institution where 95 percent of the students own their own laptop, 75 percent of the use of electronic resources is remote, and public computers in the library are seldom used by people not affiliated with the institution, it’s probably fine for the library to provide fewer readers spaces and public computers than the guidelines suggest. However, on a residential campus where few students own computers, few computers are available for student use elsewhere on campus, and many people unaffiliated with the institution use the computers in the library, it is probably not good for the library to have fewer reader spaces and public computers than the guidelines propose.
Traditional measures or standards for library space allocation have been ineffective if not irrelevant in efforts to convince university and college administrators that the Internet and digitization are not a near-term method of accommodating collection growth or solving a space shortage in the library. Rough calculations performed at Carnegie Mellon indicate that licensing and creating digital content will not begin to alleviate overcrowding in the libraries until 2006, with meager space savings of 16 percent. Libraries must digitize 100–125 volumes at a cost of (at least) $2500 to free ten square feet of floor space. [20] We need to understand how the licensing and creation of digital content affect use of library facilities, the need for collection, reader and staff spaces, and the appropriate allocation of library space for the campus.
Trends in Traditional and Emerging Output Measures
Materials circulated. Use of video and other media appears to be increasing. Depending on the local context, circulation and in-house use of print resources may be increasing or decreasing. Aggregate ARL data indicate an overall decline in circulation in large libraries. However, Oberlin Group libraries report substantial increases in circulation, which suggests that library size may be a factor. We lack sufficient contextual information and data on the use of digital resources to interpret and to understand trends in the use of physical library materials. If users can find an electronic book or journal or something comparable or good enough for their purposes on the Web, when and why would they come to the library to check out a printed book or use a printed journal? In the absence of composite measures of traditional and digital library resource use and any data about student and faculty use of information resources provided by entities other than the library, what does a decline in circulation really mean in terms of supporting education and research? Traditional library performance measures cannot explain the trends they reveal.
Reserve items circulated. The circulation of print reserves appears to be declining rapidly, even in institutions that do not offer electronic reserves. Data on use of electronic reserves are not readily available. In some institutions faculty are putting fewer materials on reserve. The availability of full-text resources on the Web—provided by the library or by other entities—may account for the decline in traditional reserve items and usage. Faculty may be providing pre-printed course packs or mounting full-text materials in course management software like BlackBoard, thereby eliminating use of the library but accomplishing the same purpose. [21] The decline in reserve use could also simply reflect the behavior of current students. [22] In the absence of usage data on other information sources, what does a decline in the use of reserves mean for the quality of education and research? Again, traditional library performance measures cannot explain the trends they reveal.
Electronic reserves are popular with many students and faculty, but the added value of desktop delivery is accompanied by added costs and the loss of revenue. The cost of providing electronic reserves entails equipment, staff training, and staff time to scan, store, link, and track use of the materials. At some institutions, additional costs are incurred in seeking copyright permission to digitize the materials and to support free printing of e-reserves items in the library. Revenue declines from photocopying traditional reserve items and fines for overdue materials. Concerns about user acceptance, technical problems that could interfere with access, and the loss of revenue in some cases are sufficient to warrant running a dual system of both traditional and electronic reserves.
Reference questions answered. Use of reference service has been fluctuating for several years. It appears to have dropped significantly in the past year, but one year of data does not constitute a trend. Furthermore, gathering reference statistics is fraught with difficulties and disparities. Traditional face-to-face reference service with a librarian is being transformed by information technologies deployed to reach an increasingly remote audience, for example, electronic mail, web-based forms, and “chat” or “see you, see me” videoconferencing software. Usage statistics from these various venues may not be compiled and reported. Reference statistics may reflect only questions asked at the reference desk. Whatever venues are included in reference statistics, the annual statistical report may be projected based on sample data gathered periodically. The result can be an incomplete and misleading picture of reference service use.
Why is reference service fluctuating and what does it mean? Traditional library performance measures cannot answer this question. Is reference service being challenged by reference-like services provided by entities outside of the library? Why, for example, would users ask a reference librarian when they can Ask Jeeves, Allexperts.com, or one of the many other Ask-A services proliferating on the Web? In the absence of usage data and quality assessments of these other services, what does fluctuation or decline in the number of reference questions answered in or by the library mean for the quality of education and research? Libraries appear to have incomplete data about the reference service we provide and insufficient contextual information to interpret the data we do have.
Interlibrary loan transactions. Use of interlibrary loan (ILL) is increasing, in many cases dramatically. We have substantial information about the use, costs, and quality of ILL. [23] We know, for example, that new technologies are transforming ILL, blurring the lines between ILL and document delivery services, shifting costs, and increasing user self-reliance. Initial increases in costs for staff training, hardware, and software (like ILLiad and ARIEL) exceed decreases in staff costs associated with photocopying and mailing. However, after the start-up costs are absorbed, staff productivity improves and the cost of patron-initiated ILL is significantly less than the cost of traditional ILL. [24] The added value to users of initiating their own ILL requests, having materials delivered to the desktop, and the ability to track their requests online no doubt increase user satisfaction. What we don’t know with certainty is why ILL is increasing. What is the relationship between the use of ILL and cancelled journal subscriptions, purchasing fewer monographs, the provision of online library catalogs and citation databases that index materials not owned by the library, improved service quality, and increased user self-reliance? Furthermore, how do the nature and mission of the institution and the character of the users it serves influence this relationship?
Library instruction classes. The number of traditional library instruction sessions and participants was increasing until recently, but now appears to be on the decline, perhaps because distance-learning technologies are being deployed to deliver library instruction. Libraries are beginning to gather statistics on use of online instructional materials, quizzes, and tutorials. Other environmental factors that may be affecting library instruction are the increasing technological savvy of users who can transfer skills from one vendor’s databases to another, or student and faculty use of information resources and services not provided by the library, in which case library instruction is not in step with user needs or behavior. Again, we lack the necessary contextual information to interpret what a decline in library instruction sessions or participants really means or why it is happening.
Gate counts. Gate counts in many libraries are declining. Why go to the library if you can find the information you need using your personal computer, create an Internet chat room to discuss your group project, or use a similar discussion facility provided in course management software like Blackboard that integrates your class syllabus, assignments, readings, quizzes, and grades? Yet gate counts seem to increase following renovation of library spaces, which suggests that the library is still valued as a physical place. What is the relationship between gate counts and the attractiveness and suitability of library space for student and faculty work or recreation? What is the relationship between gate counts, the volume of electronic resources provided by the library, the penetration of computer ownership, and the quality of the network infrastructure on campus? We can speculate all we want, but traditional library performance measures do not provide the contextual information we need to understand trends in gate counts or the impact of gate counts on the quality of life, student learning, or faculty research.
Electronic resource use. The demand for desktop delivery of materials appears to be increasing everywhere. Use of electronic resources is growing, often more rapidly than expected. Use of older journal volumes in electronic format is increasing even more rapidly than use of electronic resources overall. [25] Users want access to more full-text electronic resources and many, including some faculty, appear to believe that everything is available in electronic format and that access to the information is free. Libraries are trying to educate their constituencies about the costs of electronic resources, and collaborating in consortia to negotiate with vendors for better prices and more meaningful usage statistics. The problems associated with vendor statistics are well known and many organizations, including the Council on Library and Information Resources (CLIR) and the International Coalition of Library Consortia (ICOLC), are working with vendors to create standard definitions, methods of measurement, and delivery formats to facilitate comparative analyses.
Printing and photocopying. The trend in many libraries is that the volume of printing is increasing and the volume of photocopying is decreasing. Librarians speculate that this is because of the increased availability and printing of full-text e-journals and ereserves. However, as with all of the measures described in this article, additional contextual information is needed to understand what is actually happening in any given institution. In cases where the library recovers costs for printing, but printing elsewhere on campus is free, printing may drastically decline, while photocopying remains constant. [26] At some institutions where printing is free campus wide, the volume of printing in the library has leveled off or slightly declined over the past year or two, but photocopying continues to decline dramatically. For example, at Carnegie Mellon University Libraries, photocopying has declined by approximately 50 percent over the past four years. Printing was increasing rapidly until 1998–1999, but has been slowly declining since then. At the University of Washington, the decrease in photocopying does not equal the increase in printing; fewer total document reproductions are being made in the library. We do not understand the relationship between photocopying and printing, or the relationship between printing and the availability of electronic resources or user readiness to read these materials online. We do know that regardless of the volume of printing, more expensive printers and supplies are required now than in the past to accommodate printing color and new file formats like PDF, TIFF, GIF, and JPEG. Assuming that libraries do not offer free photocopying, if they can recover costs for printing, the new income may eventually offset the decrease in revenue from photocopying. The equipment that enables cost recovery for networked printing is expensive. “Cost recovery” in this context probably means strictly the cost of paper and toner. The revenue from printing will probably not fund the replacement cycle for the cost-recovery equipment or the printers themselves.
Trends in Outcomes Assessment
Efforts to measure the impact or influence effect of library collections and services are hampered somewhat because they rely on output measures that are difficult to interpret and on institutional performance objectives that frequently are only vaguely defined and perhaps even contested on campus. Outcomes assessments rely on output measures in the sense that they attempt to measure the achievement of some objective based on the use of library resources—though there’s no way to control or monitor mitigating factors like the use of resources not provided by the library that may influence the objective being measured. Clearly defined performance objectives and methods for measuring their achievement are essential to demonstrate that libraries are accomplishing their mission and contributing to the mission of their institution. Some academic libraries are being pressured to demonstrate the impact of their efforts on student learning and faculty research. Others are not. If libraries are serious about outcomes assessment, librarians need to engage administrators, faculty, and students in a discussion of the outcomes they expect libraries to achieve, then take the lead in articulating measurable objectives and proficiencies to be derived from use of library collections and services given the nature and mission of the college or university.
Libraries are beginning to measure the effect of their collections and services on users. In the absence of pressure or methods to measure clearly articulated objectives or proficiencies, the effort to date has focused on assessments of user satisfaction and service quality. The easiest and most popular measurement is a survey of user satisfaction, but this assessment alone is a “facile outcome” because it provides little if any insight into what contributes to user dissatisfaction. [27] Nevertheless, the trend in these assessments indicates that user satisfaction is a function of the individual user’s perception of the quality of the library’s resources, the competence and demeanor of library staff, and the physical appearance of library facilities. [28] In contrast, service quality focuses on reducing the gap between users’ expectations of excellent service and their perception of the service delivered. Studies of service quality assess the collective experience of many users and suggest that reliability is the most important characteristic of service quality. [29] Though these approaches to outcomes assessment are sufficiently interesting and informative to warrant continued use and development, they may be doomed to failure in terms of winning or bolstering support for the library because the outcomes they assess are not based on objectives or proficiencies aligned with the nature and mission of the institution or integrated with the campus’s overall assessment efforts. Assessments of user satisfaction and service quality are not viable substitutes for assessments of learning and research outcomes. Conducting meaningful and convincing outcomes assessment is the shared responsibility of librarians, faculty, and university and college administrators.
Other dimensions of outcomes assessments relate to costs. Libraries want to analyze both the effectiveness and the benefits of their financial investments in collections, services, staff, and facilities. By identifying areas where expenditures are not efficient or sufficiently beneficial, such analyses can prompt libraries to explore alternatives, streamline internal workflow, and improve the users’ experience. Such analyses also help library directors keep university and college administrators aware of the cost of quality library operations.
Assessments of cost-effectiveness measure the ability of the library to deliver outputs and outcomes with economic efficiency. Understanding the costs associated with library resources and operations is extremely difficult and time consuming. The difficulty of allocating the costs of a particular collection or service to content, staffing, facilities, hardware, and overhead is compounded by the difficulty of distinguishing between start-up costs and ongoing expenditures in an era of rapidly changing technologies, prices, and workflows. A thorough cost analysis of an electronic product, for example, must include purchasing or licensing costs and the operational costs of selecting, ordering, cataloging, networking, printing, associated reference questions, instruction, technical support, and maintaining related web pages and links. Furthermore, such analyses must be repeated to keep abreast of changes. For example, a 1993 functional cost analysis of interlibrary loan indicated that staff costs accounted for 77 percent of the total cost of the service. [30] Given the technological changes and accompanying cost shifts in providing patron-initiated ILL, this analysis needs to be repeated.
Assessments of cost-benefits measure how users subjectively value library resources. Cost-benefit analyses are also difficult and time consuming to conduct. They require a clear definition or understanding of what users value, a method to assess how a particular resource, service, or product feature provides what users value, and some meaningful way to relate this qualitative indicator to a quantitative indication of costs. Service quality assessments can uncover some of what users value, but what is the appropriate role of academic librarians when what users want (value) is not what they need? For example, undergraduate students appear to value convenience and delivery speed more than the quality of the information delivered. Purchasing electronic resources or digitizing information add value, but often increase costs. In an environment where faculty want electronic access but are reluctant to dispense with print, the additional expense of acquiring and maintaining both print and electronic collections is difficult to determine but predicted to be exorbitant and unsustainable. [31] What values can or should libraries support and at what cost?
A study of costs associated with electronic journals conducted at Drexel University illustrates the complexity of doing cost analyses. [32] The Drexel study reveals that providing electronic journals creates significant shifts in staffing and operational costs. While the purchasing power of an electronic journal dollar is greater than that of a print journal dollar because of bundling and backfiles, providing the requisite infrastructure of hardware, software, and systems staff significantly increases operating costs. The cost of staffing increases with electronic journals, partly because library administrators must be involved in negotiating licenses, joining consortia to get better pricing, developing collection strategies, and managing change (restructuring workflow, reorganizing staff positions, and building staff with appropriate skills, including the skills needed to track usage statistics and produce meaningful reports). The cost per unit of processing electronic journals is less than print, but requires staff with computer skills and the ability to adjust to continuous change in procedures. Even if the physical journals are no longer maintained, the increased cost of system maintenance, license negotiation, printing, and reference is greater than the decreased cost in physical collection acquisition and maintenance (including check-in, claiming, circulation, reshelving, weeding, binding, converting to microform or offsite storage). The question remains, is the overall increase in costs incurred by providing electronic journals offset by the added value of desktop delivery to users? We evidently believe that it is, but how do we effectively make that case to university and college administrators to secure the funding required to purchase needed electronic journals?
Difficult tasks take a significant amount of time—which means they are expensive— to accomplish. Librarians and staff are already overburdened with increasing and changing responsibilities. Even if the desired objectives or proficiencies have been articulated and the library can allocate human and financial resources to conduct outcomes assessments, many libraries do not have people with the requisite skills to conduct the research or to present the results effectively in the limited time typically allotted for such presentations to university and college administrators.
In the absence of clear guidelines and supportive models, libraries appear to be doing what is simple. For example, libraries perform simple calculations of the cost-per- search or cost-per-session of electronic resource use, based on the license cost alone, to determine whether to maintain a subscription, regardless of the fact that the calculation is misleading because it ignores the many associated costs and the user-centered qualitative dimension that ought to be the focus of any outcomes assessment. Usage is not synonymous with value. Libraries need to explore alternatives and conduct more cost analyses like the 1998 journal study at Wellesley College to compare the cost-effectiveness of subscribing to a title with the cost of purchasing articles from that journal on an as-needed basis. [33] Such comparative analyses would ideally be accompanied by assessments of user satisfaction or service quality with the different alternatives. Fully informed decisions interpret costs in light of what users value and what libraries are willing to pay to provide that value.
What value do users place on library collections and services? The commercial enterprise called Questia may provide a measure of the library’s dollar value to students. Questia’s success depends on students being willing to pay $20–22 a month for access to 50,000 digital volumes and software tools that facilitate writing their papers. If Questia succeeds, will it help or hinder academic library efforts to illustrate their value and to win support for digital library development? We don’t know the answer to this question in large part because we do not really know what the library is expected to contribute to the institution’s mission. Though libraries have traditionally operated and received budget increases without having clearly articulated objectives and performance measures, we have to wonder whether we will continue to win support for our initiatives in the current strained economy and networked environment of freely accessible web-based information without conducting meaningful and convincing outcomes assessments.