Digital Libraries and User Needs: Negotiating the Future

Anita Coleman and Tamara Sumner*, special issue editors
School of Information Resources and Library Science, University of Arizona, Tucson, AZ, USA
Email: asc@u.arizona.edu
*Department of Computer Science and Institute of Cognitive Science, University of Colorado at Boulder, Boulder, CO, USA
Email: sumner@colorado.edu

Over the past decade there have been many new initiatives in scholarly communication, digital libraries, and the development of educational, scientific and cultural heritage institutions with significant commitments to online resources and online services. For instance, preprint services have transformed the practices of some research communities such as high energy physics (Ginsparg 2001). Other disciplinary communities such as earth science (Domenico et al. 2002), biology (Bowker 2001), and neuroscience (Amari et al. 2002) are focusing on the development of library collections and services aimed at providing real time and archival data to support the conduct of science and sharing of scientific results. Within the United States, new forms of educational institutions, such as the National Science Digital Library (NSDL) program, are aiming to transform primary, secondary and tertiary science education through the provision of educational digital library collections and services (Wattenberg 1998, Zia 2001). Within the NSDL, a rich array of innovative services not traditionally associated with bricks-and-mortar libraries are being developed to support teaching and learning practices, not just information access. These include services supporting personalized content delivery, services for creating digital resources, and communication and collaboration services, to name just a few (NSDL 2003).

The diverse approaches are encompassed under the broad heading of digital library or digital repository initiatives. Yet, each community is embracing different suites of technologies and using them in very different ways. Kling and McKim (2000) note that field differences in the selection and use of digital library and scholarly communication media are not surprising. They describe how large-scale social shaping processes, taking place over many years, influence the selection of new information technologies by scientific and academic disciplines. New technologies are selected very differently based on the community's disciplinary tradition, historical practices, and current ways of working.

As such, social shaping perspectives highlight the critical role that practices, i.e. work practices, social networking practices, and other culturally shaped behaviors, play in influencing technology use and adoption. Indeed, Lynch (2003) argues that the future of digital libraries lies not in supporting generic, broadly useful services such as information access to large collections and knowledge stores, but in supporting "customization by community", i.e. the development of services tailored to support the specific, and real, practices of different user constituencies. Similarly, Borgman (1999) defines digital libraries as being constructed by and for a community of users, with functionalities specifically designed to support their particular information needs.

These insights and definitions are not new. In the early 1990s Project ENVISION attempted to build a prototype digital library of computer science literature by studying user needs of computer science professionals over a four month period. They articulated task-oriented access as an important principle for digital library development (Fox et al. 1993). Project CORE, which stands for the Chemistry Online Retrieval Experiment, was a unique collaboration of many different types of organizations: the American Chemical Society (a learned society), Chemical Abstracts Service (indexing and bibliographic database publisher), Bellcore (a technology research lab), Cornell University and OCLC (a bibliographic utility). They tried to build the chemist's workstation wherein information discovery, browsing, searching, and task-oriented access were enabled through a single, seamless, unified interface. In recent years, instances of such community-based digital libraries are taking hold in a wider variety of settings and disciplines, ranging from K-12 mathematics (Math Forum), Earth system science (Marlino et al. 2001), and higher education (MERLOT).

Within the landscape of community-based digital libraries, an enduring challenge is negotiating the future with diverse community members, i.e. establishing plans and strategies to guide future library technologies, collections, and services. In this context, design and planning requires organizations to take into account the day-to-day practices of individual users, and the needs and desires of the larger community as the library strives to reflect the values and long-term vision that the participants have for their community or academic discipline as a whole. However, community members do not necessarily share the same worldview with respect to values, vision and library priorities, and careful attention must be paid to negotiating diverse worldviews to arrive at a consensus for action (Shumar 2003).

The purpose of this special issue is to consider the spectrum of approaches being used by different libraries and service providers as they negotiate the future with their user communities. At a time when a digital information future is increasingly certain, this timely and much needed collection of articles explores, documents and reflects on the theories, practices, and experiments focusing on digital library users.

More information than ever before is born-digital, i.e. most information produced today has no print equivalent. Moreover, the news about users and their consumption of information in this digital future is troubling. Statistics for the past eight years show that even while the information supply is growing, the amount consumed is barely changing, and is in fact becoming a smaller fraction of what is produced (Lyman and Varian 2003). "We're drowning in a sea of information" according to Varian. "When you look at the challenge we face, how do we manage all this information? Ability to capture all that information has outrun our ability to utilize it effectively." Digital library research, HCI or information design research and individual digital library projects will all benefit from the broader theme that this special issue highlights: the building of digital libraries in cooperation with how users are affected and how their use of information can be improved and transformed by the increasing information supply that the digital library or repository makes possible.

Three approaches to negotiating the future

Each paper in this special issue illustrates a particular approach to understanding user needs, bringing organizational capabilities to bear on these needs, and engaging a broad spectrum of community members in design and governance. These approaches fall into three broad categories:

Each of these are discussed below in further detail.

User-centered design

The central tenets of user-centered design, originating from human factors, are the early and continual involvement of users in the design process, a focus on real users and their tasks, and iterative design guided by frequent formative evaluations (Meister 1985, Booth 1989, Gould et al. 1991). Numerous user-centered design methodologies based on these tenets have been proposed, including task-centered design (Lewis and Rieman 1993), goal-based design (Cooper 1999), and scenario-based design (Rosson and Carroll 2002). All of these methodologies draw on suites of common design practices and formative evaluation techniques, e.g. interviews, observations, prototyping, usability inspections, and usability studies. Many efforts have investigated how standard design and evaluation techniques can be modified to better serve the specific needs of digital libraries and other information technologies such as hypertext/hypermedia for education (Blandford et al. 2004, Budhu and Coleman 2002, McKnight et al. 1996).

Irrespective of the particular methodology chosen, the objectives of a user-centered design process are to develop a deep understanding of user requirements for technology design and planning, and to get systematic user feedback on evolving library systems throughout the design process. It is the responsibility of library designers to generate possible design options and to devise appropriate protocols and studies to elicit user feedback on these options.

In this issue Shreeves and Kirkham describe an early pilot study to test the viability of a search portal created specifically for discovering primary source materials by K-12 teachers-in-training and for use in a classroom. They tested their portal early with users and are candid and blunt in their assessment of the challenges that aggregation of heterogeneous metadata, despite being OAI-compliant, poses to users and service providers. Focus groups, written evaluation, and transaction logs were used to gather data and understand actual usage and user needs. They found that multiple levels of intervention are necessary if aggregation of metadata is to lead to successful user experiences. These findings, because they were early in the development of the portal, helped them redesign it to offer a more successful user experience to a specific community.

Participatory design

Participatory design methods seek to involve users more deeply in the process as co-designers, i.e. users also propose and generate design alternatives as opposed to simply providing feedback on designer-generated options (Greenbaum and Kyung 1991, Schuler and Namioka 1993). An important ideal of participatory design is industrial democracy (Ehn 1989), which implies that users should be politically empowered with a broad design remit that includes: co-designing the technology, co-determining the nature of the design process itself (schedules, processes, outcomes), and co-designing the social conditions of technology use (i.e. work practices, organizational structures and responsibilities). Contextual inquiry, where the designer works/interacts with the user in the work environment, is a particular technique and is intended to support the development of products that support, extend and transform user work (Wixon et al. 1990).

Participatory design requires significant and ongoing collaboration between designers and users throughout the design process. A key challenge to any participatory design process is access to users: it is difficult organizationally, logistically, and financially to bring users and developers together to engage in collaborative design on a frequent and sustained basis. As such, participatory design has historically been most successful in in-house software development contexts where users and developers are part of the same organization and are often co-located (Grudin 1991, Grudin 1993).

In this issue Bartolo et al. describe the work being done to integrate material from scientists/researchers' workspaces within a digital library (MatDL). Their project is an alliance of many geographically distributed institutions, and they are adapting open source tools such as CVS to enable collaboration on tasks. At the same time as tools are modified and developed, use and users of the library are supported using techniques such as kernel groups and student feedback from courses. A kernel group is an interesting form of participatory design as it encompasses notions of collaboration (access to designers and users among different partners), interdisciplinarity and multidisciplinarity (partners come from many disciplines) and practice and use environments (partners and users include teachers, students, researchers, etc.).

Unsurprisingly, many have seized on the Internet and Web as offering the technical means to enable wider user participation in the design process (Divitini et al. 1999, Hepso et al. 1999, Weatherley et al. 2002). Collaboration systems based on these ubiquitous technical infrastructures could, in theory, provide a virtual commons where distributed stakeholders overcome cultural and geographic separation to get together and engage in collaborative design on a regular basis, without having to travel to distant locations and without taking large chunks of time away from their other work responsibilities. Needless to say, given the inherently distributed nature of digital library users, approaches that enable such users to participate meaningfully in the design and planning of library systems would be invaluable. Despite the promise, results to date have been mixed, with some groups reporting reasonable success (Farschchian and Divitini 1999, Weatherley et al. 2002) and others reporting little participation by users (Hepso et al. 1999).

Khoo's contribution to this issue provides valuable insight into the formidable barrier posed by the variable understanding of developers and users about the same artifact they are building together. A study in organizational communication, individual participant understandings about the word 'resource' are described in detail in order to highlight the different tacit understandings that existed among participants of the same digital library project. The human-based effort to create concept maps that represented the collection development process also resulted in the creation of an online communication tool, the DWEL WorkHub, to represent the knowledge of the developers to the users and increase understanding between both groups.

While participatory design embraces the importance of co-designing the social conditions of technology use, the ideals of industrial democracy often get lost when applied to North American contexts (Schuler and Namioka 1993). Participatory design projects tend to focus on technology innovations as opposed to emphasizing social change (Bishop et al. 2003). Participatory action research builds on the participatory design tradition, but instead emphasizes working with community groups to effect immediate social change and building capacity within these groups to frame and investigate issues of their own concern. Thus, the design and use of library technologies and services takes a back seat to helping community groups affect local change. Participatory action research contexts can often yield invaluable, possibly profound, insights in the design and planning of library technologies and services as this approach requires designers to focus on the transformational needs of specific community groups.

Bishop et al., writing in this issue, emphasize participatory action research with the goal of effecting change through the use of Inquiry Labs (iLabs). This approach is about stimulating, motivating, improving, integrating learning within a community, often a marginalized group, by infusing library technologies and capabilities into community organizations rather than bringing users to the library. They describe the iLabs suite as "simultaneously a Web site, a community of learners, and a locus for knowledge construction". A simple blend of informational and communication tools is offered for diverse sets of users: users of iLabs inquiry activities come from all walks of life such as neighborhood activism, university courses, research projects, committee work, K-12 education, conference presentations, developing international professional associations, and art projects. iLabs reflect the successful fulfillment of user needs that have surfaced in many different arenas over and over again, and are helping in the transformation of people's ability to process and use information on both sides of the digital divide.

This is especially relevant in view of a report about the digital divide that claims the Internet has no social impact on low-income adults but may increase learning motivation and cognitive competencies (Jackson et al. 2004). Low income African Americans and Caucasians included in the study, when using the Internet for the first time used it as an informational tool, not as a communication tool. Email, the most popular application was useless to them because they had no friends or co-workers to whom they could send email.

Shared governance models

The third model for negotiating the future has its roots not in design research but in educational administration and, more broadly, organizational development (Shafritz and Ott 1992). "The process of consultation", wrote John J. Corson in 1941, "strengthens the allegiance to the institution and their individual zeal and satisfaction" (Douglass 1998). In a nutshell, that is the essence of the shared governance model: respect for the different roles played by students, faculty, staff, administrators, and the general public, and ensuring that all continue to feel a sense of ownership for the institution through the process of consultation, with the benefit of greater excellence and improvement for all. While shared governance is an idea that is generally considered to have originated in academic governance, it has been found to be effective as a business management strategy for services such as IT security risk management, and is promoted as a tool for global and sustainable management of our environment (Sachs 2004). Shared governance is also practised by another venerable academic endeavor: the publishing of scholarly journals. Consider the peer review process: most scholarly refereeing is unpaid labor that the individual offers as a service to the discipline and as part of her responsibility and commitment to help maintain the quality of the work published in the discipline.

Two articles in this special issue use shared governance quite innovatively and in different ways from each other to inform the design of the digital library, deliver needed services to users, and ensure sustainability. In the process they are also effecting social and educational changes. Foulke et al. describe the creation of Connecticut History Online (CHO), a new non-profit entity to support cross-institution collaboration, incorporating different advisory groups, including one for teachers and one for educator specialists, into a formal governance structure. Partnership is a core value, while ownership and consensus building are integral to all the project's efforts. CHO holds promise for changing the digital futures of libraries, archives, museums, and educational institutions from objects on separate trajectories to partners in a more seamlessly integrated world of multimedia information.

McMartin emphasizes how shared governance is an inherent part of MERLOT's business model, perhaps even a direct influence on its sustainability. Again, partnership is emphasized: in sharing the governance of the MERLOT collection among a large and distributed group, the responsibility for the integrity and quality of the collection is distributed among the faculty, similar to the scholarly journal's editorial and peer review model. Besides encouraging digital library research projects and initiatives to incorporate shared governance models for long-term success and sustainability, MERLOT could influence changes in social processes such as academic peer review and the promotion and tenure process.

Conclusions

The users of digital libraries like MERLOT number in the tens of thousands. Others are much smaller, but no matter the size, digital libraries almost always involve distributed, semi-virtual communities of users. In these communities, traditional users - scientists, researchers, educators, lifelong learners, and the general public - work alongside stakeholders (funding organizations, policy makers, publishers), together with software developers, librarians, archivists, records managers, and museum curators. The collaborative work of these groups ranges widely: they create and share data/content, conduct scientific research, develop metadata schemas, create metadata for objects, develop policies, software, and protocols for services and management. The articles in this issue describe some major experiments with integrated and sustainable building of digital libraries in ways that meet user needs.

We have come a long way from early digital library development efforts where the predominant model was often system-oriented or followed structured development design. Approaches such as the waterfall model, where users had a specific place in the design process, are no longer the only option for building digital libraries. Almost all the efforts reported here mix and match approaches. Users are involved and integral to the entire lifecycle of the digital library: design, use and sustainability. In the participatory design iLabs, one important community using the Inquiry Page is the Water CAMPWS collaboratory which links researchers, teachers, students, government agencies, industry and general public on the topic of water purification. Water purification is often a local government policy question that needs input from a shared governance model for effective resolution or management. Similarly, in MERLOT educators have clear ideas about information quality. These differ by discipline, and furthermore must be contextualized with use information (when was the resource used, by whom, etc.). Thus, hybrid approaches integrating users in design and development - whether using a design approach like user-centered design, participatory design, or an organizational development approach such as shared governance - appear to offer more promise for successful digital information use, enhancing such use beyond mere information access to transforming users, and ensuring library sustainability.

Acknowledgements

The prompt responses of reviewers and their helpful feedback is very much appreciated. Sumner's research is funded in part by the National Science Foundation under NSF Award #0301213 and under Cooperative Agreements #ATM-0301213 between the NSF and UCAR, and by NSF SPO AWD No. DUE-0227656, SPO 035. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

References

Amari, S-i., Beltrame, F., et al. (2002) "Neuroinformatics: The integration of shared databases and tools towards integrative neuroscience". Journal of Integrative Neuroscience, 1(2): 117-128

Bishop, A. P., Mehra, B., et al. (2003) "Participatory Action Research and Digital Libraries: Reframing Evaluation". Digital Library Use: Social Practice in Design and Evaluation, edited by A. P. Bishop, N. A. Van House and B. Buttenfield (Cambridge, MA: MIT Press), 161-189

Blandford, A., Keith, S., Connell, I. and Edwards, H. (2004) "Analytical Usability Evaluation for Digital Libraries: A Case Study". ACM/IEEE Joint Conference on Digital Libraries (JCDL '04), Tucson, AZ, June 7-11 (ACM Press), pp. 27-36 http://www.uclic.ucl.ac.uk/annb/DLUsability/p023-Blandford.pdf

Booth, P. (1989) An Introduction to Human Computer Interaction (Hillsdale, NJ: Lawrence Erlbaum Associates)

Borgman, C. L. (1999) "What are digital libraries? Competing visions". Information Processing & Management, 35(3): 227-243

Bowker, G. (2001) "Biodiversity, datadiversity". Social Studies of Science, 30(5): 643-683

Budhu, M. and Coleman, A. (2002) "The Design and Evaluation of Interactivities in a Digital Library". D-Lib Magazine, 8 (11), November http://www.dlib.org/dlib/november02/coleman/11coleman.html

Cooper, A. (1999) The Inmates are Running the Asylum: Why High-tech Products Drive us Crazy and How to Restore the Sanity (Indianapolis: SAMS)

Divitini, M., Farschchian, B., et al. (1999) "Internet-based Groupware for User Participation in Product Development: a CSCW '98/PDC '98 Workshop Report". ACM SIGGROUP Bulletin, 20(1), April: 4-55

Domenico, B., Caron, J., Davis, E., Kambic, R. and Nativi, S. (2002) "Thematic Real-time Environmental Distributed Data Services (THREDDS): Incorporating Interactive Analysis Tools into NSDL". Journal of Digital Information, 2(4), Article No. 114, May 29 hdl.handle.net/2249.2/jodi-58

Douglass, J. A. (1998) "Shared Governance at UC: An Historical Review", Center for Studies in Higher Education, University of California, Berkeley http://ishi.lib.berkeley.edu/cshe/jdouglass/gov.html

Ehn, P. (1989) Work-Oriented Design of Computer Artifacts (Stockholm: arbetslivscentrum)

Farschchian, B. and Divitini, M. (1999) "Using Email and WWW in a Distributed Participatory Design Project". ACM SIGGROUP Bulletin, 20(1), April: 10-15

Fox, E. A., et al. (1993) "Users, user interfaces, and objects: Envision, a digital library". Journal of the American Society for Information Science, 44(8), September: 480-491

Ginsparg, P. (2001) "Creating a global knowledge network". Second Joint ICSU Press - UNESCO Expert Conference on Electronic Publishing in Science, Paris, February 19-23 http://arxiv.org/blurb/pg01unesco.html

Gould, J. D., Boies, S. J., et al. (1991) "Making Usable, Useful, Productivity-Enhancing Computer Applications". Communications of the ACM, 34(1): 74-85

Greenbaum, J. and Kyung, M. (1991) Design at Work: Cooperative Design of Computer Systems (Hillsdale, NJ: Lawrence Erlbaum Associates)

Grudin, J. (1991) "Interactive Systems: Bridging the Gaps Between Developers and Users. Computer, 24(4): 59-69

Grudin, J. (1993) "Obstacles to Participatory Design in Large Product Development Organizations". Participatory Design: Principles and Practices, edited by D. Schuler and A. Namioka (Hillsdale, NJ: Lawrence Erlbaum Associates), 99-119

Hepso, V. and Juul, M., et al. (1999) "The Development and Coaching of VISOK: A Web-Based Project at Statoil". ACM SIGGROUP Bulletin, 20(1), April: 25-30

Jackson, L. A., et al. (2004) "The Impact of Internet Use on the Other Side of the Digital Divide". Communications of the ACM, 47(7), July: 43-47

Kling, R. and McKim, G. (2000) "Not Just a Matter of Time: Field Differences and the Shaping of Electronic Media in Supporting Scientific Communication". Journal of the American Society for Information Science and Technology, 51(14): 1306-1320 http://arxiv.org/abs/cs.CY/9909008

Lesk, M. (1991) "The CORE Electronic Chemistry Library". In Proceedings of the 14th Annual International ACM/SIGIR Conference on R&D in Information Retrieval, Chicago, IL, pp. 92-112

Lewis, C. H. and Rieman, J. (1993) Task-centered User Interface Design: A Practical Guide http://hcibib.org/tcuid/index.html

Lyman, P., Varian, H., et al. (2003) How much information? http://www.sims.berkeley.edu/research/projects/how-much-info-2003/

Lynch, C. (2003) "Colliding with the Real World: Heresies and Unexplored Questions about Audience, Economics, and Control of Digital Libraries". Digital Library Use: Social Practice in Design and Evaluation, edited by A. P. Bishop, N. A. Van House and B. Buttenfield (Cambridge, MA: MIT Press), pp. 191-216

McKnight, C., Dillon, A. and Richardson, J. (1996) "User-centered Design for Hypertext/Hypermedia Education". In Handbook of Research for Educational Communications and Technology, Section III: Hard Technologies: Media-Related Research, edited by R. B. Kozma (Bloomington, IN: Association for Educational Communications and Technology), pp. 622-633

Marlino, M., Sumner, T. R., et al. (2001) "The Digital Library for Earth System Education: Building Community, Building the Library". Communications of the ACM, special issue on Digital Libraries, May, 80-81

Meister, D. (1985) Behavioral Analysis and Measurement Methods (New York: Wiley)

NSDL (2003) National Science Digital Library Progress Report, Fall 2000 - Summer 2003, p. 29 http://nsdloutreach.comm.nsdl.org/annual_report/2003/NSDL.Rpt.final.Pgs_25-32.pdf

Rosson, M. B. and Carroll, J. (2002) Usability Engineering: Scenario-Based Development of Human-Computer Interaction (San Francisco: Morgan Kaufmann)

Sachs, J. D. (2004) "Sustainable Development: Editorial". Science, 304 (5671): 649

Schuler, D. and Namioka, A. (1993) Participatory Design: Principles and Practices (Hillsdale, NJ: Lawrence Erlbaum Associates)

Shafritz, Jay M. and Ott, J. S. (1992) Classics of Organization Theory, third edition (Pacific Grove, CA: Brooks/Cole Publishing)

Shumar, W. (2003) "The Role of Community and Belonging in On-line Learning: The ESCOT Project at the Math Forum". Developing Digital Libraries for K-12 Education, edited by M. Mardis (Syracuse, NY: ERIC), pp. 174-187

Wattenberg, F. (1998) "A National Digital Library for Science, Mathematics, Engineering, and Technology Education". D-Lib Magazine, October http://www.dlib.org/dlib/october98/wattenberg/10wattenberg.html

Weatherley, J., Sumner, T., et al. (2002) "Partnership Reviewing: A Cooperative Approach for Peer Review of Complex Educational Resources". ACM/IEEE Joint Conference on Digital Libraries (JCDL '02), Portland, OR, July 14-18 (ACM Press), pp. 106-114

Wixon, D., Holtzblatt, K. and Knox, S. (1990) "Contextual design: an emergent view of system design". In Conference on Human Factors in Computing Systems archive: Proceedings of the SIGCHI conference on Human Factors in Computing Systems: Empowering People, Seattle, WA, pp. 329 - 336

Zia, L. L. (2001) "The NSF National Science, Technology, Engineering, and Mathematics Education Digital Library (NSDL) program: New projects and a progress report". D-Lib Magazine, 7(11), November http://www.dlib.org/dlib/november01/zia/11zia.html

Links

Math Forum @ Drexel University http://mathforum.org/

MERLOT, Multimedia Educational Resource for Learning and Online Teaching http://www.merlot.org/Home.po