Vocabulary Mapping for Terminology Services

Diane Vizine-Goetz, Carol Hickey, Andrew Houghton and Roger Thompson
OCLC Research, OCLC Online Computer Library Center, Inc.
Email: vizine@oclc.org; hickeyc@oclc.org; houghton@oclc.org; thompson@oclc.org
Project Web Site: http://www.oclc.org/research/projects/termservices/

Abstract

The paper describes a project to add value to controlled vocabularies by making inter-vocabulary associations. A methodology for mapping terms from one vocabulary to another is presented in the form of a case study applying the approach to the Educational Resources Information Center (ERIC) Thesaurus and the Library of Congress Subject Headings (LCSH). Our approach to mapping involves encoding vocabularies according to Machine-Readable Cataloging (MARC) standards, machine matching of vocabulary terms, and categorizing candidate mappings by likelihood of valid mapping.  Mapping data is then stored as machine links. Vocabularies with associations to other schemes will be a key component of Web-based terminology services. The paper briefly describes how the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) is used to provide access to a vocabulary with mappings.

A majority of tools and features for accessing the names, subjects, and classification categories assigned to content objects are not easily accessed by people or computers. The knowledge organization schemes and the features found in cataloging and retrieval systems are often deeply embedded in proprietary formats and software. Even when knowledge organization resources are openly available, they are rarely linked with other compatible schemes or services. This paper describes a project to add value to controlled vocabularies through vocabulary mapping. The vocabulary associations are then made accessible through Web services.

In this paper, 'terminology services' is used to describe Web services involving various types of knowledge organization resources, including authority files, subject heading systems, thesauri, Web taxonomies, and classification schemes. The term 'vocabulary' is used to refer to these knowledge organization resources. Vocabularies with associations to other schemes will be a key component of Web-based terminology services. Web services are modular, Web-based, machine-to-machine applications that can be combined in various ways. For background information on Web services, see  Gardner (2001) and Tennant (2002). Web services can be accessed at various points in the metadata lifecycle, for example, when a work is authored or created, at the time an object is indexed or cataloged, or during search and retrieval. A Web service that provides mappings from a term in one vocabulary to one or more terms in another vocabulary is an example of a terminology service.

Researchers have been interested in achieving compatibility among controlled vocabularies for many years. Lancaster and Smith (1983) published an overview of the issues involved in integrating vocabularies, which is still relevant today. They describe several factors that influence how successfully one vocabulary can be associated with another, including:

• Extent of overlap in the subject matter
• Level of specificity of terms
• Degree of pre/post-coordination
• How the vocabulary codes equivalence, hierarchical, and other relationships

Researchers involved in more recent efforts to integrate vocabularies have identified additional factors affecting vocabulary compatibility:

• Differences in word use, e.g. common versus scientific names (Doerr 2001, Olson and Strawn 1997)
• Differences in meaning resulting from different classifications of terms (Doerr 2001, Whitehead 1990)

Zeng and Chan (2003) review the primary methodologies used to associate and integrate vocabularies. Of the approaches they describe, two are relevant to this paper:

• Direct mapping-establishing equivalence between terms in different controlled vocabularies or between verbal terms and classification numbers.
• Co-occurrence mapping-establishing mappings from the co-occurrence of terms from different schemes in the same metadata or catalog record.

Co-occurrence mappings are considered to be more loosely mapped than direct mappings, which usually have an intellectual review component.

The scope of OCLC's vocabulary mapping research projects has expanded to include additional classification schemes, subject heading systems, and thesauri. A list of OCLC vocabulary associations and the mapping approach used (direct, co-occurrence, or both) is shown in Table 1. In addition to DDC/LCSH co-occurrence mappings, direct mappings have been made between selected classes from the Library of Congress Classification (LCC) and the National Library of Medicine Classification (NLMC) and DDC. The LCC/DDC mappings and NLMC/DDC mappings are used to profile questions and expertise for virtual reference services. Project staff members have also made direct mappings of genre terms for fiction and drama (GSAFD) to LCSH and to LCSHac (headings for children's materials) using the procedures outlined in this paper. Because the GSAFD vocabulary is quite small - only 153 preferred terms - and based largely on LCSH, the GSAFD mapping effort was not considered a suitable test of our mapping approach. For these reasons, the approach was applied to another vocabulary.

Table 1. OCLC vocabulary associations

From	To
Vocabulary	DDC	ERIC	GSAFD	LCC	LCSH	LCSHac	MeSH	NLMC
DDC (Dewey Decimal Classification)				Direct	Direct & Co-occur	Direct & Co-occur	Direct	Direct
ERIC Thesaurus					Direct
GSAFD (Genre terms for fiction)					Direct	Direct
LCC (Library of Congress Classification)	Direct
LCSH (LC Subject Headings)	Direct & Co-occur	Direct	Direct	Co-occur			Direct
LCSHac (LC Children's Headings)	Direct & Co-occur
MeSH (Medical Subject Headings)	Direct				Direct
NLMC (National Library of Medicine Classification)	Direct

The GSAFD vocabulary terms with mappings are accessible using the OAI-PMH. The OAI protocol specifies a simple HTTP protocol for automated sharing of metadata, but as the OAI-Cat effort has shown, the approach works equally well for sharing other XML content. The content of the GSAFD records is MARC in XML (MARC Standards). The records are accessible to users via a browser (http://alcme.oclc.org/gsafd/) and to machines through the OAI-PMH Web services mechanisms. See Van de Sompel et al. (2003) for a more complete description of the how the file can be accessed using the OAI-PMH. The GSAFD/LCSH mapping file can also be downloaded from our project Web site. The file is encoded in MARC in XML and also according to version 0.5 of the Zthes schema. We have also prototyped some experimental Web services using co-occurrence mappings between the GSAFD vocabulary and LCSH.

As Table 1 shows, much of our mapping activity involves LCSH. Describing the relationship between the Art and Architecture Thesaurus (AAT) and LCSH, Whitehead (1990, p. 82) asks: "Why map to LCSH?" and replies:

Despite the weaknesses and the critical assessments that have plagued LCSH over the years, the fact remains that LCSH is the standard vocabulary used by the majority of information resources, especially libraries, in the United States.

She also notes that efforts to improve or replace LCSH must take into account its widespread use and the probability that it will be maintained for a long time. Others have reached similar conclusions. For example, the FAST project sponsored by OCLC selected LCSH as the basis for creating a faceted vocabulary for metadata. O'Neill and Chan (2003) cite the following reasons for choosing the LCSH scheme:

• [LCSH] is by far the most commonly used and widely accepted subject vocabulary for general application.
• It is the de facto universal controlled vocabulary and has been translated or adapted by many countries around the world.
• It is the largest general indexing vocabulary in the English language.

LCSH are also among the recommended encoding schemes that can be used to qualify the Dublin Core subject element. Several prominent projects that use Dublin Core metadata create subject elements based on LCSH, including the Colorado Digitization Program, DSpace, and ePrints UK.

Many standards exist for encoding vocabularies: see Koch (2003) and the SWAD-Europe Thesaurus Activity thesaurus link page for listings of some current standards. For authority files, subject headings and thesauri, we have decided to use the MARC21 Format for Authority Data. For classification data, we use the MARC21 Format for Classification Data. MARC was chosen because many large vocabularies are available in the MARC formats, and the MARC Authority format supports inter-vocabulary relationships, which are particularly important to us because of our mapping work. Some examples of vocabularies available in the MARC format include:

• Library of Congress Subject Headings (LCSH) > 277,000/263,524 concepts/terms
• The Getty vocabularies (Art & Architecture Thesaurus; Union List of Artist Names; Thesaurus of Geographic Names) > 1.6 million concepts/names/terms
• Medical Subject Headings (MeSH) > 21,973/125,858 concepts/terms
• Canadian Subject Headings (CSH) > 6,000 concepts
• Library of Congress Classification data (LCC) > 595,000 categories

The MARC authority format enables us to provide detailed coding for many common controlled vocabulary elements. Preferred terms are coded in the block of MARC tags labeled 1XX. The tag 150 is used for topical terms. Non-preferred terms are coded in the 4XX range. The MARC authority format provides for the coding of some relationships between a preferred term and non-preferred terms, including earlier forms and acronyms. Broader term/narrower term relationships and associative relationships (related terms) are coded in MARC tags 5XX. Subfield $w is used to code relationships between 1XX and 4XX fields and 1XX and 5XX fields. Tags 7XX are used to provide links between equivalent terms in the same vocabulary and equivalent terms in different vocabularies. Section 5 provides a detailed explanation of MARC 7XX linking fields.

In the remainder of this paper we describe our approach to mapping the ERIC Thesaurus to LCSH. The ERIC Thesaurus was chosen because it is a well-established vocabulary, publicly accessible on the Web, and large enough to provide a meaningful test of our mapping approach. The ERIC Thesaurus is produced by the Educational Resources Information Center, an education information network, sponsored by the U.S. Department of Education, and provides public access to education literature (ERIC 2004).

Vocabularies to be mapped are first converted to the MARC21 Authority Format. The effort involved in this step varies depending on the format of the source vocabulary (vocabulary being mapped). We have converted vocabularies from formats primarily intended for display, e.g. word processing documents without extensive use of styles and vocabularies in more structured formats such as the ERIC file (Figure 1).

Multiple instances of broader terms (BT), narrower terms (NT), and related terms (RT) stored in single ERIC fields are encoded as separate fields in the MARC format (Figure 2). The RT field shown below generates 14 fields in the MARC record. These are the fields labeled with MARC tag 550 (without $w subfields). The field labeled UF is similarly converted into two MARC fields (tag 450). One of the terms, Student ability, represents a formerly valid term. The notation in parentheses in the ERIC record indicates this and gives the lifespan of the term. When this data is converted to MARC, a 688 field (Application History Note) is constructed for this data. In the 450 field, subfield $w is added to indicate the term was formerly valid. By encoding the source and target vocabularies in the MARC Authorities Format we are able to standardize the representation of similar information and improve our ability to match vocabularies.

After the ERIC file is encoded in the MARC Authority format, the ERIC vocabulary is matched to the LCSH vocabulary. Using a series of computer programs, all preferred terms (MARC tag 150) and non-preferred terms (MARC tag 450) in the source and target vocabularies are matched. Differences in spacing, capitalization, and punctuation are ignored during the matching process. The following terms are considered matches:
 

ERIC Thesaurus Term	LCSH Term
Alzheimers Disease	Alzheimer's disease
Nurses Aides	Nurses' aides

Currently, plural versus singular forms, terms that differ only by the presence or absence of a parenthetical qualifier, and terms with a qualifier introduced by a comma are not being matched. These refinements would likely improve the match rate and will be employed in the next phase of the project.

ERIC Thesaurus Term	LCSH Term
Echolocation	Echolocation (Physiology)
Crack	Crack (Drug)
Radiology	Radiology, Medical
Rh factors	Rh factor

A total of 3,797 ERIC terms were matched to LCSH and categorized according to the following match types:

• PT/PT - An exact match (after normalization) of a preferred term (PT) in the source vocabulary to a preferred term (PT) in the target vocabulary
• PT/NPT - An exact match of a preferred term (PT) (source) to a non-preferred term (NPT) (target)
• NPT/NPT - An exact match between a non-preferred term (NPT) (source) and a non-preferred term (NPT) (target)
• NPT/PT - An exact match between a non-preferred term (NPT) (source) and a preferred term (PT) (target)

Four categories of ERIC terms were reviewed and analyzed (numbers in parentheses are ERIC category codes):

• Learning & perception (110)
• Individual development & characteristics (120)
• Health & safety (210)
• Disabilities (220)

This subset comprises about 12% of ERIC preferred terms. Statistics for PT/PT matches and PT/NPT matches are shown in Table 2. Columns 3 and 4 show the number of term matches and concept matches for PT/PT matches and columns 5 and 6 present this information for PT/NPT matches.

Table 2. PT/PT and PT/NPT matches

ERIC Category	Total Preferred Terms	PT/PTTermMatch	PT/PTConcept Match	PT/NPT Term Match	PT/NPT Concept Match
Learning & perception (110)	164	49	49	10	8
Individual development & characteristics (120)	269	83	81	25	23
Health & safety (210)	227	129	127	31	30
Disabilities (220)	113	37	37	12	10
Total	773	298	294	78	71

(PT matches NPT)

ERIC	LCSH	Match Type	Valid Mapping
PT: Ametropia	PT: Eye-Refractive errors NPT: Ametropia	PT/NPT	Yes

(NPT matches NPT) and (NPT matches PT)

ERIC	LCSH	Match Type	Valid Mapping
PT: Extraversion  Introversion NPT: Ambiversion NPT: Extroversion NPT: Introversion	PT: Extraversion NPT: Extroversion	NPT/NPT	Yes
	PT: Introversion	NPT/PT	Yes

(PT matches NPT) and two (NPT matches PT)

ERIC	LCSH	Match Type	Valid Mapping
PT: Adolescents	PT: Teenagers NPT: Adolescents	PT/NPT	Yes
NPT: Adolescence	PT: Adolescence	NPT/PT	No
NPT: Teenagers	PT: Teenagers	NPT/PT	Yes

Vocabulary links are stored in MARC fields 7XX. Using these fields, we can encode the following:

• name or code of the target vocabulary
• mapped term
• control number or unique identifier for the mapped term
• identity of the mapping organization

In the following example, the first two 750 fields are mappings to LCSH. This information is coded in the indicator value. The first two character positions at the beginning of a field are called indicators. These character positions can contain information that interprets or supplements the data found in the field. The unique identifier for the term is coded in the $0 subfield and the organization that supplied the mapping data in the $5 subfield (e.g. OCoLC-O). The code, OCoLC-O, is the MARC organization code for OCLC's Office of Research. MARC organization codes are used to represent names of libraries and other organizations that need to be identified in the bibliographic environment. The last 750 field is a mapping to MeSH with the unique MeSH identifier coded in $0 subfield. No information about the mapping organization is provided. The unique identifiers for the mapped terms are linked to Web-accessible versions of LCSH and MeSH. Mapping data of this type can be used to create high quality terminology services. For example, terminology services that support search and retrieval might use the full range of available mappings, while services invoked during indexing or cataloging might use only mappings produced by a specific organization or for a given vocabulary.

A legitimate concern about vocabulary mapping is how the mappings will be maintained. Although not a trivial task, mappings can be maintained with the help of software that tracks changes to vocabulary term records. Changes to vocabulary terms are recorded in a number of ways, e.g. by data in a vocabulary record that indicates when the record was last modified, by notes fields that chronicle changes to a vocabulary term (see field 688 in the MARC record examples), and through notifications of additions and changes distributed by vocabulary owners. Depending on the nature of the changes, human review may be needed to determine if mappings are still valid when a vocabulary term changes.

In this example, the LCSH terms are linked to LC subject authority records accessible through the OAI-Cat framework. These records are accessible to users via a browser and to machines through the OAI-PMH Web services mechanisms. The MeSH link generates a search of the MeSH vocabulary using the search features of the MeSH Browser.

6 Next steps

Our plans for the near term include refining the matching software and developing improved tools for reviewers. When the review of the ERIC/LCSH matches is complete, the file of mappings will be made available to other researchers. The file will be available in MARC in XML and also encoded according to version 0.5 of the Zthes schema. We also anticipate making this file available via OAI-PMH and for searching using SRU/SRW and the Zthes profile. See the Terminology Services project Web site for details.

Acknowledgements

We thank the reviewers of this paper for their many helpful comments and suggestions.

References

Doerr, M. (2001) "Semantic Problems of Thesaurus Mapping". Journal of Digital Information 1(8) http://jodi.tamu.edu/Articles/v01/i08/Doerr/

Gardner, T. (2001) "An Introduction to Web Services". Ariadne (29) http://www.ariadne.ac.uk/issue29/gardner/

Koch, T. (2003) "Activities to advance the powerful use of vocabularies in the digital environment - Structured overview" http://www.lub.lu.se/~traugott/drafts/seattlespec-vocab.html

Lancaster, F. W. and L. Smith (1983) "Compatibility Issues Affecting Information Systems and Services". General Information Programme and UNISIST, PGI-83/WS/23 (Paris: UNESCO)

Mandel, C. (1987) "Multiple Thesauri in Online Library Bibliographic Systems". Cataloging Distribution Service (Library of Congress: Washington, D.C.)

Olson, T. and G. Strawn (1997) "Mapping the LCSH and MeSH Systems". Information Technology and Libraries, 16(1), 5-19

O'Neill, E. and L. Chan (2003) "FAST (Faceted Application of Subject Terminology): A Simplified LCSH-based Vocabulary". World Library and Information Congress: 69th IFLA General Conference and Council, 1-9 August, Berlin http://www.ifla.org/IV/ifla69/papers/010e-ONeill_Mai-Chan.pdf

Tennant, R. (2002) "Digital Libraries-What To Know About Web Services". Library Journal 12 (July 15) http://www.libraryjournal.com/index.asp?layout=articleArchive&articleid=CA231639

Van de Sompel, H., Young, J. and T. Hickey (2003) "Using the OAI-PMH... Differently". D-Lib Magazine 9(7/8) http://www.dlib.org/dlib/july03/young/07young.html

Vizine-Goetz, D. (1998) "Popular LCSH with Dewey Numbers". In Annual Review of OCLC Research 1997 http://digitalarchive.oclc.org/da/ViewObject.jsp?objid=0000003449

Whitehead, C. (1990) "Mapping LCSH into Thesauri: the AAT Model". In Beyond the Book: Extending MARC for Subject Access, edited by T. Petersen and P. Molholt (Boston: G.H. Hall), p. 81

Zeng, M. and L. Chan (2003) "Trends and issues in establishing interoperability among knowledge organization systems". Journal of the American Society for Information Science and Technology, published online 16 Dec 2003

Links

Canadian Subject Headings (CSH) http://www.nlc-bnc.ca/6/23/index-e.html

Colorado Digitization Program Western States Dublin Core Metadata Best Practices http://www.cdpheritage.org/westerntrails/wt_bpmetadata.html

Dspace "Metadata" http://dspace.org/technology/metadata.html

ePrints UK. "Using simple Dublin Core to describe eprints" http://www.rdn.ac.uk/projects/eprints-uk/docs/simpledc-guidelines/

ERIC (2004) Educational Resources Information Center (ERIC) http://www.eric.ed.gov/index.html

Getty Vocabulary Program http://www.getty.edu/research/conducting_research/vocabularies/

GSAFD experimental Web serviceshttp://research.oclc.org/WebServices/GenreTermsAndSubjectHeadings/GenreTermsAndSubjectHeadings.asmx

Library of Congress Subject Headings (LCSH) http://www.loc.gov/cds/lcsh.html

Library of Congress Classification (LCC) http://lcweb.loc.gov/cds/mds.html#lccr

MARC21 Format for Authority Data (2003) Concise edition http://www.loc.gov/marc/authority/ecadhome.html

MARC21 Format for Classification Data (2002) Concise edition http://www.loc.gov/marc/classification/eccdhome.html

MARC Code List for Organizations (2004) http://www.loc.gov/marc/organizations/orgshome.html

MARC Standards: MARC in XML http://www.loc.gov/marc/marcxml.html

Medical Subject Headings (MeSH) http://www.nlm.nih.gov/pubs/factsheets/mesh.html

MeSH Browser http://www.nlm.nih.gov/mesh/mbinfo.html

OCLC Research: FAST: Faceted Application of Subject Terminology http://www.oclc.org/research/projects/fast/default.htm

OCLC Research: OAICat repository frameworkhttp://www.oclc.org/research/software/oai/cat.htm

OCLC Research: Search & Retrieve on the Webhttp://www.oclc.org/research/projects/webservices/default.htm

OCLC Research: Terminology Services http://www.oclc.org/research/projects/mswitch/4_termservs.htm

SWAD-Europe Thesaurus Activity http://www.w3c.rl.ac.uk/SWAD/thes_links.html

Zthes: a Z39.50 Profile for Thesaurus Navigation http://zthes.z3950.org/

Appendices

Definitions for column labels 

Tag	3-character MARC field tag
Occ	Total number of this field in all records
%Recs	Percent of records that contain this field
Occ/Rec	Occurrence of this field divided by the total number of records
Len/Occ	Average length of this field
Sub	1-chracter MARC subfield code
Occ	Total number of this subfield in all records
Occ/Rec	Occurrence of this subfield divided by the total number of records
Len/Occ	Average length of this subfield

ERIC Thesaurus encoded in MARC Authority Format 

Field and Subfield Statistics
Tag	Occ	%Recs	Occ/Rec	Len/Occ	Sub	Occ	Occ/Rec	Len/Occ
001	6080	100.00	1.00	9.00	-
003	6080	100.00	1.00	7.00	-
040	6080	100.00	1.00	29.00	a	6080	100.00	7.00
					b	6080	100.00	3.00
					c	6080	100.00	7.00
					d	6080	100.00	7.00
					e	6080	100.00	5.00
072	6080	100.00	1.00	3.00	a	6080	100.00	3.00
150	6080	100.00	1.00	16.30	a	6080	100.00	16.30
450	4562	43.90	0.75	18.05	a	4562	43.90	17.86
					w	873	11.71	1.00
550	68725	100.00	11.30	16.42	a	68725	100.00	16.24
					w	11878	91.71	1.00
680	3774	62.07	0.62	148.18	i	3774	62.07	148.18
688	873	11.71	0.14	29.95	a	873	11.71	29.95

ERIC Thesaurus encoded in MARC Authority Format 
773 Record Subset without mapping data 

Field and Subfield Statistics
Tag	Occ	%Recs	Occ/Rec	Len/Occ	Sub	Occ	Occ/Rec	Len/Occ
001	773	0.00	1.00	9.00	-
003	773	0.00	1.00	7.00	-
040	773	0.00	1.00	29.00	a	773	0.00	7.00
					b	773	0.00	3.00
					c	773	0.00	7.00
					d	773	0.00	7.00
					e	773	0.00	5.00
072	773	0.00	1.00	3.00	a	773	0.00	3.00
150	773	0.00	1.00	15.93	a	773	0.00	15.93
450	668	0.00	0.86	17.47	a	668	0.00	17.26
					w	139	0.00	1.00
550	9365	0.00	12.12	15.94	a	9365	0.00	15.77
					w	1595	0.00	1.00
680	520	0.00	0.67	140.28	i	520	0.00	140.28
688	139	0.00	0.18	29.43	a	139	0.00	29.43

ERIC Thesaurus encoded in MARC Authority Format 
773 Record Subset with mapping data 

Field and Subfield Statistics
Tag	Occ	%Recs	Occ/Rec	Len/Occ	Sub	Occ	Occ/Rec	Len/Occ
001	773	100.00	1.00	9.00	-
003	773	100.00	1.00	7.00	-
005	773	100.00	1.00	16.00	-
040	773	100.00	1.00	29.00	a	773	100.00	7.00
					b	773	100.00	3.00
					c	773	100.00	7.00
					d	773	100.00	7.00
					e	773	100.00	5.00
072	773	100.00	1.00	3.00	a	773	100.00	3.00
150	773	100.00	1.00	15.93	a	773	100.00	15.93
450	668	49.94	0.86	17.47	a	668	49.94	17.26
					w	139	13.71	1.00
550	5777	99.61	7.47	15.81	a	5777	99.61	15.62
					w	1098	76.33	1.00
680	520	67.27	0.67	140.28	i	520	67.27	140.28
688	139	13.71	0.18	29.43	a	139	13.71	29.43
750	404	50.19	0.52	30.18	0	404	50.19	16.00
					a	404	50.19	13.71
					x	12	1.55	15.75

LCSH (updated in November 2003) in MARC Authority Format 

Field and Subfield Statistics
Tag	Occ	%Recs	Occ/Rec	Len/Occ	Sub	Occ	Occ/Rec	Len/Occ