The challenges of data ingest, transformation and aggregation at the National Bibliographic Knowledgebase

Abstract

Large-scale library data aggregations provide additional discovery opportunities for users and benefit collection management, metadata and interlending work in libraries. However, they present significant delivery challenges around the matching and deduplication of records and require continuous improvement and maintenance to keep pace with changes in the way the sector creates and shares its metadata. 
This article describes how contributors to Jisc’s National Bibliographic Knowledgebase (NBK) supply their data and how the Jisc Library Hub team matches and deduplicates the data for use via Library Hub Discover, Library Hub Cataloguing, and Library Hub Compare services. It will also explore some of the challenges faced with data transfer, metadata formats, non-standard metadata, and future developments.
This article is based on a paper given at CILIP’s Rare Books & Special Collections Group conference in September 2025.