Developing a Born-Digital Preservation Workflow

Presenters: Bill Donovan and Jack Kearney, Boston College

Bell Tower image
Postcard image of the Boston College Bell Tower, ca. 1930-1945. From the Tichnor Brothers Postcard Collection at the Boston Public Library.

Our presenters described the workflow followed to access records on an external hard drive included in the personal papers of Irish soprano and harpist Mary O’Hara, their first dive into the sea of digital preservation. They described how workflows start as baseline best practices. What happens when the unanticipated occurs? Hearing about the steps taken at Boston College to appraise, ingest and clear unanticipated hurdles along the way reinforced that processing plans/workflows are a starting point. What you find when you open the files can and will drive changes to workflows – sound familiar? Tags: Writeblocker, UNIX, 8.3 Constraint, Fixity (software), Identity Finder (software), XENA tool, Policy writing, FITS tool, JHOVE tool, LOCKSS, DP in a box, Digital Forensics.

Digital Commonwealth 2.0: It’s Alive!

Presenters: Steven Anderson and Eben English

Despite the migration to our new platform in Fedora and Hydra literally happening while we met, our intrepid presenters gave before & after comparisons of the repository website with its streamlined visual presentation and enhanced search capabilities. If you haven’t already, check it out!

Rapid Fire Inspiring Projects

Benjamin Sewall Blake jumping, ca. 1888. From the Francis Blake photographs at the Massachusetts Historical Society.
Benjamin Sewall Blake jumping, ca. 1888. From the Francis Blake photographs at the Massachusetts Historical Society.

Presenters: Christine Clayton, Worcester Art Museum (WAM); Abigail Cramer, Historic New England (HNE); Sean M. Fisher, Department of Conservation (DCR) and Recreation and Rebecca Kenney, Massachusetts Water Resources Authority (MWRA); Larissa Glasser, Arnold Arboretum Horticultural Library (AAHL); Nancy Heywood, Massachusetts Historical Society (MHS); Michael Lapides, New Bedford Whaling Museum; Sara Slymon, Turner Free Library

WOW! Our presenters offered up a smorgasbord of formats, collections and projects they undertook to make records available to their users. For some, their users were internal, like the WAM, which digitized exhibition catalogs, HNE digitized their collection of photographs by Nathaniel Stebbins, DCR and MWRA digitized 8800 images, the largest collection undertaken by Digital Commonwealth so far. AAHL digitized a collection of glass plate negatives…the results? Unanticipated revenue streams – from interior decorators, increased hits on websites, object provenance authentications, access to the identities of early American movers and shakers as reported in contemporary newspapers, accessible Town Reports and High School yearbooks. Several of these projects are still in the pipelines, so not yet searchable on the Digital Commonwealth website.

Submitted by guest reporter Elizabeth Cousins, First Parish in Brookline.

Over the past few months, the development work on the new Digital Commonwealth repository at the Boston Public Library has focused on functionality for ingesting metadata records via the Open Access Initiative Protocol for Metadata Harvesting (OAI-PMH). This functionality enables Digital Commonwealth to include metadata created by institutions around the state in the central search interface, with links that point back to the original item hosted by the provider. (Digital Commonwealth currently harvests records from ten institutions and consortia, including the State Library of Massachusetts, NOBLE, SAILS, and C/W MARS to name a few.)

BPL development staff have been working closely with each OAI provider to tailor the ingest process to their preferred metadata format (Dublin Core, PBcore, MODS, etc.) as well as the system used by each institution to provide the records (CONTENTdm, Omeka, etc.) The crosswalking process, which converts the incoming metadata records into MODS, also involves a number of data standardization routines, including the transformation of date data into a facet-able and sortable date format based on W3C Date-Time Format, and the conversion of geographic subject/coverage data into hierarchical geographic subjects (state, county, city, etc.) and numeric latitude/longitude coordinates using data from the Getty Thesaurus of Geographic Names. Whenever possible, the ingest process also generates thumbnail images for each object which are then stored in the Digital Commonwealth repository, along with an archival copy of the original metadata record prior to crosswalking.

While all of this involves significant time and effort, the result will be more accurate and more complete metadata records from these providers, and a better search and discovery experience for users as well as better representation of the data within larger shared contexts such as DPLA.

So far the OAI harvesting has been restricted to a test platform. By late February the BPL expects to finish the work on the OAI feeds at which point the feeds will be added to the public repository site ( The focus will then turn to migrating the last few remaining collections from the DSpace repository into the new repository, and integrating the informational content on the current Omeka site into the new design. While no official date has been set for when the new repository will replace the existing systems and be launched as the “official” Digital Commonwealth site, it is anticipated that this milestone will be completed sometime in March.

Over the last month or so, the development of the new Digital Commonwealth repository currently ongoing at BPL has focused on refining the batch upload process. The repository developers have been working closely with the BPL Digital Services metadata team to create a standardized spreadsheet format for ingest that will offer institutions the ability to provide rich metadata about their digital objects, while also being flexible, intuitive, and simple to use. This work has brought the goal of allowing institutions to do self-mediated batch uploads much closer, though there are still several issues to tackle before this functionality is ready to roll out.

Meanwhile, the beta testing phase of both the “Search” and “Admin” applications is ongoing and has received quite a bit of helpful feedback from a number of institutions/individuals that have taken the system for a test-drive.

The URLs are:
Search (public discovery):
Admin (ingest & management):

In late September, development of the workflow for ingesting material into the repository via OAI-PMH will begin in order to aggregate records from the numerous institutions around the state that provide access to digital objects through their own repository systems. The BPL will be reaching out to institutions that currently contribute material to Digital Commonwealth via OAI-PMH feed to learn a more about existing data structures, preferred metadata formats for harvesting, back-end systems being used, and other details that will help this phase of the project move forward more smoothly.

Lastly, the BPL has set up a public Google Group email list for institutions and users to provide feedback or report issues with the new repository system. Anyone may read content posted to the group; membership is required to send messages to the list. See!forum/digitalcommonwealth for details.