Recently, Digital Repository Developer Steven Anderson and Web Services Developer Eben English presented at the Open Repositories 2014 conference in Helsinki and at the Northeast Fedora Users Group (NEFUG) meeting in Boston.
Open Repositories is an annual international conference that brings together people and institutions responsible for the development, implementation, and management of digital repositories to share information and strategies for long-term preservation and access. Steven’s presentation was entitled “When Metadata Collides: Lessons on Combining Records from Multiple Repository Systems.” It summarizes the practical challenges involved in combining diverse descriptions, authorities, and technologies into the shared Digital Commonwealth repository and highlights the imaginative ways Steven and Eben have addressed them with the help of the Digital Projects department. Watch the seven-minute presentation online. Move the slider to the 52 minute mark to start with Steven’s talk. (Editor’s note: the previous link has had intermittent connection issues. Please continue to try the link until it resolves correctly.)
During the NEFUG meeting, Eben and Steven gave a presentation titled “digital_commonwealth_presentation” during the Hydra session. Steven presented on slides, that can be viewed here, and Eben gave a 10 minute demonstrations of teh actual portal. Steven also gave a lightening talk (aka “Dork Short”) about metadata combination challenges.
Over the past few months, the development work on the new Digital Commonwealth repository at the Boston Public Library has focused on functionality for ingesting metadata records via the Open Access Initiative Protocol for Metadata Harvesting (OAI-PMH). This functionality enables Digital Commonwealth to include metadata created by institutions around the state in the central search interface, with links that point back to the original item hosted by the provider. (Digital Commonwealth currently harvests records from ten institutions and consortia, including the State Library of Massachusetts, NOBLE, SAILS, and C/W MARS to name a few.)
BPL development staff have been working closely with each OAI provider to tailor the ingest process to their preferred metadata format (Dublin Core, PBcore, MODS, etc.) as well as the system used by each institution to provide the records (CONTENTdm, Omeka, etc.) The crosswalking process, which converts the incoming metadata records into MODS, also involves a number of data standardization routines, including the transformation of date data into a facet-able and sortable date format based on W3C Date-Time Format, and the conversion of geographic subject/coverage data into hierarchical geographic subjects (state, county, city, etc.) and numeric latitude/longitude coordinates using data from the Getty Thesaurus of Geographic Names. Whenever possible, the ingest process also generates thumbnail images for each object which are then stored in the Digital Commonwealth repository, along with an archival copy of the original metadata record prior to crosswalking.
While all of this involves significant time and effort, the result will be more accurate and more complete metadata records from these providers, and a better search and discovery experience for users as well as better representation of the data within larger shared contexts such as DPLA.
So far the OAI harvesting has been restricted to a test platform. By late February the BPL expects to finish the work on the OAI feeds at which point the feeds will be added to the public repository site (https://search.digitalcommonwealth.org). The focus will then turn to migrating the last few remaining collections from the DSpace repository into the new repository, and integrating the informational content on the current Omeka site into the new design. While no official date has been set for when the new repository will replace the existing systems and be launched as the “official” Digital Commonwealth site, it is anticipated that this milestone will be completed sometime in March.
The Boston Public Library received an award for its digitization work for Digital Commonwealth members at last month’s Griffin Museum of Photography’s eighth annual Focus Awards ceremony. The Focus Awards recognize contributions to the promotion, curation, and presentation of photography. The BPL received the Commonwealth Award, which is given to an organization that brings prominence to the local photographic scene.
“We are honored to receive this award for our digitization work,” said Amy E. Ryan, President of the Boston Public Library. “It is our great pleasure to contribute to Digital Commonwealth and help increase access to photos archives, cultural treasures, and other historical materials for people across Massachusetts and around the world.”
The annual Focus Awards was created by the Griffin Museum in 2006 in order to recognize critical contributions to the promotion of photography made by institutions and individuals. Tom Blake, Digital Projects Manager for the BPL, accepted the Commonwealth Award on the library’s behalf.
The award was presented to Tom by Bob Cullum, the grandson of photographer Leslie Jones (1886-1967). The Leslie Jones collection of nearly 40,000 glass negatives was digitized by the BPL and is now available for viewing in the new Digital Commonwealth repository that the BPL designed and built and now hosts — https://search.digitalcommonwealth.org/collections/commonwealth:2j62s484w.
The award is certainly very well deserved, not just for the work the BPL has done for the membership and organization of Digital Commownwealth, but the enormouse value this work provides the reputation of the Commonwealth as a whole. Congratulations!!
Over the last month or so, the development of the new Digital Commonwealth repository currently ongoing at BPL has focused on refining the batch upload process. The repository developers have been working closely with the BPL Digital Services metadata team to create a standardized spreadsheet format for ingest that will offer institutions the ability to provide rich metadata about their digital objects, while also being flexible, intuitive, and simple to use. This work has brought the goal of allowing institutions to do self-mediated batch uploads much closer, though there are still several issues to tackle before this functionality is ready to roll out.
Meanwhile, the beta testing phase of both the “Search” and “Admin” applications is ongoing and has received quite a bit of helpful feedback from a number of institutions/individuals that have taken the system for a test-drive.
The URLs are:
Search (public discovery): http://search.digitalcommonwealth.org/
Admin (ingest & management): http://admin.digitalcommonwealth.org/
In late September, development of the workflow for ingesting material into the repository via OAI-PMH will begin in order to aggregate records from the numerous institutions around the state that provide access to digital objects through their own repository systems. The BPL will be reaching out to institutions that currently contribute material to Digital Commonwealth via OAI-PMH feed to learn a more about existing data structures, preferred metadata formats for harvesting, back-end systems being used, and other details that will help this phase of the project move forward more smoothly.
Lastly, the BPL has set up a public Google Group email list for institutions and users to provide feedback or report issues with the new repository system. Anyone may read content posted to the group; membership is required to send messages to the list. See https://groups.google.com/forum/#!forum/digitalcommonwealth for details.
The BPL is pleased to announce that they have now moved into the “beta launch” phase of the rollout of the Hydra-based Digital Commonwealth repository platform.
The new URLs are:
Search (public discovery): http://search.digitalcommonwealth.org/
Admin (ingest & management): http://admin.digitalcommonwealth.org/
Not all features are fully implemented as yet. Here is what’s available:
Public Search app:
- keyword search
- faceted browsing of search results by format, subject, date
- browse by collection, institution, or geographic location
- image viewer with zooming functionality for viewing hi-res images in detail
- users can create bookmarks and personalized folders of their favorite items
- users can create an account, or log in via their BPL/MBLN library card or Facebook account
- easily share items on Facebook, Twitter, Pinterest, and other social media
- site designed to play well with tablets and phones
Member Admin app:
- create digital collections
- upload images
- add metadata
- edit existing objects (might be of special interest to members with items in DSpace)
For admin access, contact Tom Blake (email@example.com) to get started.
Features to be added soon:
- batch uploads
- support for other content types, such as postcards, books, and audio
Content from the initial test (alpha) server is being migrated to the new production-server repository. Upon completion all data from the current Digital Commonwealth DSpace server, http://repository.digitalcommonwealth.org, will be available in the new Fedora/Hydra repository. So far about 80% of the DSpace content is available. More is added every day. Once the complete migration is assured, the process will begin to shut down the DSpace server, currently hosted at UMass Amherst Libraries.
As the new repository is now in “beta,” the public link can be shared with colleagues both inside and outside your institution(s). The BPL will be doing a small amount of promotion for this, but intend to save the grand ribbon-cutting for when the system finally replaces digitalcommonwealth.org. Coming soon! Stay tuned!
We are still actively seeking feedback, suggestions, etc., so feel free to send comments by using the feedback form at http://search.digitalcommonwealth.org/feedback.
Earlier this month the Governor approved the inclusion of $147,532 in the state FY14 budget to continue digitization for Digital Commonwealth members under the BPL’s Library for the Commonwealth program.
This money will continue the digital-imaging services that the BPL has provided in partnership with Digital Commonwealth since 2011 through funding provided by a LSTA grant awarded through the MBLC. The state expenditure attests to the valuable work achieved these past two years by the BPL and Digital Commonwealth. Thousands of significant historical items have been digitized and will soon be made available collectively through a new state-of-the-art repository under development at the BPL that just this past week celebrated its initial public beta release.
With this new funding, the BPL will continue to accept project applications and hope to grow this program to include metadata creation and other services in the near future. The BPL and Digital Commonwealth look forward to working with DC members on their digital projects throughout FY14 and beyond.
For more information about participating in this program, details are provided at the BPL’s project blog.
Over the next several months, until the end of the calendar year, Digital Commonwealth will be providing metadata production and quality control assistance for collections digitized under the BPL’s LSTA-funded statewide digitization program.
The goal is to assure that at least 40,000 records are prepared for ingest into the new repository system from at least 40 different organizations. A team of metadata workers, led by Amy Benson and the BPL Digital Services Department, has already begun to evaluate the current states and various forms of descriptive records and how to best transform and optimize them into standards and systems compliant metadata. This work is being supported with grant funds associated with the Digital Hubs Pilot Project of the Digital Public Library of America (http://blogs.law.harvard.edu/dplaalpha/about/digital-hubs-pilot-project/).
Amy Benson is currently the Librarian/Archivist for Digital Projects at Radcliffe’s Schlesinger Library. She is familiar to many due to the multiple workshops and presentations she has delivered, often in association with the NEDCC. Amy also served on the original Digital Commonwealth Executive Committee when the organization was first established.
If your organization has had anything digitized by the BPL for Digital Commonwealth and your descriptive records need to be created, transformed, or enhanced for ingest, then you will be contacted by Amy and/or the BPL team between now and December 2013. Please direct any questions about this initiative to Tom Blake (firstname.lastname@example.org; 617 859-2039).
Here is the latest update from Eben English, Web Services Developer at the BPL, who is helping develop the new Fedora/Hydra repository for the Digital Commonwealth:
The initial development phase for the new digital object management system to store and provide access to digital collections from Digital Commonwealth members is currently underway and will soon be complete. Items and metadata from the existing DSpace repository (http://repository.digitalcommonwealth.org) have begun to be migrated into the new system, and functionality for the item ingest forms as well as the end-user search interface is in the process of being implemented. It is anticipated that all basic objects from the DSpace repository will be ingested by the end of April, and support for (and ingest of) more complex digital objects (items with multiple images, books, oral histories, etc.) will be added by the end of May.
The new repository will be debuted at the Digital Commonwealth annual conference on May 1 – the 3:00 breakout session, “Digital Commonwealth 2.0: Creating Online Digital Collections with the Redesigned Repository System,” includes a full demonstration of the application, while a related session at 1:45, “Digital Commonwealth 2.0 and Metadata,” will demonstrate how to create metadata for digital objects using the new system.
For a full schedule of the conference and program descriptions, view this PDF document.