On Square Pegs, Round Holes, PREMIS Rights Statements and Apollo 13

As mentioned before in a previous post on PREMIS and PREMIS Rights Statements, we've been exploring ways that we can create rights statements as we're processing SIPs in Archivematica and then use those rights statements to set access profiles for the AIPs in our DSpace repository.

At that point, our our thinking was mostly theoretical. Since then, we've had some time to think about it, to confer with our MLibrary colleagues as well as those at the Rockefeller Archive Center and even reflect on Ed Pinsent's comments on the last post (thanks, everyone!). In this post, I'd like to give an update on how we plan (yes, still just a plan--things could change!) to actually do it. Before I dive in, though, I should remind our readers that what we're proposing here is a bit like trying, as the expression goes, to fit a "square peg in a round hole." Here's a quote from the PREMIS Data Dictionary for Preservation Metadata:

PREMIS primarily defines characteristics of Rights and permissions concerned with preservation activities, not those associated with access and/or distribution.

Yikes. "Not those associated with access and/or distribution." hrm...

The Access Profiles

Let's start at the end. In DeepBlue, our DSpace repository, we have some amount of control over both a digital object--or, to use DSpace-speak, bitstream(s)--and its associated metadata--or item. We can associate each of them (independently of one another) with one of four (or actually as many as we care to create) of what are called groups. In practice, we apply a handful of common combinations of item and bitream(s) groups when we deposit AIPs in our DSpace repository:

• Open: Both items and bitstreams are open to be viewed/downloaded by anyone in the whole world.
• Bentley Reading Room users: While items can be viewed by anyone, downloading of bitstreams must be done from within the Bentley's IP range. This can be from a wired or wireless connection.
• University of Michigan users: Items can be viewed by anyone, but only University of Michigan affiliates may download bitstreams.
• Totally restricted/embargoed items: Nobody (except Bentley archivists, who may also fulfill reference requests) can view or download anything, item or bitstream(s). Typically, these types of things are embargoed until a particular date (based on local policies), at which point in time both item and bitstream(s) will become open.[1]
• Audio/visual items with copyright or other types of concerns: Items can be viewed by anyone, but only Bentley archivists can download bitstreams. Heretofore this profile consists mostly of audio/visual material that is preserved in DSpace but made available for streaming (not downloading) in the Bentley Digital Media Library.

A Quick Refresher on Act[ion]s in PREMIS Rights Statements

As a reminder, PREMIS Rights Statements are made up of one basis (the raison d'être of the rights statement, something like copyright or policy) and one or more actions associated with that basis (these are very specific actions the repository is or isn't allowed to do). Since the basis won't have an impact on its associated action, I won't go into much detail about them here.

Actions come from a controlled vocabulary, made up of things like:

• replicate: make an exact copy
• migration: make a copy identical in content in a different file format
• modify: make a version different in content
• use: read without copying or modifying
• disseminate: create a copy or version for use outside of the preservation repository
• delete: remove from the repository

As you can see, these have a very "digital preservation" feel (in the most narrow sense of the word[2]). Hence the data dictionary's warning above.

Actions may be allowed in all cases or, of course, they may have restrictions. These express situations where, for instance, dissemination is permitted, but only to a specific type of person (say, one that's affiliated with your institution), or, taken to the extreme, that dissemination is not permitted, period. At least in Archivematica's case, you've got three choices to that express such restrictions: allow, disallow or conditional. This may sound like it covers a lot, but as you'll see, we had to get a little creative with this, as we end up using "conditional" to describe a number of different conditions.

Other than that, there are some begin and end dates associated with that action, and a note containing a textual description of the right granted if additional description is needed.

Mapping PREMIS Rights Statements in Archivematica to DSpace Groups

SIP rights template--second page

Now on to mapping the PREMIS Rights Statements implementation in Archivematica to the groups in DSpace. There's a couple different ways we might have approached this.

One way might have been to try to use an Act to tell the repository exactly what it was allowed to do with both the item and the bitstream for a particular AIP. While this approach gave us the granularity we'd need for machine-actionable PREMIS Rights Statements, we worried that it would be overly cumbersome for our human processors that would be, for the most part, manually adding them to SIPs and keying in the data.

Another way might have been to use a local controlled vocabulary for the Act field, something like "disseminate-bentley", "disseminate-umich", etc. However, associating a particular target with an action seemed, in the word of one of our DSpace gurus, "somewhat contrary to the spirit of the allowed actions" (see this sample controlled vocabulary for the 'act' element, some of which I listed above). You'll also notice if you click that link that "disseminate-bentley", "disseminate-umich", etc. are not on that list, and for good reason! We even thought briefly about using the Restriction field to specify the target audience before realizing that it too has a controlled vocabulary (one that's actually enforced by Archivematica and then used later on in some logic).

In the end, we settled on using the Note field to specify audience. Now, we know this isn't the most elegant solution--in general, the intention of the notes is specifically to not be machine-actionable, but we felt that since this PREMIS Rights Statement would ultimately be preserved in the AIP (in the METS!), and since there's a chance someone might run across it outside of our repository environment, that this was the way to go.

So here's our plan, at least an overview:

DeepBlue Groups		Archivematica PREMIS Rights Statements
Item	Bitstream	Act	Restriction	Restriction note
Anonymous	Anonymous	None	None	None
Anonymous	Reading Room only	disseminate	Conditional	Reading Room
Anonymous	University of Michigan only	disseminate	Conditional	University of Michigan
Anonymous	Archivists only	disseminate	Conditional	BDML
Archivists only	Archivists only	disseminate	Disallow	Executive records (ER) Personnel records (PR) Student records (SR) Patient/client records (CR)

A couple of notes here:

• We will not use PREMIS Rights Statements (at least those that apply to access/distribution) for AIPs that don't have restrictions.
• When we do use PREMIS Rights Statements, they will be as minimal as we can make them with the intention that they will only be used by machines, not humans. Human readable rights statements will be recorded elsewhere, like ArchivesSpace Conditions Governing Access and Use notes. 
• Most of the time, the End Date field will be OPEN, except when a Bentley policy is involved (ER, PR, SR and CR above). In those cases, an end date will let the repository know when a particular restriction expires.

Once an AIP with some sort of restriction is ready to go to DeepBlue, we'll park it somewhere temporarily[2], parse the METS file in the AIP, determine (based on the rights statements) the item and bitstream permissions, convert it to the DSpace Simple Archive Format and upload in batch to DeepBlue from there. It's sounding like the identifier for the Digital Object in ArchivesSpace will be in the AIP, so we're pretty confident we'll also be able to add the Handle back to ArchivesSpace farily easily as well.

We also think (hope!) that this approach, as long as we're consistent, would allow us to change our minds relatively easily in the future, say, if we decided after all that a more granular approach was the way to go.

But Wait! "disseminate" is Hard to Spell!

It occurred to us that in order for this approach to work, our processors can never make typos. We've all been there... this is a pretty unrealistic expectation.

For the time being, we're planning to use Greasemonkey (in Firefox) and Tampermonkey (in Chrome) to help us out with this particular problem. These are browser extensions that customize the way a web page displays or behaves using small bits of JavaScript.

We've written a fairly basic script (you can see our draft here), that looks for URL patterns that match the Add Act pages in Archivematica (as you can see in that script, http://sandbox.archivematica.org/transfer/*/rights/grants/*/ and http://sandbox.archivematica.org/ingest/*/rights/grants/*/). When it finds one, it adds an additional dropdown, like so...

It even has a nice logo!

When an option is chosen (Reading Room was chosen above), it automatically fills out the rest of the form, just like we need it. When a Bentley policy is involved (that requires an end date), it asks a processor for a creation or accession date (still working on a nice datepicker option for this), does some math, and calculates the appropriate end date. It's not the most elegant solution but we think it works for now!

Conclusion

In the end, it's perhaps a little clearer as to why PREMIS wasn't really meant for this kind of thing. Still, maybe square pegs sometimes do fit into round holes...

Seriously, though, let us know what you think!


[1] Although these types of things are not viewable, downloadable or even searchable in DSpace, typically we still provide a link to them in the collections finding aid. 
[2] Philosophically, I'd argue that access and distribution is a fundamental part of digital preservation... maybe the most fundamental part.
[3] At the end of the grant, AIPs without restrictions will be automatically uploaded to DSpace and recorded in ArchivesSpace without any more human intervention!

A Primer on PREMIS and PREMIS Rights

In today's post, I'd like to talk a little about PREMIS (the data dictionary, not the working group--although I'm sure they're all great people, like Evelyn!). We've been using something akin to "PREMIS Lite" as part of our digital archiving workflow for a while now. As part of our work on the ArchivesSpace-Archivematica-DSpace Workflow Integration project, however, and in thinking about our eventual move to Hydra, we're gearing up to implement something more like PREMIS Proper, especially for PREMIS Rights Statements.

An Overview of PREMIS

PREMIS Data Dictionary for Preservation Metadata, Version 3.0

Let's dive in with an overview of PREMIS. The following is lifted straight from their website:

The PREMIS Data Dictionary for Preservation Metadata is the international standard for metadata to support the preservation of digital objects and ensure their long-term usability. Developed by an international team of experts, PREMIS is implemented in digital preservation projects around the world, and support for PREMIS is incorporated into a number of commercial and open-source digital preservation tools and systems. The PREMIS Editorial Committee coordinates revisions and implementation of the standard, which consists of the Data Dictionary, an XML schema, and supporting documentation.

Leaving descriptive metadata, especially domain-specific descriptive metadata, to the many existing descriptive metadata schemes and encoding standards (like MARC, MODS, Dublin Core, EAD, etc.), and leaving super format-specific technical metadata to those who would get super nerdy about format-specific technical metadata (you know who you are), what I love about PREMIS is that it focuses specifically on one of my favorite things, digital preservation. It allows the digital archivist to record and, in some ways it event defines--although they might not like that I said that--the common denominator of preservation actions a preservation repository might perform on a digital object.

As of PREMIS 3.0, the data model consists of four entities:

The PREMIS Data Model

• Digital Objects:  Discrete units of information subject to digital preservation. This could also be an "Environment," that is, hardware or software that support a Digital Object in some way, like rendering or executing it. Digital Objects are further broken down into the following subcategories.

Conceptual view between object categories

• Intellectual Entity: A set of content that makes up a single unit for purposes of management and description. Pretty much anything, at any level, can be an intellectual entity, and intellectual entities may (or may not) be made up of other intellectual entities. A website (an intellectual entity you might describe or manage as an aggregate) may have a webpage (another intellectual entity) which may have an image (another intellectual entity, especially if you apply some additional descriptive metadata to it  or migrate its file it to a new format).
• Representation: A set of files, including structural information, needed for a "complete" or at least "reasonable" rendition of the Intellectual Entity. To take a book as an example, consider different ways you might represent it: a single PDF file, 10 images, one for each page (which must be read in a particular order), etc.
• File: A sequence of bytes. This is the thing that has a format, access permissions, last modification date, etc.
• Bitstream: Contiguous and non-contiguous data within a file that has meaningful common properties for preservation purposes. Makes perfect sense, right?
• Events: Actions, like preservation actions, that involve Digital Objects or Agents.
• Agents: People, organizations or software that are associated with Events (or the rights attached to them). That is, people like me, organizations like the Bentley Historical Library or software like Archivematica or FIDO.
• Rights Statements: Assertion of one (or more, which is the exciting part, but more on that later) rights or permissions statements pertaining to a Digital Object or Agent.

That's about it. If you want more information you can read the latest PREMIS Data Dictionary yourself!

PREMIS at the Bentley

We currently record information (that is, identifier, date and time, detail and outcome) about the following PREMIS event types (with ourselves as the agent as well as the piece of software we used):

• Virus Scan/Check: The process of scanning a file for malicious programs.
• Personally Identifiable Information Scan: Hmm, I can't really seem to find this one in the controlled vocabulary. Maybe we made this one up? In any case, internal consistency is what's really important, right?
• Identify Missing File Extensions/File Extension Change: Assignment of a new filetype extension to a file object; typically done only if the existing extension was found to be incorrect.
• Compression of Files: The process of coding data to save storage space or transmission time. We go for lossless and don't actually compress anything.
• Technical Metadata Extraction: Extraction of technical (or non-technical) metadata like the resolution, colordepth etc. from a file using tools such as JHOVE.

That's it! PREMIS Lite! This information gets recorded in the humble CSV format (no fancy XML for us!). We also record additional information that technically counts as PREMIS Events, like checksums (and the algorithms we used to calculate them), as well as what files got normalized (and what they got normalized to). While this information doesn't make it into this particular CSV, it does get recorded elsewhere.

The Future: PREMIS Rights Statements

By virtue of us moving to Archivematica, within mere months we'll become much more PREMIS Proper-compliant (and we'll have that PREMIS in XML!). But that's not all--there's also been a lot of discussion around here (and around MLibrary, Artefactual, ArchivesSpace, the Hydra folks and various listservs) about PREMIS Rights, especially the communication of PREMIS Rights Statements between systems, and the enforcement of them by repositories.

First, the Why

Before I jump into an outline of PREMIS Rights and all of their granular goodness, I wanted to talk briefly about the problem that PREMIS Rights Statements solve. Take, for example, two familiar standards, Dublin Core and EAD:

<dc>
  ...
  <rights>Some random free text rights statement.</rights>
  ...
</dc>

<ead>
  ...
  <accessrestrict>Some free text Conditions Governing Access note that ends on this <date>date</date>.</accessrestrict>
  <userestrict>Some free text Conditions Governing Use note.</userestrict>
  ...
</ead> 


Look familiar? Now, I'm not trying to bash Dublin Core or EAD and, certainly, some rights statements are better than no rights statements. However, there's a big issue I see with these examples. Because they are free text fields, they aren't really machine-actionable. Of course, an institution can try to be as prescriptive as possible about how to fill these out (and, in fact, we are one of those institutions), and that can arguably lead to rudimentary machine-actionability, but over time it's hard to make the case that this is a sustainable approach. It's just too hard to keep folks entering metadata on the straight path. Eventually, dates get entered in all kinds of formats and non-standard text proliferates. DPLA knows this perhaps like no other institution, because they get rights statements from institutions all over the US: "For DPLA content alone, contributing organizations have used over 87,000 different rights statements," part of the motivation behind rightsstatements.org. This variety makes it hard for a computer to predict or parse and, ultimately, enforce rights statements. 

So, for example, you can say that a particular component of a collection has an "Executive Restriction until February 5, 2036," and while that works just fine for the reference archivist or researcher in your reading room looking at a finding aid or box list, if you're talking about digital content you still have to tell the repository separately (in our case, we "e-mail José") to embargo this content for 20 years, and you still have to go back in 20 years and update the original rights statement because it has expired. Human error abounds in these types of situations, and I'm sure we're not alone in having had a researcher let us know that a particular restriction has expired. To add a layer of complexity, "Executive Restriction," when taken out of the context of a University of Michigan collection at the Bentley Historical Library, doesn't mean much to a researcher halfway across the world who might be accessing our content through something like DPLA. To put on my librarian hat for a second, free text fields also don't lend themselves to explicitly identification with standard licenses such as Creative Commons, but perhaps that's hardly ever the case for archival material.

PREMIS Rights Statements, by contrast, were designed to be machine-actionable and interoperable between software systems. They do allow for some free text explanation of what would probably be an human-unreadable rights statement, but first and foremost PREMIS Rights Statements are about solving the problem I outlined above.

PREMIS Rights Statements

PREMIS rights statements provide a flexible framework for describing both rights ("entitlements allowed to agents by copyright or other intellectual property law") and permissions ("powers or privileges granted by agreement between a rights holder and another party").

The data dictionary outlines four types of rights, or rights bases to which permissions can be linked: copyright, license, statute and other for everything else, including all those weird donor requests. Acts get applied a basis, and restrictions can refine an act. A simple example is given in Implementing the Rights Entity in Archivematica by Evelyn McClellan, a draft chapter for an upcoming book on PREMIS we got to sneak a peak at. 

Thus, a simple rights statement in a PREMIS implementation can consist of a rightsBasis (such as copyright); an act (such as replicate), and information about any restriction on the act (for example, replication permitted only for the purpose of making preservation copies).

The other cool thing about PREMIS is that you can apply multiple rights bases and associated acts and restrictions to a given digital object. For example, you may have an institutional policy (rightsBasis is Other) that allows you to have acts like migrating or deleting an item, while your transfer of copyright (rightsBasis is Copyright) allows you to have acts like modifying, publishing or disseminating an item. You can sort of do this in DACS and EAD since there are spots for both Conditions Governing Use and Access, but in those scenarios you only compound the issue I outlined above since now a machine won't be able to understand multiple human-readable rights statements.

Other Considerations

As part of our work on the grant project, I mentioned that we've been thinking a lot about PREMIS Rights. Here are some questions we're asking: 

• Could we use PREMIS Rights Statements for access? According to the data dictionary, this is slightly out of scope for PREMIS and PREMIS Rights Statements, as the latter are meant to apply more to preservation actions. However, it seems feasible and desirable to use granular PREMIS Rights Statements to tell our repository, for example, to embargo or not embargo a particular digital object, or to restrict access to a particular IP address. Artefactual has expressed interest in developing this functionality between Archivematica and AtoM, and we're interested in being able to record PREMIS Rights Statements in Archivematica, management them in ArchivesSpace, and have those be acted upon by DSpace or  Fedora/Hydra, for example.
• Is anybody using PREMIS Rights Statements in the way I just described? That is, is anyone actually using PREMIS Rights Statements to tell a repository what to do, particularly if those rights statements were recorded in a separate digital preservation or archival management system (particularly if those are Archivematica and ArchivesSpace, respectively)? So far as we know, the answer to that question is negative. If you are, let us know! We want to steal your stuff.
• Can we pass PREMIS Rights Statements from Archviematica to ArchivesSpace for subsequent maintenance of PREMIS Rights Statements? We want ArchivesSpace to be the system of record for this type of information, and we may sometimes need to edit rights statements, for example, when a restriction is based on donor death date and the donor is living when we process the collection. So far, the answer to this question is also negative; rights in ArchivesSpace are PREMIS-ish (they have a spot for rightsbasis but associated acts are one-to-one not one-to-many), while rights in Archivematica are fully PREMIS. Basically, there's nowhere (at least not yet, but there are rumors that this is changing!), to put this information in ArchivesSpace.

Conclusion

Thats about it for this Primer on PREMIS and PREMIS Rights! It's a Brave New World out there for rights statements and machine-actionability!

As an FYI, PREMIS is constantly being updated. Just this week there was a call to review the controlled vocabulary for preservation events. Let them know what you think!