How IIIF resources get their content

Tom Crane, Digirati, May 2017
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Annotations: how IIIF resources get their content

Here’s how the Oxford English Dictionary defines annotation:

1. A note by way of explanation or comment added to a text or diagram.
   'marginal annotations'

IIIF has a broader definition:

Annotations = Content

What is the content of a IIIF manifest? The Presentation API provides scaffolding to describe the structure of the object - a manifest has (usually one) sequence of canvases, each of which is an empty space with an aspect ratio and a coordinate system. Each canvas is analogous to a Powerpoint slide that is yet to have anything drawn on it. The larger scale structure of the object is described with ranges, to indicate chapters, articles, or any other relevant elements - but there’s still no content. There’s nothing to actually look at or read, other than the labels and descriptions on the Presentation API resources themselves.

All the content of the object is then hung on the scaffolding through the mechanism of annotation. As shown in the introduction, even images are annotations on the canvas. In a simple case, the scaffolding is clad with one image annotation per canvas, giving us the pages of the book to look at. More complex examples could have huge amounts of content associated with the object and each of its images - the full text, commentary, alternative images, videos, documents and more. Any content you have about an object can be connected to it through annotation, and made available to clients from the manifest.

In common usage, the word annotation might be reserved for comment-like content - short explanatory notes, tags and links. We could use annotation for that type of content, and some other mechanism to associate the "real" content like images and videos. But the annotation data model gives us all we need to connect any content with IIIF resources, there’s no need to have two mechanisms. And it does this as part of a distributed system, the web itself, that allows others to align content with a resource, not just the publisher of that resource.

Motivation

Annotations have a property called motivation that tells us why the annotation exists. Typical values of this property in general use are terms like commenting, tagging, and describing. IIIF adds a new motivation, painting. A painting motivation on an annotation means that the content should be rendered on the canvas (or a part of the canvas if the annotation targets a particular region). An image is an obvious example of content to be painted on the canvas (hung on the scaffolding), but a transcription of a line, word or paragraph would also have a painting motivation, as the body of the annotation - the transcribed text - can be overlaid on the image, and corresponds directly to the canvas at that region. On the other hand, an annotation that linked an object or a region of a canvas to a blog post or article about the object on another web page would not have a painting motivation, as it should not be rendered on the canvas surface. It is up to the client web page or viewer how to render painting and non-painting annotations.

The mechanism of annotation is how we get to see content in IIIF. Annotations are also for saying things about objects and collections:

• Why is this object in the exhibition?
• Here is a blog post about this painting
• There’s no video available here, but this is the screenplay for this missing 20 minutes of The Magnificent Ambersons
• …and here are some production stills for 49m23s - 51m17s
• “Tourteletes in fryture” means “Small fig pies basted with honey”
• Here is the curator’s essay for this collection
• Here is a transcription of this letter
• Here is an article about this correspondence
• These four engravings are all views from this location
• This person is Amelia Earhart
• This object is tagged “Aviation”, “Amelia Earhart”, “Honolulu”
• This fragment of text is about cats

All these things are content. Their bodies can be any web resource, but sometimes they are inline. The body of a tagging annotation might be a string value, but it can be a URL such as a Library of Congress subject heading or a Wikipedia page. If it’s your URL, you can use tagging annotations to build topic pages and navigation.

Where do annotations come from?

The content of an IIIF manifest doesn’t need to exist as annotations (e.g., in an annotation server) to be expressed as annotations in the model. In fact, for a typical manifest that contains images and text transcriptions, it’s likely that the content is assembled from sources like METS files, OCR data and catalogue records and then transformed into annotations to deliver the manifest and any linked annotation lists. That content is transformed so that it can be asserted as content.

You could supply comments, tags, and other content in the same way, if you already have that information in some other form. You’re more likely to be dealing with annotations natively with more dynamic content, especially when that content is coming from end users. Tags, comments, notes, bookmarks etc - these are all good candidates for native treatment as annotations, in an annotation server.

Given that all content is conveyed through annotations, any search of that content should return annotations as results. Which is the subject of the next article. Searching annotations means searching content.