Summary

How do we promote the use of archival audiovisual (AV) artifacts held at libraries, archives, and museums by researchers, teachers, and librarians? AVAnnotate is an open-access digital application and workflow that facilitates creating digital AV exhibits with archival audiovisual artifacts that include time-stamped notes and contextualizing essays and images. Teachers have developed AVAnnotate exhibits with hard-to-hear oral histories, lengthy interviews, or choppy videos to provide learners with context that is more easily indexable and discoverable. Researchers have used AVAnnotate to contextualize a range of digitized AV artifacts from bilingual editions of archival radio to annotated silent films within scholarship. Using AVAnnotate empowers users to explore alternative ways of analyzing and presenting AV artifacts. In this lesson, students learn to build simple digital exhibits with AVAnnotate using digitized video recordings of interviews from the Voces Oral History Center across different curricular contexts. During the lesson, instructors are encouraged to pause after step 3 and during steps 4 and 5 to assess student progress with questions included in the lesson outline. 

Authors

Zoe Bursztajn-Illingworth, Texas State University & Trent Wintermeier, University of Texas at Austin

Learning Outcomes

1. By using AVAnnotate to build digital exhibits with archival audio and video, students will develop information literacy around organizing annotations and discovering facets of archival audiovisual materials that could otherwise go overlooked. 

2. Students will learn through independent practice how to categorize, conceptualize, and describe what they see and hear in archival recordings in order to ethically frame the artifact in its historical context.

3. Create a digital exhibit using HTML and Markdown that combines transcription and descriptive annotations alongside optional contextual essays with embedded links and images. 

Audience

This lesson will be of interest to librarians and archivists working to contextualize digitized audio and video in institutional collections, instructors in humanities and communications classrooms working with audio and video artifacts who want to incorporate a digital method or project into their course, and undergraduate and graduate students looking for a flexible workflow that allows them to build digital exhibits with digitized audio and video. To build more elaborate digital exhibits with additional pages and images students will need to be familiar with Markdown or HTML. However, this is not required to build a basic annotation page. All that is required to build an AVAnnotate project is a computer, an internet connection, a GitHub account, and access to Google Sheets or Microsoft Excel.

Curricular Context

This lesson took place in Maggie Rivas-Rodriguez’s graduate course, “Qualitative Interviewing Methods,” in the Department of Journalism and Media in Fall 2023. The authors of this lesson were tasked with teaching Dr. Rivas-Rodriguez’s students how to create projects with AVAnnotate that included transcriptions and annotations of digitized video recordings of interviews from the Voces Oral History Center. The students in Dr. Rivas-Rodriguez’s course used these digital projects to present their critiques of the interviews they selected. The students were given AI-generated transcripts of the recordings to start and were told to focus on the interviewing methods and their outcomes. Students annotated the interviewer’s use of follow-up questions, their body language, and responsiveness, as well as the responses these elicited from the interviewees including the depth of their answers, their emotions, and implied context. 

The instructor had previously discussed qualitative interviewing techniques, which provided them with the necessary skills to create exhibits focused on aspects of the interviewer-interviewee relationship. Due to the course’s goal to help students become better interviewers, the students’ knowledge allowed them to focus on the interviewer’s tone, the quality of interviewee responses, and other parts of the video recording which would inform their practice of different interviewing techniques. For other courses in different disciplines, instructors should carefully decide what they want students to focus on in their annotations (for film studies, this might be elements of film form or aspects of film preservation; for sound studies, feedback, musical notation, and audio quality). The possibilities for audiovisual materials to be annotated are almost endless—from bird calls to home video recordings to electronic music.   

Students benefited from prior knowledge of using spreadsheets in Google Sheets or Excel and understanding the format and function of a time-stamp. For instance, we clarified that one should indicate a moment in a recording that occurs one hour and one minute into it, such as 01:01:00. It is also helpful to understand the function of rows, columns, and cells in a spreadsheet. To properly organize the data that is integrated into the AVAnnotate application, students should understand that columns organize information into distinct categories while rows describe information belonging to each individual entity. 

Preparation

Preparation prior to an AVAnnotate lesson is minimal. The lesson does require a laptop, computer, or tablet with access to the internet. Headphones are also suggested, as the process of annotation requires listening to AV which frequently includes audio. 

Plus, having identified AV for a project can be done in class or during the lesson, but it is helpful to have a file prepared prior to the lesson. If an instructor, student, or librarian has an MP3 or MP4 to use for the lesson, they will have to upload the file to the Internet Archive and retrieve the generated stable link to integrate into AVAnnotate. In Dr. Rivas-Rodriguez’s course, students had interviews that they were assigned, and their files were accessible and ready to integrate into the AVAnnotate software, and this technical preparation was helpful for the lesson. 

It may be helpful for the instructor to have a projector to work through the AVAnnotate workflow (as described in the lesson below) with students. Showing students how to input data into a spreadsheet, how to create a III manifest, or how to develop context in GitHub may be better understood visually. IIIF is a standard that was originally developed for publishing digitized manuscripts in a way that facilitated reuse. It didn’t define what metadata should apply to the images, nor define a new image format; rather it standardized ways to present existing image files and metadata for re-use. In the last few years, IIIF has been applied to AV material, or “time-based media.”  This promises the same kind of interoperability, allowing a researcher to mix segments of media from different sources without needing to download them and create new files.

Materials 

Materials that may be pertinent to the construction of an AVAnnotate project include a laptop or tablet with internet access, which is required for using the software and application. A GitHub account is required to log into the AVAnnotate application. To develop annotations, a Google or Microsoft Office account is required to access Sheets or Excel where annotations are created. AVAnnotate has a template available that outlines the format of annotations, which is not pertinent to developing a project, but it may be helpful. The template is linked here. Further, an audiovisual item is necessary; an MP3 or MP4 file must be accessible to develop a IIIF manifest within the AVAnnotate application. Or, if an audiovisual artifact has a IIIF manifest already, then such a manifest can be linked in the AVAnnotate application. 

Lesson Outline

This lesson will take one hour to one hour and thirty minutes to complete. The lesson begins with an introduction to the goals and affordances of the AVAnnotate software, which are to promote the sustainability and discoverability of AV artifacts and to increase access to the knowledge and information of those materials through annotation. Please refer to the lesson summary for information about the particular affordances of AVAnnotate as an open-access workflow for creating digital exhibits. Each of these steps contributes to the goal of developing critical digital literacy through the curation of a digital exhibit of annotated AV artifacts with AVAnnotate.

Following this discussion, the lesson includes:

1. Setting up a GitHub account/ logging into AVAnnotate via GitHub (10 minutes)

First, students will begin by creating a GitHub account and using those credentials to log into the AVAnnotate application. Since AVAnnotate uses GitHub to store code and GitHub Pages to create the static sites, a GitHub account is required to begin a project.

2. Making an AVAnnotate exhibit shell/GitHub repository (15 minutes)

Once in the AVAnnotate interface, students can “Create New Project,” which will include inputting information such as the title, description, and slug. This step creates a GitHub repository for the new project being created. 

3. Creating/integrating a IIIF manifest for an AV item (5 minutes)

Students may integrate their AV file in two different ways: First, they can create a IIIF manifest for their AV material. AVAnnotate allows students to create a IIIF manifest in the application, and this process requires inputting some information about the artifact, such as the name, duration, URL (which must end with .mp3 or .mp4, and other information about where the material is located online or sourced from. If students do not have an AV artifact ending with .mp3 or .mp4, it’s suggested that they upload the file to the Internet Archive (instructions on how to do so can be found here) and retrieve the URL from their interface. 

The second way of integrating a file is by pasting in a IIIF manifest for the AV file, which requires there to be a IIIF manifest already associated with the AV material. If there is a IIIF manifest available, this can be uploaded directly to the AVAnnotate application. An understanding of IIIF manifests is not required for making an AVAnnotate project with a stable link.

At this point, the instructor should pause to ask students the following questions: Have you been able to successfully attach a page with a direct link to your recording or import a IIIF manifest? If not, how have you input the project slug? Where did you find the link or IIIF manifest? Does the link end in a .mp3 or .mp4? If you used The Internet Archive, what kind of information (metadata) did you input that might be useful to help others discover this audiovisual artifact? 

Summary

How do we promote the use of archival audiovisual (AV) artifacts held at libraries, archives, and museums by researchers, teachers, and librarians? AVAnnotate is an open-access digital application and workflow that facilitates creating digital AV exhibits with archival audiovisual artifacts that include time-stamped notes and contextualizing essays and images. Teachers have developed AVAnnotate exhibits with hard-to-hear oral histories, lengthy interviews, or choppy videos to provide learners with context that is more easily indexable and discoverable. Researchers have used AVAnnotate to contextualize a range of digitized AV artifacts from bilingual editions of archival radio to annotated silent films within scholarship. Using AVAnnotate empowers users to explore alternative ways of analyzing and presenting AV artifacts. In this lesson, students learn to build simple digital exhibits with AVAnnotate using digitized video recordings of interviews from the Voces Oral History Center across different curricular contexts. During the lesson, instructors are encouraged to pause after step 3 and during steps 4 and 5 to assess student progress with questions included in the lesson outline. 

Authors

Zoe Bursztajn-Illingworth and Trent Wintermeier

Learning Outcomes

1. By using AVAnnotate to build digital exhibits with archival audio and video, students will develop information literacy around organizing annotations and discovering facets of archival audiovisual materials that could otherwise go overlooked. 

2. Students will learn through independent practice how to categorize, conceptualize, and describe what they see and hear in archival recordings in order to ethically frame the artifact in its historical context.

3. Create a digital exhibit using HTML and Markdown that combines transcription and descriptive annotations alongside optional contextual essays with embedded links and images. 

Audience

This lesson will be of interest to librarians and archivists working to contextualize digitized audio and video in institutional collections, instructors in humanities and communications classrooms working with audio and video artifacts who want to incorporate a digital method or project into their course, and undergraduate and graduate students looking for a flexible workflow that allows them to build digital exhibits with digitized audio and video. To build more elaborate digital exhibits with additional pages and images students will need to be familiar with Markdown or HTML. However, this is not required to build a basic annotation page. All that is required to build an AVAnnotate project is a computer, an internet connection, a GitHub account, and access to Google Sheets or Microsoft Excel.

Curricular Context

This lesson took place in Maggie Rivas-Rodriguez’s graduate course, “Qualitative Interviewing Methods,” in the Department of Journalism and Media in Fall 2023. The authors of this lesson were tasked with teaching Dr. Rivas-Rodriguez’s students how to create projects with AVAnnotate that included transcriptions and annotations of digitized video recordings of interviews from the Voces Oral History Center. The students in Dr. Rivas-Rodriguez’s course used these digital projects to present their critiques of the interviews they selected. The students were given AI-generated transcripts of the recordings to start and were told to focus on the interviewing methods and their outcomes. Students annotated the interviewer’s use of follow-up questions, their body language, and responsiveness, as well as the responses these elicited from the interviewees including the depth of their answers, their emotions, and implied context. 

The instructor had previously discussed qualitative interviewing techniques, which provided them with the necessary skills to create exhibits focused on aspects of the interviewer-interviewee relationship. Due to the course’s goal to help students become better interviewers, the students’ knowledge allowed them to focus on the interviewer’s tone, the quality of interviewee responses, and other parts of the video recording which would inform their practice of different interviewing techniques. For other courses in different disciplines, instructors should carefully decide what they want students to focus on in their annotations (for film studies, this might be elements of film form or aspects of film preservation; for sound studies, feedback, musical notation, and audio quality). The possibilities for audiovisual materials to be annotated are almost endless—from bird calls to home video recordings to electronic music.  

Students benefited from prior knowledge of using spreadsheets in Google Sheets or Excel and understanding the format and function of a time-stamp. For instance, we clarified that one should indicate a moment in a recording that occurs one hour and one minute into it, such as 01:01:00. It is also helpful to understand the function of rows, columns, and cells in a spreadsheet. To properly organize the data that is integrated into the AVAnnotate application, students should understand that columns organize information into distinct categories while rows describe information belonging to each individual entity. 

Preparation

Preparation prior to an AVAnnotate lesson is minimal. The lesson does require a laptop, computer, or tablet with access to the internet. Headphones are also suggested, as the process of annotation requires listening to AV which frequently includes audio. 

Plus, having identified AV for a project can be done in class or during the lesson, but it is helpful to have a file prepared prior to the lesson. If an instructor, student, or librarian has an MP3 or MP4 to use for the lesson, they will have to upload the file to the Internet Archive and retrieve the generated stable link to integrate into AVAnnotate. In Dr. Rivas-Rodriguez’s course, students had interviews that they were assigned, and their files were accessible and ready to integrate into the AVAnnotate software, and this technical preparation was helpful for the lesson. 

It may be helpful for the instructor to have a projector to work through the AVAnnotate workflow (as described in the lesson below) with students. Showing students how to input data into a spreadsheet, how to create a III manifest, or how to develop context in GitHub may be better understood visually. IIIF is a standard that was originally developed for publishing digitized manuscripts in a way that facilitated reuse. It didn’t define what metadata should apply to the images, nor define a new image format; rather it standardized ways to present existing image files and metadata for re-use. In the last few years, IIIF has been applied to AV material, or “time-based media.”  This promises the same kind of interoperability, allowing a researcher to mix segments of media from different sources without needing to download them and create new files.

Materials 

Materials that may be pertinent to the construction of an AVAnnotate project include a laptop or tablet with internet access, which is required for using the software and application. A GitHub account is required to log into the AVAnnotate application. To develop annotations, a Google or Microsoft Office account is required to access Sheets or Excel where annotations are created. AVAnnotate has a template available that outlines the format of annotations, which is not pertinent to developing a project, but it may be helpful. The template is linked here. Further, an audiovisual item is necessary; an MP3 or MP4 file must be accessible to develop a IIIF manifest within the AVAnnotate application. Or, if an audiovisual artifact has a IIIF manifest already, then such a manifest can be linked in the AVAnnotate application. 

Supplementary Materials 

Lesson Outline

This lesson will take one hour to one hour and thirty minutes to complete. The lesson begins with an introduction to the goals and affordances of the AVAnnotate software, which are to promote the sustainability and discoverability of AV artifacts and to increase access to the knowledge and information of those materials through annotation. Please refer to the lesson summary for information about the particular affordances of AVAnnotate as an open-access workflow for creating digital exhibits. Each of these steps contributes to the goal of developing critical digital literacy through the curation of a digital exhibit of annotated AV artifacts with AVAnnotate. Following this discussion, the lesson includes:

1. Setting up a GitHub account/ logging into AVAnnotate via GitHub (10 minutes)

First, students will begin by creating a GitHub account and using those credentials to log into the AVAnnotate application. Since AVAnnotate uses GitHub to store code and GitHub Pages to create the static sites, a GitHub account is required to begin a project.

2. Making an AVAnnotate exhibit shell/GitHub repository (15 minutes)

Once in the AVAnnotate interface, students can “Create New Project,” which will include inputting information such as the title, description, and slug. This step creates a GitHub repository for the new project being created. 

3. Creating/integrating a IIIF manifest for an AV item (5 minutes)

Students may integrate their AV file in two different ways: First, they can create a IIIF manifest for their AV material. AVAnnotate allows students to create a IIIF manifest in the application, and this process requires inputting some information about the artifact, such as the name, duration, URL (which must end with .mp3 or .mp4, and other information about where the material is located online or sourced from. If students do not have an AV artifact ending with .mp3 or .mp4, it’s suggested that they upload the file to the Internet Archive (instructions on how to do so can be found here) and retrieve the URL from their interface. 

The second way of integrating a file is by pasting in a IIIF manifest for the AV file, which requires there to be a IIIF manifest already associated with the AV material. If there is a IIIF manifest available, this can be uploaded directly to the AVAnnotate application. An understanding of IIIF manifests is not required for making an AVAnnotate project with a stable link.

At this point, the instructor should pause to ask students the following questions: Have you been able to successfully attach a page with a direct link to your recording or import a IIIF manifest? If not, how have you input the project slug? Where did you find the link or IIIF manifest? Does the link end in a .mp3 or .mp4? If you used The Internet Archive, what kind of information (metadata) did you input that might be useful to help others discover this audiovisual artifact? 

4. Developing and uploading annotations (30 minutes to one hour)

Developing annotations for the AV file happens through Google Sheets or Microsoft Excel. Either way, AVAnnotate provides a template for users to start annotating. This template, and all AVAnnotate projects for that matter, use the following columns: timestamp start, time stamp end, annotation, layer, and index (optional). Students should use the first two columns to put in the time stamp that they wish to be associated with their annotation. For example, if the annotation is related to minute one to minute two of a recording, the first column should display “1:00” and the second column “2:00.” 

The third column is the annotation itself, which may serve almost any purpose in regard to the student’s project. The annotation can be any length (shorter is suggested, though, due to the interface) and any content; the content of the annotation can vary depending on the purpose and goals of the project. A shorter annotation may be one to two sentences in length. Another way to think about this is a shorter interval of time in the recording like annotating in thirty seconds intervals for a five-minute recording, rather than in three-minute intervals. 

At this point, once students have had a chance to annotate a significant portion of the AV artifact, the following question should be considered: How did the process of annotation assist with the discovery of facets of audiovisual materials that could have otherwise gone overlooked or unnoticed?

The layer serves as a way to categorize the annotation. For example, if the annotation, such as in Maggie Rivas-Rodriguez’s course, concerns dialogue from an interviewee, then the layer for that annotation may be “Interviewee.” The layer functions as a way to describe the individual annotation it’s associated with. This allows students and other viewers of a project to more efficiently work across the annotations of a recording, and it also allows students to make an argument through annotation by pointing out specific qualities of the AV that they seek to analyze.

The index is optional but suggested, and it functions as an index would in the back of a physical book. The index works well for projects that have multiple AV items. For example, if a course included five students working with five different recordings, each exhibit and AV item could be integrated into a single project. Then, a collective index could be applied to each individual exhibit so that the students or any other viewer could identify similarities and differences in content across the exhibits.

At this point, the discussion may concern the conceptual development of the layers and index columns: How have the layer and index functions helped with the categorization and organization of the annotations? How did these functions help the ethical framing of the project? 

5. Adding page material via GitHub (to be completed by students following the workshop if the instructor’s assignment requires contextualizing essays or images)

Finally, individual pages, without AV material, can be added to exhibits. They may include photos, links, introductory material, context, or any other details. Since AVAnnotate uses GitHub, adding details to a page happens in the project’s GitHub repository, which can be accessed from the AVAnnotate interface. This requires some knowledge of HTML or Markdown, which may be required for students, depending on what they seek to accomplish with these pages. The more complex the goals, the more they will need to know about a language. For instance, if a student would like to embed images in their project pages, they will need to know more complex HTML or Markdown. A guide to basic writing and formatting Markdown for GitHub is available here, which can assist students in the curation of contextual and introductory information.

At this point, familiarity with HTML and Markdown should be considered alongside the annotations and/or transcription: What contextual materials are required in this exhibit? How do additional images or text add to the information or ideas represented in the annotations? How can knowledge of HTML and Markdown accomplish the goals of the exhibit, based on what was discovered during the process of exhibit development and annotation?

After this process, users can explore and troubleshoot aspects of their AVAnnotate exhibit. Holding space and time for this during the lesson is an essential and productive part of the lesson. After following the steps of this lesson, students should have a complete AVAnnotate project that is published online. For any troubleshooting, support is available from the AVAnnotate team through Google Groups or via the emails listed here.  

Assessment

At the end of step three of the lesson and during steps four and five, we have included several questions that will help instructors assess student understanding of the material as it is presented. At the end of Maggie Rivas-Rodriguez’s course, most students were able to build an AVAnnotate project just in a single class period. While the completion of a project is part of student learning, too are the concepts that AVAnnotate uses important to assessment. Understanding the index, layer, and annotation function is essential to understanding the goals for each exhibit. 

Therefore, the final assessment can be measured in how each student is able to employ index terms, layers, and annotations to meet the goals of their exhibit. Since how these three components are used will differ based on the AV material and purpose of the project, the assessment must be tailored to the individual student.

Adaptability 

If this lesson were used with a particular institutional archive, librarians working with the collection would also need to collaborate to make their audiovisual artifacts accessible through IIIF. Due to AVAnnotate’s use of IIIF, the video exhibits that students created can be edited collaboratively. AVAnnotate allows users to create IIIF manifests in the software’s application, allowing students to engage with each other’s annotations. Discussion on the process of creating a IIIF manifest is discussed in step three above, and information about uploading these materials to the Internet Archive is discussed in the context of preparing for this lesson. 

The types of instructors that may use AVAnnotate to accomplish curricular and pedagogical goals may be from fields such as film studies. For example, Bursztajn-Illingworth has also taught students in film studies using a version of this lesson that builds on knowledge of film form. Her undergraduate students annotated a video hosted on the Internet Archive while being taught a lesson on how to build projects in AVAnnotate.

Other instructors from disciplines such as Rhetoric and Composition, or instructors teaching courses covering topics of digital humanities, archival practices, library studies, preservation methods, and digital and emerging media. However, this lesson is designed to be broadly applicable across disciplines working with audiovisual material. The objectives of instructors, researchers, and archivists across disciplines may vary, which is why specific ideas and suggestions are left open to the needs of whoever is facilitating this lesson. 

Reflection

While this lesson can be taught virtually, ideally it will be taught in person. In-person instruction allowed the instructors to circulate and correct any minor issues with individual spreadsheet formatting as needed before students build their annotation projects. Due to AVAnnotate being a web-based software and application, there are a variety of modalities suited for creating digital exhibits in classrooms and other scholarly settings. In a classroom setting, having a collaborative space to work on different aspects of the annotation process and exhibit construction proves to be helpful for designing individual projects. Holding space for students to ask questions while working collaboratively through each step of creating a project allows for ideas and concerns to circulate. 

In a virtual setting, the workshop may be adapted to include breakout rooms during certain parts of the workshop so that instructors can both troubleshoot with students and allow students to ask each other questions. Annotation may still happen collaboratively in breakout rooms, and this can happen between students and without the instructor. A virtual setting can provide unique benefits to the use of the application, in this way. However, a virtual setting is best suited for multiple instructors as a virtual workshop would require engaging with each breakout room through screen sharing, Q&A, and potentially other modes of troubleshooting. In a classroom setting, the room should be set up to have students work in small groups on one or multiple AV files to discuss the process of annotation with one another. 

Acknowledgments

The authors would like to thank Tanya Clement for reviewing an early draft of this submission. We thank Maggie Rivas-Rodriguez for giving us the opportunity to teach a workshop in her classroom and providing recordings from the Voces Oral History Center. We would also like to thank our reviewers Kelsey Brown and Seul Lee for their generous and thoughtful feedback.