Audiographics: Annotating Sound

August 29, 2017 | Posted in Data Visualization, JavaScript

Over the last several weeks, I’ve been working on several prototypes to help facilitate different types of annotations. Most educators are already pretty familiar with the typical textual annotation, but as new media becomes more important, we ought to have tools that facilitate annotation on other types of media as well.

What is an audiographic?

Which leads me to my first prototype of audio annotation, something that I’m calling an audiographic, a play on the omni-popular infographic but also a nod to the powerful contradiction we get when we combine something that was solely auditory with a visual component. One of the reasons that annotation can be so powerful as a learning strategy is that it immediately moves someone from passive consumption to active creation. Even if we are scrawling in the margins of a book, we are participating in the creation of the narrative around the text, whether or not we are the only ones with access to the annotations themselves. For most people this is a straightforward interaction with text or the written word. We are used to taking notes while reading, but for many people other media types like audio, video, and images remain a bit more inaccessible. Making music, painting, drawing, or sculpting are all things that OTHER people do, those creative folks lucky enough to be born with an artistic impulse. Hopefully, audiographics will allow people other than typical artists to join the conversation as well and articulate what they are hearing. You can see an example of the plugin below: [audiographic id=1331] One of the cool things about the idea behind an audiographic is the ability to add different types of media to particular audio segments: we can include text to add additional context to a particular segment, images to help build a cultural understanding of the movement or period, and even other audio segments that might be similar to/different from the main piece we are analyzing.

Other Applications

Down the road, there are also lots of other applications for this type of tool outside music. People involved in oral histories could use this pattern to add context to a narrative. People involved in discourse analysis could use this tool to segment and code different pieces of spoken discourse. With some additional tooling, groups could collectively annotate other audio resources, like popular podcasts, as a part of a class assignment.

Open Source Technology

Since this project is a larger initiative, and not just a bespoke tool, I’ve decided to document the technical details in a longer post dedicated to that subject, but the open source ethos behind it is important enough to mention hear as well. To make this tool widely available and easy to use, we decided to make it into a WordPress Plugin, which you can find in an evolving form here on GitHub. It builds on and extends other open source technology created by the BBC. It’s amazing to me how the tech community can be altruistic and entrepreneurial at the same time. While most enterprises focus solely on commercial viability, the development community has smartly realized that strong open source practices allow for wider dissemination of its chief products. Because of this, we can start to leverage this technology into some real cost savings from an educational perspective. All coolness of the audiographic aside, this plugin gives music instructors the ability to create and publish their own listening guides, something students would have typically shelled out $100’s of dollars for to a textbook company. However, perhaps more importantly, they can now own the means of production. The web-based portals with annotated audio files are no longer something produced for them, rather they are now something within their ability to create as a larger process of demonstrating competency in close listening and appreciation. All edu-rants aside, this is a very fun project that has challenged me to learn some additional stuff about JavaScript Web APIs, engage with and modify open source projects form a big name player like the BBC, and work through how to structure a WordPress plugin designed for wide usage and distribution.