Workshop Proceedings: Short Papers

Jack W. Bradbury and Carol A. Bloomgarden
Macaulay Library of Natural Sounds, Cornell Laboratory of Ornithology
1 59 Sapsucker Woods Road, Ithaca, N.Y. 14850
macaulaylibrary@cornell.edu

Establishing an archive of fish and other underwater biological sounds will meet many of the long-standing challenges faced by marine acousticians — the restoration and preservation of deteriorating recordings, the ability to catalogue their sounds and data in a way that fosters exchange and sharing of data comparative studies, easy access to the sounds for analysis and identification, and the capacity to search through passive recordings for sounds of particular interest. The Macaulay Library of Natural Sounds (MLNS), with a long history of working toward these goals in ornithology and animal behavior, recently launched into the realm of a new Internet-accessible archive of underwater sounds with the help of over sixty individual recordists and institutions worldwide. Researchers will be able to annotate their sounds with detailed and extensive data through an online database application, summarize search results in exportable tables and maps, and download copies of recordings for research, teaching, and conservation. MLNS is committed to dual goals of maintaining open access to allow other researchers to listen and help identify sounds, while protecting recordists’ copyrights and restricting access during the publication process. Detailed and extensive metadata are needed, however, to create the functionality such an archive requires.

A recent survey of suitable original recordings resulted in commitments by more than 60 researchers or institutions to supply original tapes and metadata for archival at MLNS. These consist of over 8000 hours of audio tape and 800 hours of video and include recordings of 95 species of marine mammals and more than 200 species of fish (representing 36 families) and marine invertebrates. Upon request, MLNS staff will visit participating institutions to help organize and pack original tapes, collect the metadata and any information required to import it, and then carefully track the status and location of all contributed material through the shipping and archival processes.

Many older recordings exist on deteriorating tape stocks. These must be treated before copying. MLNS has extensive experience and an excellent track record in tape restoration. Many of these tapes can be restored in-house using controlled baking, burnishing, and vacuum treatments. A few may be so deteriorated that they must be out-sourced to specialists who examine the molecular structure with electron microscopes before undertaking situation-specific restoration procedures.

Most analog sound tapes will beare digitized once at high resolution (96 kHz/24 bits). Although the high sampling and bit rates are not necessary for all recordings (given ambient noise levels and frequency composition of the sounds), these high-resolution settings greatly accelerate the archival process by freeing technicians from detailed monitoring of signal levels and inadvertent aliasing. They also preserve any high frequency sounds that are unnoticed or in the background but that may later prove of interest. Proper digitization of Odontocete sonar signals will require a combination of replay at reduced speeds and even higher digitization rates. Digital recordings will beare copied at their original rates. Analog video will be converted to Ddigital Betacam tape replicas. All digitized materials will be stored on local hard disks until transferred to hard media.

Whereas terrestrial recordists can limit recording time by watching their subjects, marine researchers often record blindly for long periods, and their recordings therefore often have a much smaller fraction of useful content than do terrestrial ones. At least a third of the contributed audio material consists of such unedited continuous recordings. Once digitized, these must be examined in real-time and the relevant sounds extracted. We will work with developers to adapt prototype detector software for real-time extraction of appropriate marine sound tapes. This tool will be essential for reducing raw audio streams to separate sound files.

All high-resolution digital copies of extracted sounds and videos will be preserved in a deep archive. Sounds will be stored as AIFF files on DVD-ROM discs in a computer-controlled jukebox array. Video will be encoded as MPEG-2 files and stored in a near-line digital AIT (Advanced Intelligent Tape) tape library. Copies of each sound and video will be created in a variety of down-sampled popular formats (compact disc quality, RealAudio, QuickTime, Windows Media, and perhaps MP3) and stored on CLO’s new EMC Symmetrix hard drive system. These latter copies will be the ones available over the Web. High-resolution copies can be obtained by special order. Automatic routines will randomly monitor the integrity of the DVD-ROM copiesdeep archive and the EMC system copies. Damaged files can be robotically regenerated from backup high-resolution copies.

Remote experts will be able to examine copies of extracted sounds or annotate copies of longer behavioral sequences (both audio and video). Given our ability to create a lower resolution copy of any sound or video clip, it will be easy to download these to the consultant over the Web along with software for recording identifications or annotations that would then be uploaded to our database. Our software will be specifically designed to accept annotation data created remotely and synchronize it with all subsequent copies of the files. These tools would will thus create a much larger pool of participating consultants and eliminate the need to transport them all to MLNS.

CLO has adopted one of the industry standards for its database, (Oracle's relational system), and is designing a data architecture that conforms with the Dublin Core (http://www.dublincore.org) protocols. These define a set of metadata and XML tags that allow our databases to be accessible and compatible with other libraries and museums worldwide. Lower levels in this architecture allow for taxon-specific data for behavioral repertoires or habitat use, sophisticated GIS links using ESRI routines, complex and high level data mining protocols, and within-file (e.g. annotation and extracted parametric data) searches. CLO is developing general tools for data importation and searching through a wide variety of database formats. All data, whether entered by hand or ported, must will be checked for accuracy and veracity before being published online.

CLO's sound and video libraries have a wide diversity of users. These include private individuals, scientific researchers, conservationists and, wildlife managers, education programs at all levels, Website website owners, military and government agencies, the media and film industries, and various commercial companies. Our major goal in the recent acquisition of an EMC enterprise storage and Web delivery system was the provision of rapid, direct, and reliable access to the CLO archives through the Internet. This requires JAVA and HTML/XML programs for the Web pages and underlying engines for a variety of online services including a) powerful searches of our metadata and within-file annotations; b) provision of search results as data tables with hot links or maps with including links to a variety of online GIS tools; c) the ability to hear any selected cut online; d) the ability to collect a series of multimedia selections onto a worktable for comparison, sequencing, or editing; e) creation of a shopping cart with secure credit-card payment protocols; and f) tracked delivery of requested resources through streaming, Web downloads, or shipment of hard copies (CD, DVD). Some of these features are being developed CLO-wide, but others will require specific adaptations for the marine animal sound collections.

In addition to search, selection, and retrieval, CLO intends to provide various sound analysis tools online. These may include the abilities to a) see a playable spectrogram or waveform (or both time-aligned) of any sound in the archive; b) select and play any part of a visible spectrogram or waveform; c) select a large number of sounds in the archive and submit a batch job to compare each sound with every other sound using any of several alternative tools (spectrographic cross-correlation, temporal cross-correlation, multiple measurement and PCA, etc.); and d) submit an unknown sound and associated metadata and receive a likely identification (or list of alternative suspects).

Funding for the Marine Animal Sounds Archive is through the U.S. Office of Naval Research. Many thanks to Bob Gisiner at ONR for his support encouragement and support on this project. Carol Bloomgarden’s participation in this workshop was generously funded by the workshop sponsors.

Return to Top | Workshop Proceedings: Short Papers