Application programs are responsible for providing their own user interface and performing I/O. For information on how to read audio files in common file formats, see A Programmer's Guide to Sound by Tim Kientzle (Addison-Wesley, 1998). This outstanding reference includes source code for reading audio files in the following formats: WAVE, AU, AIFF, VOC, MPEG, and MIDI.

Indexing is the process of creating signatures to characterize the content of audio clips. The application program first opens an audio file and determines from the file header the sample rate of the digital audio samples stored in the file. The application program reads the audio samples into memory as needed using whatever buffering mechanism is convenient and efficient. To characterize an entire file or a segment of a file, the following API functions are called in sequence:

1. The sample rate is passed to CS_create_sample_processor and a pointer to a newly-allocated internal data structure is returned.
2. CS_process_sample is called once for each sample value of the file or segment. In each call, the pointer from step 1 is passed along with the next sample value in sequence.
3. The pointer from step 1 is passed to CS_get_signature and a signature is returned that characterizes the given file or segment.
4. To clean up, the pointer from step 1 is passed to CS_destroy_sample_processor to de-allocate the internal data structure.

For most applications, it is best to divide an audio file into segments and the four steps above are repeated to create a signature for each segment. The duration of segments is chosen by the application program. When indexing rapidly-changing audio such as music, 50-millisecond intervals work well; on the other hand, for ambient sound, each segment could be several seconds in duration or longer. Normally, the segments are defined to be consecutive and non-overlapping within the file, but this is not a requirement. Creating one signature to characterize an entire file is useful for locating copies of the file.

Given any two signatures, the CS_compare_signatures function determines the similarity of the two audio clips characterized by those signatures. This function returns a similarity score ranging from 0.0 (least similar) to 100.0 (most similar) indicating the extent to which the two clips sound alike.

To search a collection of audio clips, the clips must first be indexed, that is, signatures must be created to characterize them. Then any sound, including one selected from the collection, may be used as the prototype in a sound-matching query. To determine which sounds in the collection are most like the prototype, signatures derived from the prototype are compared with the signatures derived from the collection. The resulting similarity scores can be sorted to produce a ranked list of hits.

To search for sounds within a single audio file, the file is first divided into segments and a signature is created to characterize each segment. The user selects a particular segment, or sequence of segments, as the prototype (by clicking on it within a waveform display, for example). The prototype's signatures are then compared with other signatures in the file to locate matches. A region of the file is considered to be a match if its similarity to the prototype exceeds the value of a threshold parameter specified by the user.

The CS_get_color function is used to derive a color from a signature. This color can be used to represent the audio clip that is characterized by the signature. The most effective use of this function is to color the waveform display.

An audio file is first divided into segments. For each segment, a signature is created, and a color is derived from the signature and used to paint the segment in the waveform display. The resulting display is a rainbow of colors reflecting the frequency content of the audio file.

The Comparisonics API consists of six functions. Four functions are used in the indexing process:

• CS_create_sample_processor
• CS_process_sample
• CS_get_signature
• CS_destroy_sample_processor

• CS_compare_signatures

• CS_get_color