Comparisonics API

comparisonics api

The Comparisonics API Software Library provides an application program interface to C and C++ programs. This library enables a wide variety of applications to be developed that incorporate the Comparisonics technologies for audio comparison and display. It includes functions for

creating signatures to characterize audio;
comparing signatures to determine the similarity of sounds; and
deriving colors from signatures for displaying audio.

The Comparisonics API Software Library Version 2.0 is currently available for Microsoft Windows, Macintosh, and Linux platforms. Comparisonics Corporation licenses this library to companies for the development of specific applications. Send licensing inquiries to [email protected].

Indexing

Indexing is the process of creating signatures to characterize the content of audio clips. The application program first opens an audio file and determines from the file header the sample rate of the digital audio samples stored in the file. The application program reads the audio samples into memory as needed using whatever buffering mechanism is convenient and efficient. To characterize an entire file or a segment of a file, the following API functions are called in sequence:

The sample rate is passed to CS_create_sample_processor( ) and a pointer to a newly-allocated internal data structure is returned.
CS_process_sample( ) is called once for each sample value of the file or segment. In each call, the pointer from step 1 is passed along with the next sample value in sequence.
The pointer from step 1 is passed to CS_get_signature( ) and a signature is returned that characterizes the given file or segment.
To clean up, the pointer from step 1 is passed to CS_destroy_sample_processor( ) to de-allocate the internal data structure.

The application program is responsible for storing the signatures. They may be saved in memory, in disk files, or in a database. The application program must also preserve information indicating which audio file/segment is characterized by each signature.

For most applications, it is best to divide an audio file into segments and the four steps above are repeated to create a signature for each segment. The duration of segments is chosen by the application program. When indexing rapidly-changing audio such as music, 50-millisecond intervals work well; on the other hand, for ambient sound, each segment could be several seconds in duration or longer. Normally, the segments are defined to be consecutive and non-overlapping within the file, but this is not a requirement. Creating one signature to characterize an entire file is useful for locating copies of the file.

Sound Matching

Given any two signatures, CS_compare_signatures( ) determines the similarity of the two audio clips characterized by those signatures. This function returns a similarity score ranging from 0.0 (least similar) to 100.0 (most similar) indicating the extent to which the two clips sound alike.

To search a collection of audio clips, the clips must first be indexed, that is, signatures must be created to characterize them. Then any sound, including one selected from the collection, may be used as the prototype in a sound-matching query. To determine which sounds in the collection are most like the prototype, signatures derived from the prototype are compared with the signatures derived from the collection. The resulting similarity scores can be sorted to produce a ranked list of hits.

To search for sounds within a single audio file, the file is first divided into segments and a signature is created to characterize each segment. The user selects a particular segment, or sequence of segments, as the prototype (by clicking on it within a waveform display, for example). The prototype's signatures are then compared with other signatures in the file to locate matches. A region of the file is considered to be a match if its similarity to the prototype exceeds the value of a threshold parameter specified by the user.

Coloring the Waveform Display

CS_get_color( ) is used to derive a color from a signature. This color can be used to represent the audio clip that is characterized by the signature. The most effective use of this function is to color the waveform display.

An audio file is first divided into segments. For each segment, a signature is created, and a color is derived from the signature and used to paint the segment in the waveform display. The resulting display is a rainbow of colors reflecting the frequency content of the audio file.

Summary

The Comparisonics API consists of six functions. Four functions are used in the indexing process:

CS_create_sample_processor( )
CS_process_sample( )
CS_get_signature( )
CS_destroy_sample_processor( )

One function is used for sound comparison:

CS_compare_signatures( )

And one function is used for mapping sound to color:

CS_get_color( )

The Comparisonics API Software Library is distributed in the form of a C-language include file, and a binary object file that gets linked to the application program.