module pixeltable.functions.audio

Pixeltable UDFs for AudioType.

iterator audio_splitter()

Signature

@pxt.iterator
audio_splitter(
    audio: pxt.Audio,
    *,
    duration: pxt.Float | None = None,
    max_size: pxt.Int | None = None,
    overlap: pxt.Float = 0.0,
    min_segment_duration: pxt.Float = 0.0,
    min_silence_len: pxt.Float | None = None,
    silence_thresh: pxt.Float = -40.0,
    trim_leading_silence: pxt.Bool = False
)

Iterator over segments of an audio file. The audio file is split into smaller segments, sized either by duration (seconds) or by max_size (bytes). Exactly one of the two must be specified.

With max_size, every emitted segment is guaranteed not to exceed the given number of bytes.
With duration, each segment is approximately that long (a segment may run over by up to one packet).

Outputs: One row per audio segment, with the following columns:

segment_start (pxt.Float): Start time of the audio segment in seconds
segment_end (pxt.Float): End time of the audio segment in seconds
audio_segment (pxt.Audio | None): The audio content of the segment

When min_silence_len is set, segment boundaries are snapped to silences so that segments do not end in the middle of speech: each segment ends at the latest silence at or before its duration or max_size budget. trim_leading_silence additionally drops silence from the start of every segment. Parameters:

duration (pxt.Float | None): Audio segment duration in seconds. Mutually exclusive with max_size.
max_size (pxt.Int | None): Maximum audio segment size in bytes. Mutually exclusive with duration.
overlap (pxt.Float): Overlap between consecutive segments in seconds
min_segment_duration (pxt.Float): Drop the last segment if it is smaller than min_segment_duration (in seconds).
min_silence_len (pxt.Float | None): If set, enables silence-aware boundaries; the minimum length in seconds of a quiet stretch that counts as a usable cut point.
silence_thresh (pxt.Float): Level in dBFS at or below which audio is considered silent. Used when min_silence_len is set or trim_leading_silence is true.
trim_leading_silence (pxt.Bool): If true, drop leading silence from each segment so it starts at audible content.

Examples: This example assumes an existing table tbl with a column audio of type pxt.Audio. Create a view that splits all audio files into segments of 30 seconds with 5 seconds overlap:

pxt.create_view(
    'audio_segments',
    tbl,
    iterator=audio_splitter(tbl.audio, duration=30.0, overlap=5.0),
)

Create a view that splits all audio files into segments of at most 24 MB, for transcription:

pxt.create_view(
    'audio_segments',
    tbl,
    iterator=audio_splitter(tbl.audio, max_size=24 * 1024 * 1024),
)

udf encode_audio()

Signature

@pxt.udf
encode_audio(
    audio_data: pxt.Array[float32],
    *,
    input_sample_rate: pxt.Int,
    format: pxt.String,
    output_sample_rate: pxt.Int | None = None
) -> pxt.Audio

Encodes an audio clip represented as an array into a specified audio format. Parameters:

audio_data (pxt.Array[float32]): An array of sampled amplitudes. The accepted array shapes are (N,) or (1, N) for mono audio or (2, N) for stereo.
input_sample_rate (pxt.Int): The sample rate of the input audio data.
format (pxt.String): The desired output audio format. The supported formats are ‘wav’, ‘mp3’, ‘flac’, and ‘mp4’.
output_sample_rate (pxt.Int | None): The desired sample rate for the output audio. Defaults to the input sample rate if unspecified.

Examples: Add a computed column with encoded FLAC audio files to a table with audio data (as arrays of floats) and sample rates:

t.add_computed_column(
    audio_file=encode_audio(
        t.audio_data, input_sample_rate=t.sample_rate, format='flac'
    )
)

udf get_metadata()

Signature

@pxt.udf
get_metadata(audio: pxt.Audio) -> ContainerMetadata

Gets various metadata associated with an audio file and returns it as a ContainerMetadata dictionary. Parameters:

audio (pxt.Audio): The audio to get metadata for.

Returns:

ContainerMetadata: A ContainerMetadata with typical structure:

{
    'size': 2568827,
    'streams': [
        {
            'type': 'audio',
            'frames': 0,
            'duration': 2646000,
            'metadata': {},
            'time_base': 2.2675736961451248e-05,
            'codec_context': {
                'name': 'flac',
                'profile': None,
                'channels': 1,
                'codec_tag': '\x00\x00\x00\x00',
            },
            'duration_seconds': 60.0,
        }
    ],
    'bit_rate': 342510,
    'metadata': {'encoder': 'Lavf61.1.100'},
    'bit_exact': False,
}

Examples: Extract metadata for files in the audio_col column of the table tbl:

tbl.select(tbl.audio_col.get_metadata()).collect()

​module pixeltable.functions.audio

​iterator audio_splitter()

​udf encode_audio()

​udf get_metadata()

module pixeltable.functions.audio

iterator audio_splitter()

udf encode_audio()

udf get_metadata()