Skip to main content
Pixeltable UDFs for VideoType. View source on GitHub

​
UDFs

​
clip() udf

Extract a clip from a video, specified by start_time and either end_time or duration (in seconds). If start_time is beyond the end of the video, returns None. Can only specify one of end_time and duration. If both end_time and duration are None, the clip goes to the end of the video. Requirements:
  • ffmpeg needs to be installed and in PATH
Signature: 
clip(
    video: Video,
    start_time: Float,
    end_time: Optional[Float],
    duration: Optional[Float]
)-> Optional[Video]
Parameters:
  • video (Video): Input video file
  • start_time (Float): Start time in seconds
  • end_time (Optional[Float]): End time in seconds
  • duration (Optional[Float]): Duration of the clip in seconds
Returns:
  • Optional[Video]: New video containing only the specified time range or None if start_time is beyond the end of the video.

​
concat_videos() udf

Merge multiple videos into a single video. Requirements:
  • ffmpeg needs to be installed and in PATH
Signature: 
concat_videos(videos: Json)-> Video
Parameters:
  • videos (Json): List of videos to merge.
Returns:
  • Video: A new video containing the merged videos.

​
extract_audio() udf

Extract an audio stream from a video. Signature: 
extract_audio(
    video_path: Video,
    stream_idx: Int,
    format: String,
    codec: Optional[String]
)-> Audio
Parameters:
  • stream_idx (Int): Index of the audio stream to extract.
  • format (String): The target audio format. ('wav', 'mp3', 'flac').
  • codec (Optional[String]): The codec to use for the audio stream. If not provided, a default codec will be used.
Returns:
  • Audio: The extracted audio.
Example: Add a computed column to a table tbl that extracts audio from an existing column video_col: 
tbl.add_computed_column(extracted_audio=tbl.video_col.extract_audio(format='flac'))

​
extract_frame() udf

Extract a single frame from a video at a specific timestamp. Signature: 
extract_frame(
    video: Video,
    timestamp: Float
)-> Optional[Image]
Parameters:
  • video (Video): The video from which to extract the frame.
  • timestamp (Float): Extract frame at this timestamp (in seconds).
Returns:
  • Optional[Image]: The extracted frame as a PIL Image, or None if the timestamp is beyond the video duration.
Example: Extract the first frame from each video in the video column of the table tbl: 
tbl.select(tbl.video.extract_frame(0.0)).collect()
Extract a frame close to the end of each video in the video column of the table tbl: 
tbl.select(tbl.video.extract_frame(tbl.video.get_metadata().streams[0].duration_seconds - 0.1)).collect()

​
get_duration() udf

Get video duration in seconds. Signature: 
get_duration(video: Video)-> Optional[Float]
Parameters:
  • video (Video): The video for which to get the duration.
Returns:
  • Optional[Float]: The duration in seconds, or None if the duration cannot be determined.

​
get_metadata() udf

Gets various metadata associated with a video file and returns it as a dictionary. Signature: 
get_metadata(video: Video)-> Json
Parameters:
  • video (Video): The video for which to get metadata.
Returns:
  • Json: A dict such as the following:
{
    'bit_exact': False,
    'bit_rate': 967260,
    'size': 2234371,
    'metadata': {
        'encoder': 'Lavf60.16.100',
        'major_brand': 'isom',
        'minor_version': '512',
        'compatible_brands': 'isomiso2avc1mp41',
    },
    'streams': [
        {
            'type': 'video',
            'width': 640,
            'height': 360,
            'frames': 462,
            'time_base': 1.0 / 12800,
            'duration': 236544,
            'duration_seconds': 236544.0 / 12800,
            'average_rate': 25.0,
            'base_rate': 25.0,
            'guessed_rate': 25.0,
            'metadata': {
                'language': 'und',
                'handler_name': 'L-SMASH Video Handler',
                'vendor_id': '[0][0][0][0]',
                'encoder': 'Lavc60.31.102 libx264',
            },
            'codec_context': {'name': 'h264', 'codec_tag': 'avc1', 'profile': 'High', 'pix_fmt': 'yuv420p'},
        }
    ],
}
Example: Extract metadata for files in the video_col column of the table tbl: 
tbl.select(tbl.video_col.get_metadata()).collect()

​
overlay_text() udf

Overlay text on a video with customizable positioning and styling. Requirements:
  • ffmpeg needs to be installed and in PATH
Signature: 
overlay_text(
    video: Video,
    text: String,
    font: Optional[String],
    font_size: Int,
    color: String,
    opacity: Float,
    horizontal_align: String,
    horizontal_margin: Int,
    vertical_align: String,
    vertical_margin: Int,
    box: Bool,
    box_color: String,
    box_opacity: Float,
    box_border: Optional[Json]
)-> Video
Parameters:
  • video (Video): Input video to overlay text on.
  • text (String): The text string to overlay on the video.
  • font (Optional[String]): Font family or path to font file. If None, uses the system default.
  • font_size (Int): Size of the text in points.
  • color (String): Text color (e.g., 'white', 'red', '#FF0000').
  • opacity (Float): Text opacity from 0.0 (transparent) to 1.0 (opaque).
  • horizontal_align (String): Horizontal text alignment ('left', 'center', 'right').
  • horizontal_margin (Int): Horizontal margin in pixels from the alignment edge.
  • vertical_align (String): Vertical text alignment ('top', 'center', 'bottom').
  • vertical_margin (Int): Vertical margin in pixels from the alignment edge.
  • box (Bool): Whether to draw a background box behind the text.
  • box_color (String): Background box color as a string.
  • box_opacity (Float): Background box opacity from 0.0 to 1.0.
  • box_border (Optional[Json]): Padding around text in the box in pixels.
    • [10]: 10 pixels on all sides
    • [10, 20]: 10 pixels on top/bottom, 20 on left/right
    • [10, 20, 30]: 10 pixels on top, 20 on left/right, 30 on bottom
    • [10, 20, 30, 40]: 10 pixels on top, 20 on right, 30 on bottom, 40 on left
Returns:
  • Video: A new video with the text overlay applied.
Example: Add a simple text overlay to videos in a table: 
tbl.select(tbl.video.overlay_text('Sample Text')).collect()
Add a YouTube-style caption: 
tbl.select(
    tbl.video.overlay_text(
        'Caption text',
        font_size=32,
        color='white',
        opacity=1.0,
        box=True,
        box_color='black',
        box_opacity=0.8,
        box_border=[6, 14],
        horizontal_margin=10,
        vertical_align='bottom',
        vertical_margin=70,
    )
).collect()
Add text with a semi-transparent background box: 
tbl.select(
    tbl.video.overlay_text(
        'Important Message',
        font_size=32,
        color='yellow',
        box=True,
        box_color='black',
        box_opacity=0.6,
        box_border=[20, 10],
    )
).collect()

​
segment_video() udf

Split a video into segments. Requirements:
  • ffmpeg needs to be installed and in PATH
Signature: 
segment_video(
    video: Video,
    duration: Optional[Float],
    segment_times: Optional[Json],
    mode: String,
    video_encoder: Optional[String],
    video_encoder_args: Optional[Json]
)-> Json
Parameters:
  • video (Video): Input video file to segment
  • duration (Optional[Float]): Duration of each segment (in seconds). For mode='fast', this is approximate; for mode='accurate', segments will have exact durations. Cannot be specified together with segment_times.
  • segment_times (Optional[Json]): List of timestamps (in seconds) in video where segments should be split. Note that these are not segment durations. If all segment times are less than the duration of the video, produces exactly len(segment_times) + 1 segments. Cannot be empty or be specified together with duration.
  • mode (String): Segmentation mode:
    • 'fast': Quick segmentation using stream copy (splits only at keyframes, approximate durations)
    • 'accurate': Precise segmentation with re-encoding (exact durations, slower)
  • video_encoder (Optional[String]): Video encoder to use. If not specified, uses the default encoder for the current platform. Only available for mode='accurate'.
  • video_encoder_args (Optional[Json]): Additional arguments to pass to the video encoder. Only available for mode='accurate'.
Returns:
  • Json: List of file paths for the generated video segments.
Example: Split a video at 1 minute intervals using fast mode: 
tbl.select(segment_paths=tbl.video.segment_video(duration=60)).collect()
Split video into exact 10-second segments with accurate mode, using the libx264 encoder with a CRF of 23 and slow preset (for smaller output files): 
tbl.select(
    segment_paths=tbl.video.segment_video(
        duration=10, mode='accurate', video_encoder='libx264', video_encoder_args={'crf': 23, 'preset': 'slow'}
    )
).collect()
Split video into two parts at the midpoint: 
duration = tbl.video.get_duration()
tbl.select(segment_paths=tbl.video.segment_video(segment_times=[duration / 2])).collect()

​
with_audio() udf

Creates a new video that combines the video stream from video and the audio stream from audio. The start_time and duration parameters can be used to select a specific time range from each input. If the audio input (or selected time range) is longer than the video, the audio will be truncated. Requirements:
  • ffmpeg needs to be installed and in PATH
Signature: 
with_audio(
    video: Video,
    audio: Audio,
    video_start_time: Float,
    video_duration: Optional[Float],
    audio_start_time: Float,
    audio_duration: Optional[Float]
)-> Video
Parameters:
  • video (Video): Input video.
  • audio (Audio): Input audio.
  • video_start_time (Float): Start time in the video input (in seconds).
  • video_duration (Optional[Float]): Duration of video segment (in seconds). If None, uses the remainder of the video after video_start_time. video_duration determines the duration of the output video.
  • audio_start_time (Float): Start time in the audio input (in seconds).
  • audio_duration (Optional[Float]): Duration of audio segment (in seconds). If None, uses the remainder of the audio after audio_start_time. If the audio is longer than the output video, it will be truncated.
Returns:
  • Video: A new video file with the audio track added.
Example: Add background music to a video: 
tbl.select(tbl.video.with_audio(tbl.music_track)).collect()
Add audio starting 5 seconds into both files: 
tbl.select(tbl.video.with_audio(tbl.music_track, video_start_time=5.0, audio_start_time=5.0)).collect()
Use a 10-second clip from the middle of both files: 
tbl.select(
    tbl.video.with_audio(
        tbl.music_track, video_start_time=30.0, video_duration=10.0, audio_start_time=15.0, audio_duration=10.0
    )
).collect()