Extract frames from videos

This documentation page is also available as an interactive notebook. You can launch the notebook in Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the above links.

Pull frames from video files at specified intervals for analysis, thumbnails, or training data. This notebook uses pxt.create_table() so you can run cells without a project. An app puts the same columns on a TableModel in app.py and creates the tables with pxt schema update.

Problem

You have video files and need to extract frames for:

Object detection on video content
Creating thumbnails or previews
Building training datasets
Scene analysis and classification

Solution

What’s in this recipe:

Extract frames at a fixed rate (FPS)
Extract a specific number of frames
Extract only keyframes for efficiency

You create a view with a frame_iterator that automatically extracts frames from each video. New videos are processed without extra code.

Setup

%pip install -qU pixeltable

Load videos

import pixeltable as pxt
from pixeltable.functions.video import frame_iterator

# Create a fresh directory
pxt.drop_dir('video_demo', force=True)
pxt.create_dir('video_demo')

Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory ‘video_demo’.
<pixeltable.catalog.dir.Dir at 0x1695d2650>

# Create table for videos
videos = pxt.create_table('video_demo/videos', {'video': pxt.Video})

Created table ‘videos’.

# Insert a sample video
videos.insert(
    [
        {
            'video': 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/bangkok.mp4'
        }
    ]
)

Inserting rows into `videos`: 1 rows [00:00, 212.90 rows/s]
Inserted 1 row with 0 errors.
1 row inserted, 2 values computed.

Extract frames at fixed rate

Create a view that extracts 1 frame per second:

# Extract 1 frame per second
frames = pxt.create_view(
    'video_demo/frames',
    videos,
    iterator=frame_iterator(
        videos.video,
        fps=1.0,  # 1 frame per second
    ),
)

Inserting rows into `frames`: 19 rows [00:00, 8687.65 rows/s]

# View extracted frames
frames.select(frames.frame, frames.pos).head(3)

Extract keyframes only

For faster processing, extract only keyframes (I-frames):

# Extract only keyframes (much faster for long videos)
keyframes = pxt.create_view(
    'video_demo/keyframes',
    videos,
    iterator=frame_iterator(videos.video, keyframes_only=True),
)
keyframes.select(keyframes.frame).head(3)

Inserting rows into `keyframes`: 7 rows [00:00, 3277.53 rows/s]

Explanation

Extraction options:

Only one of fps, num_frames, or keyframes_only can be specified. When to use keyframes:

Quick video scanning and thumbnails
Initial content classification
Processing very long videos

Frame metadata: Each frame includes:

frame: The extracted image
pos: Frame position in the video
pts: Presentation timestamp

Use Cases

Topics

Problem

Solution

Setup

Load videos

Extract frames at fixed rate

Extract keyframes only

Explanation

See also

​Problem

​Solution

​Setup

​Load videos

​Extract frames at fixed rate

​Extract keyframes only

​Explanation

​See also

Problem

Solution

Setup

Load videos

Extract frames at fixed rate

Extract keyframes only

Explanation

See also