Media processing

Insert media. Iterators explode it into rows. Computed columns run on the new rows. Export when you are ready to train. Create the tables with pxt schema update app.py my_app, then open them with t = pxt.get_table('my_app.videos'). Cookbooks on this topic use pxt.create_table() so you can run cells without a project. An app puts the same columns on a TableModel in app.py and creates the tables with pxt schema update.

import pixeltable as pxt
import pixeltable.functions as pxtf

TableModel = pxt.model_base()


class Videos(TableModel, name='videos'):
    video: pxt.Video


class Frames(
    TableModel,
    name='frames',
    base=Videos,
    iterator=pxtf.video.frame_iterator(Videos.video, fps=1),
):
    thumb = frame.resize((256, 256))  # type: ignore[name-defined]  # iterator output

pxt schema update app.py my_app

videos = pxt.get_table('my_app.videos')
videos.insert([{'video': 'demo.mp4'}])
frames = pxt.get_table('my_app.frames')
frames.select(frames.thumb).head(10)

Add Whisper, YOLOX, and other models from the cookbooks as computed columns on these tables. When the rows look ready for training, export with to_pytorch_dataset(). Skip pxt service update on this path. Self-hosting. A video search app with endpoints: uvx pixeltable-new myapp --video, then pass videointel to pxt schema update. Live search over chunks: RAG and live APIs.

Extract video frames

Frames as rows from a pxt.Video column.

Transcribe audio

Transcripts as computed columns.

Export to PyTorch

to_pytorch_dataset() when the rows are ready.

Use Cases

Topics

Extract video frames

Transcribe audio

Export to PyTorch