Cloud Storage

Pixeltable supports storing media files (images, videos, audio, documents) in external cloud storage providers instead of local disk. This is essential for production deployments, enabling scalable storage, team collaboration, and integration with existing data infrastructure.

Supported providers

Pixeltable Cloud

Free managed storage, no bucket setup required

Amazon S3

Native S3 storage with full feature support

Google Cloud Storage

GCS buckets with gs:// URI scheme

Azure Blob Storage

Azure containers with wasb:// or abfs:// schemes

Cloudflare R2

S3-compatible storage with zero egress fees

Backblaze B2

Cost-effective S3-compatible storage

Tigris

Globally distributed S3-compatible storage

How it works

When you configure a storage destination, Pixeltable automatically:

Uploads computed media — AI-generated images, extracted video frames, and other computed media files are stored in your bucket
Copies input media — Optionally persists referenced media files for durability
Manages file lifecycle — Cleans up files when table data is deleted
Handles caching — Downloads files on-demand with intelligent local caching

Configuration

There are two ways to configure cloud storage destinations:

Global default destinations

Set default destinations for all media columns in your config.toml (see Configuration for details):

[pixeltable]
# For input media (inserted/referenced files)
input_media_dest = "s3://my-bucket/input/"

# For computed media (AI-generated outputs)
output_media_dest = "s3://my-bucket/output/"

Or via environment variables:

export PIXELTABLE_INPUT_MEDIA_DEST="s3://my-bucket/input/"
export PIXELTABLE_OUTPUT_MEDIA_DEST="s3://my-bucket/output/"

Configure these before creating tables. All media columns will automatically use the configured destinations.

Per-column destination (computed columns only)

For computed columns, you can override the default with a specific destination:

import pixeltable as pxt

# Create a table with input media column
# (uses global input_media_dest if configured)
t = pxt.create_table('my_app/images', {'image': pxt.Image})

# Add computed column with explicit destination
t.add_computed_column(
    thumbnail=t.image.resize((128, 128)),
    destination='s3://my-bucket/thumbnails/'
)

The destination parameter only applies to stored computed columns. For input columns, use the global input_media_dest configuration.

Precedence rules

Destinations are resolved in this order:

Explicit column destination — highest priority (computed columns only)
Global default — input_media_dest for input columns, output_media_dest for computed columns
Local storage — fallback if no destination is configured

Provider configuration

Pixeltable Cloud (home bucket)

Every Pixeltable Cloud account includes a free managed storage bucket. No bucket creation, no credentials file, no cloud provider account needed.

URI Format
Authentication
Example

pxtfs://org-slug:db-slug/home

Replace org-slug and db-slug with your Pixeltable Cloud organization and database names.

Set your Pixeltable API key:

[pixeltable]
api_key = "your-pixeltable-api-key"

Or via environment variable:

export PIXELTABLE_API_KEY="your-pixeltable-api-key"

Pixeltable automatically fetches and refreshes temporary credentials from the Pixeltable cloud management API.

import pixeltable as pxt

t = pxt.create_table('app/images', {'photo': pxt.Image})

t.add_computed_column(
    thumbnail=t.photo.resize((256, 256)),
    destination='pxtfs://myorg:mydb/home'
)

Or set it as your global default in config.toml:

[pixeltable]
output_media_dest = "pxtfs://myorg:mydb/home"

Or as an environment variable:

export PIXELTABLE_OUTPUT_MEDIA_DEST="pxtfs://myorg:mydb/home"

This is the fastest way to get cloud storage working. No AWS/GCP/Azure account required.

You can browse, search, and preview the contents of your home bucket directly from the Pixeltable Cloud dashboard. Navigate to Storage & Buckets in the sidebar, then select your home bucket to explore files by type (images, docs, video, audio), view metadata, and inspect individual objects.

https://www.pixeltable.com/dashboard/{org-slug}/{db-slug}/storage/home/browse

Amazon S3

URI Format
Authentication
Example

s3://bucket-name/optional/prefix/

Uses standard AWS credential chain:

Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
AWS credentials file (~/.aws/credentials)
IAM role (when running on AWS)

Optionally specify a profile in config.toml:

[pixeltable]
s3_profile = "my-aws-profile"

import pixeltable as pxt

# With global config: output_media_dest = "s3://my-bucket/output/"
t = pxt.create_table('app/images', {'photo': pxt.Image})

# Or set destination per computed column
t.add_computed_column(
    thumbnail=t.photo.resize((256, 256)),
    destination='s3://my-production-bucket/thumbnails/'
)

Google Cloud Storage

URI Format
Authentication
Requirements
Example

gs://bucket-name/optional/prefix/

Uses Google Cloud Application Default Credentials:

Service account key file (GOOGLE_APPLICATION_CREDENTIALS)
gcloud CLI authentication
GCE metadata service (when running on GCP)

pip install google-cloud-storage

# With global config: output_media_dest = "gs://my-gcs-bucket/output/"
t = pxt.create_table('app/videos', {'video': pxt.Video})

# Or set destination per computed column
t.add_computed_column(
    frames=pxt.functions.video.frame_iterator(t.video, fps=1),
    destination='gs://my-gcs-bucket/frames/'
)

Azure Blob Storage

URI Formats
Authentication
Requirements
Example

Azure supports multiple URI schemes:

wasbs://[email protected]/prefix/
abfss://[email protected]/prefix/

Configure in config.toml:

[azure]
storage_account_name = "myaccount"
storage_account_key = "your-key-here"

Or via environment variables:

export AZURE_STORAGE_ACCOUNT_NAME="myaccount"
export AZURE_STORAGE_ACCOUNT_KEY="your-key-here"

pip install azure-storage-blob

# With global config: output_media_dest = "wasbs://[email protected]/output/"
t = pxt.create_table('app/docs', {'document': pxt.Document})

# Or set destination per computed column
t.add_computed_column(
    chunks=pxt.functions.video.document_splitter(t.document),
    destination='wasbs://[email protected]/chunks/'
)

Cloudflare R2

URI Format
Authentication
Example

https://account-id.r2.cloudflarestorage.com/bucket-name/prefix/

Create an R2 API token and configure AWS-style credentials.In ~/.aws/credentials:

[r2]
aws_access_key_id = your-r2-access-key
aws_secret_access_key = your-r2-secret-key

In config.toml:

[pixeltable]
r2_profile = "r2"

t = pxt.create_table('app/images', {'image': pxt.Image})

t.add_computed_column(
    rotated=t.image.rotate(90),
    destination='https://abc123.r2.cloudflarestorage.com/my-bucket/processed/'
)

Backblaze B2

URI Format
Authentication
Example

https://s3.region.backblazeb2.com/bucket-name/prefix/

Create B2 application keys and configure AWS-style credentials.In ~/.aws/credentials:

[b2]
aws_access_key_id = your-b2-key-id
aws_secret_access_key = your-b2-application-key

In config.toml:

[pixeltable]
b2_profile = "b2"

from pixeltable.functions.video import extract_audio

t = pxt.create_table('app/videos', {'video': pxt.Video})

t.add_computed_column(
    audio=extract_audio(t.video, format='mp3'),
    destination='https://s3.us-west-004.backblazeb2.com/my-bucket/audio/',
)

Tigris

URI Format
Authentication
Example

https://t3.storage.dev/bucket-name/prefix/

Configure AWS-style credentials for Tigris.In ~/.aws/credentials:

[tigris]
aws_access_key_id = your-tigris-access-key
aws_secret_access_key = your-tigris-secret-key

In config.toml:

[pixeltable]
tigris_profile = "tigris"

t = pxt.create_table('app/media', {'file': pxt.Image})

t.add_computed_column(
    thumbnail=t.file.resize((128, 128)),
    destination='https://t3.storage.dev/my-bucket/thumbnails/'
)

Complete example

Here’s a full example using S3 for both input and computed media. First, configure your global destinations in ~/.pixeltable/config.toml:

[pixeltable]
input_media_dest = "s3://my-app-bucket/uploads/"
output_media_dest = "s3://my-app-bucket/generated/"

s3_profile = "my-aws-profile"  # optional, uses default credentials if not set

Then create your table and add computed columns:

import pixeltable as pxt
from pixeltable.functions import openai

# Create a table — input media automatically goes to input_media_dest
t = pxt.create_table('production/photos', {'photo': pxt.Image})

# Add a computed column for thumbnails
# Uses output_media_dest by default, or specify a custom destination
t.add_computed_column(
    thumbnail=t.photo.resize((256, 256)),
    destination='s3://my-app-bucket/thumbnails/'  # override default
)

# Add AI-generated descriptions (uses output_media_dest)
messages = [
    {
        'role': 'user',
        'content': [
            {'type': 'text', 'text': 'Describe this image briefly.'},
            {'type': 'image_url', 'image_url': t.photo},
        ],
    }
]
t.add_computed_column(
    description=openai.chat_completions(messages, model='gpt-4o-mini')
)

# Insert data — Pixeltable handles all uploads automatically
t.insert([
    {'photo': 'https://example.com/image1.jpg'},
    {'photo': '/local/path/to/image2.png'},
])

# Query as usual — files are streamed/cached as needed
t.select(t.photo, t.thumbnail, t.description).collect()

Best practices

Use prefixes to organize data

Structure your bucket with prefixes that reflect your application:

s3://my-bucket/
  ├── production/
  │   ├── uploads/
  │   └── generated/
  └── staging/
      ├── uploads/
      └── generated/

Separate input and output destinations

Use different prefixes or buckets for input vs computed media:

Easier to set different retention policies
Clearer cost attribution
Simpler backup strategies

Configure lifecycle policies

Set up bucket lifecycle policies to automatically:

Transition old data to cheaper storage tiers
Delete temporary/staging data after a period
Enable versioning for critical data

Use IAM roles in production

When running on cloud infrastructure, use IAM roles instead of access keys:

More secure (no key rotation needed)
Automatic credential refresh
Better audit trails

Troubleshooting

Access Denied errors

Verify your credentials have the necessary permissions:

s3:GetObject, s3:PutObject, s3:DeleteObject
s3:ListBucket for the bucket

For GCS: storage.objects.create, storage.objects.get, storage.objects.delete

Bucket not found

Ensure the bucket exists and the name is spelled correctly
Check the region matches your credential configuration
For S3-compatible providers, verify the endpoint URL is correct

Slow uploads

Pixeltable uses connection pooling and parallel uploads automatically
Consider using a bucket in the same region as your compute
Check your network bandwidth and latency

Configuration Reference

See the complete list of storage configuration options including profiles for S3, R2, B2, Tigris, and Azure.

Built-in Integrations

Infrastructure

Bring Your Own

Providers

Supported providers

Pixeltable Cloud

Amazon S3

Google Cloud Storage

Azure Blob Storage

Cloudflare R2

Backblaze B2

Tigris

How it works

Configuration

Global default destinations

Per-column destination (computed columns only)

Precedence rules

Provider configuration

Pixeltable Cloud (home bucket)

Amazon S3

Google Cloud Storage

Azure Blob Storage

Cloudflare R2

Backblaze B2

Tigris

Complete example

Best practices

Troubleshooting

Configuration Reference

​Supported providers

Pixeltable Cloud

Amazon S3

Google Cloud Storage

Azure Blob Storage

Cloudflare R2

Backblaze B2

Tigris

​How it works

​Configuration

​Global default destinations

​Per-column destination (computed columns only)

​Precedence rules

​Provider configuration

​Pixeltable Cloud (home bucket)

​Amazon S3

​Google Cloud Storage

​Azure Blob Storage

​Cloudflare R2

​Backblaze B2

​Tigris

​Complete example

​Best practices

​Troubleshooting

Configuration Reference

Supported providers

How it works

Configuration

Global default destinations

Per-column destination (computed columns only)

Precedence rules

Provider configuration

Pixeltable Cloud (home bucket)

Amazon S3

Google Cloud Storage

Azure Blob Storage

Cloudflare R2

Backblaze B2

Tigris

Complete example

Best practices

Troubleshooting