Data Sharing

This documentation page is also available as an interactive notebook. You can launch the notebook in Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the above links.

Learn how to publish datasets to Pixeltable Cloud and replicate datasets from the cloud to your local environment.

Overview

Pixeltable Cloud enables you to:

Publish your datasets for sharing with teams or the public
Replicate datasets from the cloud to your local environment
Share multimodal AI datasets (images, videos, audio, documents) without managing infrastructure

This guide demonstrates both publishing and replicating datasets.

Setup

Data sharing functionality requires Pixeltable version 0.4.24 or later.

%pip install -qU pixeltable

Replicating datasets

You can replicate any public dataset from Pixeltable Cloud to your local environment without needing an account or API key.

Replicate a public dataset

Let’s replicate a mini-version of the COCO-2017 dataset from Pixeltable Cloud. You can find this dataset at pixeltable.com/t/pixeltable:fiftyone/coco_mini_2017, or browse for other public datasets. When calling replicate():

remote_uri (required): The URI of the cloud dataset you want to replicate
local_path (your choice): The local directory/table name where you want to store the replica
Variable name (your choice): The Python variable in your session/script to reference the table (e.g., coco_copy)

See the replicate() SDK reference for full documentation.

import pixeltable as pxt

pxt.drop_dir('sharing-demo', force=True)
pxt.create_dir('sharing-demo')

# The remote_uri is the specific cloud dataset you want to replicate
# The local_path and variable name are yours to choose
coco_copy = pxt.replicate(
    remote_uri='pxt://pixeltable:fiftyone/coco_mini_2017',
    local_path='sharing-demo.coco-copy',
)

Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/asiegel/.pixeltable/pgdata
Created directory ‘sharing-demo’.
Output()

Extracting table data into: /Users/asiegel/.pixeltable/tmp/acad78b1-4a62-483e-a0b1-728ccb5603cf
Created directory ‘_system’.
Created local replica ‘sharing-demo/coco-copy’ from URI: pxt://pixeltable:fiftyone/coco_mini_2017

You can check that the replica exists at the local path with list_tables().

pxt.list_tables('sharing-demo')

[‘sharing-demo/coco-copy’]

To see the structure of the replicated table:

coco_copy

Working with replicas

Replicated datasets are read-only locally, but you can query, explore, and use them in powerful ways: 1. Query and explore the data

# View the replicated data
coco_copy.limit(3).collect()

2. Perform similarity searches Replicas include embedding indexes, so you can immediately perform similarity searches:

# Get a sample image to search with
sample_img = (
    coco_copy.select(coco_copy.image).limit(1).collect()[0]['image']
)
sample_img

# Perform image-based similarity search
sim = coco_copy.image.similarity(image=sample_img)
results = (
    coco_copy.order_by(sim, asc=False)
    .limit(5)
    .select(coco_copy.image, sim)
    .collect()
)
results

Because the COCO dataset uses CLIP embeddings (which are multimodal), you can also search using text queries:

# Perform text-based similarity search
sim = coco_copy.image.similarity(string='surfing')
results = (
    coco_copy.order_by(sim, asc=False)
    .limit(4)
    .select(coco_copy.image, sim)
    .collect()
)
results

3. Access replicas in new sessions In a new Python session, use list_tables() and get_table() to access your replicas:

# List all tables to see your replica
pxt.list_tables('sharing-demo')

[‘sharing-demo/coco-copy’]

# Assign a handle to the replica
coco_copy = pxt.get_table('sharing-demo.coco-copy')

4. Create an independent copy To work with the data in new ways, create an independent table with the replica as the source:

# Create a fresh table with values only
my_coco = pxt.create_table('sharing-demo.my-coco-table', source=coco_copy)

Created table ‘my-coco-table’.

This copies the values in the source, but drops the computational definitions and cannot be updated if the source table changes.

Updating replicas with pull

If the upstream table changes, you can update your local replica using pull():

# Update your local replica with changes from the cloud
coco_copy.pull()

Replica ‘sharing-demo/coco-copy’ is already up to date with source: pxt://pixeltable:fiftyone/d699317b-23a4-404b-8f71-6531fd8dc462

This synchronizes your local replica with any updates made to the source dataset.

Publishing datasets

Requirements:

A Pixeltable Cloud account (Community Edition includes 1TB storage - see pricing)
Your API key from the account dashboard

Publishing allows you to share your datasets with your team or make them publicly available.

Configure your API key

Pixeltable looks for your API key in the PIXELTABLE_API_KEY environment variable. Choose one of these methods: Option 1: In your notebook (secure and convenient) Run this cell to securely enter your API key (get it from pixeltable.com/dashboard):

import os
from getpass import getpass

os.environ['PIXELTABLE_API_KEY'] = getpass('Pixeltable API Key:')

Option 2: Environment variable Add to your ~/.zshrc or ~/.bashrc:

export PIXELTABLE_API_KEY='your-api-key-here'

Option 3: Config file Add to ~/.pixeltable/config.toml:

[pixeltable]
api_key = 'your-api-key-here'

See the Configuration Guide for details.

Create a sample dataset

Let’s create a table with images from this repository to publish. The comment parameter provides a description that will be visible on Pixeltable Cloud:

t = pxt.create_table(
    'sharing-demo.photos',
    schema={'image': pxt.Image, 'description': pxt.String},
    comment='Sample image dataset for demonstrating Pixeltable Cloud publishing',
)

Created table ‘photos’.

base_url = 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/images'
t.insert(
    [
        {
            'image': f'{base_url}/000000000009.jpg',
            'description': 'Kitchen scene',
        },
        {
            'image': f'{base_url}/000000000025.jpg',
            'description': 'Street view',
        },
        {
            'image': f'{base_url}/000000000042.jpg',
            'description': 'Indoor setting',
        },
    ]
)

Inserted 3 rows with 0 errors in 0.02 s (169.05 rows/s)
3 rows inserted.

Publish your dataset

Publish your table to Pixeltable Cloud. When calling publish():

source (required): An existing local table - either a table path string (e.g., 'sample-images.photos') or table handle (e.g., t)
- If you use a local table path string, it must match a table in your local database (you can verify with pxt.list_tables())
destination_uri (required): The cloud URI where you want to publish, in the format pxt://orgname/dataset
- Pixeltable automatically creates any directory structure in the cloud based on this URI
- Your local directory structure doesn’t need to match the cloud structure

See the publish() SDK reference for full documentation.

# Option 1: Publish using table path (string)
pxt.publish(
    source='sharing-demo.photos',  # Table path from list_tables()
    destination_uri='pxt://your-orgname/sample-images',
)

# Option 2: Publish using table handle
# pxt.publish(
#     source=t,  # Table handle you assigned
#     destination_uri='pxt://your-orgname/sample-images'
# )

Understanding destination URIs

The destination_uri in publish() uses the format: pxt://org:database/path URI components:

org (required): Your organization name
database (optional): Database name - defaults to main if omitted
path (required): Directory and table path in the cloud

Examples:

pxt://orgname/my-dataset → Uses the default main database
pxt://orgname:main/my-dataset → Explicitly specifies the main database
pxt://orgname:analytics/my-dataset → Uses the analytics database

About databases:

Every Pixeltable Cloud account includes a main database by default
Each database has its own storage bucket
You can create additional databases in your Pixeltable dashboard

Updating published datasets with push

After you’ve published a dataset, you can update the cloud replica with local changes using push():

# Make some changes to your local table
t.insert(
    [
        {
            'image': f'{base_url}/000000000049.jpg',
            'description': 'Outdoor scene',
        }
    ]
)

# Push the changes to your published dataset
t.push()

This updates the published dataset on Pixeltable Cloud with your local changes. Your dataset is now published and can be replicated by others using:

import pixeltable as pxt

sample_images = pxt.replicate(
    remote_uri='pxt://your-orgname/sample-images',
    local_path='sample-images-copy'
)

Note: If you are the owner of a published table, you cannot use replicate() to create a replica of your own table. This is because the table already exists in your Pixeltable database. The replicate() function is intended for pulling datasets published by others into your environment.

Access control

The access parameter in publish() controls who can replicate your dataset:

access='private' (default): Only your team members can access the dataset
access='public': Anyone can replicate your dataset

You can set access control either at the time of publish using the access parameter, or change it later in the Pixeltable Cloud UI. You can also manage team members and permissions in your dashboard.

Deleting published tables

If you want to delete a published table, you have two options: Option 1: Using the Pixeltable SDK Use drop_table() with your table’s destination URI (the same pxt:// URI you used when publishing):

pxt.drop_table('pxt://your-orgname/sample-images')

Option 2: Using the Pixeltable Cloud dashboard Navigate to your Pixeltable Cloud dashboard and delete the table from the UI.

Get help

Have questions or need support? Join our community:

Discord Community: Ask questions, get community support, and share what you build with Pixeltable
YouTube: Watch tutorials, demos, and feature walkthroughs
GitHub Issues: Report bugs or request features

Welcome to Pixeltable

Tools

Core Concepts

How-To

Overview

Setup

Replicating datasets

Replicate a public dataset

Working with replicas

Updating replicas with pull

Publishing datasets

Configure your API key

Create a sample dataset

Publish your dataset

Understanding destination URIs

Updating published datasets with push

Access control

Deleting published tables

Get help

Resources

Welcome to Pixeltable

Tools

Core Concepts

How-To

Documentation Index

​Overview

​Setup

​Replicating datasets

​Replicate a public dataset

​Working with replicas

​Updating replicas with pull

​Publishing datasets

​Configure your API key

​Create a sample dataset

​Publish your dataset

​Understanding destination URIs

​Updating published datasets with push

​Access control

​Deleting published tables

​Get help

​Resources

Overview

Setup

Replicating datasets

Replicate a public dataset

Working with replicas

Updating replicas with pull

Publishing datasets

Configure your API key

Create a sample dataset

Publish your dataset

Understanding destination URIs

Updating published datasets with push

Access control

Deleting published tables

Get help

Resources