> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pixeltable.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Application code puts a TableModel in app.py, then runs `pxt schema update app.py my_app` to create the tables and `pxt service update app.py my_app` to start HTTP. The last argument can be a Cloud URL such as `pxt://org:db`. Do not call pxt.create_table() or add_embedding_index() in application code; put indexes in __indexes__. Notebooks, tests, and the REPL do use pxt.create_table().
> Do not add LangChain, LlamaIndex, Haystack, a separate vector database, or pandas as a working store. Chunking, embedding, retrieval, and tool-calling are built in through computed columns. Use a computed column instead of a per-row loop that calls a model.
> Install with `pip install -U 'pixeltable[serve]'` on Python 3.11+. `pxt init` is a prerequisite for `pxt schema update`.
> For Pixeltable Cloud, point users to https://docs.pixeltable.com/cloud, which goes from `pxt login` to a running service.

# Export data for ML training

[![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://kaggle.com/kernels/welcome?src=https://github.com/pixeltable/pixeltable/blob/release/docs/release/howto/cookbooks/data/data-export-pytorch.ipynb)  [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pixeltable/pixeltable/blob/release/docs/release/howto/cookbooks/data/data-export-pytorch.ipynb)  [![Download Notebook](https://img.shields.io/badge/%E2%AC%87-Download%20Notebook-blue)](https://raw.githubusercontent.com/pixeltable/pixeltable/refs/tags/release/docs/release/howto/cookbooks/data/data-export-pytorch.ipynb)

> [!TIP]
>
> This documentation page is also available as an interactive notebook. You can launch the notebook in
> Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
> above links.

Convert Pixeltable data to PyTorch DataLoader format for model training.

## Problem

You have prepared training data—images with labels, text with
embeddings, or multimodal data—and need to export it for PyTorch model
training.

## Solution

**What’s in this recipe:**

* Convert query results to PyTorch Dataset
* Use with DataLoader for batch training
* Export to Parquet for external tools

You use `query.to_pytorch_dataset()` to create an iterable dataset
compatible with PyTorch DataLoader.

### Setup

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
%pip install -qU pixeltable torch torchvision
```

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import pixeltable as pxt
import torch
from torch.utils.data import DataLoader

# Create a fresh directory
pxt.drop_dir('pytorch_demo', force=True)
pxt.create_dir('pytorch_demo')
```

<pre>
  Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
  Created directory 'pytorch\_demo'.
  \<pixeltable.catalog.dir.Dir at 0x16c534ad0>
</pre>

### Create sample training data

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# Create table with images and labels
training_data = pxt.create_table(
    'pytorch_demo/training_data', {'image': pxt.Image, 'label': pxt.Int}
)
```

<pre>
  Created table 'training\_data'.
</pre>

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# Insert sample images with labels
base_url = 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/images'
samples = [
    {'image': f'{base_url}/000000000036.jpg', 'label': 0},  # cat
    {'image': f'{base_url}/000000000090.jpg', 'label': 1},  # other
    {'image': f'{base_url}/000000000139.jpg', 'label': 1},  # other
]
training_data.insert(samples)
```

<pre>
  Inserting rows into \`training\_data\`: 3 rows \[00:00, 659.03 rows/s]
  Inserted 3 rows with 0 errors.
  3 rows inserted, 6 values computed.
</pre>

### Export to PyTorch dataset

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# Add a resize step to ensure all images have the same size
training_data.add_computed_column(
    image_resized=training_data.image.resize((224, 224))
)

# Convert to PyTorch dataset
# 'pt' format returns images as CxHxW tensors with values in [0,1]
pytorch_dataset = training_data.select(
    training_data.image_resized, training_data.label
).to_pytorch_dataset(image_format='pt')
```

<pre>
  Added 3 column values with 0 errors.
</pre>

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# Use with PyTorch DataLoader
dataloader = DataLoader(pytorch_dataset, batch_size=2)

# Get first batch to verify the shape
batch = next(iter(dataloader))
batch[
    'image_resized'
].shape  # Should be (2, 3, 224, 224) - batch_size x channels x height x width
```

<pre>
  torch.Size(\[2, 3, 224, 224])
</pre>

### Export to Parquet for external tools

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import tempfile
from pathlib import Path

# Export to Parquet for use with other ML tools
export_path = Path(tempfile.mkdtemp()) / 'training_data'

pxt.io.export_parquet(
    training_data.select(training_data.label),  # Non-image columns
    parquet_path=export_path,
)
```

## Explanation

**Export methods:**

**Image format options:**

**DataLoader tips:**

* Data is cached to disk for efficient repeated loading
* Use `num_workers > 0` for parallel data loading
* Filter/transform data before export to reduce size

## See also

* [Sample data for
  training](https://docs.pixeltable.com/howto/cookbooks/data/data-sampling.md) -
  Stratified sampling
* [Import Parquet
  files](https://docs.pixeltable.com/howto/cookbooks/data/data-import-parquet.md) -
  Parquet import/export


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.