---
name: Pixeltable
description: Use when building multimodal backends, agentic workflows, or media processing pipelines. Agents should reach for this skill when working with tables, computed columns, embeddings, media files, or serving endpoints from a single Python file.
metadata:
    mintlify-proj: pixeltable
    version: "1.0"
---

# Pixeltable Skill

## Product summary

Pixeltable is a unified multimodal database that combines tables, computed columns, embedding indexes, and HTTP endpoints in a single Python file (`app.py`). It eliminates the need to glue together separate blob stores, vector databases, and orchestrators. Agents use Pixeltable to build backends that process images, video, audio, and documents alongside structured data, with automatic incremental updates and built-in serving.

**Key files and commands:**
- `app.py` — declares `TableModel` classes (tables/views) and `FastAPIRouter` instances (endpoints)
- `pxt schema update app.py my_app` — creates or migrates tables
- `pxt service update app.py my_app` — starts HTTP endpoints
- `pxt ls`, `pxt describe`, `pxt rows` — inspect tables
- `~/.pixeltable/config.toml` — configuration for API keys, storage, and system settings

**Primary docs:** https://docs.pixeltable.com

## When to use

Reach for this skill when:
- Building a backend that processes media (images, video, audio, documents) and structured data together
- Creating agentic workflows where tool calls, memory, and reasoning are columns in a table
- Setting up RAG pipelines with document chunking, embeddings, and semantic search
- Serving model inference endpoints without a separate orchestrator
- Needing incremental updates: insert a row, computed columns run automatically, indexes stay current
- Migrating from LangChain, Pinecone, or hand-rolled FastAPI stacks

Do not use Pixeltable for:
- Purely analytical workloads (use a data warehouse instead)
- Applications that don't need media or AI inference
- Scenarios where you need a separate vector database (Pixeltable includes embedding indexes)

## Quick reference

### Project setup

```bash
pip install 'pixeltable[serve]'  # [serve] for HTTP endpoints
pxt init                          # Mark directory as project root
pxt schema update app.py my_app   # Create tables
pxt service update app.py my_app  # Start endpoints
pxt service list                  # Print URL and port
```

### Table declaration (in app.py)

```python
import pixeltable as pxt
import pixeltable.functions as pxtf
from pixeltable.serving import FastAPIRouter

TableModel = pxt.model_base()

class Docs(TableModel, name='docs'):
    title: pxt.String                          # Stored column (you insert)
    body: pxt.String | None                    # Optional column
    title_upper = pxtf.string.upper(title)     # Computed column (auto-run)
    embedding = openai.embedding(title)        # AI-backed computed column
    __indexes__ = [pxt.EmbeddingIndex(embedding)]  # Vector index

class Titled(TableModel, name='titled', base=Docs.where(Docs.title != '')):
    pass  # View: filtered subset of Docs
```

### Common operations

| Task | Command |
|------|---------|
| List tables | `pxt ls my_app` |
| Show schema | `pxt describe my_app/docs` |
| Peek rows | `pxt rows my_app/docs -n 5` |
| Count rows | `pxt count my_app/docs` |
| Find errors | `pxt errors my_app/docs` |
| Drop table | `pxt drop my_app/docs -f` |
| Revert changes | `pxt revert my_app/docs --steps 1 -f` |

### Media types

```python
class Media(TableModel, name='media'):
    image: pxt.Image                    # PIL.Image.Image at runtime
    video: pxt.Video                    # Local path string at runtime
    audio: pxt.Audio                    # Local path string at runtime
    document: pxt.Document              # PDF, DOCX, etc.
    # Insert as file path or URL; Pixeltable handles caching
```

### Computed columns and UDFs

```python
@pxt.udf
def my_function(text: str) -> str:
    return text.upper()

class MyTable(TableModel, name='mytable'):
    input: pxt.String
    output = my_function(input)  # Runs on insert/update
```

### Queries and filtering

```python
t = pxt.get_table('my_app.docs')
t.where(t.title != '').select(t.title, t.embedding).collect()
t.order_by(t.title).limit(10).collect()
t.group_by(t.category).select(t.category, pxtf.count()).collect()
```

### Embedding search

```python
# Create index on model
class Docs(TableModel, name='docs'):
    text: pxt.String
    embedding = openai.embedding(text)
    __indexes__ = [pxt.EmbeddingIndex(embedding)]

# Query by similarity
query_embedding = openai.embedding('search term')
t.where(t.embedding.similarity(query_embedding) > 0.8).collect()
```

### HTTP endpoints

```python
ingest = FastAPIRouter(name='ingest')
ingest.add_insert_route(
    Docs,
    path='/docs',
    inputs=[Docs.title, Docs.body],
    outputs=[Docs.title, Docs.title_upper]
)

# POST /docs with {"title": "...", "body": "..."}
# Returns computed columns
```

## Decision guidance

| Scenario | Use | Avoid |
|----------|-----|-------|
| **Storing media** | `pxt.Image`, `pxt.Video`, `pxt.Audio`, `pxt.Document` in table columns | Separate blob store + metadata table |
| **Incremental transforms** | Computed columns (auto-run on insert/update) | Batch pipelines or manual re-runs |
| **Vector search** | `EmbeddingIndex` on table (stays in sync) | Separate vector database |
| **Tool calling in agents** | `@pxt.udf` columns that call LLM tools | LangChain or LangGraph |
| **Serving inference** | `pxt service update` with `FastAPIRouter` | Separate FastAPI app |
| **Local vs. Cloud** | Local: `pxt schema update app.py my_app`; Cloud: `pxt schema update app.py pxt://org:db` | Different code for each |

## Workflow

### 1. Set up project
```bash
pxt init
pxt service example --out app.py  # Generate starter file
```

### 2. Declare schema in app.py
- Define `TableModel` classes (stored and computed columns)
- Add `FastAPIRouter` for endpoints (optional)
- Use `@pxt.udf` for custom logic
- Declare `__indexes__` for embeddings or B-tree indexes

### 3. Create tables
```bash
pxt schema update app.py my_app
pxt schema diff app.py my_app  # Review changes first
```

### 4. Start endpoints (if using FastAPIRouter)
```bash
pxt service update app.py my_app
pxt service list  # Get URL
```

### 5. Insert data
```bash
curl -X POST http://127.0.0.1:<port>/docs \
  -H 'Content-Type: application/json' \
  -d '{"title": "Hello", "body": "world"}'
```

### 6. Query and inspect
```bash
pxt rows my_app/docs -n 5
pxt errors my_app/docs  # Check for failures
```

### 7. Iterate
- Modify `app.py` (add columns, change UDFs)
- Run `pxt schema diff` to review
- Run `pxt schema update` to apply
- Run `pxt service update` if routes changed

## Common gotchas

- **UDFs require type hints.** `@pxt.udf def f(x: str) -> str:` works; `def f(x):` fails.
- **Local UDFs are serialized.** Changes to a local UDF don't affect existing columns; only new columns use the updated code. Module UDFs (imported from another file) pick up changes on next execution.
- **Computed columns are stored by default.** Use `pxt.Column(type=..., stored=False)` to compute on-demand only.
- **Media files are external.** `pxt.Image` stores a reference; the actual file lives in local cache or cloud storage (S3, GCS, Azure). Configure with `PIXELTABLE_INPUT_MEDIA_DEST` and `PIXELTABLE_OUTPUT_MEDIA_DEST`.
- **Views auto-update.** When you insert into a base table, views that depend on it are automatically updated. No manual refresh needed.
- **Indexes are declarative.** Declare `__indexes__` on the model; do not call `add_embedding_index()` after `pxt schema update`.
- **Primary keys are optional.** If you don't declare one, Pixeltable auto-generates a rowid. Declare with `pxt.Column(primary_key=True)`.
- **Errors are stored.** If a computed column fails, the row is inserted but the cell has an error. Query with `pxt errors my_app/table` to find them.
- **Config file is required.** `~/.pixeltable/config.toml` must include `file_cache_size_g` (in GiB). Set it based on available disk space.
- **Daemon runs in background.** First `pxt` command starts a daemon on `127.0.0.1:22089`. It persists across shells. Use `pxt daemon stop` to kill it.

## Verification checklist

Before submitting work:

- [ ] Run `pxt schema diff app.py my_app` and review all changes
- [ ] Confirm no destructive operations (column drops) unless intentional
- [ ] Test computed columns on sample rows with `.select(...).collect()` before adding to schema
- [ ] Verify UDFs have type hints on all parameters and return value
- [ ] Check `pxt errors my_app/table` for any failed computations
- [ ] Confirm `pxt service list` shows endpoints running (if using FastAPIRouter)
- [ ] Test HTTP routes with curl or a client before deployment
- [ ] Run `pxt schema update -n` (dry-run) to confirm plan before applying
- [ ] For Cloud: confirm `PIXELTABLE_API_KEY` is set and `pxt db update pxt://org:db` succeeded

## Resources

**Comprehensive navigation:** https://docs.pixeltable.com/llms.txt

**Critical pages:**
1. [Quick Start](https://docs.pixeltable.com/overview/quick-start) — Install, write app.py, create tables, start endpoints
2. [CLI Reference](https://docs.pixeltable.com/platform/cli) — Every command, flag, and exit code
3. [How It Works](https://docs.pixeltable.com/overview/how-it-works) — What `pxt schema` and `pxt service` do, database options

---

> For additional documentation and navigation, see: https://docs.pixeltable.com/llms.txt