> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pixeltable.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Application code puts a TableModel in app.py, then runs `pxt schema update app.py my_app` to create the tables and `pxt service update app.py my_app` to start HTTP. The last argument can be a Cloud URL such as `pxt://org:db`. Do not call pxt.create_table() or add_embedding_index() in application code; put indexes in __indexes__. Notebooks, tests, and the REPL do use pxt.create_table().
> Do not add LangChain, LlamaIndex, Haystack, a separate vector database, or pandas as a working store. Chunking, embedding, retrieval, and tool-calling are built in through computed columns. Use a computed column instead of a per-row loop that calls a model.
> Install with `pip install -U 'pixeltable[serve]'` on Python 3.11+. `pxt init` is a prerequisite for `pxt schema update`.
> For Pixeltable Cloud, point users to https://docs.pixeltable.com/cloud, which goes from `pxt login` to a running service.

# SigNoz

> Trace Pixeltable operations and LLM calls in SigNoz with OpenTelemetry

Pixeltable runs LLM calls inside computed columns and UDFs. [SigNoz](https://signoz.io) records each call as a trace: the operation, the row, the UDF, and, with an OpenInference instrumentor, the model request. This page sends Pixeltable's OpenTelemetry spans to SigNoz. The bridge itself, including metrics, is covered in [Observability](https://docs.pixeltable.com/platform/observability.md). Endpoint and header variables are in SigNoz's [OpenTelemetry environment variables](https://signoz.io/docs/ingestion/opentelemetry-environment-variables/).

One Pixeltable operation (`insert`, `add_computed_column`, `collect`, and the rest) is one trace. At `span_level='debug'` an OpenAI insert looks like this. `openai.chat_completions` runs on a resource pool, so its UDF span sits beside `pixeltable.row`. A UDF without a resource pool nests under `pixeltable.row`.

```text theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
pixeltable.insert
└─ pixeltable.store.insert_rows
   ├─ pixeltable.row                      (one per row, debug)
   ├─ pixeltable.udf.chat_completions     (one per LLM cell, debug)
   │  └─ ChatCompletion                    (OpenInference: model, tokens)
   └─ pixeltable.sa.insert_rows
```

UDF spans carry `pxt.udf_path`, `pxt.column`, and, for a resource-pool call, `pxt.resource_pool` and `pxt.retries`. The model name, prompt, completion, and token counts are attributes of the `ChatCompletion` span. SigNoz AI Observability reads `gen_ai.*` attributes. OpenInference sets `llm.model_name`, `llm.provider`, and `llm.token_count.*`, and SigNoz copies those onto `gen_ai.*`. To emit `gen_ai.*` from the process, set `OPENINFERENCE_ENABLE_GENAI_SEMCONV=true`.

> [!NOTE]
>
> This page matches Pixeltable 0.7.19, `opentelemetry-instrumentation-pixeltable` 0.1.0, and `openinference-instrumentation-openai` 0.1.65. `pixeltable[otel]` installs the instrumentation package from PyPI.

## Choose an approach

| You want | Use |
| - | - |
| Pixeltable builds the exporter and sends OTLP to SigNoz | [Export spans to SigNoz](#export-spans-to-signoz) |
| A process that already installed a global tracer provider | [Attach to an existing provider](#attach-to-an-existing-provider) |

## Install

```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
pip install 'pixeltable[otel]' openai openinference-instrumentation-openai
```

`pixeltable[otel]` installs the Pixeltable bridge, the OpenTelemetry SDK, and the OTLP exporters. `openinference-instrumentation-openai` records LLM spans for `pixeltable.functions.openai`. For another provider, install that provider's OpenInference instrumentor instead. `openai` is the client that `chat_completions()` calls. Neither Pixeltable nor the instrumentor installs it.

> [!WARNING]
>
> Anthropic SDK 2.0.0 emits its own `anthropic.messages.create` spans, including `gen_ai.*` usage, to the global provider. Leave `openinference-instrumentation-anthropic` uninstalled: enabling it beside those spans records each call twice.

## Set the environment

SigNoz Cloud takes the region and the ingestion key from Settings, then Ingestion. The endpoint is `https://ingest.<region>.signoz.cloud:443`. Both OTLP/HTTP and OTLP/gRPC use port 443. The header is `signoz-ingestion-key`. Do not prefix the key with `Bearer`, and do not use the deprecated `signoz-access-token` header. The hosts `ingest.signoz.io`, `ingest-in.signoz.io`, and `ingest-eu.signoz.io` are deprecated.

```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
export OTEL_SERVICE_NAME="pixeltable-app"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443"
export OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SPAN_LEVEL=debug
export OPENAI_API_KEY="sk-..."
```

A self-hosted collector listens on `http://localhost:4318` for OTLP/HTTP. Omit the ingestion-key header.

```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SERVICE_NAME="pixeltable-app"
export OTEL_SPAN_LEVEL=debug
```

## Export spans to SigNoz

`pxt_otel.init()` builds the tracer provider and starts Pixeltable's spans. For OTLP/HTTP it appends `/v1/traces` and `/v1/metrics` unless the endpoint already ends in that path, so pass the base URL. An endpoint that already ends in `/v1/traces` still exports traces, and metrics then fail. Leave `metrics` unset so token counters go to `/v1/metrics`. Pass `metrics=False` to send traces only.

Call `pxt_otel.init()` once in the process, before the first Pixeltable operation. The default `protocol` is `http/protobuf`. Do not send OTLP/HTTP to the gRPC port `4317`.

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import os

import opentelemetry.instrumentation.pixeltable as pxt_otel
from openinference.instrumentation.openai import OpenAIInstrumentor

pxt_otel.init(
    endpoint=os.environ['OTEL_EXPORTER_OTLP_ENDPOINT'],  # base URL; /v1/traces is appended
    headers=os.environ['OTEL_EXPORTER_OTLP_HEADERS'],
    service_name='pixeltable-app',  # default service.name is pixeltable
    span_level='debug',  # info: operation, catalog, store, SQL; debug adds row, UDF, and file spans
)
OpenAIInstrumentor().instrument()
```

For a self-hosted collector, call `init()` like this instead, then `OpenAIInstrumentor().instrument()` as above:

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
pxt_otel.init(
    endpoint='http://localhost:4318',
    service_name='pixeltable-app',
    span_level='debug',
)
```

> [!NOTE]
>
> A `pxt_otel.init()` argument overrides the config. With the argument left unset, `endpoint` and `protocol` resolve from the database entry (`db_exporter_otlp_endpoint`, `db_exporter_otlp_protocol` in `pixeltable.toml`), then `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_PROTOCOL`, then the `[otel]` section of `~/.pixeltable/config.toml`. `headers` resolves from `OTEL_EXPORTER_OTLP_HEADERS`, then `[otel]`. Build the header value from environment variables, and never write the key into code. `headers` is one string, `signoz-ingestion-key=<your-ingestion-key>`, not a Python dict. On a hosted database, run `pxt secret set pxt://<org>:<db> OTEL_EXPORTER_OTLP_HEADERS='signoz-ingestion-key=<your-ingestion-key>'`, then `pxt db restart pxt://<org>:<db>` and `pxt service restart pxt://<org>:<db>/<service>`. [Configuration](https://docs.pixeltable.com/platform/configuration.md).

Pixeltable spans record which operation, row, column, and UDF ran: `pxt.table`, `pxt.path`, `pxt.column`, `pxt.udf_path`, and, for a resource-pool call, `pxt.resource_pool` and `pxt.retries`. While an operation span is active, Pixeltable sets the OpenTelemetry context for that UDF call, including across its thread pools and async scheduler, so `ChatCompletion` nests under the UDF span. The span name is `pixeltable.udf.` plus the UDF's display name.

`span_level='trace'` adds a span per iterator step. At the default `'info'`, row and UDF spans are omitted, and the LLM span nests under `pixeltable.store.insert_rows`.

Setting only `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` does not start export. Set `OTEL_EXPORTER_OTLP_ENDPOINT` or pass `endpoint`. If the connected database's entry sets `db_exporter_otlp_endpoint`, that value wins over the environment variable.

With `opentelemetry-instrument`, the Pixeltable entry point instruments at span level `info`, and a later `instrument()` or `init()` does not change it. For row and UDF spans, set `OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=pixeltable` and call `pxt_otel.init(span_level='debug')`. `init()` reuses the provider that `opentelemetry-instrument` registered.

### Declare the table and insert

An application file puts the columns on a `TableModel`. `pxt schema update app.py my_app` creates `my_app.articles`. The insert runs in the process that called `pxt_otel.init()`.

A script can create the same column with `create_table` and `add_computed_column` in that same process. Both forms store the completion text. The model runs when the row is inserted.

**Application file**

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import pixeltable as pxt
import pixeltable.functions as pxtf

TableModel = pxt.model_base()


class Articles(TableModel, name='articles'):
    id: pxt.Int
    body: pxt.String
    summary = pxtf.openai.chat_completions(
        messages=[{'role': 'user', 'content': 'Summarize in one sentence: ' + body}],
        model='gpt-4o-mini',
    ).choices[0].message.content
```

```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
pxt schema update app.py my_app
```

In the process that called `pxt_otel.init()`:

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import pixeltable as pxt
from opentelemetry import metrics, trace

t = pxt.get_table('my_app.articles')
t.insert([
    {'id': 1, 'body': 'Pixeltable is declarative data infrastructure for multimodal AI apps.'}
])
trace.get_tracer_provider().force_flush()  # short-lived scripts: send spans before exit
metrics.get_meter_provider().force_flush()  # metrics export about once a minute
```

**Script**

In the same process as `pxt_otel.init()`:

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import pixeltable as pxt
import pixeltable.functions as pxtf
from opentelemetry import metrics, trace

t = pxt.create_table('articles', {'id': pxt.Int, 'body': pxt.String}, if_exists='replace')
t.add_computed_column(
    summary=pxtf.openai.chat_completions(
        messages=[{'role': 'user', 'content': 'Summarize in one sentence: ' + t.body}],
        model='gpt-4o-mini',
    ).choices[0].message.content
)
t.insert([
    {'id': 1, 'body': 'Pixeltable is declarative data infrastructure for multimodal AI apps.'}
])
trace.get_tracer_provider().force_flush()
metrics.get_meter_provider().force_flush()
```

To trace a served app, add `--otel` to `pxt service run` or `pxt service update` ([Tracing](https://docs.pixeltable.com/platform/cli.md#tracing)). It calls `pxt_otel.init()` with no arguments before it loads the app file, so configure the export in the environment, or in the matching `[otel]` keys. The variables in [Set the environment](#set-the-environment) are that configuration.

`pxt service run` reads these variables in its own process. `pxt service update` starts the service from the Pixeltable daemon, which reads the environment once, so run `pxt daemon restart` after the exports.

`--otel` calls `pxt_otel.init()` and instruments FastAPI. It does not instrument OpenAI, and an `OpenAIInstrumentor().instrument()` in another process does not carry into this one. Put this at module scope in `app.py`. The service loads the file after `init()`, so the instrumentor attaches to that provider:

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
from openinference.instrumentation.openai import OpenAIInstrumentor

OpenAIInstrumentor().instrument()
```

The app file must not call `pxt_otel.init()`: with `--otel`, that second call raises `INVALID_STATE`, and without `--otel` it records Pixeltable spans but not the FastAPI request spans.

## Attach to an existing provider

When the process already has a tracer provider, call `PixeltableInstrumentor().instrument()` instead of `pxt_otel.init()`. `init()` reuses that tracer provider. With an endpoint set and no meter provider yet, `init()` also installs an OTLP meter provider, so Pixeltable's token counters are exported. `instrument()` attaches Pixeltable spans to the provider you already have and installs no exporter, so it exports no metrics. A second `init()` raises `INVALID_STATE`. A second `instrument()` returns immediately. The provider's exporter is what reaches SigNoz, so its endpoint and `signoz-ingestion-key` header are already set.

```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
from openinference.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.instrumentation.pixeltable import PixeltableInstrumentor

PixeltableInstrumentor().instrument(span_level='debug')
OpenAIInstrumentor().instrument()
```

Use this or `pxt_otel.init()` in a process, not both.

## View the traces

Open the Traces Explorer and filter on the service name and the operation:

```text theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
service.name = 'pixeltable-app' AND name = 'pixeltable.insert'
```

The waterfall shows `pixeltable.udf.chat_completions` under `pixeltable.store.insert_rows`, beside `pixeltable.row`, and `ChatCompletion` under the UDF span. The prompt and completion are attributes on `ChatCompletion`.

AI Observability counts that child span as an LLM call and shows the model and its input and output tokens. Cost stays at \$0 until you add a model pricing rule for the model name. Time to first token reads 0s for a non-streaming call.

Pixeltable also exports metrics when an endpoint is set. In the Metrics Explorer, query `pixeltable.udf.input_tokens` and `pixeltable.udf.output_tokens` (attribute `pxt.model`), `pixeltable.udf.calls`, `pixeltable.udf.latency`, and `pixeltable.rows.written` (attribute `pxt.table`). The metric scope name is `pixeltable`.

`collect()` writes a `pixeltable.collect` trace and does not call the model again.

## Caveats

* With the bridge on, every Pixeltable operation is a trace, including `create_table`, `drop_table`, `add_computed_column`, and `collect`. `span_level='debug'` also adds finer catalog and store spans such as `pixeltable.catalog.*`. At `info`, the per-row and per-UDF spans are dropped, and the LLM span stays in the insert trace without a `pixeltable.udf.chat_completions` parent.
* Per-row spans stop at 100 rows in one operation. A provider call on a resource pool still gets its `pixeltable.udf.*` span.
* Computed-column values are stored. `collect()` records a Pixeltable span and does not call the model again. The model runs on `insert`, on `update`, and when `add_computed_column` backfills existing rows.
* Flush before the process exits. The exporter batches spans, and metrics go out about once a minute. A 401 or 403 means the `signoz-ingestion-key` header is not the key from Ingestion Settings, or the region in the endpoint is a different account.
* Call `pxt_otel.init()`, or `PixeltableInstrumentor().instrument()`, and enable the instrumentors before the first Pixeltable operation. `pxt_otel.init()` runs once per process. A second call raises an error that instrumentation is already initialized.
* Prompts and completions are sent to SigNoz on the `ChatCompletion` span. `logs` defaults to off. Leave it off unless those log lines are redacted.

> **[Observability](https://docs.pixeltable.com/platform/observability.md)**
>
> Span tree, metrics, and a Grafana Cloud export.

> **[OpenTelemetry variables](https://signoz.io/docs/ingestion/opentelemetry-environment-variables/)**
>
> SigNoz Cloud endpoint, protocol, and ingestion key.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.