SigNoz

Pixeltable runs LLM calls inside computed columns and UDFs. SigNoz records each call as a trace: the operation, the row, the UDF, and, with an OpenInference instrumentor, the model request. This page sends Pixeltable’s OpenTelemetry spans to SigNoz. The bridge itself, including metrics, is covered in Observability. Endpoint and header variables are in SigNoz’s OpenTelemetry environment variables. One Pixeltable operation (insert, add_computed_column, collect, and the rest) is one trace. At span_level='debug' an OpenAI insert looks like this. openai.chat_completions runs on a resource pool, so its UDF span sits beside pixeltable.row. A UDF without a resource pool nests under pixeltable.row.

pixeltable.insert
└─ pixeltable.store.insert_rows
   ├─ pixeltable.row                      (one per row, debug)
   ├─ pixeltable.udf.chat_completions     (one per LLM cell, debug)
   │  └─ ChatCompletion                    (OpenInference: model, tokens)
   └─ pixeltable.sa.insert_rows

UDF spans carry pxt.udf_path, pxt.column, and, for a resource-pool call, pxt.resource_pool and pxt.retries. The model name, prompt, completion, and token counts are attributes of the ChatCompletion span. SigNoz AI Observability reads gen_ai.* attributes. OpenInference sets llm.model_name, llm.provider, and llm.token_count.*, and SigNoz copies those onto gen_ai.*. To emit gen_ai.* from the process, set OPENINFERENCE_ENABLE_GENAI_SEMCONV=true.

This page matches Pixeltable 0.7.19, opentelemetry-instrumentation-pixeltable 0.1.0, and openinference-instrumentation-openai 0.1.65. pixeltable[otel] installs the instrumentation package from PyPI.

Choose an approach

You want	Use
Pixeltable builds the exporter and sends OTLP to SigNoz	Export spans to SigNoz
A process that already installed a global tracer provider	Attach to an existing provider

Install

pip install 'pixeltable[otel]' openai openinference-instrumentation-openai

pixeltable[otel] installs the Pixeltable bridge, the OpenTelemetry SDK, and the OTLP exporters. openinference-instrumentation-openai records LLM spans for pixeltable.functions.openai. For another provider, install that provider’s OpenInference instrumentor instead. openai is the client that chat_completions() calls. Neither Pixeltable nor the instrumentor installs it.

Anthropic SDK 2.0.0 emits its own anthropic.messages.create spans, including gen_ai.* usage, to the global provider. Leave openinference-instrumentation-anthropic uninstalled: enabling it beside those spans records each call twice.

Set the environment

SigNoz Cloud takes the region and the ingestion key from Settings, then Ingestion. The endpoint is https://ingest.<region>.signoz.cloud:443. Both OTLP/HTTP and OTLP/gRPC use port 443. The header is signoz-ingestion-key. Do not prefix the key with Bearer, and do not use the deprecated signoz-access-token header. The hosts ingest.signoz.io, ingest-in.signoz.io, and ingest-eu.signoz.io are deprecated.

export OTEL_SERVICE_NAME="pixeltable-app"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443"
export OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SPAN_LEVEL=debug
export OPENAI_API_KEY="sk-..."

A self-hosted collector listens on http://localhost:4318 for OTLP/HTTP. Omit the ingestion-key header.

export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_SERVICE_NAME="pixeltable-app"
export OTEL_SPAN_LEVEL=debug

Export spans to SigNoz

pxt_otel.init() builds the tracer provider and starts Pixeltable’s spans. For OTLP/HTTP it appends /v1/traces and /v1/metrics unless the endpoint already ends in that path, so pass the base URL. An endpoint that already ends in /v1/traces still exports traces, and metrics then fail. Leave metrics unset so token counters go to /v1/metrics. Pass metrics=False to send traces only. Call pxt_otel.init() once in the process, before the first Pixeltable operation. The default protocol is http/protobuf. Do not send OTLP/HTTP to the gRPC port 4317.

import os

import opentelemetry.instrumentation.pixeltable as pxt_otel
from openinference.instrumentation.openai import OpenAIInstrumentor

pxt_otel.init(
    endpoint=os.environ['OTEL_EXPORTER_OTLP_ENDPOINT'],  # base URL; /v1/traces is appended
    headers=os.environ['OTEL_EXPORTER_OTLP_HEADERS'],
    service_name='pixeltable-app',  # default service.name is pixeltable
    span_level='debug',  # info: operation, catalog, store, SQL; debug adds row, UDF, and file spans
)
OpenAIInstrumentor().instrument()

For a self-hosted collector, call init() like this instead, then OpenAIInstrumentor().instrument() as above:

pxt_otel.init(
    endpoint='http://localhost:4318',
    service_name='pixeltable-app',
    span_level='debug',
)

A pxt_otel.init() argument overrides the config. With the argument left unset, endpoint and protocol resolve from the database entry (db_exporter_otlp_endpoint, db_exporter_otlp_protocol in pixeltable.toml), then OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_PROTOCOL, then the [otel] section of ~/.pixeltable/config.toml. headers resolves from OTEL_EXPORTER_OTLP_HEADERS, then [otel]. Build the header value from environment variables, and never write the key into code. headers is one string, signoz-ingestion-key=<your-ingestion-key>, not a Python dict. On a hosted database, run pxt secret set pxt://<org>:<db> OTEL_EXPORTER_OTLP_HEADERS='signoz-ingestion-key=<your-ingestion-key>', then pxt db restart pxt://<org>:<db> and pxt service restart pxt://<org>:<db>/<service>. Configuration.

Pixeltable spans record which operation, row, column, and UDF ran: pxt.table, pxt.path, pxt.column, pxt.udf_path, and, for a resource-pool call, pxt.resource_pool and pxt.retries. While an operation span is active, Pixeltable sets the OpenTelemetry context for that UDF call, including across its thread pools and async scheduler, so ChatCompletion nests under the UDF span. The span name is pixeltable.udf. plus the UDF’s display name. span_level='trace' adds a span per iterator step. At the default 'info', row and UDF spans are omitted, and the LLM span nests under pixeltable.store.insert_rows. Setting only OTEL_EXPORTER_OTLP_TRACES_ENDPOINT does not start export. Set OTEL_EXPORTER_OTLP_ENDPOINT or pass endpoint. If the connected database’s entry sets db_exporter_otlp_endpoint, that value wins over the environment variable. With opentelemetry-instrument, the Pixeltable entry point instruments at span level info, and a later instrument() or init() does not change it. For row and UDF spans, set OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=pixeltable and call pxt_otel.init(span_level='debug'). init() reuses the provider that opentelemetry-instrument registered.

Declare the table and insert

An application file puts the columns on a TableModel. pxt schema update app.py my_app creates my_app.articles. The insert runs in the process that called pxt_otel.init(). A script can create the same column with create_table and add_computed_column in that same process. Both forms store the completion text. The model runs when the row is inserted.

Application file
Script

import pixeltable as pxt
import pixeltable.functions as pxtf

TableModel = pxt.model_base()


class Articles(TableModel, name='articles'):
    id: pxt.Int
    body: pxt.String
    summary = pxtf.openai.chat_completions(
        messages=[{'role': 'user', 'content': 'Summarize in one sentence: ' + body}],
        model='gpt-4o-mini',
    ).choices[0].message.content

pxt schema update app.py my_app

In the process that called pxt_otel.init():

import pixeltable as pxt
from opentelemetry import metrics, trace

t = pxt.get_table('my_app.articles')
t.insert([
    {'id': 1, 'body': 'Pixeltable is declarative data infrastructure for multimodal AI apps.'}
])
trace.get_tracer_provider().force_flush()  # short-lived scripts: send spans before exit
metrics.get_meter_provider().force_flush()  # metrics export about once a minute

In the same process as pxt_otel.init():

import pixeltable as pxt
import pixeltable.functions as pxtf
from opentelemetry import metrics, trace

t = pxt.create_table('articles', {'id': pxt.Int, 'body': pxt.String}, if_exists='replace')
t.add_computed_column(
    summary=pxtf.openai.chat_completions(
        messages=[{'role': 'user', 'content': 'Summarize in one sentence: ' + t.body}],
        model='gpt-4o-mini',
    ).choices[0].message.content
)
t.insert([
    {'id': 1, 'body': 'Pixeltable is declarative data infrastructure for multimodal AI apps.'}
])
trace.get_tracer_provider().force_flush()
metrics.get_meter_provider().force_flush()

To trace a served app, add --otel to pxt service run or pxt service update (Tracing). It calls pxt_otel.init() with no arguments before it loads the app file, so configure the export in the environment, or in the matching [otel] keys. The variables in Set the environment are that configuration. pxt service run reads these variables in its own process. pxt service update starts the service from the Pixeltable daemon, which reads the environment once, so run pxt daemon restart after the exports. --otel calls pxt_otel.init() and instruments FastAPI. It does not instrument OpenAI, and an OpenAIInstrumentor().instrument() in another process does not carry into this one. Put this at module scope in app.py. The service loads the file after init(), so the instrumentor attaches to that provider:

from openinference.instrumentation.openai import OpenAIInstrumentor

OpenAIInstrumentor().instrument()

The app file must not call pxt_otel.init(): with --otel, that second call raises INVALID_STATE, and without --otel it records Pixeltable spans but not the FastAPI request spans.

Attach to an existing provider

When the process already has a tracer provider, call PixeltableInstrumentor().instrument() instead of pxt_otel.init(). init() reuses that tracer provider. With an endpoint set and no meter provider yet, init() also installs an OTLP meter provider, so Pixeltable’s token counters are exported. instrument() attaches Pixeltable spans to the provider you already have and installs no exporter, so it exports no metrics. A second init() raises INVALID_STATE. A second instrument() returns immediately. The provider’s exporter is what reaches SigNoz, so its endpoint and signoz-ingestion-key header are already set.

from openinference.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.instrumentation.pixeltable import PixeltableInstrumentor

PixeltableInstrumentor().instrument(span_level='debug')
OpenAIInstrumentor().instrument()

Use this or pxt_otel.init() in a process, not both.

View the traces

Open the Traces Explorer and filter on the service name and the operation:

service.name = 'pixeltable-app' AND name = 'pixeltable.insert'

The waterfall shows pixeltable.udf.chat_completions under pixeltable.store.insert_rows, beside pixeltable.row, and ChatCompletion under the UDF span. The prompt and completion are attributes on ChatCompletion. AI Observability counts that child span as an LLM call and shows the model and its input and output tokens. Cost stays at $0 until you add a model pricing rule for the model name. Time to first token reads 0s for a non-streaming call. Pixeltable also exports metrics when an endpoint is set. In the Metrics Explorer, query pixeltable.udf.input_tokens and pixeltable.udf.output_tokens (attribute pxt.model), pixeltable.udf.calls, pixeltable.udf.latency, and pixeltable.rows.written (attribute pxt.table). The metric scope name is pixeltable. collect() writes a pixeltable.collect trace and does not call the model again.

Caveats

With the bridge on, every Pixeltable operation is a trace, including create_table, drop_table, add_computed_column, and collect. span_level='debug' also adds finer catalog and store spans such as pixeltable.catalog.*. At info, the per-row and per-UDF spans are dropped, and the LLM span stays in the insert trace without a pixeltable.udf.chat_completions parent.
Per-row spans stop at 100 rows in one operation. A provider call on a resource pool still gets its pixeltable.udf.* span.
Computed-column values are stored. collect() records a Pixeltable span and does not call the model again. The model runs on insert, on update, and when add_computed_column backfills existing rows.
Flush before the process exits. The exporter batches spans, and metrics go out about once a minute. A 401 or 403 means the signoz-ingestion-key header is not the key from Ingestion Settings, or the region in the endpoint is a different account.
Call pxt_otel.init(), or PixeltableInstrumentor().instrument(), and enable the instrumentors before the first Pixeltable operation. pxt_otel.init() runs once per process. A second call raises an error that instrumentation is already initialized.
Prompts and completions are sent to SigNoz on the ChatCompletion span. logs defaults to off. Leave it off unless those log lines are redacted.

Observability

Span tree, metrics, and a Grafana Cloud export.

OpenTelemetry variables

SigNoz Cloud endpoint, protocol, and ingestion key.

Built-in Integrations

Observability

Infrastructure

Bring Your Own

Providers

Choose an approach

Install

Set the environment

Export spans to SigNoz

Declare the table and insert

Attach to an existing provider

View the traces

Caveats

Observability

OpenTelemetry variables

​Choose an approach

​Install

​Set the environment

​Export spans to SigNoz

​Declare the table and insert

​Attach to an existing provider

​View the traces

​Caveats

Observability

OpenTelemetry variables

Choose an approach

Install

Set the environment

Export spans to SigNoz

Declare the table and insert

Attach to an existing provider

View the traces

Caveats