Skip to main content

​
module  pixeltable.functions.gemini

Pixeltable UDFs that wrap various endpoints from the Google Gemini API. In order to use them, you must first pip install google-genai and configure your Gemini credentials, as described in the Working with Gemini tutorial. Supports two authentication methods:
  • Google AI Studio: set GOOGLE_API_KEY or GEMINI_API_KEY (or put api_key in the gemini section of the Pixeltable config file).
  • Vertex AI: set GOOGLE_GENAI_USE_VERTEXAI=true and GOOGLE_CLOUD_PROJECT (and optionally GOOGLE_CLOUD_LOCATION), then authenticate via Application Default Credentials (e.g. gcloud auth application-default login).

​
func  invoke_tools()

Signature
invoke_tools(
    tools: pixeltable.func.tools.Tools,
    response: pixeltable.exprs.expr.Expr
) -> pixeltable.exprs.inline_expr.InlineDict
Converts an OpenAI response dict to Pixeltable tool invocation format and calls tools._invoke().

​
udf  embed_content()

Signatures
# Signature 1:
@pxt.udf
embed_content(
    contents: pxt.String,
    model: pxt.String,
    config: pxt.Json | None,
    use_batch_api: pxt.Bool
) -> pxt.Array[(None,), float32]

# Signature 2:
@pxt.udf
embed_content(
    contents: pxt.Image,
    model: pxt.String,
    config: pxt.Json | None
) -> pxt.Array[(None,), float32]

# Signature 3:
@pxt.udf
embed_content(
    contents: pxt.Audio,
    model: pxt.String,
    config: pxt.Json | None
) -> pxt.Array[(None,), float32]

# Signature 4:
@pxt.udf
embed_content(
    contents: pxt.Video,
    model: pxt.String,
    config: pxt.Json | None
) -> pxt.Array[(None,), float32]

# Signature 5:
@pxt.udf
embed_content(
    contents: pxt.Document,
    model: pxt.String,
    config: pxt.Json | None
) -> pxt.Array[(None,), float32]
Generate embeddings for text, images, video, and other content. For more information on Gemini embeddings API, see: https://ai.google.dev/gemini-api/docs/embeddings Requirements:
  • pip install google-genai
Parameters:
  • contents (String): The string, image, audio, video, or document to embed.
  • model (String): The Gemini model to use.
  • config (Json | None, default: Literal(None)): Configuration for embedding generation, corresponding to keyword arguments of genai.types.EmbedContentConfig. For details on the parameters, see: https://googleapis.github.io/python-genai/genai.html#genai.types.EmbedContentConfig
  • use_batch_api (Bool, default: Literal(False)): If True, use Gemini’s Batch API that provides a higher throughput at a lower cost at the expense of higher latency.
Returns:
  • pxt.Array[(None,), float32]: The corresponding embedding vector.
Examples: Add a computed column with embeddings to an existing table with a text column: 
t.add_computed_column(
    embedding=embed_content(t.text, model='gemini-embedding-001')
)
Add an embedding index on text column: 
t.add_embedding_index(
    t.text, embedding=embed_content.using(model='gemini-embedding-001')
)

​
udf  generate_content()

Signature
@pxt.udf
generate_content(
    contents: pxt.Json,
    *,
    model: pxt.String,
    config: pxt.Json | None = None,
    tools: pxt.Json | None = None
) -> pxt.Json
Generate content from the specified model. Request throttling: Applies the rate limit set in the config (section gemini.rate_limits; use the model id as the key). If no rate limit is configured, uses a default of 600 RPM. Requirements:
  • pip install google-genai
Parameters:
  • contents (pxt.Json): The input content to generate from. Can be a prompt, or a list containing images and text prompts, as described in: https://ai.google.dev/gemini-api/docs/text-generation
  • model (pxt.String): The name of the model to use.
  • config (pxt.Json | None): Configuration for generation, corresponding to keyword arguments of genai.types.GenerateContentConfig. For details on the parameters, see: https://googleapis.github.io/python-genai/genai.html#genai.types.GenerateContentConfig
  • tools (pxt.Json | None): An optional list of Pixeltable tools to use. It is also possible to specify tools manually via the config['tools'] parameter, but at most one of config['tools'] or tools may be used.
Returns:
  • pxt.Json: A dictionary containing the response and other metadata.
Examples: Add a computed column that applies the model gemini-2.5-flash to an existing Pixeltable column tbl.prompt of the table tbl: 
tbl.add_computed_column(
    response=generate_content(tbl.prompt, model='gemini-2.5-flash')
)

​
udf  generate_images()

Signature
@pxt.udf
generate_images(
    prompt: pxt.String,
    *,
    model: pxt.String,
    config: pxt.Json | None = None
) -> pxt.Image
Generates images based on a text description and configuration. For additional details, see: https://ai.google.dev/gemini-api/docs/image-generation Request throttling: Applies the rate limit set in the config (section imagen.rate_limits; use the model id as the key). If no rate limit is configured, uses a default of 600 RPM. Requirements:
  • pip install google-genai
Parameters: Returns:
  • pxt.Image: The generated image.
Examples: Add a computed column that applies the model imagen-4.0-generate-001 to an existing Pixeltable column tbl.prompt of the table tbl: 
tbl.add_computed_column(
    response=generate_images(tbl.prompt, model='imagen-4.0-generate-001')
)

​
udf  generate_videos()

Signature
@pxt.udf
generate_videos(
    prompt: pxt.String | None = None,
    image: pxt.Image | None = None,
    *,
    model: pxt.String,
    config: pxt.Json | None = None
) -> pxt.Video
Generates videos based on a text description and configuration. For additional details, see: https://ai.google.dev/gemini-api/docs/video At least one of prompt or image must be provided. Request throttling: Applies the rate limit set in the config (section veo.rate_limits; use the model id as the key). If no rate limit is configured, uses a default of 600 RPM. Requirements:
  • pip install google-genai
Parameters:
  • prompt (pxt.String | None): A text description of the videos to generate.
  • image (pxt.Image | None): An image to use as the first frame of the video.
  • model (pxt.String): The model to use.
  • config (pxt.Json | None): Configuration for generation, corresponding to keyword arguments of genai.types.GenerateVideosConfig. For details on the parameters, see: https://googleapis.github.io/python-genai/genai.html#genai.types.GenerateVideosConfig
Returns:
  • pxt.Video: The generated video.
Examples: Add a computed column that applies the model veo-3.0-generate-001 to an existing Pixeltable column tbl.prompt of the table tbl: 
tbl.add_computed_column(
    response=generate_videos(tbl.prompt, model='veo-3.0-generate-001')
)
Last modified on March 15, 2026