Skip to main content

​
module  pixeltable.functions.twelvelabs

Pixeltable UDFs that wrap various endpoints from the TwelveLabs API. In order to use them, you must first pip install twelvelabs and configure your TwelveLabs credentials, as described in the Working with TwelveLabs tutorial.

​
udf  embed()

Signatures
# Signature 1:
embed(
    model_name: pxt.String,
    text: pxt.String,
    image: pxt.Image | None
) -> pxt.Array[float32] | None

# Signature 2:
embed(
    model_name: pxt.String,
    image: pxt.Image
) -> pxt.Array[float32] | None

# Signature 3:
embed(
    model_name: pxt.String,
    audio: pxt.Audio,
    start_sec: pxt.Float | None,
    end_sec: pxt.Float | None,
    embedding_option: pxt.Json | None
) -> pxt.Array[float32] | None

# Signature 4:
embed(
    model_name: pxt.String,
    video: pxt.Video,
    start_sec: pxt.Float | None,
    end_sec: pxt.Float | None,
    embedding_option: pxt.Json | None
) -> pxt.Array[float32] | None
Creates an embedding vector for the given text, audio, image, or video input. Each UDF signature corresponds to one of the four supported input types. If text is specified, it is possible to specify an image as well, corresponding to the text_image embedding type in the TwelveLabs API. This is (currently) the only way to include more than one input type at a time. Equivalent to the TwelveLabs Embed API: https://docs.twelvelabs.io/v1.3/docs/guides/create-embeddings Request throttling: Applies the rate limit set in the config (section twelvelabs, key rate_limit). If no rate limit is configured, uses a default of 600 RPM. Requirements:
  • pip install twelvelabs
Parameters:
  • model_name (String): The name of the model to use. Check the TwelveLabs documentation for available models.
  • text (String): The text to embed.
  • image (Image | None, default: Literal(None)): If specified, the embedding will be created from both the text and the image.
Returns:
  • pxt.Array[float32] | None: The embedding.
Examples: Add a computed column embed for an embedding of a string column input: 
tbl.add_computed_column(
    embed=embed(model_name='marengo3.0', text=tbl.input)
)