Working with Voyage AI in Pixeltable

This documentation page is also available as an interactive notebook. You can launch the notebook in Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the above links.

Pixeltable’s Voyage AI integration enables you to access state-of-the-art embedding and reranker models via the Voyage AI API.

Prerequisites

A Voyage AI account with an API key (https://www.voyageai.com/)

Important notes

Voyage AI usage may incur costs based on your Voyage AI plan.
Be mindful of sensitive data and consider security measures when integrating with external services.

First you’ll need to install required libraries and enter your Voyage AI API key.

%pip install -qU voyageai

import getpass
import os

if 'VOYAGE_API_KEY' not in os.environ:
    os.environ['VOYAGE_API_KEY'] = getpass.getpass(
        'Enter your Voyage AI API key:'
    )

Now let’s create a Pixeltable directory to hold the tables for our demo.

import pixeltable as pxt

# Remove the 'voyageai_demo' directory and its contents, if it exists
pxt.drop_dir('voyageai_demo', force=True)
pxt.create_dir('voyageai_demo')

Created directory ‘voyageai_demo’.
<pixeltable.catalog.dir.Dir at 0x106daf2d0>

Text embeddings

Voyage AI provides state-of-the-art embedding models for semantic search and RAG applications.

from pixeltable.functions import voyageai

# Create a table for document embeddings
docs_t = pxt.create_table('voyageai_demo/documents', {'text': pxt.String})

# Add computed column with Voyage embeddings
docs_t.add_computed_column(
    embedding=voyageai.embeddings(
        docs_t.text, model='voyage-3.5', input_type='document'
    )
)

Created table ‘documents’.
Added 0 column values with 0 errors.
No rows affected.

# Insert some sample documents
documents = [
    'The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.',
    'Photosynthesis in plants converts light energy into glucose and produces essential oxygen.',
    '20th-century innovations, from radios to smartphones, centered on electronic advancements.',
    'Rivers provide water, irrigation, and habitat for aquatic species, vital for ecosystems.',
    "Apple's conference call to discuss fourth fiscal quarter results is scheduled for Thursday, November 2, 2023.",
    "Shakespeare's works, like 'Hamlet' and 'A Midsummer Night's Dream,' endure in literature.",
]

docs_t.insert({'text': doc} for doc in documents)

Inserting rows into `documents`: 6 rows [00:00, 2561.67 rows/s]
Inserted 6 rows with 0 errors.
6 rows inserted, 12 values computed.

# View the embeddings
docs_t.select(docs_t.text, docs_t.embedding).head(3)

Embedding index for similarity search

You can use Voyage AI embeddings with Pixeltable’s embedding index for efficient similarity search.

# Create a table with an embedding index
search_t = pxt.create_table('voyageai_demo/search', {'text': pxt.String})

# Add embedding index for similarity search
embed_fn = voyageai.embeddings.using(
    model='voyage-3.5', input_type='document'
)
search_t.add_embedding_index('text', string_embed=embed_fn)

Created table ‘search’.

# Insert documents
search_t.insert({'text': doc} for doc in documents)

Inserting rows into `search`: 6 rows [00:00, 973.68 rows/s]
Inserted 6 rows with 0 errors.
6 rows inserted, 12 values computed.

# Perform similarity search
sim = search_t.text.similarity(
    string='What are the health benefits of Mediterranean food?'
)
search_t.order_by(sim, asc=False).limit(3).select(
    search_t.text, score=sim
).collect()

Reranking

Voyage AI’s rerankers can refine search results by providing more accurate relevance scores.

# Create a table for reranking
rerank_t = pxt.create_table(
    'voyageai_demo/rerank', {'query': pxt.String, 'documents': pxt.Json}
)

# Add computed column with reranking results
rerank_t.add_computed_column(
    reranked=voyageai.rerank(
        rerank_t.query, rerank_t.documents, model='rerank-2.5', top_k=3
    )
)

Created table ‘rerank’.
Added 0 column values with 0 errors.
No rows affected.

# Insert query and documents to rerank
rerank_t.insert(
    [
        {
            'query': "When is Apple's conference call scheduled?",
            'documents': documents,
        }
    ]
)

Inserting rows into `rerank`: 1 rows [00:00, 343.65 rows/s]
Inserted 1 row with 0 errors.
1 row inserted, 2 values computed.

# Add computed column to extract top results using JSON path
rerank_t.add_computed_column(top_results=rerank_t.reranked['results'])

Added 1 column value with 0 errors.
1 row updated, 1 value computed.

# Extract the top result's document and score
rerank_t.select(
    rerank_t.query,
    top_document=rerank_t.top_results[0]['document'],
    top_score=rerank_t.top_results[0]['relevance_score'],
).collect()

# View reranking results
rerank_t.select(rerank_t.query, rerank_t.top_results).collect()

Multimodal Embeddings

Voyage AI’s multimodal model (voyage-multimodal-3) can embed both images and text into the same vector space, enabling cross-modal similarity search.

# Create a table for multimodal embeddings
mm_t = pxt.create_table(
    'voyageai_demo/multimodal',
    {'image': pxt.Image, 'caption': pxt.String},
    if_exists='replace',
)

# Add computed columns for image and text embeddings
# multimodal_embed can embed either images or text independently
mm_t.add_computed_column(
    image_embedding=voyageai.multimodal_embed(
        mm_t.image, model='voyage-multimodal-3.5', input_type='document'
    )
)
mm_t.add_computed_column(
    text_embedding=voyageai.multimodal_embed(
        mm_t.caption, model='voyage-multimodal-3.5', input_type='document'
    )
)

Created table ‘multimodal’.
Added 0 column values with 0 errors.
Added 0 column values with 0 errors.
No rows affected.

# Insert a sample image with caption
mm_t.insert(
    [
        {
            'image': 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/images/000000000139.jpg',
            'caption': 'A person standing next to an elephant',
        }
    ]
)

Inserting rows into `multimodal`: 1 rows [00:00, 520.00 rows/s]
Inserted 1 row with 0 errors.
1 row inserted, 5 values computed.

# View the multimodal embeddings
mm_t.select(
    mm_t.image, mm_t.caption, mm_t.image_embedding, mm_t.text_embedding
).head()

Learn more

To learn more about RAG operations in Pixeltable, check out the RAG Operations in Pixeltable tutorial. For more information about Voyage AI models and features, visit:

Built-in Integrations

Observability

Infrastructure

Bring Your Own

Providers

Prerequisites

Important notes

Text embeddings

Embedding index for similarity search

Reranking

Multimodal Embeddings

Learn more

​Prerequisites

​Important notes

​Text embeddings

​Embedding index for similarity search

​Reranking

​Multimodal Embeddings

​Learn more

Prerequisites

Important notes

Text embeddings

Embedding index for similarity search

Reranking

Multimodal Embeddings

Learn more