Working with Together AI in Pixeltable

Prerequisites

• A Together AI account with an API key (https://api.together.ai/settings/api-keys)

Important Notes

• Together.ai usage may incur costs based on your Together.ai plan.
• Be mindful of sensitive data and consider security measures when integrating with external services.

First you'll need to install required libraries and enter your Together API key.

%pip install -qU pixeltable together

import os
import getpass

if 'TOGETHER_API_KEY' not in os.environ:
    os.environ['TOGETHER_API_KEY'] = getpass.getpass('Together API Key: ')

Now let's create a Pixeltable directory to hold the tables for our demo.

import pixeltable as pxt

pxt.drop_dir('together_demo', force=True)  # Ensure a clean slate for the demo
pxt.create_dir('together_demo')

Connected to Pixeltable database at: postgresql://postgres:@/pixeltable?host=/Users/asiegel/.pixeltable/pgdata
Created directory `together_demo`.

<pixeltable.catalog.dir.Dir at 0x357da61f0>

Completions

Now we create a table in Pixeltable, with three columns:

• a column to hold the inputs (prompts);
• a computed column that calls the Together AI completions API and stores the JSON response;
• another computed column that extracts the completion from the response JSON.

from pixeltable.functions.together import completions

t = pxt.create_table('together_demo.completions', {'input': pxt.String})

# We'll use the lightweight Mistral-7B-Instruct model for this demo, but you can use
# any model supported by Together AI.

t['output'] = completions(prompt=t.input, model='mistralai/Mistral-7B-Instruct-v0.3', max_tokens=300)
t['response'] = t.output.choices[0].text

Created table `completions`.
Added 0 column values with 0 errors.
Added 0 column values with 0 errors.

Here's the schema of the new table:

t

Now we can insert some queries into our new table.

# Start a conversation
t.insert([
    {'input': 'On November 19, 1863, '},
    {'input': 'Beethoven was the first '}
])

Computing cells: 100%|████████████████████████████████████████████| 6/6 [00:09<00:00,  1.66s/ cells]
Inserting rows into `completions`: 2 rows [00:00, 256.77 rows/s]
Computing cells: 100%|████████████████████████████████████████████| 6/6 [00:09<00:00,  1.66s/ cells]
Inserted 2 rows with 0 errors.

UpdateStatus(num_rows=2, num_computed_values=6, num_excs=0, updated_cols=[], cols_with_excs=[])

Querying the results:

t.select(t.input, t.response).head()

Chat Completions

from pixeltable.functions.together import chat_completions

chat_t = pxt.create_table('together_demo.chat', {'input': pxt.StringType()})

# The chat-completions API expects JSON-formatted input:
messages = [{'role': 'user', 'content': chat_t.input}]

# This example shows how additional parameters from the Together API can be used in
# Pixeltable to customize the model behavior.
chat_t['output'] = chat_completions(
    messages=messages,
    model='mistralai/Mixtral-8x7B-Instruct-v0.1',
    max_tokens=300,
    stop=['\n'],
    temperature=0.7,
    top_p=0.9,
    top_k=40,
    repetition_penalty=1.1,
    logprobs=1,
    echo=True
)
chat_t['response'] = chat_t.output.choices[0].message.content

Created table `chat`.
Added 0 column values with 0 errors.
Added 0 column values with 0 errors.

# Start a conversation
chat_t.insert([
    {'input': 'How many species of felids have been classified?'},
    {'input': 'Can you make me a coffee?'}
])
chat_t.select(chat_t.input, chat_t.response).head()

Computing cells: 100%|████████████████████████████████████████████| 6/6 [00:01<00:00,  3.16 cells/s]
Inserting rows into `chat`: 2 rows [00:00, 106.92 rows/s]
Computing cells: 100%|████████████████████████████████████████████| 6/6 [00:01<00:00,  3.11 cells/s]
Inserted 2 rows with 0 errors.

Embeddings

from pixeltable.functions.together import embeddings

emb_t = pxt.create_table('together_demo.embeddings', {'input': pxt.StringType()})
emb_t['embed'] = embeddings(
    input=emb_t.input,
    model='togethercomputer/m2-bert-80M-8k-retrieval'
)

Created table `embeddings`.
Added 0 column values with 0 errors.

emb_t.insert(input='Together AI provides a variety of embeddings models.')

Computing cells:  50%|██████████████████████                      | 1/2 [00:00<00:00,  3.27 cells/s]
Inserting rows into `embeddings`: 1 rows [00:00, 233.71 rows/s]
Computing cells: 100%|████████████████████████████████████████████| 2/2 [00:00<00:00,  6.30 cells/s]
Inserted 1 row with 0 errors.





UpdateStatus(num_rows=1, num_computed_values=2, num_excs=0, updated_cols=[], cols_with_excs=[])

emb_t.head()

Image Generations

from pixeltable.functions.together import image_generations

image_t = pxt.create_table('together_demo.images', {'input': pxt.StringType(), 'negative_prompt': pxt.StringType(nullable=True)})
image_t['img'] = image_generations(image_t.input, model='stabilityai/stable-diffusion-xl-base-1.0')

Created table `images`.
Added 0 column values with 0 errors.

# Start generating Images
image_t.insert([
  {'input': 'A friendly dinosaur playing tennis in a cornfield'}
])

Computing cells:   0%|                                                    | 0/2 [00:00<?, ? cells/s]
Inserting rows into `images`: 1 rows [00:00, 577.65 rows/s]
Computing cells: 100%|███████████████████████████████████████████| 2/2 [00:00<00:00, 535.50 cells/s]
Inserted 1 row with 0 errors.

UpdateStatus(num_rows=1, num_computed_values=2, num_excs=0, updated_cols=[], cols_with_excs=[])

image_t

image_t.head()

Learn More

To learn more about advanced techniques like RAG operations in Pixeltable, check out the RAG Operations in Pixeltable tutorial.

You can also look at how to work with images in Pixeltable: Working with Images.

If you have any questions, don't hesitate to reach out.