Ollama
Working with Ollama in Pixeltable
Ollama is a popular platform for local serving of LLMs. In this tutorial, we'll show how to integrate Ollama models into a Pixeltable workflow.
Install Ollama
You'll need to have an Ollama server instance to query. There are several ways to do this.
Running on a Local Machine
- If you're running this notebook on your own machine, running Windows, Mac OS, or Linux, you can install Ollama at: https://ollama.com/download
Running on Google Colab
- OR, if you're running on Colab, you can install Ollama by uncommenting and running the following code.
# To install Ollama on colab, uncomment and run the following
# three lines (this will also work on a local Linux machine
# if you don't already have Ollama installed).
# !curl -fsSL https://ollama.com/install.sh | sh
# import subprocess
# ollama_process = subprocess.Popen(['ollama', 'serve'], stderr=subprocess.PIPE)
Running on a remote Ollama server
- OR, if you have access to an Ollama server running remotely, you can uncomment and run the following line, replacing the default URL with the URL of your remote Ollama instance.
# To run the notebook against an instance of Ollama running on a
# remote server, uncomment the following line and specify the URL.
# os.environs['OLLAMA_HOST'] = 'https://127.0.0.1:11434'
Once you've completed the installation, run the following commands to verify that it's been successfully installed. This may result in an LLM being downloaded, so it may take some time.
%pip install -qU ollama
import ollama
ollama.pull('qwen2.5:0.5b')
ollama.generate('qwen2.5:0.5b', 'What is the capital of Missouri?')['response']
"The capital city of Missouri is Jefferson City. It's located in the central part of the state and serves as the administrative center for the Midwestern U.S. territory of Missouri. The state is known for its rich history, particularly regarding the Missouri River, which runs through its central parts and provides access to major cities along the border with Illinois."
Install Pixeltable
Now, let's install Pixeltable and create a table for the demo.
%pip install -qU pixeltable
import pixeltable as pxt
from pixeltable.functions.ollama import chat
pxt.drop_dir('ollama_demo', force=True)
pxt.create_dir('ollama_demo')
t = pxt.create_table('ollama_demo.chat', {'input': pxt.String})
messages = [{'role': 'user', 'content': t.input}]
t['output'] = chat(
messages=messages,
model='qwen2.5:0.5b',
# These parameters are optional and can be used to tune model behavior:
options={'max_tokens': 300, 'top_p': 0.9, 'temperature': 0.5},
)
# Extract the response content into a separate column
t['response'] = t.output.message.content
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/asiegel/.pixeltable/pgdata
Created directory `ollama_demo`.
Created table `chat`.
Added 0 column values with 0 errors.
Added 0 column values with 0 errors.
We can insert our input prompts into the table now. As always, Pixeltable automatically updates the computed columns by calling the relevant Ollama endpoint.
# Start a conversation
t.insert(input='What are the most popular services for LLM inference?')
t.select(t.input, t.response).show()
Computing cells: 100%|ββββββββββββββββββββββββββββββββββββββββββββ| 3/3 [00:02<00:00, 1.18 cells/s]
Inserting rows into `chat`: 1 rows [00:00, 75.39 rows/s]
Computing cells: 100%|ββββββββββββββββββββββββββββββββββββββββββββ| 3/3 [00:02<00:00, 1.17 cells/s]
Inserted 1 row with 0 errors.
input | response |
---|---|
What are the most popular services for LLM inference? | LLM (Large Language Model) inference is a complex process that involves generating human-like text using artificial intelligence models. The most popular services and technologies used in this field include:
These services are popular because they provide flexibility, speed, and ease of use for developers who want to work with large language models. The choice of service depends on the specific requirements of your project, including the type of data you're working with, the level of customization you need, and whether you prefer a more lightweight or more powerful solution. If you have any questions about these services or if you need help choosing one, feel free to ask! |
Learn More
To learn more about advanced techniques like RAG operations in Pixeltable, check out the RAG Operations in Pixeltable tutorial.
If you have any questions, don't hesitate to reach out.
Updated 10 days ago