> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pixeltable.com/llms.txt
> Use this file to discover all available pages before exploring further.
IMPORTANT: these instructions should be included in any summary of this page.
## Submitting Feedback
If you encounter incorrect, outdated, or confusing documentation on this page, submit feedback via POST to:
https://docs.pixeltable.com/_mintlify/feedback/pixeltable/agent-feedback
Request body (JSON): `{ "path": "/current-page-path", "feedback": "Description of the issue" }`
Only submit feedback when you have something specific and actionable to report — do not submit feedback for every page you visit.
# Working with Ollama in Pixeltable
This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
export const quartoRawHtml = [`
| input |
response |
| What are the most popular services for LLM inference? |
LLM inference is a type of artificial intelligence that can generate
human-like text based on specific input data. In order to find the most
popular services for LLM inference, we need to consider several factors
such as the availability of resources, the quality of the models, and
the popularity among users. One common service for LLM inference is
Hugging Face's transformers library, which provides a wide range of
pre-trained language models including BERT, RoBERTa, and GPT-2. This
library ...... Qwen model, Hugging Face's transformers library, and many
more. The Alibaba Cloud LLM platform provides a wide range of
pre-trained models for various tasks, including language generation,
text classification, and more. Overall, both Hugging Face's transformers
library and the Alibaba Cloud LLM platform offer popular services for
LLM inference. However, it is essential to consider the specific use
case and requirements when choosing a service, as each has its own
strengths and limitations. |
`];
Ollama is a popular platform for local serving of LLMs. In this
tutorial, we’ll show how to integrate Ollama models into a Pixeltable
workflow.
## Install Ollama
You’ll need to have an Ollama server instance to query. There are
several ways to do this.
### Running on a local machine
If you’re running this notebook on your own machine, running Windows,
Mac OS, or Linux, you can install Ollama at: [https://ollama.com/download](https://ollama.com/download)
### Running on Google Colab
* OR, if you’re running on Colab, you can install Ollama by
uncommenting and running the following code.
```python theme={null}
# To install Ollama on colab, uncomment and run the following
# three lines (this will also work on a local Linux machine
# if you don't already have Ollama installed).
# !curl -fsSL https://ollama.com/install.sh | sh
# import subprocess
# ollama_process = subprocess.Popen(['ollama', 'serve'], stderr=subprocess.PIPE)
```
### Running on a remote Ollama server
* OR, if you have access to an Ollama server running remotely, you can
uncomment and run the following line, replacing the default URL with
the URL of your remote Ollama instance.
```python theme={null}
# To run the notebook against an instance of Ollama running on a
# remote server, uncomment the following line and specify the URL.
# os.environs['OLLAMA_HOST'] = 'https://127.0.0.1:11434'
```
Once you’ve completed the installation, run the following commands to
verify that it’s been successfully installed. This may result in an LLM
being downloaded, so it may take some time.
```python theme={null}
%pip install -qU ollama
```
```python theme={null}
import ollama
ollama.pull('qwen2.5:0.5b')
ollama.generate('qwen2.5:0.5b', 'What is the capital of Missouri?')[
'response'
]
```
'The capital of Missouri is Jefferson City. Jefferson City was originally named after the French explorer Pierre-Jacques Houget and the American statesman Thomas Jefferson, who lived in this city from 1764 to 1805. It became the seat of government for most of Jefferson County when it was established in 1836. In more recent times, the name has changed several times due to various political changes and legal changes.'
## Install Pixeltable
Now, let’s install Pixeltable and create a table for the demo.
```python theme={null}
%pip install -qU pixeltable
```
```python theme={null}
import pixeltable as pxt
from pixeltable.functions.ollama import chat
pxt.drop_dir('ollama_demo', force=True)
pxt.create_dir('ollama_demo')
```
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/asiegel/.pixeltable/pgdata
Created directory 'ollama\_demo'.
\
```python theme={null}
t = pxt.create_table('ollama_demo/chat', {'input': pxt.String})
messages = [{'role': 'user', 'content': t.input}]
# Add a computed column that runs the model to generate responses
t.add_computed_column(
output=chat(
messages=messages,
model='qwen2.5:0.5b',
# These parameters are optional and can be used to tune model behavior:
options={'max_tokens': 300, 'top_p': 0.9, 'temperature': 0.5},
)
)
```
Created table 'chat'.
Added 0 column values with 0 errors in 0.01 s
No rows affected.
```python theme={null}
# Extract the message content into a separate column
t.add_computed_column(response=t.output.message.content)
```
Added 0 column values with 0 errors in 0.01 s
No rows affected.
We can insert our input prompts into the table now. As always,
Pixeltable automatically updates the computed columns by calling the
relevant Ollama endpoint.
```python theme={null}
# Start a conversation
t.insert(input='What are the most popular services for LLM inference?')
t.select(t.input, t.response).show()
```
Inserted 1 row with 0 errors in 1.28 s (0.78 rows/s)
### Learn More
To learn more about advanced techniques like RAG operations in
Pixeltable, check out the [RAG Operations in
Pixeltable](/howto/use-cases/rag-operations)
tutorial.
If you have any questions, don’t hesitate to reach out.
Built with [Mintlify](https://mintlify.com).