Use case	Tools needed
Data assistant	`get_data`, `run_query`
Customer support	`lookup_order`, `check_status`
Research agent	`search_web`, `fetch_article`

query	tool_results
What's the weather in Tokyo?	{"get_weather": ["Rainy, 65\u00b0F"], "get_stock_price": null}
What's the stock price of Apple?	{"get_weather": null, "get_stock_price": ["\$178.50"]}
What's the weather in Paris and the price of Microsoft stock?	{"get_weather": ["Partly cloudy, 68\u00b0F"], "get_stock_price": ["\$378.90"]}

Benefit	Description
Standardized interface	Connect to any MCP-compliant server
External services	Integrate APIs, databases, and custom logic
Reusable tools	Share tools across different applications
Dynamic discovery	Automatically discover available tools

Column Name	Type	Computed With
query	String
response	Required[Json]	chat_completions(messages=[{'role': 'user', 'content': query}], model='gpt-4o-mini', tools=[{'name': 'SearchPixeltableDocumentation', 'description': 'Search across the Pixeltable Documentation knowledge base to find relevant information, code examples, API references, and guides. Use this tool when you need to answer questions about Pixeltable Documentation, find specific documentation, understand how features work, or locate implementation details. The search returns contextual content with titles and direct links to the documentation pages.', 'parameters': {'type': 'object', 'properties': {'query': {'type': 'string'}}}, 'required': ['query'], 'additionalProperties': False}])
tool_results	Required[Json]	{'SearchPixeltableDocumentation': map(_openai_response_to_pxt_tool_calls(response).SearchPixeltableDocumentation[*], lambda R: SearchPixeltableDocumentation(query=_extract_str_tool_arg(ObjectRef(JsonMapperDispatch(), 5263294452192566501, 88144469474745989).args, param_name='query')))}

query	tool_results
What is Pixeltable?	{"SearchPixeltableDocumentation": ["Title: pixeltable.DataFrame\nLink: https://docs.pixeltable.com/sdk/v0.4.22/dataframe\nContent: pixeltable.DataFrame\nRepresents a query for retrievin ...... fault: 10 ): Number of rows to select. Default is 10. Returns: DataFrameResultSet : A DataFrameResultSet with the first n rows of the DataFrame.\n\n"]}
How to use OpenAI in Pixeltable?	{"SearchPixeltableDocumentation": ["Title: Working with OpenAI in Pixeltable\nLink: https://docs.pixeltable.com/howto/providers/working-with-openai\nContent: Working with OpenAI in Pix ...... s pxt\n\n# Remove the 'openai_demo' directory and its contents, if it exists\npxt.drop_dir('openai_demo', force=True)\npxt.create_dir('openai_demo')\n\n"]}

search_result
Title: pixeltable.DataFrame Link: https://docs.pixeltable.com/sdk/v0.4.22/dataframe Content: pixeltable.DataFrame Represents a query for retrieving and transforming data from Pixeltable tables. View source on GitHub Represents a query for retrieving and transforming data from Pixeltable tables. View source on GitHub method head() Return the first n rows of the DataFrame, in insertion order of the underlying Table. head() is not supported for joins. Parameters: n ( int , default: 10 ): Number of rows to select. Default is 10. Returns: DataFrameResultSet : A DataFrameResultSet with the first n rows of the DataFrame.
Title: Working with OpenAI in Pixeltable Link: https://docs.pixeltable.com/howto/providers/working-with-openai Content: Working with OpenAI in Pixeltable This documentation page is also available as an interactive notebook. You can launch the notebook in Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the above links. Pixeltable’s OpenAI integration enables you to access OpenAI models via the OpenAI API. This documentation page is also a ...... API. Important notes OpenAI usage may incur costs based on your OpenAI plan. Be mindful of sensitive data and consider security measures when integrating with external services. First you’ll need to install required libraries and enter your OpenAI API key. Now let’s create a Pixeltable directory to hold the tables for our demo. import pixeltable as pxt # Remove the 'openai_demo' directory and its contents, if it exists pxt.drop_dir('openai_demo', force=True) pxt.create_dir('openai_demo')

Component	Purpose
`@pxt.udf`	Define tool functions
`pxt.tools()`	Bundle functions into Tools object
`tools=` parameter	Pass tools to LLM
`invoke_tools()`	Execute tool calls from LLM response
`pxt.mcp_udfs()`	Load tools from an MCP server

Provider	Function
OpenAI	`openai.invoke_tools()`
Anthropic	`anthropic.invoke_tools()`
Groq	`groq.invoke_tools()`
Gemini	`gemini.invoke_tools()`
Bedrock	`bedrock.invoke_tools()`

## Solution **What’s in this recipe:** * Define tools as Python functions * Let LLMs decide which tool to call * Automatically execute tool calls with `invoke_tools` * Use MCP servers to load external tools You define tools with JSON schemas, pass them to the LLM, and use `invoke_tools` to execute the function calls. ### Setup ```python theme={null} %pip install -qU pixeltable openai mcp ``` ```python theme={null} import getpass import os if 'OPENAI_API_KEY' not in os.environ: os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key: ') ``` ```python theme={null} import pixeltable as pxt from pixeltable.functions import openai ``` ```python theme={null} # Create a fresh directory pxt.drop_dir('tools_demo', force=True) pxt.create_dir('tools_demo') ```

  Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
  Created directory 'tools\_demo'.
  \

### Define tools as UDFs ```python theme={null} # Define tool functions as Pixeltable UDFs @pxt.udf def get_weather(city: str) -> str: """Get the current weather for a city.""" # In production, call a real weather API weather_data = { 'new york': 'Sunny, 72°F', 'london': 'Cloudy, 58°F', 'tokyo': 'Rainy, 65°F', 'paris': 'Partly cloudy, 68°F', } return weather_data.get( city.lower(), f'Weather data not available for {city}' ) @pxt.udf def get_stock_price(symbol: str) -> str: """Get the current stock price for a symbol.""" # In production, call a real stock API prices = { 'AAPL': '$178.50', 'GOOGL': '$141.25', 'MSFT': '$378.90', 'AMZN': '$185.30', } return prices.get(symbol.upper(), f'Price not available for {symbol}') ``` ```python theme={null} # Create a Tools object with our functions tools = pxt.tools(get_weather, get_stock_price) ``` ### Create tool-calling pipeline ```python theme={null} # Create table for queries queries = pxt.create_table('tools_demo/queries', {'query': pxt.String}) ```

  Created table 'queries'.

```python theme={null} # Add LLM call with tools queries.add_computed_column( response=openai.chat_completions( messages=[{'role': 'user', 'content': queries.query}], model='gpt-4o-mini', tools=tools, # Pass tools to the LLM ) ) ```

  Added 0 column values with 0 errors in 0.00 s
  No rows affected.

```python theme={null} # Automatically execute tool calls and get results queries.add_computed_column( tool_results=openai.invoke_tools(tools, queries.response) ) ```

  Added 0 column values with 0 errors in 0.01 s
  No rows affected.

### Run tool-enabled queries ```python theme={null} # Insert queries that require tool calls sample_queries = [ {'query': "What's the weather in Tokyo?"}, {'query': "What's the stock price of Apple?"}, { 'query': "What's the weather in Paris and the price of Microsoft stock?" }, ] queries.insert(sample_queries) ```

  Inserted 3 rows with 0 errors in 4.16 s (0.72 rows/s)
  3 rows inserted.

```python theme={null} # View results queries.select(queries.query, queries.tool_results).collect() ```

## Using MCP Servers as Tools The [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) is an open protocol that standardizes how applications provide context to LLMs. Pixeltable can connect to MCP servers and use their exposed tools as UDFs. ### Why MCP?

### Create an MCP Server First, create an MCP server with tools you want to expose. Save this as `mcp_server.py`: ```python theme={null} from mcp.server.fastmcp import FastMCP mcp = FastMCP('PixeltableDemo', stateless_http=True) @mcp.tool() def calculate_discount(price: float, discount_percent: float) -> float: """Calculate the discounted price.""" return price * (1 - discount_percent / 100) @mcp.tool() def check_inventory(product_id: str) -> str: """Check inventory status for a product.""" # In production, query your inventory database inventory = { 'SKU001': 'In stock (42 units)', 'SKU002': 'Low stock (3 units)', 'SKU003': 'Out of stock', } return inventory.get(product_id, f'Unknown product: {product_id}') if __name__ == '__main__': mcp.run(transport='streamable-http') ``` Run the server: `python mcp_server.py` (it will listen on `http://localhost:8000/mcp`) ### Connect to MCP Server and Use Tools ```python theme={null} # Connect to the MCP server using pxt.mcp_udfs() # This creates a Pixeltable UDF for each tool exposed by the server # See: https://docs.pixeltable.com/platform/custom-functions#5-mcp-udfs mcp_tools = pxt.mcp_udfs('https://docs.pixeltable.com/mcp') # View available tools - each is now a callable Pixeltable function for tool in mcp_tools: print(f'- {tool.name}: {tool.comment()}') ```

  - SearchPixeltableDocumentation: Search across the Pixeltable Documentation knowledge base to find relevant information, code examples, API references, and guides. Use this tool when you need to answer questions about Pixeltable Documentation, find specific documentation, understand how features work, or locate implementation details. The search returns contextual content with titles and direct links to the documentation pages.

```python theme={null} # Bundle MCP tools for LLM use mcp_toolset = pxt.tools(*mcp_tools) # Create a table with MCP tool-calling pipeline mcp_queries = pxt.create_table( 'tools_demo/mcp_queries', {'query': pxt.String} ) # Add LLM call with MCP tools mcp_queries.add_computed_column( response=openai.chat_completions( messages=[{'role': 'user', 'content': mcp_queries.query}], model='gpt-4o-mini', tools=mcp_toolset, ) ) # Execute MCP tool calls mcp_queries.add_computed_column( tool_results=openai.invoke_tools(mcp_toolset, mcp_queries.response) ) # View the schema - note that mcp_toolset is stored as persistent metadata # Every subsequent insert will use these same tools automatically mcp_queries.describe() ```

  Created table 'mcp\_queries'.
  Added 0 column values with 0 errors in 0.00 s
  Added 0 column values with 0 errors in 0.01 s

```python theme={null} # Test with e-commerce queries mcp_queries.insert( [ {'query': 'What is Pixeltable?'}, {'query': 'How to use OpenAI in Pixeltable?'}, ] ) mcp_queries.select(mcp_queries.query, mcp_queries.tool_results).collect() ```

```python theme={null} # Extract the search result with a named column mcp_queries.select( search_result=mcp_queries.tool_results[ 'SearchPixeltableDocumentation' ][0] ).collect() ```

## Explanation **Tool calling flow:**

  Query → LLM decides tool → invoke\_tools executes → Results

**Key components:**

**MCP integration:**

  MCP Server → pxt.mcp\_udfs() → pxt.tools() → LLM tool calling

MCP servers expose tools via a standardized protocol. Pixeltable’s `mcp_udfs()` connects to any MCP server and returns the tools as callable UDFs that can be bundled with `pxt.tools()` for LLM use. **Supported providers:**

## See also * [Build a RAG pipeline](/howto/cookbooks/agents/pattern-rag-pipeline) - Retrieval-augmented generation * [Run local LLMs](/howto/providers/working-with-ollama) - Local model inference * [Multimodal MCP Servers](/libraries/mcp) - Pixeltable’s MCP server collection * [Custom Functions](/platform/custom-functions) - More about UDFs and MCP integration Built with [Mintlify](https://mintlify.com).