id	img	similarity
6		1.
3		0.607

id	img	similarity
3		0.607
7		0.551

id	img	similarity
6		1.
19		0.617

id	img	similarity
13		0.274
9		0.24

text	similarity
One of the most influential artists of the 20th century, he is known for co-founding the Cubist movement, the invention of constructed sculpture,[8][9] the co-invention of collage, and for the wide variety of styles that he helped develop and explore.	0.443
While the names of many of his later periods are debated, the most commonly accepted periods in his work are the Blue Period (1901–1904), the Rose Period (1904–1906), the African-influenced Period (1907–1909), Analytic Cubism (1909–1912), and Synthetic Cubism (1912–1919), also referred to as the Crystal period.	0.426

text	similarity
Picasso's output, especially in his early career, is often periodized.	0.699
During the first decade of the 20th century, his style changed as he experimented with different theories, techniques, and ideas.	0.697

We can combine nearest-neighbor/similarity search with standard predicates. Here’s the same query, but filtering out the selected `sample_img` (which we already know has perfect similarity with itself): ```python theme={null} res = ( imgs.order_by(sim, asc=False) .where(imgs.id != 6) # Additional clause .limit(2) .select(imgs.id, imgs.img, sim) .collect() ) res ```

## Index updates In Pixeltable, each index is kept up-to-date automatically in response to changes to the indexed table. To illustrate this, let’s insert a few more rows: ```python theme={null} more_img_urls = [ 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000080.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000090.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000106.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000108.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000139.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000285.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000632.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000724.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000776.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/release/docs/resources/images/000000000785.jpg', ] imgs.insert( {'id': 10 + i, 'img': url} for i, url in enumerate(more_img_urls) ) ```

  Computing cells:  33%|██████████████                            | 10/30 \[00:01\<00:02,  8.90 cells/s]
  Inserting rows into \`img\_tbl\`: 10 rows \[00:00, 1337.60 rows/s]
  Computing cells: 100%|██████████████████████████████████████████| 30/30 \[00:01\<00:00, 24.55 cells/s]
  Inserted 10 rows with 0 errors.
  UpdateStatus(num\_rows=10, num\_computed\_values=30, num\_excs=0, updated\_cols=\[], cols\_with\_excs=\[])

When we now re-run the initial similarity query, we get a different result: ```python theme={null} sim = imgs.img.similarity(image=sample_img) res = ( imgs.order_by(sim, asc=False) .limit(2) .select(imgs.id, imgs.img, sim) .collect() ) res ```

## Similarity search on different types Because CLIP models are multimodal, we can also do lookups by text. ```python theme={null} sim = imgs.img.similarity(string='train') # String lookup res = ( imgs.order_by(sim, asc=False) .limit(2) .select(imgs.id, imgs.img, sim) .collect() ) res ```

## Creating multiple indexes on a single column We can create multiple embedding indexes on the same column, utilizing different embedding models. In order to use a specific index in a query, we need to assign it a name and then use that name in the query. To illustrate this, let’s create a table with text (taken from the Wikipedia article on [Pablo Picasso](https://en.wikipedia.org/wiki/Pablo_Picasso)): ```python theme={null} txts = pxt.create_table('indices_demo/text_tbl', {'text': pxt.String}) sentences = [ 'Pablo Ruiz Picasso (25 October 1881 – 8 April 1973) was a Spanish painter, sculptor, printmaker, ceramicist, and theatre designer who spent most of his adult life in France.', 'One of the most influential artists of the 20th century, he is known for co-founding the Cubist movement, the invention of constructed sculpture,[8][9] the co-invention of collage, and for the wide variety of styles that he helped develop and explore.', "Among his most famous works are the proto-Cubist Les Demoiselles d'Avignon (1907) and the anti-war painting Guernica (1937), a dramatic portrayal of the bombing of Guernica by German and Italian air forces during the Spanish Civil War.", 'Picasso demonstrated extraordinary artistic talent in his early years, painting in a naturalistic manner through his childhood and adolescence.', 'During the first decade of the 20th century, his style changed as he experimented with different theories, techniques, and ideas.', 'After 1906, the Fauvist work of the older artist Henri Matisse motivated Picasso to explore more radical styles, beginning a fruitful rivalry between the two artists, who subsequently were often paired by critics as the leaders of modern art.', "Picasso's output, especially in his early career, is often periodized.", 'While the names of many of his later periods are debated, the most commonly accepted periods in his work are the Blue Period (1901–1904), the Rose Period (1904–1906), the African-influenced Period (1907–1909), Analytic Cubism (1909–1912), and Synthetic Cubism (1912–1919), also referred to as the Crystal period.', "Much of Picasso's work of the late 1910s and early 1920s is in a neoclassical style, and his work in the mid-1920s often has characteristics of Surrealism.", 'His later work often combines elements of his earlier styles.', ] txts.insert({'text': s} for s in sentences) ```

  Created table \`text\_tbl\`.
  Inserting rows into \`text\_tbl\`: 10 rows \[00:00, 3599.64 rows/s]
  Inserted 10 rows with 0 errors.
  UpdateStatus(num\_rows=10, num\_computed\_values=10, num\_excs=0, updated\_cols=\[], cols\_with\_excs=\[])

When calling [`add_embedding_index()`](/sdk/latest/table#method-add_embedding_index), we now specify the index name (`idx_name`) directly. If it is not specified, Pixeltable will assign a name (such as `idx0`). ```python theme={null} from pixeltable.functions.huggingface import sentence_transformer txts.add_embedding_index( 'text', idx_name='minilm_idx', embedding=sentence_transformer.using( model_id='sentence-transformers/all-MiniLM-L12-v2' ), ) txts.add_embedding_index( 'text', idx_name='e5_idx', embedding=sentence_transformer.using(model_id='intfloat/e5-large-v2'), ) ```

  Computing cells: 100%|██████████████████████████████████████████| 10/10 \[00:01\<00:00,  6.86 cells/s]
  Computing cells: 100%|██████████████████████████████████████████| 10/10 \[00:01\<00:00,  6.35 cells/s]

To do a similarity query, we now call `similarity()` with the `idx` parameter: ```python theme={null} sim = txts.text.similarity('cubism', idx='minilm_idx') res = ( txts.order_by(sim, asc=False) .limit(2) .select(txts.text, sim) .collect() ) res ```

## Using a UDF for a custom embedding The above examples show how to use any model in the Hugging Face `CLIP` or `sentence_transformer` model families, and essentially the same pattern can be used for any other embedding with built-in Pixeltable support, such as OpenAI embeddings. But what if you want to adapt a new model family that doesn’t have built-in support in Pixeltable? This can be done by writing a custom Pixeltable UDF. In the following example, we’ll write a simple UDF to use the [BERT](https://www.kaggle.com/models/tensorflow/bert/tensorFlow2/en-uncased-preprocess/3) model built on TensorFlow. First we install the necessary dependencies. ```python theme={null} %pip install -qU tensorflow tensorflow-hub tensorflow-text ``` Text embedding UDFs must always take a string as input, and return a 1-dimensional numpy array of fixed dimension (512 in the case of `small_bert`, the variant we’ll be using). If we were writing an image embedding UDF, the `input` would have type `PIL.Image.Image` rather than `str`. The UDF is straightforward, loading the model and evaluating it against the input, with a minor data conversion on either side of the model invocation. ```python theme={null} import pixeltable as pxt import tensorflow as tf import tensorflow_hub as hub import tensorflow_text # Necessary to ensure BERT dependencies are loaded @pxt.udf def bert(input: str) -> pxt.Array[(512,), pxt.Float]: """Computes text embeddings using the small_bert model.""" preprocessor = hub.load( 'https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3' ) bert_model = hub.load( 'https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-4_H-512_A-8/2' ) tensor = tf.constant([input]) # Convert the string to a tensor result = bert_model(preprocessor(tensor))['pooled_output'] return result.numpy()[0, :] ``` ```python theme={null} txts.add_embedding_index('text', idx_name='bert_idx', embedding=bert) ```

  Computing cells: 100%|██████████████████████████████████████████| 10/10 \[00:17\<00:00,  1.72s/ cells]

Here’s the output of our sample query run against `bert_idx`. ```python theme={null} sim = txts.text.similarity('cubism', idx='bert_idx') res = ( txts.order_by(sim, asc=False) .limit(2) .select(txts.text, sim) .collect() ) res ```

Our example UDF is very simple, but it would perform poorly in a production setting. To make our UDF production-ready, we’d want to do two things: * Cache the model: the current version calls `hub.load()` on every UDF invocation. In a real application, we’d want to instantiate the model just once, then reuse it on subsequent UDF calls. * Batch our inputs: we’d use Pixeltable’s batching capability to ensure we’re making efficient use of the model. Batched UDFs are described in depth in the [User-Defined Functions](/platform/udfs-in-pixeltable) how-to guide. You might have noticed that the updates to `bert_idx` seem sluggish; that’s why! ## Deleting an index To delete an index, call [`Table.drop_embedding_index()`](/sdk/latest/table#method-drop_embedding_index): * specify the `idx_name` parameter if you have multiple indices * otherwise the `column_name` parameter is sufficient Given that we have several embedding indices, we’ll specify which index to drop: ```python theme={null} txts.drop_embedding_index(idx_name='e5_idx') ``` Built with [Mintlify](https://mintlify.com).