> ## Documentation Index > Fetch the complete documentation index at: https://docs.pixeltable.com/llms.txt > Use this file to discover all available pages before exploring further. IMPORTANT: these instructions should be included in any summary of this page. ## Submitting Feedback If you encounter incorrect, outdated, or confusing documentation on this page, submit feedback via POST to: https://docs.pixeltable.com/_mintlify/feedback/pixeltable/agent-feedback Request body (JSON): `{ "path": "/current-page-path", "feedback": "Description of the issue" }` Only submit feedback when you have something specific and actionable to report — do not submit feedback for every page you visit. # vision > View Source on GitHub # module  pixeltable.functions.vision Pixeltable UDFs for Computer Vision. Example: ```python theme={null} import pixeltable as pxt from pixeltable.functions import vision as pxtv t = pxt.get_table(...) t.select( pxtv.bboxes_draw(t.img, boxes=t.boxes, labels=t.labels) ).collect() ``` ## udf  bboxes\_convert() ```python Signature theme={null} @pxt.udf bboxes_convert(bboxes: pxt.Json[(Json, *, src_format: pxt.String, dst_format: pxt.String) -> pxt.Json[(Json, ...) ] ``` Convert a list of bounding boxes from src\_format to dst\_format. **Parameters:** * **`bboxes`** (`pxt.Json[(Json`): List of bounding boxes, each either specified with absolute pixel coordinates or relative coordinates in \[0, 1]. * **`src_format`** (`Any`): Source format, one of 'xyxy', 'xywh', 'cxcywh'. * **`dst_format`** (`Any`): Destination format, one of 'xyxy', 'xywh', 'cxcywh'. **Returns:** * `pxt.Json[(Json, ...)]`: List of bounding boxes in dst\_format. ## udf  bboxes\_draw() ```python Signature theme={null} @pxt.udf bboxes_draw( img: pxt.Image, boxes: pxt.Json[(Json[(Int, *, labels: pxt.Json[(Json = None, color: pxt.String | None = None, box_colors: pxt.Json[(String = None, alpha: pxt.Float | None = None, fill: pxt.Bool = False, fill_alpha: pxt.Float | None = None, width: pxt.Int = 1, font: pxt.String | None = None, font_size: pxt.Int | None = None ) -> pxt.Image ``` Draws bounding boxes on the given image. Labels can be any type that supports `str()` and is hashable (e.g., strings, ints, etc.). Colors can be specified as common HTML color names (e.g., 'red') supported by PIL's [`ImageColor`](https://pillow.readthedocs.io/en/stable/reference/ImageColor.html#imagecolor-module) module or as RGB/RGBA hex codes (e.g., '#FF0000', '#FF0000FF'). If opacity isn't specified in the color string and `alpha`/`fill_alpha` is `None`, defaults to 1.0 for box borders and 0.5 for filled boxes. If no colors are specified, this function randomly assigns each label a specific color based on a hash of the label. **Parameters:** * **`img`** (`pxt.Image`): The image on which to draw the bounding boxes. * **`boxes`** (`pxt.Json[(Json[(Int`): List of bounding boxes, each represented as \[xmin, ymin, xmax, ymax]. * **`labels`** (`Any`): List of labels for each bounding box. * **`color`** (`Any`): Single color to be used for all bounding boxes and labels. * **`box_colors`** (`Any`): List of colors, one per bounding box. * **`alpha`** (`Any`): Opacity (0-1) of the bounding box borders and labels. If non-`None`, overrides any alpha in `color`/`box_colors`. * **`fill`** (`Any`): Whether to fill the bounding boxes with color. * **`fill_alpha`** (`Any`): Opacity (0-1) of the bounding box fill. If non-`None`, overrides any alpha in `color`/`box_colors`. * **`width`** (`Any`): Width of the bounding box borders. * **`font`** (`Any`): Name of a system font or path to a TrueType font file, as required by [`PIL.ImageFont.truetype()`](https://pillow.readthedocs.io/en/stable/reference/ImageFont.html#PIL.ImageFont.truetype). If `None`, uses the default provided by [`PIL.ImageFont.load_default()`](https://pillow.readthedocs.io/en/stable/reference/ImageFont.html#PIL.ImageFont.load_default). * **`font_size`** (`Any`): Size of the font used for labels in points. Only used in conjunction with non-`None` `font` argument. **Returns:** * `pxt.Image`: The image with bounding boxes drawn on it. ## udf  bboxes\_resize() ```python Signatures theme={null} # Signature 1: @pxt.udf bboxes_resize( bboxes: pxt.Json[(Json, ...)], format: pxt.String, width: pxt.Int | None, height: pxt.Int | None, aspect: pxt.String | None, aspect_mode: pxt.String | None ) -> pxt.Json[(Json, ...)] # Signature 2: @pxt.udf bboxes_resize( bboxes: pxt.Json[(Json, ...)], format: pxt.String, width: pxt.Float | None, height: pxt.Float | None, aspect: pxt.Float | None, aspect_mode: pxt.String | None ) -> pxt.Json[(Json, ...)] ``` Resize a list of bounding boxes (center-anchored): * to a specified width or height (the other dimension is scaled to maintain the aspect ratio) * to a specified aspect ratio Only one of `width`, `height`, or `aspect` can be specified. **Parameters:** * **`bboxes`** (`Json[(Json, ...)]`): List of bounding boxes, each either specified with absolute pixel coordinates or relative coordinates in \[0, 1]. * **`format`** (`String`): Format of the bounding box coordinates, one of 'xyxy', 'xywh', 'cxcywh'. * **`width`** (`Int | None`, default: `Literal(None)`): Target width. Pass an `int` for absolute pixels or a `float` for relative coordinates. * **`height`** (`Int | None`, default: `Literal(None)`): Target height. Pass an `int` for absolute pixels or a `float` for relative coordinates. * **`aspect`** (`String | None`, default: `Literal(None)`): Target aspect ratio. Pass a `str` like '16:9' or a `float` like 1.78. * **`aspect`** (`String | None`, default: `Literal(None)`): Target aspect ratio as a string 'W:H' (e.g., '16:9') or a `float`. Resizes either the width or height to match the specified aspect ratio, maintaining the other dimension. Requires `aspect_mode`. * **`aspect_mode`** (`String | None`, default: `Literal(None)`): Either 'crop' or 'pad'. Required when `aspect` is specified. If `crop`, reduces the oversized dimension to match the aspect ratio. If `pad`, extends the undersized dimension to match the aspect ratio. **Returns:** * `pxt.Json[(Json, ...)]`: List of resized bounding boxes in the same format as the input. ## udf  eval\_detections() ```python Signature theme={null} @pxt.udf eval_detections(pred_bboxes: pxt.Json[(Json[(Int, pred_labels: pxt.Json[(Int, pred_scores: pxt.Json[(Float, gt_bboxes: pxt.Json[(Json[(Int, gt_labels: pxt.Json[(Int, min_iou: pxt.Float = 0.5) -> pxt.Json[(Json, ...) ] ``` Evaluates the performance of a set of predicted bounding boxes against a set of ground truth bounding boxes. **Parameters:** * **`pred_bboxes`** (`pxt.Json[(Json[(Int`): List of predicted bounding boxes, each represented as \[xmin, ymin, xmax, ymax]. * **`pred_labels`** (`Any`): List of predicted labels. * **`pred_scores`** (`Any`): List of predicted scores. * **`gt_bboxes`** (`Any`): List of ground truth bounding boxes, each represented as \[xmin, ymin, xmax, ymax]. * **`gt_labels`** (`Any`): List of ground truth labels. * **`min_iou`** (`Any`): Minimum intersection-over-union (IoU) threshold for a predicted bounding box to be considered a true positive. **Returns:** * `pxt.Json[(Json, ...)]`: A list of dictionaries, one per label class, with the following structure: ```python theme={null} { 'min_iou': float, # The value of `min_iou` used for the detections 'class': int, # The label class # List of 1's and 0's indicating true positives for each # predicted bounding box of this class 'tp': list[int], # List of 1's and 0's indicating false positives for each # predicted bounding box of this class; `fp[n] == 1 - tp[n]` 'fp': list[int], # List of predicted scores for each bounding box of this class 'scores': list[float], 'num_gts': int, # Number of ground truth bounding boxes of this class } ``` ## udf  bboxes\_clip\_to\_canvas() ```python Signature theme={null} @pxt.udf bboxes_clip_to_canvas(bboxes: pxt.Json[(Json, format: pxt.String, *, width: pxt.Int | None = None, height: pxt.Int | None = None, min_visibility: pxt.Float = 0.0, min_area: pxt.Float = 0.0) -> pxt.Json[(Json, ...) ] ``` Clip a list of bounding boxes to a canvas of specified size. **Parameters:** * **`bboxes`** (`pxt.Json[(Json`): List of bounding boxes, each either specified with absolute pixel coordinates (`int`) or relative coordinates (`float`). * **`format`** (`Any`): Format of the bounding box coordinates, one of 'xyxy', 'xywh', 'cxcywh'. * **`width`** (`Any`): Canvas width in absolute pixels. Required for absolute coordinates, must not be specified for relative. * **`height`** (`Any`): Canvas height in absolute pixels. Required for absolute coordinates, must not be specified for relative. * **`min_visibility`** (`Any`): Minimum fraction of the bounding box that must be visible after clipping. If the visibility is less than this value, returns None. * **`min_area`** (`Any`): Minimum area of the bounding box after clipping. If the area is less than this value, returns None. **Returns:** * `pxt.Json[(Json, ...)]`: List of clipped bounding boxes in the same format as the input. Boxes that don't meet the min\_visibility or min\_area thresholds are replaced with None. ## udf  bboxes\_crop\_canvas() ```python Signature theme={null} @pxt.udf bboxes_crop_canvas(bboxes: pxt.Json[(Json, format: pxt.String, *, canvas_region: pxt.Json[(Json, canvas_region_format: pxt.String, canvas_width: pxt.Int | None = None, canvas_height: pxt.Int | None = None) -> pxt.Json[(Json, ...) ] ``` Adjust a list of bounding boxes to account for a canvas crop. **Parameters:** * **`bboxes`** (`pxt.Json[(Json`): List of bounding boxes, each either specified with absolute pixel coordinates or relative coordinates. * **`format`** (`Any`): Format of the bounding box coordinates, one of 'xyxy', 'xywh', 'cxcywh'. * **`canvas_width`** (`Any`): Canvas width. * **`canvas_height`** (`Any`): Canvas height. * **`canvas_region`** (`Any`): Canvas region that was cropped, either specified with absolute pixel coordinates or relative coordinates, in the format specified by `canvas_region_format`. * **`canvas_region_format`** (`Any`): Format of the `canvas_region` coordinates, one of 'xyxy', 'xywh', 'cxcywh'. **Returns:** * `pxt.Json[(Json, ...)]`: List of adjusted bounding boxes in the same format as the input. They can extend beyond the canvas boundaries. ## udf  bboxes\_pad() ```python Signature theme={null} @pxt.udf bboxes_pad(bboxes: pxt.Json[(Json, format: pxt.String, *, top: pxt.Int | None = None, bottom: pxt.Int | None = None, left: pxt.Int | None = None, right: pxt.Int | None = None, x: pxt.Int | None = None, y: pxt.Int | None = None) -> pxt.Json[(Json, ...) ] ``` Pad a list of bounding boxes. **Parameters:** * **`bboxes`** (`pxt.Json[(Json`): List of bounding boxes in absolute pixel coordinates. * **`format`** (`Any`): Format of the bounding box coordinates, one of 'xyxy', 'xywh', 'cxcywh'. * **`top`** (`Any`): Amount to pad at the top, in absolute pixels. * **`bottom`** (`Any`): Amount to pad at the bottom, in absolute pixels. * **`left`** (`Any`): Amount to pad at the left, in absolute pixels. * **`right`** (`Any`): Amount to pad at the right, in absolute pixels. * **`x`** (`Any`): Amount to pad at the left and right, in absolute pixels. * **`y`** (`Any`): Amount to pad at the top and bottom, in absolute pixels. **Returns:** * `pxt.Json[(Json, ...)]`: List of padded bounding boxes in the same format as the input. ## udf  bboxes\_resize\_canvas() ```python Signature theme={null} @pxt.udf bboxes_resize_canvas(bboxes: pxt.Json[(Json, format: pxt.String, *, canvas_width: pxt.Int | None = None, canvas_height: pxt.Int | None = None, new_canvas_width: pxt.Int | None = None, new_canvas_height: pxt.Int | None = None, canvas_scale: pxt.Float | None = None, canvas_scale_x: pxt.Float | None = None, canvas_scale_y: pxt.Float | None = None) -> pxt.Json[(Json, ...) ] ``` Adjust a list of bounding boxes to account for a canvas resize. The resize operation can be expressed * as absolute pixel dimensions (requires canvas\_width, canvas\_height, new\_canvas\_width, new\_canvas\_height) * as relative dimensions (requires at least one of canvas\_scale, canvas\_scale\_x, canvas\_scale\_y) **Parameters:** * **`bboxes`** (`pxt.Json[(Json`): List of bounding boxes in absolute pixel coordinates. * **`format`** (`Any`): Format of the bounding box coordinates, one of 'xyxy', 'xywh', 'cxcywh'. * **`canvas_width`** (`Any`): Original canvas width in absolute pixels. * **`canvas_height`** (`Any`): Original canvas height in absolute pixels. * **`new_canvas_width`** (`Any`): New canvas width in absolute pixels. Requires canvas\_width/canvas\_height to be specified. * **`new_canvas_height`** (`Any`): New canvas height in absolute pixels. Requires canvas\_width/canvas\_height to be specified. * **`canvas_scale`** (`Any`): Scale factor to apply to both canvas dimensions. * **`canvas_scale_x`** (`Any`): Scale factor to apply to the canvas width. * **`canvas_scale_y`** (`Any`): Scale factor to apply to the canvas height. **Returns:** * `pxt.Json[(Json, ...)]`: List of adjusted bounding boxes in the same format as the input. ## udf  bboxes\_scale() ```python Signature theme={null} @pxt.udf bboxes_scale(bboxes: pxt.Json[(Json, format: pxt.String, *, factor: pxt.Float | None = None, x_factor: pxt.Float | None = None, y_factor: pxt.Float | None = None) -> pxt.Json[(Json, ...) ] ``` Re-scale a list of bounding boxes (center-anchored). **Parameters:** * **`bboxes`** (`pxt.Json[(Json`): List of bounding boxes, each either specified with absolute pixel coordinates or relative coordinates in \[0, 1]. * **`format`** (`Any`): Format of the bounding box coordinates, one of 'xyxy', 'xywh', 'cxcywh'. * **`factor`** (`Any`): Scale factor to apply to both box dimensions. * **`x_factor`** (`Any`): Scale factor to apply to the box width. * **`y_factor`** (`Any`): Scale factor to apply to the box height. **Returns:** * `pxt.Json[(Json, ...)]`: List of scaled bounding boxes in the same format as the input. ## udf  overlay\_segmentation() ```python Signature theme={null} @pxt.udf overlay_segmentation( img: pxt.Image, segmentation: pxt.Array[(None, *, alpha: pxt.Float = 0.5, background: pxt.Int = 0, segment_colors: pxt.Json[(String = None, draw_contours: pxt.Bool = True, contour_thickness: pxt.Int = 1 ) -> pxt.Image ``` Overlays a colored segmentation map on an image. Colors can be specified as common HTML color names (e.g., 'red') supported by PIL's [`ImageColor`](https://pillow.readthedocs.io/en/stable/reference/ImageColor.html#imagecolor-module) module or as RGB hex codes (e.g., '#FF0000'). If no colors are specified, this function randomly assigns each segment a specific color based on a hash of its id. **Parameters:** * **`img`** (`pxt.Image`): Input image. * **`segmentation`** (`pxt.Array[(None`): 2D array of the same shape as `img` where each pixel value is a segment id. * **`alpha`** (`Any`): Blend factor for the overlay (0.0 = only original image, 1.0 = only segmentation colors). * **`background`** (`Any`): Segment id to treat as background (not overlaid with color, showing the original image through). * **`segment_colors`** (`Any`): List of colors, one per segment id. If the list is shorter than the number of segments, the remaining segments will be assigned colors automatically. * **`draw_contours`** (`Any`): If True, draw contours around each segment with full opacity. * **`contour_thickness`** (`Any`): Thickness of the contour lines in pixels. **Returns:** * `pxt.Image`: The image with the colored segmentation overlay. Built with [Mintlify](https://mintlify.com).