Enhancements

• Type Checking: Significant progress made on the type-checking system, including groundwork and performance improvements.
• Documentation: Added cross-links to docstrings for easier navigation and improved readability.
• Table Creation: create_table can now accept a DataFrame directly, making table creation more flexible.
• Postgres and pgvector Update: Postgres has been updated to version 16.4, and pgvector to 0.7.4 for better compatibility and performance.
• CI/CD: Implemented Notebook CI and set up Nightly CI to ensure code quality and catch regressions early.

Fixes

• Unit Tests: Fixed a unit test related to Together AI integration.
• Notebook Regressions: Fixed several regressions in the notebooks that were causing issues.
• Psycopg3: Updated to use psycopg3 as the Postgres driver for improved performance and stability.
• Table Namespace: Cleaned up the Table class namespace and prevented conflicting column names to avoid errors and improve code clarity.
• JSON Serialization: Fixed a bug related to the serialization of JSON slices.
• JSON Literals: Enforced the requirement for keys in JSON literals to be strings to maintain data consistency.

Highlights

• Enhanced Performance: Optimized data loading with StoreBase.load_column() improvements.
• Flexible Data Handling: Support for lists, dictionaries, and other non-numpy datatypes in import_pandas.
• Precise Video Frame Extraction: Control frame extraction with FrameIterator by specifying the exact number of frames.
• Improved Visualizations: New UDF draw_bounding_boxes() for easier visualization of object detection results.
• Streamlined Codebase: Migrated to a Pixeltable-compatible pgserver fork, improved SQL handling of arithmetic expressions, and refactored for better code organization and type checking.
• Enhanced Usability: All column types are now stored by default, making it easier to persist your data.

New Features

• import_json() and import_rows(): Convenient functions for importing JSON data and inserting rows directly into tables.
• Timestamp Functions Library: Expanded library with more functions for working with timestamps.
• Aggregate make_list(): Create lists from grouped data.

Improvements

• Method Call Syntax: Simplified syntax for calling library functions.
• Notebook Experience: Improved user-friendliness with better error messages and tab completion.
• Testing: Enhanced test coverage and automation for more reliable code.

Bug Fixes and Other Changes

• Database Update: New DB version to accommodate internal changes for better compatibility.
• Type Handling: Removed support for Python datetime.date until a proper DateType is added.
• CSV Import: Nullable types are now used as defaults for better data handling during CSV import.

Let us know what you think! Your feedback is invaluable to us as we continue to build and improve Pixeltable.

• Label Studio integration:
  • Now support pre-signed URLs for S3 buckets when working with Label Studio
  • Fixes for compatibility with newer versions of label-studio-sdk
• Tutorial: We added a new tutorial about Tables and Data Operations to help get started
• Functions: We added new String functions to make it easy to work with string within Pixeltable

• New force=True Option: We added a force=True option for drop_table and drop_dir functions, providing more control over data deletion.
• API Improvements: Introduced several minor enhancements to the API for better usability and functionality.
• Tutorial Updates: Revised existing tutorial notebooks and added new ones, covering topics like OpenAI, Together, Fireworks, and indexing.