Summary

This PR adds support for Set as a feature type in Feast, allowing users to store collections of unique values.

Changes Made

1. Protobuf Definitions (Value.proto)

• Added 8 new SET type variants for primitive types:
  • INT32_SET, INT64_SET, FLOAT_SET, DOUBLE_SET
  • STRING_SET, BYTES_SET, BOOL_SET, UNIX_TIMESTAMP_SET
• Added corresponding protobuf message types (Int32Set, Int64Set, etc.)
• Updated the Value message to include set fields
• Compiled with protobuf version 4.25.1 and mypy-protobuf 3.3.0 (matching CI environment)

2. Python SDK Updates

• value_type.py: Added set types to the ValueType enum and defined SetType union
• types.py:
  • Created new Set class (ComplexFeastType) similar to Array
  • Added set type mappings in VALUE_TYPES_TO_FEAST_TYPES
  • Set types support all primitive types except Map
• type_map.py:
  • Added PYTHON_SET_VALUE_TYPE_TO_PROTO_VALUE mapping
  • Implemented Python → Proto conversions for sets with automatic duplicate removal
  • Implemented Proto → Python conversions returning native Python sets
  • Updated PROTO_VALUE_TO_VALUE_TYPE_MAP with set field mappings
  • Fixed: Corrected UNIX_TIMESTAMP_SET to use "unix_timestamp_set_val" field name
  • Fixed: Added SET types to pandas type mapping (returns "object" type)
  • Fixed: Added missing UNIX_TIMESTAMP mappings to PROTO_VALUE_TO_VALUE_TYPE_MAP
  • Fixed: Used distinct variable names (set_proto_type, set_field_name, set_valid_types) for Set type handling to avoid mypy type conflicts with List type variables
  • Fixed: Moved # type: ignore comment to the list comprehension line to suppress mypy type checking errors for bool conversion
  • Fixed: Added set type inference in python_type_to_feast_value_type to properly handle Python sets passed in entity_rows
  • Fixed: Renamed variables in set type inference (common_set_item_type, current_set_item_type) to avoid variable redefinition error with list type inference variables

3. Tests

• Added unit tests in test_types.py for Set type creation and round-trip conversion
• Added comprehensive tests in test_type_map.py for:
  • Set conversion (string, int, float, bool sets)
  • Duplicate removal from lists when converting to sets
  • Empty and null set handling
  • Multiple set value conversions
• Added test_python_transformation_with_set_types in test_on_demand_python_transformation.py for:
  • Set types in On Demand Feature Views
  • Automatic deduplication of set values
  • Conversion from lists to sets
  • Online feature retrieval with Set types

4. Documentation

• Updated docs/reference/type-system.md with:
  • Set type table showing all supported set variants
  • Usage examples demonstrating duplicate removal
  • Updated complete feature view example with set fields

5. Code Quality

• Ran make format-python and make lint-python to ensure code quality
• All ruff checks pass for modified files
• Fixed bugs found during testing
• Fixed all mypy type checking errors

Usage Example

from feast.types import Set, String, Int32
from feast import Field

# Define fields with set types
schema = [
    Field(name="visited_pages", dtype=Set(String)),
    Field(name="unique_categories", dtype=Set(Int32)),
]

# Data with duplicates
data = {
    "visited_pages": {"home", "products", "checkout", "products"},  # duplicate removed
    "unique_categories": {1, 2, 3, 2, 1}  # duplicates removed
}
# Feast stores: {"home", "products", "checkout"} and {1, 2, 3}

Benefits

• Automatic deduplication: Sets automatically remove duplicate values
• Type safety: Full type checking and validation
• Consistent API: Set types follow the same patterns as Array types
• Comprehensive support: All primitive types (except Map) have set variants

Bug Fixes

• Fixed UNIX_TIMESTAMP_SET to use correct protobuf field name ("unix_timestamp_set_val" instead of "int64_set_val")
• Added SET types to feast_value_type_to_pandas_type function to return "object" type for pandas compatibility
• Added missing UNIX_TIMESTAMP type mappings in PROTO_VALUE_TO_VALUE_TYPE_MAP (unix_timestamp_val, unix_timestamp_list_val, unix_timestamp_set_val)
• Fixed mypy type error by using distinct variable names for Set type handling (set_proto_type, set_field_name, set_valid_types) instead of reusing List type variable names
• Fixed mypy list comprehension type error by moving # type: ignore comment to the correct line
• Fixed set type inference by adding support for Python set type in python_type_to_feast_value_type function
• Fixed mypy variable redefinition error by renaming set type inference variables to avoid conflicts with list type inference variables

Checklist

• Update protobuf definitions
• Compile protobuf files with correct version (4.25.1) and mypy-protobuf (3.3.0)
• Remove @typing_extensions.final decorators to match CI environment
• Update Python SDK (value_type.py, types.py, type_map.py)
• Add unit tests
• Add On Demand Feature View tests
• Update documentation
• Run make format-python and make lint-python
• Fix UNIX_TIMESTAMP_SET field name bug
• Add SET types to pandas type mapping
• Add missing UNIX_TIMESTAMP mappings to PROTO_VALUE_TO_VALUE_TYPE_MAP
• Fix all mypy type checking errors
• Fix set type inference for request source features
• Fix variable redefinition error in set type inference

• Fixes Add set as feature type #5886

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.