Python SDK Packaging

Relevant source files

• Makefile
• pyproject.toml
• sdk/python/requirements/py3.10-ci-requirements.txt
• sdk/python/requirements/py3.10-minimal-requirements.txt
• sdk/python/requirements/py3.10-minimal-sdist-requirements.txt
• sdk/python/requirements/py3.10-requirements.txt
• sdk/python/requirements/py3.11-ci-requirements.txt
• sdk/python/requirements/py3.11-minimal-requirements.txt
• sdk/python/requirements/py3.11-minimal-sdist-requirements.txt
• sdk/python/requirements/py3.11-requirements.txt
• sdk/python/requirements/py3.12-ci-requirements.txt
• sdk/python/requirements/py3.12-minimal-requirements.txt
• sdk/python/requirements/py3.12-minimal-sdist-requirements.txt
• setup.py

This document describes the packaging configuration for the Feast Python SDK, including dependency management, build process, and distribution artifacts. For information about the overall build system orchestration, see Build System and Dependency Management. For details on release versioning and publishing, see Release Process and Versioning.

Package Overview

The Feast Python SDK is packaged as a standard Python wheel distributed via PyPI under the name feast. The packaging configuration supports multiple Python versions (3.10, 3.11, 3.12) and includes over 30 optional feature sets through extras_require, allowing users to install only the dependencies needed for their deployment scenario.

Key Packaging Components:

Component	Purpose	Primary File
Package definition	Core setuptools configuration	setup.py
Modern config	PEP 517/518 compatible metadata	pyproject.toml
Dependency locking	Pinned requirements with hashes	sdk/python/requirements/*.txt
Build orchestration	Make targets for packaging tasks	Makefile

Sources: setup.py1-397 pyproject.toml1-213 Makefile1-776

Setup.py Configuration

The primary package configuration is defined in setup.py using setuptools. The package supports dynamic versioning via setuptools_scm, which derives version numbers from Git tags.

Package Metadata

Dynamic Versioning Implementation

Sources: setup.py26-318 setup.py310-318

The package version is automatically determined from Git tags using a custom regex pattern that supports Feast's tag structure:

This allows tags like v0.30.0 or sdk/python/v0.30.0 to be parsed correctly.

Sources: setup.py310-312 setup.py315-318

Package Structure

The Python SDK code resides in sdk/python/feast/, and the setup configuration explicitly defines this structure:

• Package directory: sdk/python specified via package_dir={'': 'sdk/python'}
• Excluded packages: java, infra, sdk/python/tests, ui
• Included package data: Enabled via include_package_data=True

Sources: setup.py320-334

Dependency Management

Feast uses a sophisticated dependency management system with a minimal core and extensive optional extras. This allows users to install only what they need for their specific use case.

Core Dependencies

Core Dependencies List

Sources: setup.py32-62

The REQUIRED list in setup.py defines 27 core dependencies that are always installed. Key categories include:

Category	Dependencies	Purpose
CLI	click, colorama, tabulate	Command-line interface
Config	PyYAML, toml, jsonschema	Configuration parsing
Data	pandas, pyarrow, numpy, dask	Data manipulation
Web	fastapi, uvicorn, gunicorn	HTTP serving
Types	pydantic, protobuf, typeguard	Type validation
Utils	requests, tqdm, tenacity, mmh3	HTTP, progress, retries, hashing

Sources: setup.py32-62

Optional Extras

The package defines 34 optional extras through extras_require, organized by integration type:

Cloud Provider Extras

Extras by Category

Sources: setup.py64-298 setup.py336-380

The extras are organized into logical groups:

Category	Extras	Example Dependencies
Cloud	gcp, aws, azure, snowflake	google-cloud-bigquery, boto3, azure-storage-blob, snowflake-connector-python
Online Stores	redis, postgres, mysql, cassandra, elasticsearch, milvus, hazelcast, singlestore, qdrant	Database-specific drivers
Offline Stores	spark, trino, duckdb, ibis, mssql, clickhouse, couchbase	Query engine libraries
File Formats	delta	deltalake for Delta Lake support
ML/AI	pytorch, rag, image, docling, nlp	torch, transformers, datasets, timm
Infrastructure	k8s, grpcio, opentelemetry	kubernetes, grpcio, prometheus_client
Development	ci, dev, docs	Testing and documentation tools
Special	minimal, minimal-sdist-build	Production and build dependencies

Sources: setup.py64-298

Composite Extras

Several extras are composite, combining multiple feature sets:

Sources: setup.py195-298

Requirements Lock Files

Dependencies are locked with exact versions and SHA256 hashes for security and reproducibility. The lock files are generated using uv pip compile and stored in sdk/python/requirements/.

Lock File Types

Lock File Structure

Sources: sdk/python/requirements/py3.10-ci-requirements.txt1-2 sdk/python/requirements/py3.11-requirements.txt1-2

Each lock file includes:

• Header comment with the exact uv pip compile command used to generate it
• Package entries with exact versions and SHA256 hashes
• Dependency provenance via comments showing which package required each dependency

Example entry format:

aiohttp==3.13.3 \
    --hash=sha256:01ad2529d4b5035578f5081606a465f3b814c542882804e2e8cda61adf5c71bf \
    --hash=sha256:042e9e0bcb5fba81886c8b4fbb9a09d6b8a00245fd8d88e4d989c1f96c74164c \
    # via
    #   aiobotocore
    #   fsspec

Sources: sdk/python/requirements/py3.10-ci-requirements.txt15-138

Lock File Generation

Makefile Targets

Sources: Makefile117-140

The lock-python-dependencies-all Make target regenerates all lock files:

Sources: Makefile117-140

The process uses:

• pixi: Isolated environment per Python version
• uv pip compile: Fast, reliable resolver with hash generation
• --no-strip-extras: Preserves extras metadata for debugging
• --generate-hashes: Adds SHA256 hashes for security

Sources: Makefile121-139

Installation Targets

The Makefile provides multiple installation targets for different scenarios:

Development Installation

Installation Target Details

Sources: Makefile69-115

Target	Use Case	Method	Installation Type
install-python-dependencies-dev	Local development	uv pip sync + uv pip install -e .	Editable in venv
install-python-dependencies-minimal	Minimal local dev	uv pip sync minimal + editable	Editable in venv
install-python-dependencies-ci	GitHub Actions CI	uv pip sync --system	System Python
install-python-ci-dependencies	Legacy CI	python -m piptools sync	System Python
install-python	E2E tests	piptools sync + setup.py develop	Development mode

Sources: Makefile69-115

Special Handling for PyTorch

The CI installation includes special logic to prevent CUDA dependencies:

Sources: Makefile83-93

Build Process

The Python SDK build process involves multiple steps, coordinated through Make targets.

Build Workflow

Protobuf Compilation

Sources: Makefile142-143

The compile-protos-python target invokes a custom script that:

1. Reads .proto files from protos/feast/
2. Generates Python code using protoc with the mypy-protobuf plugin
3. Outputs to sdk/python/feast/protos/

Sources: Makefile142-143

Python Wheel Building

The package uses the build module (PEP 517) for creating distribution artifacts:

This creates both:

• Wheel: feast-{version}-py3-none-any.whl (platform-independent)
• Source distribution: feast-{version}.tar.gz

The wheel is a "pure Python" wheel (py3-none-any) because all compiled extensions (like Go bindings) are optional extras.

Sources: setup.py392-396 package metadata

Setup Requirements

Sources: setup.py392-395

These are installed automatically when building from source and are needed for:

• pybindgen: Generating C bindings (used by optional Go integration)
• setuptools_scm: Extracting version from Git tags

Multi-Python Version Support

Feast supports Python 3.10, 3.11, and 3.12, with separate lock files for each version.

Version Matrix

Version Detection

Sources: Makefile30-32

The Makefile automatically detects the Python version:

This is used to select the appropriate lock file for installation:

Sources: Makefile30-32 Makefile69-71

Conditional Dependencies

Some extras include version-specific dependencies:

Sources: setup.py190-193

This allows different dependency versions or alternatives based on Python version.

Package Entry Points

The package registers the feast CLI command as a console script:

Sources: setup.py390

This creates a feast command that invokes the cli function from feast.cli.cli when installed.

Minimal SDIST Build

The minimal-sdist-build extra includes dependencies needed to build the package from source in resource-constrained environments:

Sources: setup.py275-288

The corresponding lock files (py3.X-minimal-sdist-requirements.txt) exclude milvus-lite which has large binary dependencies, and include build tools like:

• meson and meson-python for NumPy/SciPy building
• Cython for compiling extensions
• pybindgen for C++ bindings
• setuptools>=60 for modern packaging features

Sources: sdk/python/requirements/py3.10-minimal-sdist-requirements.txt1-2 pyproject.toml189-208

Package Classifiers

The package includes PyPI classifiers for discoverability:

Sources: setup.py383-389

This metadata helps users find the package and understand compatibility constraints.

---

Key Takeaways:

1. Modular Design: Core + 34 optional extras allow flexible installations
2. Locked Dependencies: SHA256 hashes ensure reproducible builds
3. Multi-Version Support: Separate lock files for Python 3.10, 3.11, 3.12
4. Build Automation: Make targets coordinate complex multi-step builds
5. Dynamic Versioning: Git tags drive semantic versioning via setuptools_scm