-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Open
Description
Context
PR #5827 added dbt integration with basic data source support.
Problem
The generated data sources use default/inferred configurations but don't allow specifying:
- BigQuery: GCP project, dataset overrides
- Snowflake: account, warehouse, role, connection parameters
- File: base path, file format options
This means generated sources may not connect to the actual data.
Current Limitation
# BigQuery example - no way to override project
return BigQuerySource(
name=f"{model.name}_source",
table=model.full_table_name, # Uses dbt's database.schema.table
# Missing: project override, dataset override
)Proposed Enhancement
Add CLI options for data source configuration:
# BigQuery
feast dbt import -m manifest.json -e driver_id \
--data-source-type bigquery \
--bigquery-project my-gcp-project
# Snowflake
feast dbt import -m manifest.json -e driver_id \
--data-source-type snowflake \
--snowflake-account my-account \
--snowflake-warehouse my-warehouse \
--snowflake-role my-role
# File
feast dbt import -m manifest.json -e driver_id \
--data-source-type file \
--file-format parquet \
--file-base-path /data/warehouseOr support reading from feature_store.yaml:
offline_store:
type: bigquery
project: my-gcp-project
dbt:
inherit_offline_store_config: trueRelated
Metadata
Metadata
Assignees
Labels
No labels