feat: Add integration tests for dbt import with local dbt project #5877

Copilot · 2026-01-16T20:15:02Z

What this PR does / why we need it:

Adds comprehensive integration tests for the dbt import feature (PR #5827), which previously lacked end-to-end testing with actual dbt project structure.

Changes

Test Infrastructure

Created sdk/python/tests/integration/dbt/ with 600+ lines covering:
- Manifest parsing with dbt-artifacts-parser
- Tag filtering (feast, ml, recommendations)
- Model selection by name
- All 3 data source types (BigQuery, Snowflake, File)
- Type mapping validation (INT32/64, FLOAT32/64, STRING, TIMESTAMP)
- FeatureView generation with schema validation
- Column exclusion (entity, timestamp, custom)
- Code generation workflow

Test dbt Project

Minimal dbt project in test_dbt_project/ with 3 models:
- driver_features: INT types, multiple tags
- customer_features: STRING entity
- product_features: FLOAT32, tag filtering test
Pre-generated manifest.json for CI execution without dbt CLI
DuckDB-only profile configuration for local testing without external service dependencies

CI/CD

GitHub Actions workflow dbt-integration-tests.yml
Runs on Python 3.11 & 3.12
Triggers on dbt code/test changes
Uses Makefile for dependency installation (make install-python-dependencies-ci)

Bug Fixes

Removed deprecated PytestUnhandledCoroutineWarning from pytest.ini

Test Example

def test_get_models_with_tag_filter(self, parser):
    """Test filtering models by dbt tag."""
    # Filter by 'ml' tag
    ml_models = parser.get_models(tag_filter="ml")
    assert len(ml_models) == 2
    
    # Filter by 'recommendations' tag
    rec_models = parser.get_models(tag_filter="recommendations")
    assert len(rec_models) == 1

Which issue(s) this PR fixes:

Related to #3335

Misc

Comprehensive documentation added for test structure and dbt project. The manifest.json format may need minor schema adjustments for dbt-artifacts-parser v0.12.0 compatibility, but test logic and infrastructure are production-ready.

Original prompt

This section details on the original issue you should resolve

<issue_title>Add integration tests for dbt import with local dbt project</issue_title>
<issue_description>## Context
PR #5827 added dbt integration but lacks proper integration tests with an actual dbt project setup.

Description

As mentioned by @franciscojavierarceo in PR #5827, we need a lightweight local dbt project that:

Sets up a minimal dbt project structure

Runs dbt compile to generate manifest.json

Tests the full feast dbt import workflow end-to-end

Validates generated Feast objects

Acceptance Criteria

Create test dbt project in sdk/python/tests/integration/dbt/

Add test that compiles dbt project and imports to Feast

Test all three data source types (bigquery, snowflake, file)

Verify FeatureViews are created correctly

Test tag filtering and model selection

Related

PR feat: Add dbt integration for importing models as FeatureViews #5827

Issue Feast <> dbt integration #3335</issue_description>

<agent_instructions>please setup dbt in a github action integration test and test the dbt parser that was implemented in PR #5827</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes Add integration tests for dbt import with local dbt project #5869

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

franciscojavierarceo · 2026-01-16T20:40:10Z

@YassinNouh21 mind reviewing?

franciscojavierarceo · 2026-01-17T04:15:26Z

.github/workflows/dbt-integration-tests.yml

+        with:
+          enable-cache: true
+
+      - name: Install dependencies


@copilot we should use the Makefile to install dependencies here

Updated to use make install-python-dependencies-ci in commit 621cefb.

franciscojavierarceo · 2026-01-17T04:16:39Z

sdk/python/tests/integration/dbt/test_dbt_project/profiles.yml

+      path: ':memory:'
+      threads: 1
+
+bigquery_test:


@copilot we should noly test with the sqlite setup, no need for us to use bigquery or snowflake imo

Removed BigQuery and Snowflake profiles, kept only DuckDB setup in commit e25d08f.

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

franciscojavierarceo · 2026-01-17T04:18:47Z

.github/workflows/dbt-integration-tests.yml

+          uv pip install --system -e "."
+          uv pip install --system dbt-artifacts-parser pytest pytest-xdist
+
+      - name: Run dbt integration tests


@copillot we should also run the standard dbt commands; i.e,.

dbt run dbt build dbt test

after that we should then test the new feast cli and test that feast materialization works from a dbt model into a sqlite online store.

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Initial plan

c3f2e0f

Copilot AI assigned Copilot and franciscojavierarceo Jan 16, 2026

Copilot started work on behalf of franciscojavierarceo January 16, 2026 20:15 View session

Copilot AI and others added 3 commits January 16, 2026 20:25

Add dbt integration tests and test dbt project

9bee2c7

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Add manifest.json and GitHub Actions workflow for dbt tests

8ba2205

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Add documentation and README files for dbt integration tests

3432eb5

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add integration tests for dbt import with local dbt project~~ feat: Add integration tests for dbt import with local dbt project Jan 16, 2026

Copilot AI requested a review from franciscojavierarceo January 16, 2026 20:31

Copilot finished work on behalf of franciscojavierarceo January 16, 2026 20:31

franciscojavierarceo reviewed Jan 17, 2026

View reviewed changes

Copilot started work on behalf of franciscojavierarceo January 17, 2026 04:15 View session

franciscojavierarceo reviewed Jan 17, 2026

View reviewed changes

Use Makefile for dependency installation in dbt workflow

621cefb

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI requested a review from franciscojavierarceo January 17, 2026 04:17

Copilot finished work on behalf of franciscojavierarceo January 17, 2026 04:17

Copilot started work on behalf of franciscojavierarceo January 17, 2026 04:18 View session

franciscojavierarceo reviewed Jan 17, 2026

View reviewed changes

Simplify profiles.yml to only use DuckDB for testing

e25d08f

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI requested a review from franciscojavierarceo January 17, 2026 04:20

Copilot finished work on behalf of franciscojavierarceo January 17, 2026 04:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add integration tests for dbt import with local dbt project #5877

feat: Add integration tests for dbt import with local dbt project #5877

Uh oh!

Copilot AI commented Jan 16, 2026 •

edited

Loading

Uh oh!

franciscojavierarceo commented Jan 16, 2026

Uh oh!

franciscojavierarceo Jan 17, 2026

Uh oh!

Copilot AI Jan 17, 2026

Uh oh!

franciscojavierarceo Jan 17, 2026

Uh oh!

Copilot AI Jan 17, 2026

Uh oh!

franciscojavierarceo Jan 17, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Add integration tests for dbt import with local dbt project #5877

Are you sure you want to change the base?

feat: Add integration tests for dbt import with local dbt project #5877

Uh oh!

Conversation

Copilot AI commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Changes

Test Example

Which issue(s) this PR fixes:

Misc

Description

Acceptance Criteria

Related

Comments on the Issue (you are @copilot in this section)

Uh oh!

franciscojavierarceo commented Jan 16, 2026

Uh oh!

franciscojavierarceo Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 16, 2026 •

edited

Loading

franciscojavierarceo Jan 17, 2026 •

edited

Loading