feat: Add skip_validation parameter to push() for ODFVs with missing UDF dependencies #5894

Copilot · 2026-01-23T06:34:39Z

What this PR does / why we need it:

store.push() fails with ModuleNotFoundError when On-Demand Feature Views contain UDFs that reference modules unavailable in the current environment (e.g., training code not deployed to serving infrastructure). This blocks data ingestion in production environments.

Changes

Transformation deserialization bypass:

PandasTransformation.from_proto() and PythonTransformation.from_proto() accept skip_udf parameter
Returns explicit identity functions instead of deserializing with dill.loads() when enabled
Preserves type signatures for both DataFrame and Dict transformations

Call chain propagation:

push(), push_async(), write_to_online_store(), write_to_online_store_async() accept skip_validation parameter
Flows through list_all_feature_views() → registry → proto_registry_utils
OnDemandFeatureView.from_proto() propagates skip_udf to transformation parsing

Caching strategy:

Bypasses caching when skip_udf=True to prevent cache pollution with dummy UDFs
Separate cached (_list_all_feature_views_cached, _list_on_demand_feature_views_cached) and non-cached paths

Usage

# Before: Fails if UDF references unavailable modules
store.push("push_source", df)

# After: Skips UDF deserialization
store.push("push_source", df, skip_validation=True)

All parameters default to False for backward compatibility. No API breaking changes.

Which issue(s) this PR fixes:

Addresses ModuleNotFoundError when pushing data with ODFVs containing UDFs that reference environment-specific modules.

Misc

CodeQL scan: 0 alerts
Added unit tests for transformation skip logic and parameter propagation
Follows pattern suggested by @franciscojavierarceo for consistency with apply() method

Original prompt

This section details on the original issue you should resolve

<issue_title>Read On-Demand Feature View and deserialization while pushing data</issue_title>
<issue_description> ## Description

A ModuleNotFoundError occurs when calling store.push() to ingest data into the Online Store. The error is triggered when Feast attempts to synchronize the registry and encounters an On-Demand Feature View.

Because Feast uses dill to serialize/deserialize User Defined Functions (UDFs), it fails if the execution environment lacks the specific Python module (in this case, training) that was present when the UDF was originally defined and registered.
🔍 Error Traceback

File "/usr/local/lib/python3.12/site-packages/data_dataflow/core/io.py", line 201, in process
    self.store.push(push_source_name, df, to=PushMode.ONLINE)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/feature_store.py", line 1698, in push
    self.write_to_online_store(
  File "/usr/local/lib/python3.12/site-packages/feast/feature_store.py", line 1946, in write_to_online_store
    feature_view, df = self._get_feature_view_and_df_for_online_write(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/feature_store.py", line 1904, in _get_feature_view_and_df_for_online_write
    for fv_proto in self.list_all_feature_views(allow_registry_cache)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/feature_store.py", line 294, in list_all_feature_views
    return self._list_all_feature_views(allow_cache, tags=tags)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/feature_store.py", line 269, in _list_all_feature_views
    for fv in self.registry.list_all_feature_views(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/infra/registry/registry.py", line 647, in list_all_feature_views
    return proto_registry_utils.list_all_feature_views(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/infra/registry/proto_registry_utils.py", line 67, in wrapper
    cache_value = func(registry_proto, project, tags)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/infra/registry/proto_registry_utils.py", line 243, in list_all_feature_views
    + list_on_demand_feature_views(registry_proto, project, tags)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/infra/registry/proto_registry_utils.py", line 67, in wrapper
    cache_value = func(registry_proto, project, tags)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/infra/registry/proto_registry_utils.py", line 285, in list_on_demand_feature_views
    OnDemandFeatureView.from_proto(on_demand_feature_view)
  File "/usr/local/lib/python3.12/site-packages/feast/on_demand_feature_view.py", line 400, in from_proto
    transformation = PandasTransformation.from_proto(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/feast/transformation/pandas_transformation.py", line 150, in from_proto
    udf=dill.loads(user_defined_function_proto.body),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/dill/_dill.py", line 311, in loads
    return load(file, ignore, **kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/dill/_dill.py", line 297, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/dill/_dill.py", line 452, in load
    obj = StockUnpickler.load(self)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/dill/_dill.py", line 442, in find_class
    return StockUnpickler.find_class(self, module, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'training'

Steps to Reproduce

- Define an OnDemandFeatureView in an environment where put transformation code to a local module.

- The UDF inside this view references a function, class, or constant from the training module.

- Run feast apply to save the definition to the remote registry (e.g., S3, GCS, or SQL).

- Run store.push() from a different environment that does not have the training module installed in its Python path.

Root Cause Analysis

This is a serialization dependency issue. When dill (the library Feast uses for pickling) serializes a function, it often stores references to the modules where global variables or dep...

Fixes Read On-Demand Feature View and deserialization while pushing data #5893

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Initial plan

ee8a7d0

Copilot AI assigned Copilot and franciscojavierarceo Jan 23, 2026

Copilot started work on behalf of franciscojavierarceo January 23, 2026 06:35 View session

Copilot AI and others added 4 commits January 23, 2026 06:39

Add skip_validation parameter to push method and related methods

6b2bf7f

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Fix caching decorator compatibility with skip_udf parameter

f9542e5

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Apply code formatting with ruff

79d6744

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Use explicit identity functions instead of lambda for better type safety

a735bfb

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix ModuleNotFoundError in On-Demand Feature View~~ feat: Add skip_validation parameter to push() for ODFVs with missing UDF dependencies Jan 23, 2026

Copilot AI requested a review from franciscojavierarceo January 23, 2026 06:51

Copilot finished work on behalf of franciscojavierarceo January 23, 2026 06:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add skip_validation parameter to push() for ODFVs with missing UDF dependencies #5894

feat: Add skip_validation parameter to push() for ODFVs with missing UDF dependencies #5894

Uh oh!

Copilot AI commented Jan 23, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Add skip_validation parameter to push() for ODFVs with missing UDF dependencies #5894

Are you sure you want to change the base?

feat: Add skip_validation parameter to push() for ODFVs with missing UDF dependencies #5894

Uh oh!

Conversation

Copilot AI commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Changes

Usage

Which issue(s) this PR fixes:

Misc

Steps to Reproduce

Root Cause Analysis

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 23, 2026 •

edited

Loading