Skip to content

fix: Ignore data source timestamps when diffing registry objects - #6889

Draft
adarshsm wants to merge 1 commit into
feast-dev:masterfrom
adarshsm:fix/5573-plan-ignore-source-meta
Draft

adarshsm wants to merge 1 commit into
feast-dev:masterfrom
adarshsm:fix/5573-plan-ignore-source-meta

Conversation

@adarshsm

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

feast plan shows spurious changes to a feature view's batch_source / stream_source that differ only in meta.created_timestamp / meta.last_updated_timestamp. That's the noise reported in #5573.

Root cause. DataSource.__init__ sets created_timestamp and last_updated_timestamp to "now", and _set_timestamps_in_proto writes them to the source proto's meta. The registry copy of a feature view and the one just declared in the repo therefore always carry different source timestamps. diff_registry_objects only walks the spec fields once current != new, but then it compares each field as a whole proto. As soon as a feature view has any real change (tags, description, ttl, …), its embedded source is reported as changed too. The same thing happens for:

  • stream_source (and the batch_source nested inside a Push/Kafka source)
  • LabelView.source
  • ODFV request sources
  • the top-level meta field when a data source itself is diffed

Minimal reproduction on current master:

registered = FeatureView.from_proto(make_fv().to_proto())  # built earlier
declared = make_fv()                                        # built by feast plan
declared.description = "changed"
[d.property_name for d in diff_registry_objects(registered, declared, "feature view").feast_object_property_diffs]
# ['batch_source', 'description']   <- batch_source only differs in meta timestamps

It happens the same way with FileSource and BigQuerySource, so it's not BigQuery-specific.

Fix. Before comparing, diff_registry_objects now works on copies of both specs with meta cleared on every DataSource message they contain (walked recursively through message, repeated and map fields). Real source changes, like a different path, table or timestamp field, are still reported. The printed val_existing / val_declared also no longer include the timestamp noise. Whether an object counts as changed at all still comes from the objects' __eq__, so this PR doesn't change it.

Which issue(s) this PR fixes:

Fixes #5573

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

New test_diff_registry_objects_ignores_data_source_meta in tests/unit/diff/test_registry_diff.py builds the registered and declared objects from separately constructed sources, the way feast plan does. The existing tests share a single source instance, which hides the bug. It checks three things:

  • a tag-only change reports only tags, with a push source wrapping a file source so the nested batch_source is covered
  • a real path change still reports batch_source / stream_source
  • diffing a data source directly reports timestamp_field and not meta

It fails on master (['tags', 'batch_source', 'stream_source'] == ['tags']) and passes with the fix. The rest of tests/unit/diff/test_registry_diff.py passes. ruff check and ruff format --check are clean on both files.

Local environment note: my venv doesn't have the GCP extra, and tests/utils/data_source_test_creator.py imports google.cloud.bigquery at module level. To run this test file locally I stubbed that import out. I also don't have types-protobuf, so mypy only reports the same missing-stubs error it gives for every other google.protobuf import.

🤖 Generated with Claude Code

A DataSource sets its created/last updated timestamps whenever it is
constructed, and they travel in the source proto's meta. diff_registry_objects
compared whole spec fields, so whenever a feature view had any real change,
feast plan also reported its batch_source/stream_source as changed, showing
only differing meta timestamps. Strip meta from every DataSource in both
specs before comparing.

Fixes feast-dev#5573

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: adarshsm <24850536+adarshsm@users.noreply.github.com>
@codecov-commenter

codecov-commenter commented Sep 29, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.86%. Comparing base (a87c070) to head (5003e26).

Files with missing lines Patch % Lines
sdk/python/feast/diff/registry_diff.py 91.66% 1 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6889      +/-   ##
==========================================
+ Coverage   47.84%   47.86%   +0.02%     
==========================================
  Files         422      422              
  Lines       52727    52751      +24     
  Branches     7662     7670       +8     
==========================================
+ Hits        25226    25251      +25     
+ Misses      25680    25678       -2     
- Partials     1821     1822       +1     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 49.21% <91.66%> (+0.02%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/diff/registry_diff.py 56.77% <91.66%> (+7.36%) ⬆️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a87c070...5003e26. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BigQuery table metadata causing Plan Permadiff

2 participants