Conversation
A DataSource sets its created/last updated timestamps whenever it is constructed, and they travel in the source proto's meta. diff_registry_objects compared whole spec fields, so whenever a feature view had any real change, feast plan also reported its batch_source/stream_source as changed, showing only differing meta timestamps. Strip meta from every DataSource in both specs before comparing. Fixes feast-dev#5573 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: adarshsm <24850536+adarshsm@users.noreply.github.com>
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6889 +/- ##
==========================================
+ Coverage 47.84% 47.86% +0.02%
==========================================
Files 422 422
Lines 52727 52751 +24
Branches 7662 7670 +8
==========================================
+ Hits 25226 25251 +25
+ Misses 25680 25678 -2
- Partials 1821 1822 +1
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
feast planshows spurious changes to a feature view'sbatch_source/stream_sourcethat differ only inmeta.created_timestamp/meta.last_updated_timestamp. That's the noise reported in #5573.Root cause.
DataSource.__init__setscreated_timestampandlast_updated_timestampto "now", and_set_timestamps_in_protowrites them to the source proto'smeta. The registry copy of a feature view and the one just declared in the repo therefore always carry different source timestamps.diff_registry_objectsonly walks the spec fields oncecurrent != new, but then it compares each field as a whole proto. As soon as a feature view has any real change (tags, description, ttl, …), its embedded source is reported as changed too. The same thing happens for:stream_source(and thebatch_sourcenested inside a Push/Kafka source)LabelView.sourcemetafield when a data source itself is diffedMinimal reproduction on current
master:It happens the same way with
FileSourceandBigQuerySource, so it's not BigQuery-specific.Fix. Before comparing,
diff_registry_objectsnow works on copies of both specs withmetacleared on everyDataSourcemessage they contain (walked recursively through message, repeated and map fields). Real source changes, like a different path, table or timestamp field, are still reported. The printedval_existing/val_declaredalso no longer include the timestamp noise. Whether an object counts as changed at all still comes from the objects'__eq__, so this PR doesn't change it.Which issue(s) this PR fixes:
Fixes #5573
Checks
git commit -s)Testing Strategy
New
test_diff_registry_objects_ignores_data_source_metaintests/unit/diff/test_registry_diff.pybuilds the registered and declared objects from separately constructed sources, the wayfeast plandoes. The existing tests share a single source instance, which hides the bug. It checks three things:tags, with a push source wrapping a file source so the nestedbatch_sourceis coveredbatch_source/stream_sourcetimestamp_fieldand notmetaIt fails on
master(['tags', 'batch_source', 'stream_source'] == ['tags']) and passes with the fix. The rest oftests/unit/diff/test_registry_diff.pypasses.ruff checkandruff format --checkare clean on both files.Local environment note: my venv doesn't have the GCP extra, and
tests/utils/data_source_test_creator.pyimportsgoogle.cloud.bigqueryat module level. To run this test file locally I stubbed that import out. I also don't havetypes-protobuf, so mypy only reports the same missing-stubs error it gives for every othergoogle.protobufimport.🤖 Generated with Claude Code