Skip to content

Commit 0de9196

Browse files
singhhimanshu0811Himanshu Singh
andauthored
feat(cassandra): Add multi-DC support via per-datacenter execution profiles (feast-dev#6434)
* feat(cassandra): add multi-DC support via per-datacenter execution profiles Introduce a `datacenters` config list as an alternative to the flat `hosts` field. Each entry specifies the contact points, local_dc, and optional replication metadata for one datacenter. On connect, a named Cassandra execution profile is registered per DC (profile key = local_dc), enabling callers to route driver-level queries to a specific datacenter. The default profile is pinned to `load_balancing.local_dc`, or the first DC when load_balancing is absent. All existing `hosts` / `secure_bundle_path` configs are fully backward compatible — the original code path is untouched. Also updates docs/reference/online-stores/cassandra.md with a multi-DC feature_store.yaml example. Signed-off-by: singhhimanshu0811 email : itssinghhimanshu@gmail.com Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * implemented suggestions and tests Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * doc -s Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * doc -s Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * empty commit Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * trigger build Signed-off-by: Himanshu Singh <itssinghhimanshu@gmail.com> Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * secrets whitelisting Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fix: apply ruff format and fix pre-commit secret/linting issues Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fix: bump cassandra-driver minimum to 3.26.0 for execution_profile support in execute_concurrent_with_args Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fix: add type annotations to resolve mypy errors in Branch A cluster_kwargs Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fix: rename Branch A execution_profiles variable to avoid mypy no-redef error Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fix: rename Branch A cluster_kwargs to avoid mypy no-redef with Branch B Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * fix: rename Branch A cluster_kwargs to avoid mypy no-redef with Branch B Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * regenerated pixi Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> * trigger build Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> --------- Signed-off-by: singhhimanshu0811 Signed-off-by: Himanshu Singh <himanshu.singh@walmart.com> Co-authored-by: Himanshu Singh <himanshu.singh@walmart.com>
1 parent de11152 commit 0de9196

5 files changed

Lines changed: 1606 additions & 705 deletions

File tree

‎docs/reference/online-stores/cassandra.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,51 @@ online_store:
3737
```
3838
{% endcode %}
3939
40+
### Example (Cassandra — multi-DC)
41+
42+
Use `datacenters` instead of `hosts` when your cluster spans multiple datacenters.
43+
Each entry gets a named Cassandra **execution profile** keyed by its `name` field,
44+
enabling per-DC routing. The default profile is determined by `load_balancing.local_dc`
45+
(or the first datacenter entry when `load_balancing` is absent). Use the optional
46+
`routing` block to direct reads and writes to specific datacenters. The keyspace must
47+
already exist; Feast does not create it automatically.
48+
49+
`datacenters` is mutually exclusive with `hosts` and `secure_bundle_path`.
50+
51+
{% code title="feature_store.yaml" %}
52+
```yaml
53+
project: my_feature_repo
54+
registry: data/registry.db
55+
provider: local
56+
online_store:
57+
type: cassandra
58+
keyspace: KeyspaceName
59+
datacenters:
60+
- name: dc1
61+
hosts:
62+
- 192.168.1.1
63+
- 192.168.1.2
64+
replication_factor: 3 # optional, informational only
65+
replication_strategy: NetworkTopologyStrategy # optional, informational only
66+
- name: dc2
67+
hosts:
68+
- 10.0.0.1
69+
replication_factor: 2 # optional, informational only
70+
routing: # optional
71+
read_dc: dc2 # DC to use for reads (default: load_balancing.local_dc)
72+
write_dc: dc1 # DC to use for writes (default: load_balancing.local_dc)
73+
port: 9042 # optional
74+
username: user # optional
75+
password: secret # optional
76+
protocol_version: 5 # optional
77+
load_balancing: # optional
78+
local_dc: 'dc1' # sets the default execution profile
79+
load_balancing_policy: 'TokenAwarePolicy(DCAwareRoundRobinPolicy)' # optional
80+
read_concurrency: 100 # optional
81+
write_concurrency: 100 # optional
82+
```
83+
{% endcode %}
84+
4085
### Example (Astra DB)
4186

4287
{% code title="feature_store.yaml" %}

0 commit comments

Comments
 (0)