Allow customisation of the dotevn setting source to filter variables - #832
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an env_filtering option to the dotenv settings source to control which .env variables are passed into the settings model, addressing env-prefix related “extra fields” errors reported in #800.
Changes:
- Introduce
DotenvFilteringand a newenv_filteringsetting inSettingsConfigDict. - Implement dotenv-source filtering modes:
only_existingandmatch_prefix. - Add tests and documentation describing the new behavior.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_settings.py | Adds coverage for the new env_filtering modes. |
| pydantic_settings/sources/types.py | Defines DotenvFiltering literal type. |
| pydantic_settings/sources/providers/dotenv.py | Implements filtering behavior in DotEnvSettingsSource. |
| pydantic_settings/sources/init.py | Re-exports DotenvFiltering from the sources package. |
| pydantic_settings/main.py | Adds env_filtering to SettingsConfigDict. |
| docs/index.md | Documents env_filtering and updates the dotenv/env_prefix note. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Thanks @CaselIT for the PR. Copilot left some comments. Please check them. |
|
the handling of env_nested_delimiter was already kinda problematic before this patch. I've fixed the reported comments, but I haven't fixed what I would argue is another bug. Let me know if you agree, otherwise I can look into fixing it too |
| a: A | ||
|
|
||
| s = Settings() | ||
| exp = {'a': {'x': 'bar', 'b': 'y'}, 'b': 'b', 'c@a': '1', 'c@b': '2'} |
There was a problem hiding this comment.
these c@a and c@b is what I would argue is another bug, but it was already like this so I left the behaviour in plance
|
also I haven't found if you have a changelog or similar, if you want me to add a note let me know where should I |
we don't have a changelog file. We will generate it during release. |
|
Could you please add a test for |
|
updated. let me know if you want me to do something regarding the other behaviour mentioned here #832 (comment) |
It would be great if you could fix this. Better to fix it in separate PR |
|
I'll create an issue in the meantime. just to confirm, this should be |
After checking more, I think this probably is not an issue. becuase |
|
You are probably right, the docs says
since there is no c BaseModel this is likely in line with "unexpected results" Maybe an additional doc line may be worth adding? |
yes, would be good |
|
done |
|
Thanks @CaselIT Let's merge this. It will be available in the next release! Please help us in case of issues after the release. |
|
great!
will try, ping me if something comes up related to this |
This PR adds a new setting option for the dotenv setting source to filter what variables are provided to the model.
The setting
env_filteringis added:'match_prefix': only the variable that match the prefix will be passed to the model. Useful when used in conjunction withenv_prefixto "scope" a single dotenv file to a specific model.'only_existing': only the variables that have a corresponding field will be passed to the model. When using this option the dotenv setting source will behave like the env settings source.Fixes: #800