Skip to content

Commit 2028874

Browse files
adinauerclaude
adinauer
and
claude
authored
feat(core): Data Collection (#5759)
* ref(core): Rename URL query parameter resolver Keep the internal resolver and its tests aligned with the canonical public Data Collection option name. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * ref(http): Use URL query parameter option name Update URL filtering and integration coverage to consume the renamed canonical Data Collection option. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * test(spring): Bind URL query parameter policies Use the canonical URL query parameter property name in Spring Boot 2, 3, and 4 binding coverage. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): Preserve Data Collection on null setter Ignore null assignments so the always-present Data Collection configuration and its current values remain intact. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(android): Preserve installation ID independently of Data Collection Keep the Android installation ID available for distinct ID, user ID, device ID, and hybrid scope fallbacks regardless of the userInfo setting. Continue applying userInfo only to automatic user details such as IP addresses and remove the now-unused legacy-always resolver variant. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(ktor): Exclude query parameters from span descriptions Parse Ktor client request URLs through the shared URL utility so span descriptions omit query parameters and fragments. Keep the raw URL for trace propagation and avoid introducing query span data. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * test(apollo): Cover response header filtering Verify Apollo 4 applies deny-list behavior to response headers for both supported execution implementations. Co-Authored-By: Claude <noreply@anthropic.com> * fix(opentelemetry): Exclude queries from span descriptions Parse the url.full fallback through the shared URL utility before using it as an HTTP span description. This removes query parameters and fragments while preserving route and target handling. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(opentelemetry): Preserve completed request headers Do not apply Data Collection policies while converting completed OpenTelemetry attributes. Preserve the existing sendDefaultPii behavior because completed attributes may have been supplied manually by customers. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(opentelemetry): Preserve completed URL attributes Do not apply Data Collection policies while converting completed OpenTelemetry URL attributes. Preserve manually supplied values and leave attribute collection controls to OpenTelemetry. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(opentelemetry): Normalize legacy HTTP target descriptions Exclude query parameters and fragments when deriving Sentry span descriptions from the legacy http.target attribute. Leave the completed OpenTelemetry attribute unchanged. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(database): Preserve query descriptions Keep sanitized or parameterized query text independent of databaseQueryData. The option only controls bound parameters, write payloads, and result data, which the current JDBC and SQLite integrations do not collect. Remove the unused legacy resolver path and its policy-specific tests. Co-Authored-By: Claude <noreply@anthropic.com> * fix(replay): Keep Replay independent from Data Collection Restore Session Replay network privacy settings as the only controls for Replay data. Data Collection and sendDefaultPii do not affect Replay, which avoids changing behavior for existing Replay users. Add coverage that restrictive Data Collection settings leave explicitly enabled Replay network details unchanged. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * test(graphql): Cover GraphqlUtils request body filtering Add focused coverage for parsing a single GraphQL request object and independently removing document and variable content while preserving operation metadata and allowed fields. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(graphql): Filter batched GraphQL request bodies Apply document and variable collection policies to every operation in a batched GraphQL request. Fail closed when a batch contains non-object entries instead of attaching partially filtered content. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * test(graphql): Cover malformed batched request body entries Verify GraphQL request filtering fails closed when a batch contains a non-object entry. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * ref(core): Clarify forced Data Collection configuration Rename the internal override marker to explain that it forces an empty Data Collection object into explicit mode. Align the related tests with the clarified semantics. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * test(core): Clarify Data Collection resolver scenarios Separate legacy sendDefaultPii fallback coverage from configured Data Collection behavior. Give each resolver test a name that describes one configuration state. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * test(apollo): Cover request header filtering in Apollo 4 Verify that Apollo 4 applies configured Data Collection deny-list behavior to captured request headers across both supported execution paths. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * feat(core): Add Data Collection external options Parse flattened Data Collection settings from properties, system properties, and environment variables. Merge only configured values so omitted settings retain their documented or legacy behavior. Refs #5666 * fix(core): Use URL query parameter external option name Align the flattened external configuration key with the Data Collection option name used by the specification and Android manifest configuration.\n\nRefs #5666\nCo-Authored-By: Claude <noreply@anthropic.com> * feat(android): Add Data Collection manifest options Parse flattened Data Collection metadata while preserving existing option values for fields omitted from the manifest. Expose internal configuration-state helpers across SDK modules so Android can distinguish absent metadata from explicit settings.\n\nRefs #5666\nCo-Authored-By: Claude <noreply@anthropic.com> * fix(core): Reject malformed cookie pairs Validate cookie names and values before applying Data Collection filters. Fail closed for malformed values that could embed additional sensitive cookie pairs while preserving valid quoted and padded values. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * revert: fix(android): Apply user info policy to distinct ID This reverts commit b27d61d.\n\nKeep generated installation IDs independent of the userInfo policy. Restore\ndefault generation before programmatic configuration so applications can\ncontinue clearing the distinct ID in the configuration callback.\n\nRefs #5666\n\nCo-Authored-By: Claude <noreply@anthropic.com> * fix(core): Skip null filtered cookie headers Avoid adding nullable filter results to cookie header lists so downstream consumers only receive actual header values. Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): Preserve blank cookie segments Keep empty and whitespace-only cookie segments unchanged instead of replacing them with a filtered marker. Co-Authored-By: Claude <noreply@anthropic.com> * ref(core): Remove broad cookie filtering catches Let unexpected implementation errors remain visible instead of swallowing fatal JVM errors during deterministic cookie parsing. Co-Authored-By: Claude <noreply@anthropic.com> * ref(core): Extract cookie utilities from HTTP utilities Move cookie parsing and filtering into a focused internal utility and update integrations to use it. Keep generic query and header filtering in HttpUtils. Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): Avoid sharing Data Collection fallbacks Create key-value fallback behaviors for each resolver lookup so mutations cannot leak across cookie, query parameter, and header policies. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): Support WebSocket URL parsing Treat ws and wss as valid hierarchical URIs without requiring JVM URL handlers. This preserves Ktor WebSocket span descriptions and query filtering instead of falling back to an unknown URL. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * changelog * test(okhttp): Use valid Set-Cookie fixture Exercise response cookie filtering with a valid cookie and preserve its attributes in the expected output. Co-Authored-By: Claude <noreply@anthropic.com> * revert: fix(opentelemetry): Exclude queries from span descriptions Revert the OpenTelemetry span-description normalization from this stack PR. Preserve completed OpenTelemetry URL and target values when deriving descriptions. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): [Data Collection 24] Narrow utility exception handling Catch only the recoverable failures produced while filtering GraphQL bodies and decoding query parameter names. Preserve fail-closed handling for malformed GraphQL Unicode escapes without swallowing fatal JVM errors. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(spring): Preserve URL credential filtering Build WebClient span descriptions from sanitized URL details in legacy mode. Avoid exposing URL credentials when Data Collection is not explicitly configured. Co-Authored-By: Claude <noreply@anthropic.com> * ref(core): Centralize HTTP client cookie filtering Move Data Collection and legacy cookie policy selection into CookieUtils. Remove duplicated wrappers from OkHttp, Ktor, and Apollo integrations. Co-Authored-By: Claude <noreply@anthropic.com> * docs: Document Data Collection configuration Describe Data Collection defaults, migration from sendDefaultPii, and the supported configuration mechanisms. Include examples for key-value filtering and HTTP body selection so users can adopt the new controls safely. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * feat(core): [Data Collection 25] Apply file path policy Add the specification-defined filePaths option across programmatic, external, Spring Boot, and Android manifest configuration. Gate automatically captured File I/O paths through the resolved policy while preserving sendDefaultPii when Data Collection is absent.\n\nRefs #5666 * feat(logback): [Data Collection 26] Add unencoded message opt-in Allow encoder-equipped Logback appenders to include original message templates and parameters without relying on sendDefaultPii. Keep sendDefaultPii as a temporary compatibility exception while Data Collection replaces its other behavior. Co-Authored-By: Claude <noreply@anthropic.com> * docs(logback): Clarify unencoded message behavior Describe how encoder use, the integration opt-in, and the legacy PII option control original message data. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(logback): Expose mutable Kotlin appender option Align the includeUnencodedMessage getter and setter names so Kotlin exposes the option as a mutable synthetic property. Refs #5666 Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): Expose mutable Kotlin Data Collection options Align nullable Boolean getters and setters so Kotlin exposes mutable synthetic properties. Allow callers to clear an explicit override by assigning null. Co-Authored-By: Claude <noreply@anthropic.com> * fix(core): [Data Collection 27] Make opt-in explicit Stop treating DataCollection construction as an implicit opt-in. Add an explicit forceDataCollection method so callers can select the new defaults without replacing manifest or external configuration. Co-Authored-By: Claude <noreply@anthropic.com> * docs: Document explicit Data Collection opt-in Show how to opt in to the documented Data Collection defaults without configuring an individual option. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 386030d commit 2028874

119 files changed

Lines changed: 6017 additions & 444 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,120 @@
55
### Features
66

77
- Add `LocalSentrySpan` to `sentry-compose` so apps can provide a parent `ISpan` to a composable subtree and have nested `SentryTraced` spans attach to it ([#6112]https://github.com/getsentry/sentry-java/pull/6112)
8+
- Add `dataCollection`, a fine-grained replacement for `sendDefaultPii`, for controlling data collected automatically by SDK integrations ([#5759](https://github.com/getsentry/sentry-java/pull/5759))
9+
- `sendDefaultPii` remains supported for backwards compatibility. When `dataCollection` is not configured, the SDK preserves the existing `sendDefaultPii` behavior.
10+
- Configuring any `dataCollection` option makes it the source of truth. `sendDefaultPii` is then ignored, and omitted `dataCollection` options use the defaults below.
11+
- The Logback appender is a compatibility exception. When an encoder is configured, `sendDefaultPii=true` continues to include the original message template and parameters. To opt in independently of `sendDefaultPii`, set `<includeUnencodedMessage>true</includeUnencodedMessage>` on the Sentry appender in `logback.xml` or `logback-spring.xml`.
12+
- Data explicitly supplied through APIs such as `Sentry.setUser`, scopes, event processors, or `beforeSend` is not affected.
13+
14+
To opt in to the documented `dataCollection` defaults without configuring an individual option:
15+
16+
```java
17+
Sentry.init(options -> options.getDataCollection().forceDataCollection());
18+
```
19+
20+
| Option | Default | Behavior |
21+
| --- | --- | --- |
22+
| `userInfo` | `true` | Allows integrations to populate user identity and IP address information automatically. |
23+
| `cookies` | `{ mode: DENY_LIST, terms: [] }` | Collects cookies while filtering sensitive values. |
24+
| `httpHeaders.request` | `{ mode: DENY_LIST, terms: [] }` | Collects request headers while filtering sensitive values. |
25+
| `httpHeaders.response` | `{ mode: DENY_LIST, terms: [] }` | Collects response headers while filtering sensitive values. |
26+
| `httpBodies` | All supported body types | Collects supported incoming and outgoing request and response bodies. An empty set disables body collection. |
27+
| `urlQueryParams` | `{ mode: DENY_LIST, terms: [] }` | Collects URL query parameters while filtering sensitive values. |
28+
| `graphql.document` | `true` | Collects GraphQL documents. |
29+
| `graphql.variables` | `true` | Collects GraphQL variables. |
30+
| `databaseQueryData` | `true` | Allows collection of associated query data, such as bound parameters, write payloads, and results, where supported. Sanitized query statements and structural database metadata remain available. |
31+
| `filePaths` | `true` | Allows file-system instrumentation to collect file and directory paths. File extensions and byte counts remain available when disabled. |
32+
33+
Cookies, HTTP headers, and URL query parameters support three modes:
34+
35+
- `OFF`: Do not collect the category.
36+
- `DENY_LIST`: Collect values except those matching the built-in sensitive deny-list or additional configured terms.
37+
- `ALLOW_LIST`: Only send plaintext values for matching terms. The built-in sensitive deny-list still applies.
38+
39+
Matching is case-insensitive and partial. The built-in sensitive deny-list contains `auth`, `token`, `secret`, `password`, `passwd`, `pwd`, `key`, `jwt`, `bearer`, `sso`, `saml`, `csrf`, `xsrf`, `credentials`, `session`, `sid`, and `identity`. Filtered values are replaced with `"[Filtered]"`. Custom deny-list terms extend rather than replace this list.
40+
41+
Configure all HTTP body types, a custom cookie deny-list, a request-header allow-list, and disable URL query parameter and file path collection in an options callback:
42+
43+
```java
44+
Sentry.init(
45+
options -> {
46+
options
47+
.getDataCollection()
48+
.setHttpBodies(
49+
EnumSet.of(
50+
HttpBodyType.INCOMING_REQUEST,
51+
HttpBodyType.OUTGOING_REQUEST,
52+
HttpBodyType.INCOMING_RESPONSE,
53+
HttpBodyType.OUTGOING_RESPONSE));
54+
options
55+
.getDataCollection()
56+
.setCookies(
57+
KeyValueCollectionBehavior.denyList(
58+
"forwarded", "-ip", "remote-", "via", "-user"));
59+
options
60+
.getDataCollection()
61+
.getHttpHeaders()
62+
.setRequest(
63+
KeyValueCollectionBehavior.allowList("content-type", "x-request-id"));
64+
options
65+
.getDataCollection()
66+
.setUrlQueryParams(KeyValueCollectionBehavior.off());
67+
options.getDataCollection().setFilePaths(false);
68+
});
69+
```
70+
71+
Configure the same options in `sentry.properties`:
72+
73+
```properties
74+
data-collection.http-bodies=incoming_request,outgoing_request,incoming_response,outgoing_response
75+
data-collection.cookies.mode=deny_list
76+
data-collection.cookies.terms=forwarded,-ip,remote-,via,-user
77+
data-collection.http-headers.request.mode=allow_list
78+
data-collection.http-headers.request.terms=content-type,x-request-id
79+
data-collection.url-query-params.mode=off
80+
data-collection.file-paths=false
81+
```
82+
83+
Configure them with Spring Boot properties:
84+
85+
```properties
86+
sentry.data-collection.http-bodies=incoming-request,outgoing-request,incoming-response,outgoing-response
87+
sentry.data-collection.cookies.mode=deny-list
88+
sentry.data-collection.cookies.terms=forwarded,-ip,remote-,via,-user
89+
sentry.data-collection.http-headers.request.mode=allow-list
90+
sentry.data-collection.http-headers.request.terms=content-type,x-request-id
91+
sentry.data-collection.url-query-params.mode=off
92+
sentry.data-collection.file-paths=false
93+
```
94+
95+
Configure them in `AndroidManifest.xml`:
96+
97+
```xml
98+
<meta-data
99+
android:name="io.sentry.data-collection.http-bodies"
100+
android:value="incoming_request,outgoing_request,incoming_response,outgoing_response" />
101+
<meta-data
102+
android:name="io.sentry.data-collection.cookies.mode"
103+
android:value="deny_list" />
104+
<meta-data
105+
android:name="io.sentry.data-collection.cookies.terms"
106+
android:value="forwarded,-ip,remote-,via,-user" />
107+
<meta-data
108+
android:name="io.sentry.data-collection.http-headers.request.mode"
109+
android:value="allow_list" />
110+
<meta-data
111+
android:name="io.sentry.data-collection.http-headers.request.terms"
112+
android:value="content-type,x-request-id" />
113+
<meta-data
114+
android:name="io.sentry.data-collection.url-query-params.mode"
115+
android:value="off" />
116+
<meta-data
117+
android:name="io.sentry.data-collection.file-paths"
118+
android:value="false" />
119+
```
120+
121+
See the [Data Collection documentation](https://docs.sentry.io/platforms/java/configuration/options/#dataCollection) for all configuration keys, supported integrations, and migration guidance.
8122

9123
### Fixes
10124

@@ -52,6 +166,7 @@
52166

53167
### Fixes
54168

169+
- Support `ws` and `wss` URL parsing for WebSocket instrumentation ([#6064](https://github.com/getsentry/sentry-java/pull/6064))
55170
- Keep resolving the server name after `Sentry.close()` or a re-init. Closing the SDK shut down the shared hostname cache for the life of the process, so `server_name` silently froze at the value it had last resolved ([#6119](https://github.com/getsentry/sentry-java/pull/6119))
56171
- Order breadcrumbs by the timestamp they carry rather than by when they were created in the current process, so breadcrumbs restored from disk or handed over by a hybrid SDK no longer sort as if they had just happened ([#6097](https://github.com/getsentry/sentry-java/pull/6097))
57172

‎sentry-android-core/api/sentry-android-core.api‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -283,7 +283,6 @@ public final class io/sentry/android/core/DeviceInfoUtil {
283283
public fun getSplitApksInfo ()Lio/sentry/android/core/ContextUtils$SplitApksInfo;
284284
public fun getTotalMemory ()Ljava/lang/Long;
285285
public static fun isCharging (Landroid/content/Intent;Lio/sentry/SentryOptions;)Ljava/lang/Boolean;
286-
public static fun resetInstance ()V
287286
}
288287

289288
public abstract class io/sentry/android/core/EnvelopeFileObserverIntegration : io/sentry/Integration, java/io/Closeable {

‎sentry-android-core/src/main/java/io/sentry/android/core/ApplicationExitInfoEventProcessor.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -681,7 +681,7 @@ private void mergeUser(final @NotNull SentryBaseEvent event) {
681681
if (user.getId() == null) {
682682
user.setId(getDeviceId());
683683
}
684-
if (user.getIpAddress() == null && options.isSendDefaultPii()) {
684+
if (user.getIpAddress() == null && options.getDataCollectionResolver().isUserInfo()) {
685685
user.setIpAddress(IpAddressUtils.DEFAULT_IP_ADDRESS);
686686
}
687687
}

‎sentry-android-core/src/main/java/io/sentry/android/core/DefaultAndroidEventProcessor.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,7 @@ private void mergeUser(final @NotNull SentryBaseEvent event) {
178178
if (user.getId() == null) {
179179
user.setId(Installation.id(context));
180180
}
181-
if (user.getIpAddress() == null && options.isSendDefaultPii()) {
181+
if (user.getIpAddress() == null && options.getDataCollectionResolver().isUserInfo()) {
182182
user.setIpAddress(IpAddressUtils.DEFAULT_IP_ADDRESS);
183183
}
184184
}

‎sentry-android-core/src/main/java/io/sentry/android/core/DeviceInfoUtil.java‎

Lines changed: 1 addition & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,13 @@
1515
import android.os.SystemClock;
1616
import android.util.DisplayMetrics;
1717
import io.sentry.DateUtils;
18-
import io.sentry.ISentryLifecycleToken;
1918
import io.sentry.SentryLevel;
2019
import io.sentry.SentryOptions;
2120
import io.sentry.android.core.internal.util.CpuInfoUtils;
2221
import io.sentry.android.core.internal.util.DeviceOrientations;
2322
import io.sentry.android.core.internal.util.RootChecker;
2423
import io.sentry.protocol.Device;
2524
import io.sentry.protocol.OperatingSystem;
26-
import io.sentry.util.AutoClosableReentrantLock;
2725
import java.io.File;
2826
import java.util.Calendar;
2927
import java.util.Collections;
@@ -34,17 +32,10 @@
3432
import org.jetbrains.annotations.ApiStatus;
3533
import org.jetbrains.annotations.NotNull;
3634
import org.jetbrains.annotations.Nullable;
37-
import org.jetbrains.annotations.TestOnly;
3835

3936
@ApiStatus.Internal
4037
public final class DeviceInfoUtil {
4138

42-
@SuppressLint("StaticFieldLeak")
43-
private static volatile DeviceInfoUtil instance;
44-
45-
private static final @NotNull AutoClosableReentrantLock staticLock =
46-
new AutoClosableReentrantLock();
47-
4839
private final @NotNull Context context;
4940
private final @NotNull SentryAndroidOptions options;
5041
private final @NotNull BuildInfoProvider buildInfoProvider;
@@ -80,19 +71,7 @@ public DeviceInfoUtil(
8071
@NotNull
8172
public static DeviceInfoUtil getInstance(
8273
final @NotNull Context context, final @NotNull SentryAndroidOptions options) {
83-
if (instance == null) {
84-
try (final @NotNull ISentryLifecycleToken ignored = staticLock.acquire()) {
85-
if (instance == null) {
86-
instance = new DeviceInfoUtil(ContextUtils.getApplicationContext(context), options);
87-
}
88-
}
89-
}
90-
return instance;
91-
}
92-
93-
@TestOnly
94-
public static void resetInstance() {
95-
instance = null;
74+
return options.getOrCreateDeviceInfoUtil(context);
9675
}
9776

9877
// we can get some inspiration here

0 commit comments

Comments
 (0)