docs: scrub dual-path language after FFI removal

Doc comments still described two binding styles, a connector cdylib, and a separate widening library — none of which exist. Module docs, javadoc, error messages, and READMEs now describe the single path: every bridge cdylib is an export_bridge! expansion over the datafusion-spark-bridge SDK, widening included. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
timsaucer · Jun 10, 2026 · Jun 10, 2026 · Jun 10, 2026 · Jun 10, 2026 · Jun 10, 2026
commit 13bca92dc1aa8398b892f148f4a2953fd4a89c66
diff --git a/examples/python/README.md b/examples/python/README.md
@@ -110,16 +110,16 @@ root
 +---+-----+
 ```
 
-Filter row count drops from 4 → 2 because the predicate is pushed across the
-FFI boundary as a `LogicalExprNode` proto and applied inside DataFusion before
-Arrow batches cross back to Spark.
+Filter row count drops from 4 → 2 because the predicate is pushed into the
+bridge cdylib as a `LogicalExprNode` proto and applied inside DataFusion
+before Arrow batches cross back to Spark.
 
 ## Notes
 
 - `master("local[2]")` keeps driver + executor in one JVM so the example
-  cdylib loads once. Cluster mode would need the cdylib pre-staged on every
-  worker (the widening lib is bundled in `datafusion-java-spark`; only the
-  per-bridge example lib is not).
+  cdylib loads once. In cluster mode nothing extra is needed: the bridge
+  cdylib travels inside the examples jar and `NativeLibraryLoader` extracts
+  it on every worker.
 - `extraClassPath` (not `--packages` / `userClassPathFirst`) is used because
   the Spark distro ships Arrow 12, flatbuffers 1.12, and protobuf 2.5, all
   of which we need to override; userClassPathFirst splits Netty across two

diff --git a/native-common/src/lib.rs b/native-common/src/lib.rs
@@ -15,10 +15,11 @@
 // specific language governing permissions and limitations
 // under the License.
 
-//! JNI plumbing shared by this workspace's cdylibs (`datafusion-jni` and the
-//! Spark connector helper): the error-to-Java-exception mapping, the
-//! per-cdylib Tokio runtime singleton, and the async-stream-to-
-//! `FFI_ArrowArrayStream` bridge.
+//! JNI plumbing shared by this workspace's native crates (`datafusion-jni`
+//! and `datafusion-spark-bridge`, and through the latter every bridge
+//! cdylib): the error-to-Java-exception mapping, the per-cdylib Tokio
+//! runtime singleton, and the async-stream-to-`FFI_ArrowArrayStream`
+//! bridge.
 //!
 //! Each cdylib statically links its own copy of this rlib, so [`runtime`] is
 //! a per-cdylib singleton -- exactly the behaviour each crate had when this

diff --git a/native/src/lib.rs b/native/src/lib.rs
@@ -82,8 +82,8 @@ pub(crate) fn jvm() -> &'static JavaVM {
 
 pub(crate) fn runtime() -> &'static Runtime {
     // The singleton itself lives in datafusion-jni-common (shared with the
-    // Spark helper cdylib; each cdylib statically links its own copy, so the
-    // runtime stays per-library). The init hook eagerly installs the
+    // datafusion-spark-bridge SDK; each cdylib statically links its own
+    // copy, so the runtime stays per-library). The init hook eagerly installs the
     // runtime-metrics accumulator (no-op when the `runtime-metrics` Cargo
     // feature is off). Initialising here -- not lazily on the first
     // `runtimeStats()` call -- means the RuntimeMonitor's sampling baseline

diff --git a/spark/README.md b/spark/README.md
@@ -380,7 +380,8 @@ Shared-scan operational details:
 - **Schema inference** — your provider's Arrow schema, widened, becomes the
   Spark schema. Driver-side, one probe build with empty `partitionBytes`.
 - **Type widening** — Spark's columnar readers reject several Arrow types
-  DataFusion happily produces. The connector cdylib transparently casts
+  DataFusion happily produces. The SDK (inside your bridge's cdylib)
+  transparently casts
   unsigned ints → wider signed, `Float16` → `Float32`, `Time*` → wider ints,
   any-unit/tz `Timestamp` → microsecond, recursively through
   `List`/`LargeList`/`FixedSizeList` (see

diff --git a/spark/bridge/src/lib.rs b/spark/bridge/src/lib.rs
@@ -75,7 +75,7 @@ pub(crate) fn runtime_handle() -> &'static Handle {
     datafusion_jni_common::runtime().handle()
 }
 
-/// Generate the JNI entry points for a static bridge cdylib.
+/// Generate the JNI entry points for a bridge cdylib.
 ///
 /// `jni_class` is the **underscore-mangled** binary name of the Java class
 /// declaring the matching `native` methods: dots become underscores

diff --git a/spark/bridge/src/scan.rs b/spark/bridge/src/scan.rs
@@ -15,13 +15,13 @@
 // specific language governing permissions and limitations
 // under the License.
 
-//! Planning and execution of a Spark scan, provider-source-agnostic.
+//! Planning and execution of a Spark scan.
 //!
-//! Every function here is the body of one JNI entry point; the caller (the
-//! generic FFI cdylib, or a static bridge's `export_bridge!` expansion)
-//! supplies only how the provider is obtained, as a `make` closure. The
-//! provider is wrapped in a [`WideningTableProvider`] here, so both binding
-//! styles get identical Spark-compatible Arrow types.
+//! Every function here is the body of one JNI entry point generated by a
+//! bridge's `export_bridge!` expansion, which supplies only how the provider
+//! is obtained, as a `make` closure. The provider is wrapped in a
+//! [`WideningTableProvider`] here, so every bridge gets identical
+//! Spark-compatible Arrow types.
 //!
 //! [`create_scan`] registers the widened provider on a private
 //! `SessionContext` built from the caller-pinned config, applies the pruned

diff --git a/spark/src/main/scala/io/datafusion/spark/ArrowToSparkSchema.scala b/spark/src/main/scala/io/datafusion/spark/ArrowToSparkSchema.scala
@@ -35,9 +35,10 @@ import org.apache.spark.sql.types._
  * Null. No unsigned-int or Time accessor exists; we surface a clear error at schema discovery
  * for those — the alternative is silent corruption.
  *
- * The widening cdylib (connector-core/native/) inserts a `WideningTableProvider` upstream of the
- * Spark reader that casts unsupported types kernel-side (UInt*→signed wider, Float16→Float32,
- * non-µs Timestamp→µs Timestamp, Time→Int) so Spark only ever sees compatible Arrow types.
+ * The widening layer (datafusion-spark-bridge, compiled into every bridge cdylib) inserts a
+ * `WideningTableProvider` upstream of the Spark reader that casts unsupported types kernel-side
+ * (UInt*→signed wider, Float16→Float32, non-µs Timestamp→µs Timestamp, Time→Int) so Spark only
+ * ever sees compatible Arrow types.
  */
 object ArrowToSparkSchema {
 
@@ -67,7 +68,7 @@ object ArrowToSparkSchema {
           unsupported(
             f,
             s"unsigned integer UInt$bits (Spark ArrowColumnVector has no unsigned accessor; " +
-              "widening cdylib casts these before Spark sees them — this branch indicates the " +
+              "widening layer casts these before Spark sees them — this branch indicates the " +
               "WideningTableProvider was bypassed)"
           )
         case (bits, signed) => unsupported(f, s"Int(bits=$bits, signed=$signed)")
@@ -76,7 +77,7 @@ object ArrowToSparkSchema {
     case t: ArrowType.FloatingPoint =>
       t.getPrecision match {
         case FloatingPointPrecision.HALF =>
-          unsupported(f, "Float16 (widening cdylib must cast to Float32 before Spark)")
+          unsupported(f, "Float16 (widening layer must cast to Float32 before Spark)")
         case FloatingPointPrecision.SINGLE => FloatType
         case FloatingPointPrecision.DOUBLE => DoubleType
         case other => unsupported(f, s"FloatingPoint($other)")