Skip to content

Commit f0b5c49

Browse files
davisusanibarlidavidm
davisusanibar
and
lidavidm
committed
ARROW-15578: [Java][Doc] Document C Data Interface and how to interface with other languages
Document C Data and how to interface with other languages - Java - Python - Java - C++ Closes apache#12794 from davisusanibar/java-tutorial-ARROW-15578 Lead-authored-by: david dali susanibar arce <davi.sarces@gmail.com> Co-authored-by: David Li <li.davidm96@gmail.com> Signed-off-by: David Li <li.davidm96@gmail.com>
1 parent b7f7bad commit f0b5c49

2 files changed

Lines changed: 224 additions & 0 deletions

File tree

‎docs/source/java/cdata.rst‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,223 @@
1+
.. Licensed to the Apache Software Foundation (ASF) under one
2+
.. or more contributor license agreements. See the NOTICE file
3+
.. distributed with this work for additional information
4+
.. regarding copyright ownership. The ASF licenses this file
5+
.. to you under the Apache License, Version 2.0 (the
6+
.. "License"); you may not use this file except in compliance
7+
.. with the License. You may obtain a copy of the License at
8+
9+
.. http://www.apache.org/licenses/LICENSE-2.0
10+
11+
.. Unless required by applicable law or agreed to in writing,
12+
.. software distributed under the License is distributed on an
13+
.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
.. KIND, either express or implied. See the License for the
15+
.. specific language governing permissions and limitations
16+
.. under the License.
17+
18+
================
19+
C Data Interface
20+
================
21+
22+
.. contents::
23+
24+
Arrow supports exchanging data without copying or serialization within the same process
25+
through the :ref:`c-data-interface`, even between different language runtimes.
26+
27+
Java to Python
28+
--------------
29+
30+
Use this guide to implement :doc:`Java to Python <../python/integration/python_java.rst>`
31+
communication using the C Data Interface.
32+
33+
Java to C++
34+
-----------
35+
36+
Example: Share an Int64 array from C++ to Java:
37+
38+
**C++ Side**
39+
40+
Use this guide to :doc:`compile arrow <../developers/cpp/building.rst>` library:
41+
42+
.. code-block:: shell
43+
44+
$ git clone https://github.com/apache/arrow.git
45+
$ cd arrow/cpp
46+
$ mkdir build # from inside the `cpp` subdirectory
47+
$ cd build
48+
$ cmake .. --preset ninja-debug-minimal
49+
$ cmake --build .
50+
$ tree debug/
51+
debug/
52+
├── libarrow.800.0.0.dylib
53+
├── libarrow.800.dylib -> libarrow.800.0.0.dylib
54+
└── libarrow.dylib -> libarrow.800.dylib
55+
56+
Implement a function in CDataCppBridge.h that exports an array via the C Data Interface:
57+
58+
.. code-block:: cpp
59+
60+
#include <iostream>
61+
#include <arrow/api.h>
62+
#include <arrow/c/bridge.h>
63+
64+
void FillInt64Array(const uintptr_t c_schema_ptr, const uintptr_t c_array_ptr) {
65+
arrow::Int64Builder builder;
66+
builder.Append(1);
67+
builder.Append(2);
68+
builder.Append(3);
69+
builder.AppendNull();
70+
builder.Append(5);
71+
builder.Append(6);
72+
builder.Append(7);
73+
builder.Append(8);
74+
builder.Append(9);
75+
builder.Append(10);
76+
std::shared_ptr<arrow::Array> array = *builder.Finish();
77+
78+
struct ArrowSchema* c_schema = reinterpret_cast<struct ArrowSchema*>(c_schema_ptr);
79+
auto c_schema_status = arrow::ExportType(*array->type(), c_schema);
80+
if (!c_schema_status.ok()) c_schema_status.Abort();
81+
82+
struct ArrowArray* c_array = reinterpret_cast<struct ArrowArray*>(c_array_ptr);
83+
auto c_array_status = arrow::ExportArray(*array, c_array);
84+
if (!c_array_status.ok()) c_array_status.Abort();
85+
}
86+
87+
**Java Side**
88+
89+
For this example, we will use `JavaCPP`_ to call our C++ function from Java,
90+
without writing JNI bindings ourselves.
91+
92+
.. code-block:: xml
93+
94+
<?xml version="1.0" encoding="UTF-8"?>
95+
<project xmlns="http://maven.apache.org/POM/4.0.0"
96+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
97+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
98+
<modelVersion>4.0.0</modelVersion>
99+
100+
<groupId>org.example</groupId>
101+
<artifactId>java-cdata-example</artifactId>
102+
<version>1.0-SNAPSHOT</version>
103+
104+
<properties>
105+
<maven.compiler.source>8</maven.compiler.source>
106+
<maven.compiler.target>8</maven.compiler.target>
107+
<arrow.version>8.0.0</arrow.version>
108+
</properties>
109+
<dependencies>
110+
<dependency>
111+
<groupId>org.bytedeco</groupId>
112+
<artifactId>javacpp</artifactId>
113+
<version>1.5.7</version>
114+
</dependency>
115+
<dependency>
116+
<groupId>org.apache.arrow</groupId>
117+
<artifactId>arrow-c-data</artifactId>
118+
<version>${arrow.version}</version>
119+
</dependency>
120+
<dependency>
121+
<groupId>org.apache.arrow</groupId>
122+
<artifactId>arrow-vector</artifactId>
123+
<version>${arrow.version}</version>
124+
</dependency>
125+
<dependency>
126+
<groupId>org.apache.arrow</groupId>
127+
<artifactId>arrow-memory-core</artifactId>
128+
<version>${arrow.version}</version>
129+
</dependency>
130+
<dependency>
131+
<groupId>org.apache.arrow</groupId>
132+
<artifactId>arrow-memory-netty</artifactId>
133+
<version>${arrow.version}</version>
134+
</dependency>
135+
<dependency>
136+
<groupId>org.apache.arrow</groupId>
137+
<artifactId>arrow-format</artifactId>
138+
<version>${arrow.version}</version>
139+
</dependency>
140+
</dependencies>
141+
</project>
142+
143+
.. code-block:: java
144+
145+
import org.bytedeco.javacpp.annotation.Platform;
146+
import org.bytedeco.javacpp.annotation.Properties;
147+
import org.bytedeco.javacpp.tools.InfoMap;
148+
import org.bytedeco.javacpp.tools.InfoMapper;
149+
150+
@Properties(
151+
target = "CDataJavaToCppExample",
152+
value = @Platform(
153+
include = {
154+
"CDataCppBridge.h"
155+
},
156+
compiler = {"cpp11"},
157+
linkpath = {"/arrow/cpp/build/debug/"},
158+
link = {"arrow"}
159+
)
160+
)
161+
public class CDataJavaConfig implements InfoMapper {
162+
163+
@Override
164+
public void map(InfoMap infoMap) {
165+
}
166+
}
167+
168+
.. code-block:: shell
169+
170+
# Compile our Java code
171+
$ javac -cp javacpp-1.5.7.jar CDataJavaConfig.java
172+
173+
# Generate CDataInterfaceLibrary
174+
$ java -jar javacpp-1.5.7.jar CDataJavaConfig.java
175+
176+
# Generate libjniCDataInterfaceLibrary.dylib
177+
$ java -jar javacpp-1.5.7.jar CDataJavaToCppExample.java
178+
179+
# Validate libjniCDataInterfaceLibrary.dylib created
180+
$ otool -L macosx-x86_64/libjniCDataJavaToCppExample.dylib
181+
macosx-x86_64/libjniCDataJavaToCppExample.dylib:
182+
libjniCDataJavaToCppExample.dylib (compatibility version 0.0.0, current version 0.0.0)
183+
@rpath/libarrow.800.dylib (compatibility version 800.0.0, current version 800.0.0)
184+
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0)
185+
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
186+
187+
**Java Test**
188+
189+
Let's create a Java class to test our bridge:
190+
191+
.. code-block:: java
192+
193+
import org.apache.arrow.c.ArrowArray;
194+
import org.apache.arrow.c.ArrowSchema;
195+
import org.apache.arrow.c.Data;
196+
import org.apache.arrow.memory.BufferAllocator;
197+
import org.apache.arrow.memory.RootAllocator;
198+
import org.apache.arrow.vector.BigIntVector;
199+
200+
public class TestCDataInterface {
201+
public static void main(String[] args) {
202+
try(
203+
BufferAllocator allocator = new RootAllocator();
204+
ArrowSchema arrowSchema = ArrowSchema.allocateNew(allocator);
205+
ArrowArray arrowArray = ArrowArray.allocateNew(allocator)
206+
){
207+
CDataJavaToCppExample.FillInt64Array(
208+
arrowSchema.memoryAddress(), arrowArray.memoryAddress());
209+
try(
210+
BigIntVector bigIntVector = (BigIntVector) Data.importVector(
211+
allocator, arrowArray, arrowSchema, null)
212+
){
213+
System.out.println("C++-allocated array: " + bigIntVector);
214+
}
215+
}
216+
}
217+
}
218+
219+
.. code-block:: shell
220+
221+
C++-allocated array: [1, 2, 3, null, 5, 6, 7, 8, 9, 10]
222+
223+
.. _`JavaCPP`: https://github.com/bytedeco/javacpp

‎docs/source/java/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,4 +33,5 @@ on the Arrow format and other language bindings see the :doc:`parent documentati
3333
ipc
3434
algorithm
3535
dataset
36+
cdata
3637
Reference (javadoc) <reference/index>

0 commit comments

Comments
 (0)