You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 5e25af8
Browse filesBrowse the repository at this point in the historyBrowse files
The memory modules contain all the functionality that Arrow uses to manage memory (allocation and deallocation).
22
+
The memory modules contain all the functionality that Arrow uses to allocate and deallocate memory. This document is divided in two parts:
23
+
The first part, *Memory Basics*, provides a high-level introduction. The following section, *Arrow Memory In-Depth*, fills in the details.
24
+
25
+
.. contents::
26
+
27
+
Memory Basics
28
+
=============
23
29
This section will introduce you to the major concepts in Java’s memory management:
24
30
25
31
* `ArrowBuf`_
26
32
* `BufferAllocator`_
27
33
* Reference counting
28
34
29
-
.. contents::
35
+
It also provides some guidelines for working with memory in Arrow, and describes how to debug memory issues when they arise.
30
36
31
37
Getting Started
32
-
===============
38
+
---------------
33
39
34
40
Arrow's memory management is built around the needs of the columnar format and using off-heap memory.
35
-
Also, it is its own independent implementation, and does not wrap the C++ implementation.
41
+
Arrow Java has its own independent implementation. It does not wrap the C++ implementation, although the framework is flexible enough
42
+
to be used with memory allocated in C++ that is used by Java code.
36
43
37
44
Arrow provides multiple modules: the core interfaces, and implementations of the interfaces.
38
45
Users need the core interfaces, and exactly one of the implementations.
@@ -41,25 +48,29 @@ Users need the core interfaces, and exactly one of the implementations.
41
48
* ``memory-netty``: An implementation of the memory interfaces based on the `Netty`_ library.
42
49
* ``memory-unsafe``: An implementation of the memory interfaces based on the `sun.misc.Unsafe`_ library.
43
50
51
+
44
52
ArrowBuf
45
-
========
53
+
--------
46
54
47
55
ArrowBuf represents a single, contiguous region of `direct memory`_. It consists of an address and a length,
48
56
and provides low-level interfaces for working with the contents, similar to ByteBuffer.
49
57
50
58
Unlike (Direct)ByteBuffer, it has reference counting built in, as discussed later.
51
59
52
60
Why Arrow Uses Direct Memory
53
-
----------------------------
61
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
54
62
55
63
* The JVM can optimize I/O operations when using direct memory/direct buffers; it will attempt to avoid copying buffer contents to/from an intermediate buffer. This can speed up IPC in Arrow.
56
64
* Since Arrow always uses direct memory, JNI modules can directly wrap native memory addresses instead of copying data. We use this in modules like the C Data Interface.
57
65
* Conversely, on the C++ side of the JNI boundary, we can directly access the memory in ArrowBuf without copying data.
58
66
59
67
BufferAllocator
60
-
===============
68
+
---------------
61
69
62
-
The `BufferAllocator`_ interface deals with allocating ArrowBufs for the application.
70
+
The `BufferAllocator`_ is primarily an arena or nursery used for accounting of buffers (ArrowBuf instances).
71
+
As the name suggests, it can allocate new buffers associated with itself, but it can also
72
+
handle the accounting for buffers allocated elsewhere. For example, it handles the Java-side accounting for
73
+
memory allocated in C++ and shared with Java using the C-Data Interface. In the code below it performs an allocation:
63
74
64
75
.. code-block:: Java
65
76
@@ -89,51 +100,54 @@ memory from a child allocator, those allocations are also reflected in all paren
89
100
effectively sets the program-wide memory limit, and serves as the master bookkeeper for all memory allocations.
90
101
91
102
Child allocators are not strictly required, but can help better organize code. For instance, a lower memory limit can
92
-
be set for a particular section of code. When the allocator is closed, it then checks that that section didn't leak any
93
-
memory. And child allocators can be named, which makes it easier to tell where an ArrowBuf came from during debugging.
103
+
be set for a particular section of code. The child allocator can be closed when that section completes,
104
+
at which point it checks that that section didn't leak any memory.
105
+
Child allocators can also be named, which makes it easier to tell where an ArrowBuf came from during debugging.
94
106
95
107
Reference counting
96
-
==================
108
+
------------------
97
109
98
-
Direct memory is more expensive to allocate and deallocate. That's why allocators pool or cache direct buffers.
99
-
100
-
Because we want to pool/cache buffers and manage them deterministically, we use manual reference counting instead of
101
-
the garbage collector. This simply means that each buffer has a counter keeping track of the number of references to
110
+
Because direct memory is expensive to allocate and deallocate, allocators may share direct buffers. To managed shared buffers
111
+
deterministically, we use manual reference counting instead of the garbage collector.
112
+
This simply means that each buffer has a counter keeping track of the number of references to
102
113
the buffer, and the user is responsible for properly incrementing/decrementing the counter as the buffer is used.
103
114
104
-
In Arrow, each ArrowBuf has an associated `ReferenceManager`_ that tracks the reference count, which can be retrieved
105
-
with ArrowBuf.getReferenceManager(). The reference count can be updated with `ReferenceManager.release`_ and
106
-
`ReferenceManager.retain`_.
115
+
In Arrow, each ArrowBuf has an associated `ReferenceManager`_ that tracks the reference count. You can retrieve
116
+
it with ArrowBuf.getReferenceManager(). The reference count is updated using `ReferenceManager.release`_ to decrement the count,
117
+
and `ReferenceManager.retain`_ to increment it.
107
118
108
-
Of course, this is tedious and error-prone, so usually, instead of directly working with buffers, we should use
119
+
Of course, this is tedious and error-prone, so instead of directly working with buffers, we typically use
109
120
higher-level APIs like ValueVector. Such classes generally implement Closeable/AutoCloseable and will automatically
110
121
decrement the reference count when closed.
111
122
112
123
Allocators implement AutoCloseable as well. In this case, closing the allocator will check that all buffers
113
124
obtained from the allocator are closed. If not, ``close()`` method will raise an exception; this helps track
114
125
memory leaks from unclosed buffers.
115
126
116
-
As you see, reference counting needs to be handled carefully. To ensure that an
127
+
Reference counting needs to be handled carefully. To ensure that an
117
128
independent section of code has fully cleaned up all allocated buffers, use a new child allocator.
118
129
119
130
Development Guidelines
120
-
======================
131
+
----------------------
121
132
122
133
Applications should generally:
123
134
124
135
* Use the BufferAllocator interface in APIs instead of RootAllocator.
125
136
* Create one RootAllocator at the start of the program.
126
137
* ``close()`` allocators after use (whether they are child allocators or the RootAllocator), either manually or preferably via a try-with-resources statement.
127
138
128
-
Debugging Memory Leaks/Allocation
129
-
=================================
130
139
131
-
Allocators have a debug mode that makes it easier to figure out where a leak is originated.
132
-
To enable it, enable assertions with ``-ea`` or set the system property, ``-Darrow.memory.debug.allocator=true``.
133
-
When enabled, a log will be kept of allocations.
140
+
Debugging Memory Leaks/Allocation
141
+
---------------------------------
134
142
135
-
Arrow logs some allocation information via SLF4J; configure it properly to see these logs (e.g. via Logback/Apache Log4j).
143
+
In ``DEBUG`` mode, the allocator and
144
+
supporting classes will record additional debug tracking information to
145
+
better track down memory leaks and issues. To enable DEBUG mode, either
146
+
enable Java assertions with ``-ea`` or pass the following system
147
+
property to the VM when starting
148
+
``-Darrow.memory.debug.allocator=true``.
136
149
150
+
When DEBUG is enabled, a log will be kept of allocations. Configure SLF4J to see these logs (e.g. via Logback/Apache Log4j).
137
151
Consider the following example to see how it helps us with the tracking of allocators:
138
152
139
153
.. code-block:: Java
@@ -215,6 +229,9 @@ This will include information about allocation operations on the buffer with sta
215
229
at jdk.jshell.execution.Util.forwardExecutionControlAndIO(Util.java:137)
216
230
at jdk.jshell.execution.RemoteExecutionControl.main(RemoteExecutionControl.java:70)
217
231
232
+
The BufferAllocator also provides a ``BufferAllocator.toVerboseString()`` which can be used in
233
+
``DEBUG`` mode to get extensive stacktrace information and events associated with various Allocator behaviors.
234
+
218
235
Finally, enabling the ``TRACE`` logging level will automatically provide this stack trace when the allocator is closed:
219
236
220
237
.. code-block:: java
@@ -276,6 +293,7 @@ Finally, enabling the ``TRACE`` logging level will automatically provide this st
0 commit comments