Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add new ArrayBuffer APIs (copy(), wrap(), allocate(), toData()) #463

Merged
merged 11 commits into from
Jan 7, 2025
161 changes: 133 additions & 28 deletions docs/docs/types/array-buffers.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,44 +44,149 @@ It is important to understand the ownership, and threading concerns around such

## Ownership

- An `ArrayBuffer` that was created on the native side is **owning**, which means you can safely access it's data as long as the `ArrayBuffer` reference is alive.
There's two types of `ArrayBuffer`s, **owning** and **non-owning**:

### Owning

An `ArrayBuffer` that was created on the native side is **owning**, which means you can safely access it's data as long as the `ArrayBuffer` reference is alive.
It can be safely held strong for longer, e.g. as a class property/member, and accessed from different Threads.

```swift
func doSomething() -> ArrayBufferHolder {
let buffer = ArrayBufferHolder.allocate(1024 * 10)
let data = buffer.data // <-- ✅ safe to do because we own it!
self.buffer = buffer // <-- ✅ safe to do to use it later!
DispatchQueue.global().async {
let data = buffer.data // <-- ✅ also safe because we own it!
}
return buffer
```swift
func doSomething() -> ArrayBufferHolder {
// highlight-next-line
let buffer = ArrayBufferHolder.allocate(1024 * 10)
let data = buffer.data // <-- ✅ safe to do because we own it!
self.buffer = buffer // <-- ✅ safe to use it later!
DispatchQueue.global().async {
let data = buffer.data // <-- ✅ also safe because we own it!
}
```
return buffer
}
```

### Non-owning

- An `ArrayBuffer` that was received as a parameter from JS cannot be safely kept strong as the JS VM can delete it at any point, hence it is **non-owning**.
An `ArrayBuffer` that was received as a parameter from JS cannot be safely kept strong as the JS VM can delete it at any point, hence it is **non-owning**.
It's data can only be safely accessed before the synchronous function returned, as this will stay within the JS bounds.

```swift
func doSomething(buffer: ArrayBufferHolder) {
let data = buffer.data // <-- ✅ safe to do because we're still sync
DispatchQueue.global().async {
// code-error
let data = buffer.data // <-- ❌ NOT safe
}
```swift
func doSomething(buffer: ArrayBufferHolder) {
let data = buffer.data // <-- ✅ safe to do because we're still sync
DispatchQueue.global().async {
// code-error
let data = buffer.data // <-- ❌ NOT safe
}
```
If you need a non-owning buffer's data for longer, **copy it first**:
```swift
func doSomething(buffer: ArrayBufferHolder) {
let copy = ArrayBufferHolder.copy(of: buffer)
DispatchQueue.global().async {
let data = copy.data // <-- ✅ safe now because we have a owning copy
}
}
```
If you need a non-owning buffer's data for longer, **copy it first**:
```swift
func doSomething(buffer: ArrayBufferHolder) {
// diff-add
let copy = ArrayBufferHolder.copy(of: buffer)
let data = copy.data // <-- ✅ safe now because we have a owning copy
DispatchQueue.global().async {
let data = copy.data // <-- ✅ still safe now because we have a owning copy
}
```
}
```

## Threading

An `ArrayBuffer` can be accessed from both JS and native, and even from multiple Threads at once, but they are **not thread-safe**.
To prevent race conditions or garbage-data from being read, make sure to not read from- and write to- the `ArrayBuffer` at the same time.

## Creating Buffers

Buffers can either be created from native (**owning**), or from JS (**non-owning**).

### From native

On the native side, an **owning** `ArrayBuffer` can either **wrap-**, or **copy-** an existing buffer:

<Tabs>
<TabItem value="cpp" label="C++">
```cpp
auto myData = new uint8_t*[4096];

// wrap (no copy)
auto wrappingArrayBuffer = ArrayBuffer::wrap(myData, 4096, [=]() {
delete[] myData;
});
// copy
auto copiedArrayBuffer = ArrayBuffer::copy(myData, 4096);
// new blank buffer
auto newArrayBuffer = ArrayBuffer::allocate(4096);
```
</TabItem>
<TabItem value="swift" label="Swift">
```swift
let myData = UnsafeMutablePointer<UInt8>.allocate(capacity: 4096)

// wrap (no copy)
let wrappingArrayBuffer = ArrayBuffer.wrap(dataWithoutCopy: myData,
size: 4096,
onDelete: { myData.deallocate() })
// copy
let copiedArrayBuffer = ArrayBuffer.copy(of: wrappingArrayBuffer)
// new blank buffer
let newArrayBuffer = ArrayBuffer.allocate(size: 4096)
```
</TabItem>
<TabItem value="kotlin" label="Kotlin">
```kotlin
val myData = ByteBuffer.allocateDirect(4096)

// wrap (no copy)
val wrappingArrayBuffer = ArrayBuffer.wrap(myData)


// copy
let copiedArrayBuffer = ArrayBuffer.copy(myData)
// new blank buffer
val newArrayBuffer = ArrayBuffer.allocate(4096)
```
</TabItem>
</Tabs>

#### Language-native buffer types

ArrayBuffers also provide helper and conversion methods for the language-native conventional buffer types:

<Tabs>
<TabItem value="cpp" label="C++">
C++ often uses [`std::vector<uint8_t>`](https://en.cppreference.com/w/cpp/container/vector) to represent Data.
```cpp
std::vector<uint8_t> data;
auto buffer = ArrayBuffer::copy(data);
/* convert back to vector would be a copy. */
```
</TabItem>
<TabItem value="swift" label="Swift">
Swift often uses [`Data`](https://developer.apple.com/documentation/foundation/data) to represent Data.
```swift
let data = Data(capacity: 1024)
let buffer = ArrayBufferHolder.copy(data: data)
let dataAgain = buffer.toData(copyIfNeeded: true)
```
</TabItem>
<TabItem value="kotlin" label="Kotlin">
Kotlin often uses [`ByteBuffer`](https://developer.android.com/reference/java/nio/ByteBuffer) to represent Data.
```kotlin
val data = ByteBuffer.allocateDirect(1024)
val buffer = ArrayBuffer.copy(data)
val dataAgain = buffer.getBuffer(copyIfNeeded = true)
```
</TabItem>
</Tabs>

### From JS

From JS, a **non-owning** `ArrayBuffer` can be created via the [`ArrayBuffer`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer) web APIs, and viewed or edited using the typed array APIs (e.g. [`Uint8Array`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array)).

```ts
const arrayBuffer = new ArrayBuffer(4096)
const view = new Uint8Array(arrayBuffer)
view[0] = 64
view[1] = 128
view[2] = 255
```
Original file line number Diff line number Diff line change
Expand Up @@ -262,9 +262,8 @@ class HybridTestObjectSwift : HybridTestObjectSwiftKotlinSpec {
}

func getBufferLastItem(buffer: ArrayBufferHolder) throws -> Double {
let lastBytePointer = buffer.data.advanced(by: buffer.size - 1)
let lastByte = lastBytePointer.load(as: UInt8.self)
return Double(lastByte)
let lastByte = buffer.data.advanced(by: buffer.size - 1)
return Double(lastByte.pointee)
}

func setAllValuesTo(buffer: ArrayBufferHolder, value: Double) throws {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,19 +107,40 @@ class ArrayBuffer {
/**
* Copy the given `ArrayBuffer` into a new **owning** `ArrayBuffer`.
*/
fun copyOf(other: ArrayBuffer): ArrayBuffer {
// 1. Create a new buffer with the same size as the other
val newBuffer = ByteBuffer.allocateDirect(other.size)
// 2. Prepare the source buffer
val originalBuffer = other.getBuffer(false)
originalBuffer.rewind()
fun copy(other: ArrayBuffer): ArrayBuffer {
val byteBuffer = other.getBuffer(false)
return copy(byteBuffer)
}

/**
* Copy the given `ByteBuffer` into a new **owning** `ArrayBuffer`.
*/
fun copy(byteBuffer: ByteBuffer): ArrayBuffer {
// 1. Find out size
byteBuffer.rewind()
val size = byteBuffer.remaining()
// 2. Create a new buffer with the same size as the other
val newBuffer = ByteBuffer.allocateDirect(size)
// 3. Copy over the source buffer into the new buffer
newBuffer.put(originalBuffer)
newBuffer.put(byteBuffer)
// 4. Rewind both buffers again to index 0
newBuffer.rewind()
originalBuffer.rewind()
byteBuffer.rewind()
// 5. Create a new `ArrayBuffer`
return ArrayBuffer(newBuffer)
}

/**
* Wrap the given `ByteBuffer` in a new **owning** `ArrayBuffer`.
*/
fun wrap(byteBuffer: ByteBuffer): ArrayBuffer {
byteBuffer.rewind()
return ArrayBuffer(byteBuffer)
}

@Deprecated("Use copy(...) instead", level = DeprecationLevel.WARNING)
fun copyOf(other: ArrayBuffer): ArrayBuffer {
return copy(other)
}
}
}
17 changes: 16 additions & 1 deletion packages/react-native-nitro-modules/cpp/core/ArrayBuffer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,25 @@ using namespace facebook;

// 1. ArrayBuffer

std::shared_ptr<ArrayBuffer> ArrayBuffer::makeBuffer(uint8_t* data, size_t size, DeleteFn&& deleteFunc) {
std::shared_ptr<ArrayBuffer> ArrayBuffer::wrap(uint8_t* data, size_t size, DeleteFn&& deleteFunc) {
return std::make_shared<NativeArrayBuffer>(data, size, std::move(deleteFunc));
}

std::shared_ptr<ArrayBuffer> ArrayBuffer::copy(uint8_t* data, size_t size) {
uint8_t* copy = new uint8_t[size];
std::memcpy(copy, data, size);
return ArrayBuffer::wrap(copy, size, [=]() { delete[] copy; });
}

std::shared_ptr<ArrayBuffer> ArrayBuffer::copy(std::vector<uint8_t>& data) {
return ArrayBuffer::copy(data.data(), data.size());
}

std::shared_ptr<ArrayBuffer> ArrayBuffer::allocate(size_t size) {
uint8_t* data = new uint8_t[size];
return ArrayBuffer::wrap(data, size, [=]() { delete[] data; });
}

// 2. NativeArrayBuffer

NativeArrayBuffer::NativeArrayBuffer(uint8_t* data, size_t size, DeleteFn&& deleteFunc)
Expand Down
21 changes: 20 additions & 1 deletion packages/react-native-nitro-modules/cpp/core/ArrayBuffer.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#include "OwningReference.hpp"
#include <jsi/jsi.h>
#include <thread>
#include <vector>

namespace margelo::nitro {

Expand Down Expand Up @@ -52,7 +53,25 @@ class ArrayBuffer : public jsi::MutableBuffer {
* Create a new `NativeArrayBuffer` that wraps the given data (without copy) of the given size,
* and calls `deleteFunc` in which `data` should be deleted.
*/
static std::shared_ptr<ArrayBuffer> makeBuffer(uint8_t* data, size_t size, DeleteFn&& deleteFunc);
static std::shared_ptr<ArrayBuffer> wrap(uint8_t* data, size_t size, DeleteFn&& deleteFunc);
/**
* Create a new `NativeArrayBuffer` that copies the given data of the given size
* into a newly allocated buffer.
*/
static std::shared_ptr<ArrayBuffer> copy(uint8_t* data, size_t size);
/**
* Create a new `NativeArrayBuffer` that copies the given `std::vector`.
*/
static std::shared_ptr<ArrayBuffer> copy(std::vector<uint8_t>& data);
/**
* Create a new `NativeArrayBuffer` that allocates a new buffer of the given size.
*/
static std::shared_ptr<ArrayBuffer> allocate(size_t size);

[[deprecated("Use wrapBuffer(...) instead.")]]
static std::shared_ptr<ArrayBuffer> makeBuffer(uint8_t* data, size_t size, DeleteFn&& deleteFunc) {
return ArrayBuffer::wrap(data, size, std::move(deleteFunc));
}
};

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,17 @@ class ArrayBufferHolder {
* Once the `ArrayBuffer` is no longer in use, the given `deleteFunc` will be called with the given `deleteFuncContext`
* as an argument. The caller is responsible for deleting `data` once this is called.
*/
static ArrayBufferHolder makeBuffer(uint8_t* _Nonnull data, size_t size, SwiftClosure destroy) {
static ArrayBufferHolder wrap(uint8_t* _Nonnull data, size_t size, SwiftClosure destroy) {
std::function<void()> deleteFunc = destroy.getFunction();
auto arrayBuffer = ArrayBuffer::makeBuffer(data, size, std::move(deleteFunc));
auto arrayBuffer = ArrayBuffer::wrap(data, size, std::move(deleteFunc));
return ArrayBufferHolder(arrayBuffer);
}

public:
/**
* Gets the raw bytes the underlying `ArrayBuffer` points to.
*/
void* _Nonnull getData() const SWIFT_COMPUTED_PROPERTY {
uint8_t* _Nonnull getData() const SWIFT_COMPUTED_PROPERTY {
return _arrayBuffer->data();
}
/**
Expand Down
Loading
Loading