DEVELOPMENT.md

Installation

Before PR merge, you should use this installation step.

# Install from source
 npm i 'mybigday/transformers.js#merge'

# or prerelease maintained by BRICKS
npm i @fugood/transformers

Basic Polyfills & Fixes

react-native-transformers-example/polyfills.js

Lines 2 to 16 in a0060d1

	import 'text-encoding-polyfill'; // support TextEncoder
	import { Buffer } from 'buffer';
	import XRegExp from 'xregexp';
	global.Buffer = Buffer;
	// replace default RegExp to support unicode
	const nativeRegExp = global.RegExp;
	const newRegExp = (...args) => {
	global.RegExp = nativeRegExp;
	const result = XRegExp(...args);
	global.RegExp = newRegExp;
	return result;
	};
	global.RegExp = newRegExp;

react-native-transformers-example/babel.config.js

Line 4 in b8321a7

'@babel/plugin-proposal-export-namespace-from',

Android

• Add largeHeap to android/app/src/main/AndroidManifest.xml

<application
  android:name=".MainApplication"
  android:label="@string/app_name"
  android:icon="@mipmap/ic_launcher"
  android:roundIcon="@mipmap/ic_launcher_round"
  android:allowBackup="false"
  android:theme="@style/AppTheme"
+  android:largeHeap="true">

Onnxruntime

Version

Should ensure onnxruntime-common and onnxruntime-react-native version are same. You could using resolution (yarn) or overrides (npm) to force them.

For example.

react-native-transformers-example/package.json

Lines 71 to 77 in 16550ba

	"resolutions": {
	"jpeg-js": "^0.4.4",
	"node-fetch": "^2.6.7",
	"xml2js": "^0.5.0",
	"onnxruntime-common": "1.17.0",
	"onnxruntime-react-native": "1.17.0"
	}

ONNX Runtime patch work for LM models

If you use onnxruntime-react-native<1.17.0, you should patch the library.

• Allow the creation of boolean tensors from ByteBuffer (It included in v1.16.1, Required for language model)
• Allows the creation and extraction of zero length tensors (It included in v1.15, Required for LLM KV cache)
• [js/rn] Support create boolean tensor (It included in v1.17.0, Required for language model)
• [js/rn] Support load external data (It included in v1.18.0, Required for large model)

Performance Improvement

Image Process

Way 1: GCanvas

https://github.com/hans00/react-native-transformers-example/tree/gcanvas

May make app unstable. Should more reuse canvas instance

• @flyskywhy/react-native-browser-polyfill
• @flyskywhy/react-native-gcanvas

Known Issue

• The offscreen canvas too small will not get full decoded image data.
• The canvas too large will crash.
• Create too many canvas may cause crash (include refresh app).

Way 2: Skia

Stable, but slightly slower than gcanvas

react-native-transformers-example/polyfills.js

Line 1 in a0060d1

import 'react-native-skia-offscreencanvas/polyfill';