Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node WorkerChannel native addon #1394

Draft
wants to merge 82 commits into
base: v3
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
9a9409d
WIP: Node WorkerChannel native addon
jmillan May 2, 2024
1e59102
remove worker-channel from dependencies in package.json
jmillan May 9, 2024
ee18617
provide version
jmillan May 10, 2024
2c80482
add workerChannels to .eslintignore and .prettierignore
jmillan May 10, 2024
f925bfd
move header file to include folder
jmillan May 10, 2024
18971e0
remove blank line
jmillan May 10, 2024
02d95d8
package.json: remove worker:build-lib script
jmillan May 10, 2024
f6c222b
install workerChannel via 'npm ci'
jmillan May 10, 2024
77a67b4
properly build Debug/Release build types
jmillan May 10, 2024
23df194
lint
jmillan May 10, 2024
65ae95e
remove .eslintignore and .prettierignore
jmillan May 10, 2024
5d7ca09
Merge remote-tracking branch 'origin/v3' into node-worker-channel
jmillan May 10, 2024
6ed5f88
add workerChannel to eslint and prettier
jmillan May 10, 2024
fbe2120
fix name for libmediasoup-worker library in linux
jmillan May 10, 2024
6cec438
add *.ts files inside src/ folder
jmillan May 10, 2024
69bc91f
workerChannel: npm-scripts.ts
jmillan May 13, 2024
66261f5
eslint
jmillan May 13, 2024
1007680
npm-scripts: build worker lib before typescript
jmillan May 13, 2024
906d3ba
npm-scripts: remove typo
jmillan May 13, 2024
71d7a75
workerChannel: initialize pointer
jmillan May 14, 2024
20334b7
binding.gyp: use a common path for libmediasoup-worker library
jmillan May 14, 2024
0f8eae8
binding.gyp: use module_root_dir rather than pwd
jmillan May 14, 2024
e0c95f2
Windows: test
jmillan May 15, 2024
8362703
npm-scripts: do not build mediasoup-worker but libmediasoup-worker
jmillan May 15, 2024
e1df701
remove worker died event
jmillan May 15, 2024
82361c0
windows test
jmillan May 15, 2024
c933523
windows test cont.
jmillan May 15, 2024
33a7360
windows test
jmillan May 15, 2024
439874f
workflow remove every OS but windows
jmillan May 15, 2024
d9f6bf7
Windows: test
jmillan May 15, 2024
e331bd4
binding.gyp: remove unneeded dependency
jmillan May 15, 2024
8eadd6b
worker-channel: remove unused dependency
jmillan May 15, 2024
7329f63
npm-scripts: build addon with --verbose option
jmillan May 15, 2024
fc527eb
meson.build: set b_vscrt option to use /MT runtime library for windows
jmillan May 15, 2024
93fe295
windows: test
jmillan May 15, 2024
d1897f5
windows: test, add more needed libraries for windows
jmillan May 15, 2024
3246f84
workerChannel: include uv.h rather than DepLibUV.hpp
jmillan May 15, 2024
99542b8
windows: WIP, do not import libuv
jmillan May 15, 2024
6e2909d
handle feedback
jmillan May 17, 2024
2062749
tests
jmillan May 17, 2024
5f64e7d
add missing include
jmillan May 17, 2024
bc98653
windows: linker option to allow multiple symbols
jmillan May 17, 2024
8c4bc93
format
jmillan May 17, 2024
95b8225
use EnhancedEventEmitter rather than EventEmitter
jmillan May 17, 2024
1fec8b9
be able to provide a specific libmediasoup_worker.a library
jmillan May 17, 2024
8a5ffe7
binding.gyp: use single quotes
jmillan May 17, 2024
7642644
Windows: only add debug symbols on Debug build type
jmillan May 17, 2024
00a2b1a
npm-scripts: use a cons for addon path
jmillan May 17, 2024
5e3b4ad
package.json: fix name and description
jmillan May 17, 2024
8f46779
Revert "Windows: only add debug symbols on Debug build type"
jmillan May 17, 2024
53025e7
make workerChannel a completely separate npm module
jmillan May 20, 2024
4237fc2
make workerChannel a completely separate npm module
jmillan May 20, 2024
54cc2a2
add missing Logger and enhancedEvents files
jmillan May 20, 2024
17bbc6c
Revert "make workerChannel a completely separate npm module"
jmillan May 20, 2024
9c170da
do not use a separate TS or npm project for WorkerChannel
jmillan May 20, 2024
4037cab
lint
jmillan May 20, 2024
1b248e8
npm-scripts: test, --forceExit
jmillan May 20, 2024
f437881
workerChannel: add into node/src
jmillan May 21, 2024
d11d964
WorkerChannel: .gitignore
jmillan May 21, 2024
e22cace
workflow: enable all SO
jmillan May 21, 2024
147b08d
lint
jmillan May 21, 2024
a258ea6
workflow: fix tabs
jmillan May 21, 2024
b892741
clang-format
jmillan May 21, 2024
19d5242
clang-format: typo
jmillan May 21, 2024
738acea
test: remove --forceExit
jmillan May 21, 2024
0c15f42
.gitignore
jmillan May 21, 2024
8902239
worker: format
jmillan May 21, 2024
85784ad
windows: set RuntimeLibrary for Debug and Release
jmillan May 21, 2024
b3044ad
workflow: enable all OS in mediasoup-node
jmillan May 21, 2024
e80a59e
tests: add comment of addon tests
jmillan May 22, 2024
c3e5d67
WIP: prebuild
jmillan May 22, 2024
95d9cd0
lint
jmillan May 22, 2024
cbf9b30
WIP: prebuilt
jmillan May 22, 2024
f83ca80
WIP: prebuilt
jmillan May 22, 2024
376bdcf
WIP: prebuilt
jmillan May 22, 2024
b559e4d
prebuilt working
jmillan May 22, 2024
9f24c50
comment
jmillan May 22, 2024
a003c63
workerChannel: Do not copy the addon binary into node/lib
jmillan May 22, 2024
9fd5dfb
package.json: files, add workerChannel files
jmillan May 22, 2024
c82c605
.gitignore
jmillan May 22, 2024
e469bf2
cleanup
jmillan May 22, 2024
d95db9d
package.json: engines, set a minimum npm version
jmillan May 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/mediasoup-worker-prebuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,4 @@ jobs:
- name: Upload mediasoup-worker prebuilt binary
uses: softprops/action-gh-release@v1
with:
files: worker/prebuild/mediasoup-worker-*.tgz
files: worker/prebuild/libmediasoup-worker-*.tgz
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
# flatc generated files.
/node/src/fbs

## Node WorkerChannel addon.
/node/src/workerChannel/build

## Rust.
/rust/examples-frontend/*/node_modules
/rust/examples-frontend/*/package-lock.json
Expand Down
78 changes: 19 additions & 59 deletions node/src/Channel.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import * as os from 'node:os';
import { Duplex } from 'node:stream';
import { info, warn } from 'node:console';
import * as flatbuffers from 'flatbuffers';
import { WorkerChannel } from './workerChannel/src';
import { Logger } from './Logger';
import { EnhancedEventEmitter } from './enhancedEvents';
import { InvalidStateError } from './errors';
Expand Down Expand Up @@ -35,11 +35,8 @@ export class Channel extends EnhancedEventEmitter {
// Closed flag.
#closed = false;

// Unix Socket instance for sending messages to the worker process.
readonly #producerSocket: Duplex;

// Unix Socket instance for receiving messages to the worker process.
readonly #consumerSocket: Duplex;
// WorkerChannel for sending and receiving messages to/from worker.
readonly #workerChannel: WorkerChannel;

// Next id for messages sent to the worker process.
#nextId = 0;
Expand All @@ -57,23 +54,20 @@ export class Channel extends EnhancedEventEmitter {
* @private
*/
constructor({
producerSocket,
consumerSocket,
workerChannel,
pid,
}: {
producerSocket: any;
consumerSocket: any;
workerChannel: WorkerChannel;
pid: number;
}) {
super();

logger.debug('constructor()');

this.#producerSocket = producerSocket as Duplex;
this.#consumerSocket = consumerSocket as Duplex;
this.#workerChannel = workerChannel;

// Read Channel responses/notifications from the worker.
this.#consumerSocket.on('data', (buffer: Buffer) => {
this.#workerChannel.on('data', (buffer: Buffer) => {
if (!this.#recvBuffer.length) {
this.#recvBuffer = buffer;
} else {
Expand Down Expand Up @@ -176,22 +170,6 @@ export class Channel extends EnhancedEventEmitter {
this.#recvBuffer = this.#recvBuffer.slice(msgStart);
}
});

this.#consumerSocket.on('end', () =>
logger.debug('Consumer Channel ended by the worker process')
);

this.#consumerSocket.on('error', error =>
logger.error(`Consumer Channel error: ${error}`)
);

this.#producerSocket.on('end', () =>
logger.debug('Producer Channel ended by the worker process')
);

this.#producerSocket.on('error', error =>
logger.error(`Producer Channel error: ${error}`)
);
}

/**
Expand All @@ -217,24 +195,6 @@ export class Channel extends EnhancedEventEmitter {
for (const sent of this.#sents.values()) {
sent.close();
}

// Remove event listeners but leave a fake 'error' hander to avoid
// propagation.
this.#consumerSocket.removeAllListeners('end');
this.#consumerSocket.removeAllListeners('error');
this.#consumerSocket.on('error', () => {});

this.#producerSocket.removeAllListeners('end');
this.#producerSocket.removeAllListeners('error');
this.#producerSocket.on('error', () => {});

// Destroy the sockets.
try {
this.#producerSocket.destroy();
} catch (error) {}
try {
this.#consumerSocket.destroy();
} catch (error) {}
}

/**
Expand Down Expand Up @@ -282,14 +242,14 @@ export class Channel extends EnhancedEventEmitter {
notificationOffset
);

// Finalizes the buffer and adds a 4 byte prefix with the size of the buffer.
this.#bufferBuilder.finishSizePrefixed(messageOffset);
// Finalizes the buffer.
this.#bufferBuilder.finish(messageOffset);

// Create a new buffer with this data so multiple contiguous flatbuffers
// do not point to the builder buffer overriding others info.
const buffer = new Uint8Array(this.#bufferBuilder.asUint8Array());
// Take a reference of the inner buffer.
const buffer = this.#bufferBuilder.asUint8Array();

// Clear the buffer builder so it's reused for the next request.
// NOTE: This merely resets the buffer offset, it does not clear the content.
this.#bufferBuilder.clear();

if (buffer.byteLength > MESSAGE_MAX_LEN) {
Expand All @@ -298,7 +258,7 @@ export class Channel extends EnhancedEventEmitter {

try {
// This may throw if closed or remote side ended.
this.#producerSocket.write(buffer, 'binary');
this.#workerChannel.send(buffer);
} catch (error) {
logger.warn(`notify() | sending notification failed: ${error}`);

Expand Down Expand Up @@ -354,22 +314,22 @@ export class Channel extends EnhancedEventEmitter {
requestOffset
);

// Finalizes the buffer and adds a 4 byte prefix with the size of the buffer.
this.#bufferBuilder.finishSizePrefixed(messageOffset);
// Finalizes the buffer.
this.#bufferBuilder.finish(messageOffset);

// Create a new buffer with this data so multiple contiguous flatbuffers
// do not point to the builder buffer overriding others info.
const buffer = new Uint8Array(this.#bufferBuilder.asUint8Array());
// Take a reference of the inner buffer.
const buffer = this.#bufferBuilder.asUint8Array();

// Clear the buffer builder so it's reused for the next request.
// NOTE: This merely resets the buffer offset, it does not clear the content.
this.#bufferBuilder.clear();

if (buffer.byteLength > MESSAGE_MAX_LEN) {
throw new Error(`request too big [method:${Method[method]}]`);
}

// This may throw if closed or remote side ended.
this.#producerSocket.write(buffer, 'binary');
this.#workerChannel.send(buffer);

return new Promise((pResolve, pReject) => {
const sent: Sent = {
Expand Down
Loading
Loading