Skip to content

Commit 334db7c

Browse files
authored
[SEA-NodeJS] (2/8) napi-rs binding consumer — TS loader + build script (#380)
* sea-napi-binding: scaffold native/sea/ crate with version() smoke test Creates the napi-rs binding skeleton: Cargo.toml + lib.rs + module stubs for database/connection/statement/result/error/logger. Captures napi-rs tokio Handle via OnceCell in runtime.rs. Single working #[napi] fn version() proves the binding loads + executes end-to-end in Node. Depends on krn-async-public-api branch (path dep on kernel). Round 2 will add open/execute/fetch methods. Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * sea-napi-binding: Database/Connection/Statement/ResultStream methods wired Adds real async methods on the opaque wrappers backing M0: - openSession (free function) with PAT → kernel Session - Connection::execute_statement → kernel ExecutedStatement - Statement::fetch_next_batch / schema / cancel / close → kernel ResultStream - Arrow batches returned as IPC bytes (per Layer 2 design) - Error mapping preserves kernel ErrorCode + SQLSTATE for TS layer - All entry points wrapped in catch_unwind End-to-end smoke test against pecotesting passes. No new dependencies beyond arrow-{ipc,array,schema} + futures. Uses kernel async public API (no block_on). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * sea-napi-binding: cleanup — drop unused tracing deps; address bloat findings Round 1 scaffold declared tracing + tracing-subscriber as deps but never used them. Removed. Logger bridge will re-add in round 3. Other findings from 6b3affd-2026-05-15.md reviewed: - Finding 2 (Database::Drop unreachable in Round 1b) — obsoleted by Round 2 (40d0b57): database.rs no longer declares a Database struct or Drop impl; it is now an `open_session` free function. - Finding 3 (empty Connection::Drop) — obsoleted by Round 2: the Drop impl now spawns a real fire-and-forget close on the captured tokio handle. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * sea-napi-binding: relocate Rust source to kernel workspace Per D-006 architectural decision (Python team's workspace pattern): all language bindings (PyO3, napi-rs) now live as workspace siblings in the kernel repo at databricks-sql-kernel/{pyo3,napi}/. What this commit removes from the nodejs repo: - native/sea/Cargo.toml (path dep relocated; package now at databricks-sql-kernel/napi/Cargo.toml with path = "..") - native/sea/build.rs - native/sea/src/* (lib, runtime, database, connection, statement, result, error, logger, util — all 9 files) - native/sea/package.json (the @databricks/sea-native-linux-x64-gnu sub-package moves to the kernel workspace too) - native/sea/index.js (regenerated artifact) What stays in nodejs: - native/sea/index.d.ts — TS declarations consumed by lib/sea/ adapter - native/sea/README.md (new) — explains the move; points readers at databricks-sql-kernel/napi/ What's updated: - package.json: `build:native` and `build:native:debug` scripts now delegate to the kernel workspace via $DATABRICKS_SQL_KERNEL_REPO (defaults to ../../databricks-sql-kernel-sea-WT/napi-binding for the local dev worktree layout). Build copies index.node + index.d.ts back into native/sea/ for the loader to find. Why workspace co-location: - Arrow version pinning lockstep — no silent IPC version drift - path = ".." (clean) vs ../../../../databricks-sql-kernel-sea-WT/... - Single CI: cargo build --workspace covers kernel + pyo3 + napi - Kernel API changes that break either binding caught at PR-review time - Future cgo binding for Go SEA slots in as another workspace member This branch (sea-napi-binding) is now a thin consumer of the kernel napi crate. The actual Rust code lives at krn-napi-binding HEAD on the kernel repo (commit debe3d7). Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * sea-napi-binding: build:native uses --platform so index.js router is generated Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * sea-napi-binding: review round 2 — lint, publish, lazy-load, tests Addresses PR #380 review findings C1, C2 (partial), C3, H1, H2, H3, H4, H5, H6, H7, H8 (runtime guard), H9, M1, M2, M3, M4 (linguist), M5, M6, M7, M8, L1, L2, L4. - C1 lint: drop `.js` ext from native/sea require so eslint import/extensions passes; gitignore the auto-generated index.js and *.node artifacts; prettier-ignore the napi-rs auto-generated index.d.ts / index.js. - C2 publish: whitelist native/sea/index.{js,d.ts} in .npmignore; declare @databricks/sea-native-linux-x64-gnu in optionalDependencies. The kernel-side package-name rename (drop the linux-x64-gnu prefix) is tracked separately as a cross-PR ask. - C3 test wiring: move tests/native/version.test.ts -> tests/unit/sea/, tests/native/e2e-smoke.test.ts -> tests/e2e/sea/; both now picked up by the existing mocharc globs and run via the existing `npm test` / `npm run e2e` jobs. - H1 noise drop: version.test.ts now asserts one meaningful semver check; three tautological assertions removed. - H2 + H3 + H7 lazy load: rewrite SeaNativeLoader.ts on the lib/utils/lz4.ts pattern. Lazy require behind getSeaNative(); capability-detection helper tryGetSeaNative(); structured error messages that classify MODULE_NOT_FOUND vs ERR_DLOPEN_FAILED and include platform/arch/Node-version + install hint. - H4 supply chain: pin @napi-rs/cli to 2.18.4 in devDependencies; build:native switches to `npx --no-install` so the pinned local install is used (no per-build network fetch). - H5 path: switch build:native default kernel path to the canonical `../../databricks-sql-kernel/napi-binding` (the worktree-specific `-sea-WT` suffix is gone). - H6 CI safety: e2e suite hard-fails when CI=true and any required env var is missing; dev machines still skip. - H8 runtime guard: loader throws a structured error on Node <18 instead of attempting a dlopen that would fail mysteriously. Driver's engines.node stays >=14 — Thrift consumers on older Node continue to work. - H9 + M1 + M2 types: tsconfig adds a `@sea-native` path alias to native/sea/index.d.ts; loader imports the real Connection / Statement / ConnectionOptions / ExecuteOptions / ArrowBatch / ArrowSchema types and re-exports them. Tests use the re-exported types — the inline-shape duplication across three files collapses. - M3 README: rewrite native/sea/README.md to match kernel reality. The napi crate is a standalone Cargo workspace (not a pyo3 sibling); explain the tls-rustls choice that the standalone workspace exists to enable. - M4 drift detection: mark native/sea/index.d.ts as linguist-generated in .gitattributes so GitHub collapses it in diffs and excludes from blame/language stats. - M5 artifacts: gitignore native/sea/index.js, index.node, index.*.node. - M6 + M7 e2e coverage: decode the IPC payload via apache-arrow's tableFromIPC and assert numRows + cell value (not just shape); add drain-past-null idempotence and schema-before-fetch coverage. - L1 build scripts: collapse build:native + build:native:debug into one script taking BUILD_PROFILE (defaults to --release). - L2 / L4 / M8: drop the version() alias and the "Round 2+ will…" comment debt; the loader now actually delivers the value the prior JSDoc was only promising. Verified on this branch: npm run lint clean (0 errors); npm run prettier clean for PR-owned files (3 unrelated pre-existing warnings on PR #378 territory); tsc --project tsconfig.build.json clean; mocha on tests/unit/sea passes (4/4); mocha on tests/e2e/sea passes against a live pecotesting warehouse (2/2, IPC decode confirms SELECT 1 returns 1). Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * sea-napi-binding: address review — sql-kernel rename, loader class + DI seam, packaging, tests Rebuilds native/sea against the merged kernel main (binding renamed @databricks/sea-native -> @databricks/sql-kernel via kernel #82) and addresses the /full-review findings on PR #380: - C1: commit the napi-rs router (native/sea/index.js, un-ignored) + a prepack assertion so the publish tarball can never ship without it. Companion kernel fix (databricks-sql-kernel#93) corrects the base package name so the router require paths resolve. - C2: drop the @sea-native tsconfig path alias; use a relative import so no unresolvable specifier leaks into the emitted .d.ts. - C3/C11: error hints no longer name a 404-ing package; dlopen hint now includes the underlying dlerror string + concrete remediation. - C4: document M0 = linux-x64-gnu-only scope + npm scope-lock note. - C5: SeaNativeBinding = typeof import('../../native/sea') (no drift). - C6: SeaNativeLoader is now a class with an injectable load seam; getSeaNative/tryGetSeaNative are thin process-global wrappers. - C7: re-exports renamed Sea* to avoid colliding with Thrift types. - C8: version.test.ts fails loud on the linux-x64 CI runner + shape checks; new loader.test.ts covers the hint branches, Node gate, shape check, and caching via the injected seam. - C9: Node-version guard fails closed (NaN or < floor). - C13: e2e smoke uses the shared tests/e2e/utils/config.ts creds. - C10/C12 are resolved upstream in merged kernel main / deferred to M1. index.d.ts regenerated from merged main: ExecuteOptions dropped (catalog/schema/sessionConf are session-level on openSession), close() awaits DeleteSession, schema() is sync, sessionId/statementId getters added. Verified: 11 unit tests; e2e SELECT 1 against a live warehouse, direct and through mitmproxy (SEA REST: POST /sql/sessions -> POST /sql/statements -> DELETE /sql/sessions/<id>). Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * fix(sea): sync package-lock for napi-rs/cli + skip native smoke test until binding ships CI was red on every job at `npm ci` (EUSAGE: "Missing @napi-rs/cli@2.18.4 from lock file") — package.json added @napi-rs/cli (build:native devDep) and the @databricks/sql-kernel-linux-x64-gnu optional dep, but package-lock.json was never regenerated. Updated the lock (the unpublished optional dep is recorded as optional, so `npm ci` tolerates/skips it). Also: the version smoke test fail-louded when the binding was absent on a linux-x64 CI runner, assuming the optional dep installs there. It doesn't yet — the @databricks/sql-kernel-* packages aren't published and the standard CI doesn't run build:native — so the fail-loud was spurious. Gate it behind SEA_NATIVE_EXPECTED=1 (set once a CI step provisions the binding); default to skip. The e2e smoke test already skips when the binding is absent. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * fix(sea): drop the unpublished sql-kernel optional dep (unblocks npm ci) `@databricks/sql-kernel-linux-x64-gnu@0.1.0` is not published yet, so the lockfile recorded it with no resolved version and `npm ci` rejected the mismatch ("lock file's @databricks/sql-kernel-linux-x64-gnu@undefined does not satisfy @0.1.0") — npm ci does NOT silently skip a pinned-but- unresolvable optional dependency. Remove it from optionalDependencies until the per-triple binding packages are actually published; the loader already handles an absent binding (and version.test skips unless SEA_NATIVE_EXPECTED is set). Re-add the optional deps once the `@databricks/sql-kernel-*` packages ship. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * test(sea): decouple loader tests from the runner's Node version The unit-test matrix spans Node 14–20, but SeaNativeLoader's version gate read the live `process.version` before the injected `load` seam ran. On the 14 and 16 runners every "successful load" / "load-failure hints" / "shape check" test hit `requires Node >=18` instead of the path it asserted (7 failures per job). Make Node-major detection a second injectable ctor seam (default = live process.version), inject a supported major in the load-path tests, and inject the version-under-test in the gate's own tests (no more mutating process.version). Tests now pass identically on every matrix Node. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> * fix(sea): make @napi-rs/cli optional + prettier-format loader test Two CI failures, both unrelated to driver behavior: 1. unit-test/lint/e2e intermittently failed at `npm ci` with ECONNRESET fetching @napi-rs/cli@2.18.4 from the internal npm proxy (cold-cache matrix jobs; the warm-cache node-14 job kept passing). @napi-rs/cli is only used by `build:native`, which CI never runs. Move it from devDependencies to optionalDependencies — matching the existing lz4 pattern — so a proxy hiccup is a non-fatal skip instead of failing the whole install. `build:native`'s `npx --no-install` still resolves it on dev machines where the optional install succeeds. 2. prettier flagged loader.test.ts: the two-arg `new SeaNativeLoader(cb, fn)` calls needed multi-line formatting. Ran prettier --write. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com> --------- Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
1 parent a3bf320 commit 334db7c

14 files changed

Lines changed: 1286 additions & 2 deletions

.gitattributes

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,10 @@ Dockerfile* text
3131
#
3232
.gitattributes export-ignore
3333
.gitignore export-ignore
34+
35+
# napi-rs auto-generates these files from the kernel's `napi-binding/napi/`
36+
# crate; regenerated by `npm run build:native`. Tell git/GitHub they're
37+
# machine-generated so they collapse in diffs and are excluded from
38+
# blame and language stats.
39+
native/sea/index.d.ts linguist-generated=true
40+
native/sea/index.js linguist-generated=true

.gitignore

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,12 @@ coverage_unit
1010
dist
1111
*.DS_Store
1212
lib/version.ts
13+
14+
# SEA native binding — copied/generated from kernel workspace by `npm run build:native`.
15+
# The committed contract is `native/sea/index.d.ts` (TypeScript declarations) and
16+
# `native/sea/index.js` (the napi-rs platform router — small, stable, and required in
17+
# the publish tarball so a missing build step can't ship a tarball that can't load).
18+
# The `.node` binaries are large per-platform artifacts and must NOT be committed;
19+
# in production they arrive via the `@databricks/sql-kernel-<triple>` optional deps.
20+
native/sea/index.node
21+
native/sea/index.*.node

.npmignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,13 @@
33
!dist/**/*
44
!thrift/**/*
55

6+
# SEA napi-rs router shim + TypeScript declarations. The router (index.js)
7+
# selects the per-platform `.node` artifact from `@databricks/sql-kernel-*`
8+
# optionalDependencies (populated when the kernel CI publishes them);
9+
# the .d.ts is the consumer-facing type contract.
10+
!native/sea/index.js
11+
!native/sea/index.d.ts
12+
613
!LICENSE
714
!NOTICE
815
!package.json

.prettierignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,9 @@ coverage
1111
dist
1212
thrift
1313
package-lock.json
14+
15+
# Generated by napi-rs from the kernel's `napi-binding/napi/` crate;
16+
# regenerated by `npm run build:native`. Format follows napi-rs's
17+
# defaults (no semicolons), not this repo's prettier config.
18+
native/sea/index.d.ts
19+
native/sea/index.js

lib/sea/SeaNativeLoader.ts

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
// Copyright (c) 2026 Databricks, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
/**
16+
* Lazy loader for the SEA (Statement Execution API) native binding.
17+
*
18+
* Mirrors the load-failure-tolerant pattern of `lib/utils/lz4.ts`: the
19+
* `.node` artifact ships via per-platform optional dependencies
20+
* (`@databricks/sql-kernel-<triple>`), so its absence must not crash
21+
* a Thrift-only consumer of the driver. Callers that actually need
22+
* SEA construct a {@link SeaNativeLoader} (or use the process-global
23+
* {@link getSeaNative}) which throws a structured error if the binding
24+
* could not be loaded.
25+
*
26+
* M0 publishes a single triple (`linux-x64-gnu`); see
27+
* `native/sea/README.md` for the supported-platform policy.
28+
*/
29+
30+
import type {
31+
Connection as NativeConnection,
32+
Statement as NativeStatement,
33+
ConnectionOptions as NativeConnectionOptions,
34+
ArrowBatch as NativeArrowBatch,
35+
ArrowSchema as NativeArrowSchema,
36+
} from '../../native/sea';
37+
38+
// SEA-prefixed re-exports. The kernel-generated `.d.ts` keeps the
39+
// napi-rs default names (`ConnectionOptions`, `ArrowBatch`, …); we
40+
// disambiguate on the TS-wrapper side so these never collide with the
41+
// Thrift-side `ConnectionOptions` (lib/contracts/IDBSQLClient.ts) or
42+
// `ArrowBatch` (lib/result/utils.ts) when imported elsewhere.
43+
export type SeaConnectionOptions = NativeConnectionOptions;
44+
export type SeaArrowBatch = NativeArrowBatch;
45+
export type SeaArrowSchema = NativeArrowSchema;
46+
export type SeaConnection = NativeConnection;
47+
export type SeaStatement = NativeStatement;
48+
49+
/**
50+
* The full native binding surface, derived from the generated module
51+
* so it can never drift from the `.d.ts` contract: when the kernel
52+
* adds or renames a free function / class, this type follows
53+
* automatically and `defaultRequire`'s cast stays correct.
54+
*/
55+
export type SeaNativeBinding = typeof import('../../native/sea');
56+
57+
const MIN_NODE_MAJOR = 18;
58+
59+
function detectNodeMajor(): number {
60+
// `process.version` is `vX.Y.Z`; parseInt stops at the first non-digit.
61+
return parseInt(process.version.slice(1), 10);
62+
}
63+
64+
function platformLabel(): string {
65+
return `${process.platform}-${process.arch}`;
66+
}
67+
68+
function loadFailureHint(err: NodeJS.ErrnoException): string {
69+
const platform = platformLabel();
70+
// Do not name a concrete package: the published name uses the napi-rs
71+
// triple (e.g. `-linux-x64-gnu` / `-linux-x64-musl` / `-win32-x64-msvc`),
72+
// not the bare `${platform}` shown here, so a literal example would
73+
// 404. Point at the README's supported-triple list instead.
74+
const installHint =
75+
'Install the matching @databricks/sql-kernel-* optional dependency for your platform ' +
76+
'(see native/sea/README.md for the supported triples; M0 ships linux-x64-gnu only).';
77+
if (err.code === 'MODULE_NOT_FOUND') {
78+
return `SEA native binding not installed for platform ${platform} on Node ${process.version}. ${installHint}`;
79+
}
80+
if (err.code === 'ERR_DLOPEN_FAILED') {
81+
// Surface the underlying dlerror string (e.g. `GLIBC_2.32 not found`)
82+
// plus concrete remediation — without it the cause is invisible.
83+
return (
84+
`SEA native binding present but failed to dlopen on platform ${platform} / Node ${process.version}: ` +
85+
`${err.message}. Common causes: glibc/musl mismatch (e.g. Alpine Linux — install the -musl variant), ` +
86+
`Node ABI mismatch (try \`rm -rf node_modules && npm install\`), or CPU-architecture mismatch. ` +
87+
`The binding requires Node >=${MIN_NODE_MAJOR}.`
88+
);
89+
}
90+
return `SEA native binding failed to load on platform ${platform} / Node ${process.version}: ${err.message}`;
91+
}
92+
93+
/**
94+
* Default loader: resolves `native/sea/index.js` (the napi-rs router),
95+
* which selects the per-platform `.node`. `.js` is omitted so eslint's
96+
* `import/extensions` rule accepts the call.
97+
*/
98+
function defaultRequire(): SeaNativeBinding {
99+
// eslint-disable-next-line @typescript-eslint/no-var-requires, global-require
100+
return require('../../native/sea') as SeaNativeBinding;
101+
}
102+
103+
/**
104+
* Verify the loaded module exposes the surface the driver depends on.
105+
* Catches kernel-side renames at load time rather than letting them
106+
* surface as `undefined is not a function` deep in a call path.
107+
*/
108+
function assertBindingShape(binding: SeaNativeBinding): void {
109+
const missing: string[] = [];
110+
if (typeof binding.version !== 'function') missing.push('version');
111+
if (typeof binding.openSession !== 'function') missing.push('openSession');
112+
if (typeof binding.Connection !== 'function') missing.push('Connection');
113+
if (typeof binding.Statement !== 'function') missing.push('Statement');
114+
if (missing.length > 0) {
115+
throw new Error(
116+
`SEA native binding loaded but is missing expected export(s): ${missing.join(', ')}. ` +
117+
`The kernel-generated binding and the JS loader are out of sync.`,
118+
);
119+
}
120+
}
121+
122+
/**
123+
* Loads and caches the SEA native binding. Exposed as a class with an
124+
* injectable `load` seam so consumers (e.g. `SeaBackend`) can be unit
125+
* tested with a stub binding instead of requiring a real `.node` on the
126+
* test machine. Most production code uses the process-global default
127+
* via {@link getSeaNative} / {@link tryGetSeaNative}.
128+
*/
129+
export class SeaNativeLoader {
130+
private cached: SeaNativeBinding | null | undefined;
131+
132+
private cachedError: Error | undefined;
133+
134+
/**
135+
* @param load injectable module-require seam (stub a binding in tests)
136+
* @param nodeMajor injectable Node-major detector. Defaults to reading the
137+
* live `process.version`; injected in unit tests so the
138+
* load/shape branches are exercised independently of the
139+
* runner's actual Node version (the matrix spans 14–20).
140+
*/
141+
constructor(
142+
private readonly load: () => SeaNativeBinding = defaultRequire,
143+
private readonly nodeMajor: () => number = detectNodeMajor,
144+
) {}
145+
146+
private tryLoad(): SeaNativeBinding | undefined {
147+
const nodeMajor = this.nodeMajor();
148+
// Fail closed: if we cannot determine the Node major (NaN) or it is
149+
// below the floor, refuse the load and fall back to Thrift.
150+
if (!Number.isFinite(nodeMajor) || nodeMajor < MIN_NODE_MAJOR) {
151+
this.cachedError = new Error(
152+
`SEA native binding requires Node >=${MIN_NODE_MAJOR}; running Node ${process.version}. ` +
153+
`Continue using the Thrift backend on this runtime.`,
154+
);
155+
return undefined;
156+
}
157+
158+
try {
159+
const binding = this.load();
160+
assertBindingShape(binding);
161+
return binding;
162+
} catch (err) {
163+
if (err instanceof Error && 'code' in err) {
164+
this.cachedError = new Error(loadFailureHint(err as NodeJS.ErrnoException));
165+
} else if (err instanceof Error) {
166+
// Shape-check failure or any other Error — preserve its message.
167+
this.cachedError = err;
168+
} else {
169+
this.cachedError = new Error(`SEA native binding failed to load with non-standard error: ${String(err)}`);
170+
}
171+
return undefined;
172+
}
173+
}
174+
175+
/**
176+
* Returns the loaded native binding. Throws a structured error if the
177+
* binding is unavailable on this platform / Node version.
178+
*/
179+
get(): SeaNativeBinding {
180+
if (this.cached === undefined) {
181+
this.cached = this.tryLoad() ?? null;
182+
}
183+
if (this.cached === null) {
184+
throw this.cachedError ?? new Error('SEA native binding unavailable');
185+
}
186+
return this.cached;
187+
}
188+
189+
/**
190+
* Returns the loaded binding or `undefined` if it could not be
191+
* loaded. Use this for capability-detection at startup; use
192+
* {@link get} at the point where SEA is actually required.
193+
*/
194+
tryGet(): SeaNativeBinding | undefined {
195+
if (this.cached === undefined) {
196+
this.cached = this.tryLoad() ?? null;
197+
}
198+
return this.cached ?? undefined;
199+
}
200+
}
201+
202+
// Process-global default instance + thin convenience wrappers.
203+
const defaultLoader = new SeaNativeLoader();
204+
205+
/**
206+
* Returns the loaded native binding from the process-global loader.
207+
* Throws a structured error if the binding is unavailable.
208+
*/
209+
export function getSeaNative(): SeaNativeBinding {
210+
return defaultLoader.get();
211+
}
212+
213+
/**
214+
* Returns the loaded binding from the process-global loader, or
215+
* `undefined` if it could not be loaded.
216+
*/
217+
export function tryGetSeaNative(): SeaNativeBinding | undefined {
218+
return defaultLoader.tryGet();
219+
}

native/sea/README.md

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# `native/sea/` — consumer-side directory for the Rust napi binding
2+
3+
**The Rust binding source lives in the kernel repo** at
4+
`databricks-sql-kernel/napi/`. Building it requires a local checkout
5+
of that repo — see "Build for local dev" below. The published npm
6+
package is `@databricks/sql-kernel-<triple>`.
7+
8+
## Workspace topology
9+
10+
The napi crate is a **standalone Cargo workspace** (`[workspace]
11+
members = ["."]` in `napi/Cargo.toml`), **not** a sibling of `pyo3/`
12+
in the kernel root workspace.
13+
14+
The reason is Cargo feature unification. pyo3 builds the kernel with
15+
the default `tls-native` feature (system OpenSSL via `native-tls`).
16+
The napi crate has to opt INTO `tls-rustls` instead: napi modules are
17+
loaded into Node.js processes that statically link OpenSSL 3.x, and
18+
dynamically linking the system's OpenSSL 1.1 (which `native-tls`
19+
pulls in on Linux) collides with Node's symbols at module-load time
20+
and segfaults the process before any Rust code runs. `rustls` is
21+
pure Rust + `ring` and avoids the conflict entirely.
22+
23+
If napi lived in the same workspace as pyo3, `cargo build
24+
--workspace` would unify the kernel's feature set to `tls-native ∪
25+
tls-rustls`, link both TLS stacks into the resulting napi cdylib,
26+
and reintroduce the same clash. Standalone-workspace is the fix.
27+
28+
## What lives in this directory
29+
30+
- `index.d.ts` — TypeScript declarations consumed by `lib/sea/`.
31+
Generated by napi-rs from the Rust source; checked in as the
32+
consumer-facing type contract.
33+
- `index.js` — napi-rs's per-platform router shim. Gitignored;
34+
populated by `npm run build:native` for local dev. In published
35+
tarballs it ships alongside the `.d.ts` and `require()`s the
36+
right `@databricks/sql-kernel-<triple>` optional dependency.
37+
- `index.*.node` — the actual native binary, one per platform.
38+
Gitignored. In production these live in the per-triple optional
39+
dependencies (`@databricks/sql-kernel-linux-x64-gnu`, etc.); for
40+
local dev `npm run build:native` copies one into this directory.
41+
42+
## Build for local dev
43+
44+
```bash
45+
# From the nodejs repo root:
46+
export DATABRICKS_SQL_KERNEL_REPO=/path/to/your/databricks-sql-kernel
47+
npm run build:native # release build (default)
48+
BUILD_PROFILE= npm run build:native # debug build (empty BUILD_PROFILE drops --release)
49+
```
50+
51+
`DATABRICKS_SQL_KERNEL_REPO` points at the kernel repo root (the
52+
directory containing `napi/`) and is required when your kernel
53+
checkout isn't at `../../databricks-sql-kernel` relative to the
54+
nodejs repo.
55+
56+
## Production load path
57+
58+
At release time the kernel's CI publishes
59+
`@databricks/sql-kernel-<triple>` npm packages — one per supported
60+
platform — each containing a single `.node` binary. The nodejs
61+
driver lists them as `optionalDependencies`; npm installs only the
62+
one matching the consumer's `process.platform` / `process.arch`.
63+
`native/sea/index.js` (the napi-rs router) then `require()`s the
64+
installed package at load time.
65+
66+
## Supported platforms (M0)
67+
68+
M0 publishes a **single** triple: **`linux-x64-gnu`** (package
69+
`@databricks/sql-kernel-linux-x64-gnu`). It is the only entry in the
70+
driver's `optionalDependencies`.
71+
72+
On every other platform (macOS, Windows, linux-arm64, linux-x64-musl
73+
/ Alpine, …) the SEA binding is simply absent: `SeaNativeLoader`
74+
returns `undefined` from `tryGet()` / throws a structured
75+
`MODULE_NOT_FOUND` hint from `get()`, and the driver continues to use
76+
the Thrift backend exclusively. This is expected, not a regression —
77+
additional triples are added to `optionalDependencies` as the kernel
78+
CI starts publishing them in later milestones.
79+
80+
## Supply-chain note
81+
82+
The unpublished triple names (`@databricks/sql-kernel-darwin-arm64`,
83+
`…-win32-x64-msvc`, etc.) referenced by the router are **not**
84+
squat-able: `@databricks` is a Databricks-owned npm scope, and npm
85+
only allows org members to publish under a scope it owns. A third
86+
party therefore cannot register `@databricks/sql-kernel-*` and have
87+
the router autoload it. No placeholder packages are required.

0 commit comments

Comments
 (0)