feat: Javascript interface and implementation of API based upon RFC 0…

…01 (#2)

* chore: ignore DS_Store files

* build: create base navdata-sdk module

* build: fix build with navdata-sdk

* build: include navdata-sdk in tsconfig

* refactor: add msfstypes to ts library

* wip(feat/restructure): centralized function call, raw sql query

heavy wip

* refactor: clean up queue processing

* refactor: move lint and lint to root

* refactor: centralize communications between wasm and js, and cleanups

also improves performance for deleting existing downloaded package

* refactor: apply clippy changes

* refactor: rename Request to Task

to not confuse with network requests

* refactor: download status update flow

* refactor: move phase matching logic

* refactor: add typescript interface typings

* refactor: use string values for NavigraphFunction enum

* refactor: use serde for all commbus related things

makes code cleaner and less prone to error

* docs: add note about the navdata interface construction

* refactor: switch instead of if chain

* chore: add temp. CommBus types, use types lib

* refactor: avoid array with `any` callbacks

* refactor: strongly type `callWasmFunction` name param

Using string parameter instead of enum to increase discoverability

* chore: use lowercase for type declaration file

* refactor: improve error handling

* refactor: move msfstypes to base package

* feat: add get airport function

* fix: use string literal instead of enum

* chore: format

* feat: unit tests and project refactor (#1)

* start node running of interface

* refactors

* fix: update `memoryBuffer` after `malloc`

* refactor: add `test_out` directory, fix our path type implementation

* refactor: project structure

* feat: jest test environment

* fix: stop lifecycle once tests completed

* feat: automatic download

* remove unnecessary package

* refactor: Cleanup setup.ts

* fix: make sh files executable

* fix: have build run with docker

* feat: test workflow

* fix: make test.sh executable

* Update test.sh

* test

* Update setup.ts

* Create push.yml

* Update push.yml

* general fixes

* fix: workspace issues

---------

Co-authored-by: Jack Lavigne <[email protected]>

* fix: use params and support number fields in raw sql query

* start new API

* feat: get airports in range

* feat: Airways by ident

* feat: Airways by range

* formatting

* feat: Departures query

* fix: sim issues

* refactor: Move database out of wasm module

* refactor: switch to fully snake_case and other spec compliance

* feat: arrivals

* refactor: procedure mapping

* feat: get_database_info

* refactor: remove uneccesary serde renames

* feat: Approaches

* fix: fix airport_ident

* pad out tests

* feat: Waypoint and VhfNavaid queries

* refactor: Generic range query generator

* feat: get runways at airport

* feat: Waypoint/VhfNavaid range queries

* feat: ndb navaids

* feat: navaid and waypoint airport queries

* feat: get airways at fix

* feat: comments

* feat: Airspace range queries

* fix: missing Database Info type

* fix: approach

* feat: gates

* feat: communications

* feat: Gls navaids

* feat: pathpoints

* improved documentation

* Update README.md

* Update README.md

* Update NavigraphLogin.tsx

* fix: js interface return types

* general fixes

* refactor workflow env

* Revert "refactor workflow env"

This reverts commit dafb9ef.

* refactors and comments in js

* feat: click on qr code to login

* test verbose

* test

* test

* Update pr.yml

* ci: run tests without docker

* ci: run jest directly

* ci: update actions, bump node version

* ci: disable cone mode for sparse-checkout

* ci: exclude blobs instead of sparse

* ci: skip installing packages in docker

This should cause the tests to fail right? I guess the reason it works locally is because it uses the dependencies installed by us on the windows-side...

* ci: run correct test script

* ci: temporarily skip WASM builds

* ci: explicit node version, install deps

* ci: install deps inside docker

* ci: make sure both commands run inside docker

* ci: revert to incorrect (but working) script

This configuration seems to result in successful test runs for some reason ¯\_(ツ)_/¯

* ci: reintroduce WASM build step

* ci: move test run to separate line for clarity

* Delete DOCS.md

---------

Co-authored-by: Malte Hallström <[email protected]>
Co-authored-by: Jack Lavigne <[email protected]>

src/wasm_navdata_interface/.cargo/config.toml → .cargo/config.toml

@@ -1,11 +1,11 @@
-[target.wasm32-wasi]
-rustflags = [
-  "-Clink-arg=--export-table",
-  "-Clink-arg=--export=malloc",
-  "-Clink-arg=--export=free",
-  "-Clink-arg=-L/usr/lib/llvm-15/lib/clang/15.0.7/lib/wasi",
-  "-Clink-arg=-lclang_rt.builtins-wasm32"
-]
-
-[build]
+[target.wasm32-wasi]
+rustflags = [
+  "-Clink-arg=--export-table",
+  "-Clink-arg=--export=malloc",
+  "-Clink-arg=--export=free",
+  "-Clink-arg=-L/usr/lib/llvm-15/lib/clang/15.0.7/lib/wasi",
+  "-Clink-arg=-lclang_rt.builtins-wasm32"
+]
+
+[build]
 target = "wasm32-wasi"

examples/gauge/.eslintrc.js → .eslintrc.js

@@ -1,6 +1,6 @@
 module.exports = {
   root: true,
-  ignorePatterns: [".eslintrc.js"],
+  ignorePatterns: [".eslintrc.js", "src/wasm/", "msfstypes/"],
   env: { browser: true, commonjs: true, es6: true },
   plugins: ["@typescript-eslint"],
   extends: [

.github/workflows/pr.yml

@@ -11,12 +11,23 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
-      - name: Build WASM module
+        uses: actions/checkout@v4
+        with:
+          filter: blob:none
+
+      - name: Create env file
         run: |
-          ./src/wasm_navdata_interface/scripts/run_docker_cmd.sh ./scripts/build.sh
+          touch .env
+          echo NAVDATA_SIGNED_URL=${{ secrets.NAVDATA_SIGNED_URL }} >> .env
+
+      - name: Build WASM module
+        run: npm run build:wasm-workflow
+
+      - name: Test
+        run: npm run test-workflow
+
       - name: Upload WASM module to GitHub
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4
         with:
-          name: navdata_interface.wasm
-          path: ./src/wasm_navdata_interface/out/navdata_interface.wasm
+          name: msfs_navdata_interface.wasm
+          path: ./out/msfs_navdata_interface.wasm

.github/workflows/pre-release.yml

@@ -12,10 +12,9 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v2
       - name: Build WASM module
-        run: |
-          ./src/wasm_navdata_interface/scripts/run_docker_cmd.sh ./scripts/build.sh
+        run: npm run build:wasm-workflow
       - name: Pre-Release
         uses: softprops/action-gh-release@v1
         with:
-          files: ./src/wasm_navdata_interface/out/navdata_interface.wasm
+          files: ./out/msfs_navdata_interface.wasm
           prerelease: true

.github/workflows/push.yml

@@ -0,0 +1,24 @@
+on:
+  push:
+    branches:
+    - main
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: create env file
+        run: |
+          touch .env
+          echo NAVDATA_SIGNED_URL=${{ secrets.NAVDATA_SIGNED_URL }} >> .env
+      - name: Build WASM module
+        run: npm run build:wasm-workflow
+      - name: Test
+        run: npm run test-workflow
+      - name: Upload WASM module to GitHub
+        uses: actions/upload-artifact@v2
+        with:
+          name: msfs_navdata_interface.wasm
+          path: ./out/msfs_navdata_interface.wasm

.github/workflows/release.yml

@@ -13,9 +13,8 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v2
       - name: Build WASM module
-        run: |
-          ./src/wasm_navdata_interface/scripts/run_docker_cmd.sh ./scripts/build.sh
+        run: npm run build:wasm-workflow
       - name: Release
         uses: softprops/action-gh-release@v1
         with:
-          files: ./src/wasm_navdata_interface/out/navdata_interface.wasm
+          files: ./out/msfs_navdata_interface.wasm

.gitignore

@@ -5,8 +5,8 @@ _PackageInt
 tsconfig.tsbuildinfo
 .vs
 examples/aircraft/PackageSources/html_ui/Pages/VCockpit/Instruments/Navigraph/NavdataInterfaceSample
-examples/aircraft/PackageSources/SimObjects/Airplanes/Navigraph_Navdata_Interface_Aircraft/panel/navdata_interface.wasm
-src/wasm_navdata_interface/out
+examples/aircraft/PackageSources/SimObjects/Airplanes/Navigraph_Navdata_Interface_Aircraft/panel/msfs_navdata_interface.wasm
+out
 
 # Rust
 # will have compiled files and executables
@@ -26,3 +26,6 @@ Cargo.lock
 .vscode
 .env
 *.local
+.DS_Store
+
+test_work/

.prettierignore

@@ -0,0 +1,8 @@
+# Rollup
+.rollup.cache
+
+package-lock.json
+
+*.gltf
+*.json
+*.yml

CODEOWNERS

@@ -3,3 +3,4 @@
 # @Navigraph/app-dev will be requested for
 # review when someone opens a pull request.
 *       @pepperoni505
+*       @professoralex13

Cargo.toml

@@ -0,0 +1,11 @@
+[workspace]
+resolver = "2"
+
+members = ["src/wasm", "src/database"]
+
+[profile.release]
+lto = true
+strip = true
+
+[patch.crates-io]
+rusqlite = { git = "https://github.com/navigraph/rusqlite", rev = "7921774" }

README.md

@@ -13,8 +13,10 @@ Here's an overview on the structure of this repository, which is designed to be
   - `aircraft/` includes a base aircraft to test in the sim
   - `gauge/` includes a very simple TypeScript instrument to communicate with the WASM module
 - `src/`
-  - Contains the source for the navdata interface (and soon the JS library)
-  - `wasm_navdata_interface` includes the Rust source code for the WASM module
+  - `database` Includes rust source code for interfacing with a DFD sqlite file (not WASM specific)
+  - `js` Includes source code for the JS interface for using the sdk
+  - `test` Includes code for testing the JS and Rust code using a Node runtime
+  - `wasm` includes the Rust source code for the WASM module which handles the downloading of the databse file, and interfacing with the database implementation
 
 ## Including in Your Aircraft
 
@@ -26,7 +28,7 @@ Here's an overview on the structure of this repository, which is designed to be
    size_mm=0,0
    pixel_size=0,0
    texture=NO_TEXTURE
-   htmlgauge00=WasmInstrument/WasmInstrument.html?wasm_module=navdata_interface.wasm&wasm_gauge=navdata_interface,0,0,1,1
+   htmlgauge00=WasmInstrument/WasmInstrument.html?wasm_module=msfs_navdata_interface.wasm&wasm_gauge=navdata_interface,0,0,1,1
    ```
    - Note that if you already have a `VCockpit` with `NO_TEXTURE` you can just add another `htmlgauge` to it, while making sure to increase the index
 
@@ -44,7 +46,93 @@ Before building, make sure you have properly created and set an `.env` file in `
 ## Building the WASM Module Yourself
 
 1. [Download](https://www.docker.com/products/docker-desktop/) Docker Desktop
-2. Open the `src/wasm_navdata_interface` folder in a terminal
-3. Run `.\build.bat` (must be on Windows)
+2. Run `npm run build:wasm` (must be on Windows)
    - This will take a while to download and build the first time, but subsequent runs will be quicker
-4. The compiled WASM module will be copied to `src/wasm_navdata_interface/out` **and** `examples/aircraft/PackageSources/SimObjects/Airplanes/Navigraph_Navdata_Interface_Aircraft/panel`
+3. The compiled WASM module will be copied to `out` **and** `examples/aircraft/PackageSources/SimObjects/Airplanes/Navigraph_Navdata_Interface_Aircraft/panel`
+
+## Interfacing with the navdata gauge manually
+
+The navdata interface acts as its own WASM gauge in sim, so in order to communicate with it, you must use the [CommBus](https://docs.flightsimulator.com/html/Programming_Tools/WASM/Communication_API/Communication_API.htm).
+
+The gauge communicates using the following event names:
+
+(Any types referenced can be found in `wasm/src/json_structs.rs`)
+
+- `NAVIGRAPH_CallFunction`: This event is received by the interface and is used to trigger one of the interfaces functions. It takes in arguments of type `CallFunction`. The available functions and their expected parameters can be found in the `json_structs.rs` file
+- `NAVIGRAPH_FunctionResult`: This event is sent by the interface as a response to a previously triggered function. Its result will have the type `FunctionResult`, with the data field containing the expected return type of the function.
+- `NAVIGRAPH_Event`: This event is sent by the interface to give indications of progress or that the interface is running correctly.
+
+### Example
+
+Below is an example of communicating with the interface in JS. (We provide a JS wrapper, the code below is just a basic example to show how it works). Please read the CommBus documentation to determine how to interface with CommBus in your chosen language. `src/js` contains our JS wrapper, it is also a useful example for implementing a fully fleshed out wrapper.
+
+```js
+const queue = []
+
+const listener = RegisterCommBusListener(() => {
+  listener.on("NAVIGRAPH_FunctionResult", jsonArgs => {
+    const args = JSON.parse(jsonArgs)
+
+    // When a FunctionResult is received, find the item in queue which matches the id, and resolve or reject it
+    const queueItem = queue.find(m => m.id === args.id)
+
+    if (queueItem) {
+      queue.splice(queue.indexOf(queueItem), 1)
+      const data = args.data
+
+      if (args.status === FunctionResultStatus.Success) {
+        queueItem.resolve(data)
+      } else {
+        queueItem.reject(new Error(typeof data === "string" ? data : "Unknown error"))
+      }
+    }
+  })
+}) // RegisterCommBusListener is a function provided by sim
+
+function getAirport(ident) {
+  const id = Utils.generateGUID() // Utils is a class provided by sim
+
+  const args = {
+    function: "GetAirport", // The name of the function being called
+    id, // CallFunctions and FunctionResults are tied together with the id field
+    data: {
+      // The parameters of the function
+      ident,
+    },
+  }
+
+  listener.callWasm("NAVIGRAPH_CallFunction", JSON.stringify(args))
+
+  return new Promise((resolve, reject) => {
+    queue.push({
+      id,
+      resolve: response => resolve(response),
+      reject: error => reject(error),
+    })
+  })
+}
+
+function executeSql(sql, params) {
+  const id = Utils.generateGUID() // Utils is a class provided by sim
+
+  const args = {
+    function: "ExecuteSQLQuery", // The name of the function being called
+    id, // CallFunctions and FunctionResults are tied together with the id field
+    data: {
+      // The parameters of the function
+      sql,
+      params,
+    },
+  }
+
+  listener.callWasm("NAVIGRAPH_CallFunction", JSON.stringify(args))
+
+  return new Promise((resolve, reject) => {
+    queue.push({
+      id,
+      resolve: response => resolve(response),
+      reject: error => reject(error),
+    })
+  })
+}
+```