[WIP] hosting the sync-server with the desktop app (POC) #3631

MikesGlitch · 2024-10-10T20:05:29Z

The goal

Free server hosting via the desktop app. We'd run the sync server inside of the desktop app, expose it to the internet via ngrok (or similar) and provide a QR code/url for the user to connect their phone to the server.

We could provide an option to start actual-server when the computer starts up. If we did this the server would be alive whenever the computer is on. If the user downloads the pwa to their phone they could budget even when the server is offline - it would sync when the computer is turned back on.

Todos:

In a follow up feature:

Provide an option to start actual-server when the computer starts up

Known issues

server /info endpoint isn't returning data - it means we can't see what version the server is on. I think it's to do with it not finding the package.json file. We're using an external package for it.

Next: Figure out the best way to bring actual-server into the actual repo

Expose the running server to the internet

Manged to get it working with ngrok. Ngrok's big and they give you a free domain. It works a treat. I haven't tested it in anger, I'm sure it will disconnect me after an hour or so so we may need to add logic to reconnect.

Thoughts on bundling the actual-server code

If actual-server was in the actual repo we could bundle using electron-builder. The package size seemed good when I tried that. This seems like the right way forward for all kinds of reasons - some not related to electron - type sharing, debugging, simlicity etc.

Old thoughts:

Actual-server bundle is huge - consider using webpack or https://github.com/vercel/ncc . Webpack seems promising, they have targets for electron and node.
BetterSqlite3 is complaining that it was compiled for server on a different version of node than electron. We'll also run into the arch issue so we'll need to fully make that external in the actual-server bundle and use the one we have in electron.
If we bundle, the bundle should only be used for electron (if we want to bundle for normal use it will be a separate bundle)

Server sync notes

We could warn the client against syncing if the client version is greater than the server version.
- This means we should never have out of sync migrations
- We'd always know the server version & client version at this point - we would have already migrated but we wont sync it with the server unless it is safe to do so.
- Downside - if it takes 2 weeks to update the server the user is unable to sync until it is updated

netlify · 2024-10-10T20:05:46Z

✅ Deploy Preview for actualbudget ready!

Name	Link
🔨 Latest commit	`6671e2c`
🔍 Latest deploy log	https://app.netlify.com/sites/actualbudget/deploys/6733a1c6707b5400088e39ba
😎 Deploy Preview	https://deploy-preview-3631.demo.actualbudget.org
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

github-actions · 2024-10-10T20:07:46Z

Bundle Stats — desktop-client

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count	Total bundle size	% Changed
9	5.34 MB → 5.34 MB (+470 B)	+0.01%

Changeset

File	Δ	Size
`src/components/manager/BudgetList.tsx`	📈 +399 B (+3.44%)	11.32 kB → 11.71 kB
`src/browser-preload.browser.js`	📈 +71 B (+1.75%)	3.96 kB → 4.03 kB

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

Asset	File Size	% Changed
static/js/index.js	3.34 MB → 3.34 MB (+470 B)	+0.01%

Smaller

No assets were smaller

Unchanged

Asset	File Size	% Changed
static/js/usePreviewTransactions.js	1.64 kB	0%
static/js/resize-observer.js	18.37 kB	0%
static/js/indexeddb-main-thread-worker-e59fee74.js	13.5 kB	0%
static/js/AppliedFilters.js	20.96 kB	0%
static/js/narrow.js	82.55 kB	0%
static/js/BackgroundImage.js	122.29 kB	0%
static/js/wide.js	242.95 kB	0%
static/js/ReportRouter.js	1.51 MB	0%

github-actions · 2024-10-10T20:07:47Z

Bundle Stats — loot-core

Hey there, this message comes from a GitHub action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Files count	Total bundle size	% Changed
1	1.26 MB	0%

Changeset

No files were changed

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

No assets were bigger

Smaller

No assets were smaller

Unchanged

Asset	File Size	% Changed
kcab.worker.js	1.26 MB	0%

Jonathan-Fang · 2024-10-12T01:37:06Z

This seems very cool!

MikesGlitch · 2024-10-14T19:51:43Z

bin/package-electron

@@ -37,6 +37,7 @@ fi
 yarn workspace loot-core build:node

 yarn workspace @actual-app/web build --mode=desktop
+yarn workspace @actual-app/crdt build


crdt is a dependency of actual-server. Build it before packaging the sync-server

youngcw · 2024-10-29T16:49:05Z

Just throwing this out there. Could the bundled server have a way to make its own ssl cert? That way we know it setup right and the user just have to know to click past the warning, or have guides on how to install the cert.

MikesGlitch · 2024-10-29T16:54:12Z

Just throwing this out there. Could the bundled server have a way to make its own ssl cert? That way we know it setup right and the user just have to know to click past the warning, or have guides on how to install the cert.

Still investigating that one. I think it can be done because dev servers do it but still experimenting. Just added an item on the todo for it.

…lectron/server-hosting

packages/sync-server/src/app-simplefin/app-simplefin.js

+    }
+
+    if (Array.isArray(accountId) != Array.isArray(startDate)) {
+      console.log(accountId, startDate);


packages/sync-server/src/app-simplefin/app-simplefin.js

+      );
+    }
+    if (Array.isArray(accountId) && accountId.length !== startDate.length) {
+      console.log(accountId, startDate);


packages/sync-server/src/app-sync.js

+  }
+
+  try {
+    await fs.writeFile(getPathForUserFile(fileId), req.body);


packages/sync-server/src/app-sync.js

+  }
+
+  res.setHeader('Content-Disposition', `attachment;filename=${fileId}`);
+  res.sendFile(getPathForUserFile(fileId));


packages/sync-server/src/util/middlewares.js

+    // Source: https://expressjs.com/en/guide/error-handling.html
+    return next(err);
+  }
+  console.log(`Error on endpoint ${req.url}`, err.message, err.stack);


…lectron/server-hosting

MikesGlitch added 3 commits October 10, 2024 20:04

start to this epic idea

20e2ef2

bits

361200e

add a note

c718b2c

actual-github-bot bot added the 🚧 WIP label Oct 10, 2024

MikesGlitch changed the title ~~[WIP] server hosting~~ [WIP] Experimental: hosting the sync-server locally Oct 10, 2024

MikesGlitch changed the title ~~[WIP] Experimental: hosting the sync-server locally~~ [WIP] hosting the sync-server locally Oct 10, 2024

MikesGlitch changed the title ~~[WIP] hosting the sync-server locally~~ [WIP] hosting the sync-server with the desktop app Oct 10, 2024

MikesGlitch added 2 commits October 11, 2024 21:11

server starting button erroring on imports due to env differences

e32fb8b

working now

1802818

MikesGlitch added 3 commits October 12, 2024 11:25

using node_modules and allowing electron-builder to bundle it

194f50d

ensuring deps are built before running server

7782f2d

unneeded mode

55e9113

MikesGlitch changed the title ~~[WIP] hosting the sync-server with the desktop app~~ [WIP] POC: hosting the sync-server with the desktop app Oct 12, 2024

MikesGlitch changed the title ~~[WIP] POC: hosting the sync-server with the desktop app~~ [WIP] hosting the sync-server with the desktop app (POC) Oct 12, 2024

MikesGlitch added 6 commits October 13, 2024 22:19

exposing sync server via ngrok

3266319

log msg for dev

b3e6d28

using official ngrok package for smaller size

ce5225e

comment

07e7730

ngrok bits

b9c14dc

correcting ngrok conditional logic

c6d2f59

MikesGlitch commented Oct 14, 2024

View reviewed changes

MikesGlitch added 5 commits October 14, 2024 20:54

ngrok settings optional

8eb145f

bits

23fdf04

moving server folders

58aa350

package process

2dc34de

merge

4873c9c

fix timeout message

312e85e

This was referenced Oct 30, 2024

[WIP] Remove unneeded file #3755

Closed

🔧 Remove test file #3756

Merged

MikesGlitch added 5 commits October 30, 2024 14:02

note

4f667d0

Merge branch 'master' of https://github.com/MikesGlitch/actual into e…

a5f2f8b

…lectron/server-hosting

use env variable instead of https agent for default fetch

79f648f

updating root ca impl to use node env variable for more support

2265164

release notes

4400ff1

MikesGlitch mentioned this pull request Nov 4, 2024

Removing node-fetch and updating root ca impl for more support #3782

Merged

2 tasks

MikesGlitch added 5 commits November 4, 2024 12:10

removing node-fetch

98174bb

clean up

26adfec

error message

8cfd39f

merg

4dd1fe0

adding the sync server into the workspace as an experiment

78fbebe

github-advanced-security bot found potential problems Nov 4, 2024

View reviewed changes

MikesGlitch added 2 commits November 4, 2024 18:37

merge

5886678

update server

6513207

github-advanced-security bot found potential problems Nov 4, 2024

View reviewed changes

MikesGlitch mentioned this pull request Nov 7, 2024

Separate build folders for easier switching between web and electron #3801

Merged

MikesGlitch added 5 commits November 7, 2024 21:03

Merge branch 'master' of https://github.com/MikesGlitch/actual into e…

99b6cb2

…lectron/server-hosting

responsive

4036b57

test

c718e67

put it back

2a1bb2c

fix rootpath

8847f64

MikesGlitch mentioned this pull request Nov 10, 2024

Option to launch Actual on startup #3816

Draft

5 tasks

MikesGlitch added 3 commits November 11, 2024 22:42

Merge branch 'master' of https://github.com/MikesGlitch/actual into e…

0ce8c57

…lectron/server-hosting

Merge branch 'master' of https://github.com/MikesGlitch/actual into e…

8037e03

…lectron/server-hosting

merge

6671e2c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] hosting the sync-server with the desktop app (POC) #3631

[WIP] hosting the sync-server with the desktop app (POC) #3631

MikesGlitch commented Oct 10, 2024 •

edited

Loading

netlify bot commented Oct 10, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024 •

edited

Loading

Jonathan-Fang commented Oct 12, 2024

MikesGlitch Oct 14, 2024

youngcw commented Oct 29, 2024

MikesGlitch commented Oct 29, 2024 •

edited

Loading

[WIP] hosting the sync-server with the desktop app (POC) #3631

Are you sure you want to change the base?

[WIP] hosting the sync-server with the desktop app (POC) #3631

Conversation

MikesGlitch commented Oct 10, 2024 • edited Loading

The goal

Known issues

Next: Figure out the best way to bring actual-server into the actual repo

Expose the running server to the internet

Thoughts on bundling the actual-server code

Server sync notes

netlify bot commented Oct 10, 2024 • edited Loading

✅ Deploy Preview for actualbudget ready!

github-actions bot commented Oct 10, 2024 • edited Loading

Bundle Stats — desktop-client

github-actions bot commented Oct 10, 2024 • edited Loading

Bundle Stats — loot-core

Jonathan-Fang commented Oct 12, 2024

MikesGlitch Oct 14, 2024

Choose a reason for hiding this comment

youngcw commented Oct 29, 2024

MikesGlitch commented Oct 29, 2024 • edited Loading

MikesGlitch commented Oct 10, 2024 •

edited

Loading

netlify bot commented Oct 10, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024 •

edited

Loading

github-actions bot commented Oct 10, 2024 •

edited

Loading

MikesGlitch commented Oct 29, 2024 •

edited

Loading