Tracking list of manual smoke tests before each release

[jessfraz]

Manual test tasklist: [add plaform here]

• The application opens
• User can sign in
• The onboarding can be replayed and looks solid
• A project can be created
• A sketch can be draw (line, arc, rect, circle).
• Lines and shapes can be modified.
• Select face and extruding works.
• Reloading the application works.
• Can create additional files and directories.
• Models can be exported to a file.
• I repeated some code done with point-and-click by typing in the code editor and everything was responsive and ok.
• Double clicking a KCL file opens ZMA, and with the CLI too on project directory or file
• Offset plane can be created and shows up
• The auto-updater works.

Please add anything you believe is missing. Try to not be too precise. We want testers to veer off a little to test edges.

[lf94]

Can we put a process in place?

[lf94]

I was thinking maybe a spreadsheet with screenshot links that prove we have done the tests manually.

[jessfraz]

this is it haha, smoke test these for each os :)

[jessfraz]

oh that seems like a lot of work, I trust you all to actually do the thing, versus wasting time dealing w excel

[lf94]

We can save time by partitioning the tests amongst ourselves using a doc tho

[jessfraz]

fair fair set up whatever process you want :)

[franknoirot]

Maybe we auto-make a tasklist on "Cut release vXX.YY.ZZ" PRs

[nadr0]

Grab the out-ubuntu-22.04 AppImage from the build-publish-apps job in the CI CD pipeline of the PR that has the cut release. Example scroll to the bottom and see the artifacts sections to download the file.

Gotcha: I had to chmod +x the file and install Fuse to get the AppImage running

Use: updater-test-ubuntu-22.04 from the artifacts page for the update tests.

• Electron tests: external links open an external browser Electron tests: external links open an external browser #3373
• coredump will open github issue with stuff populated (from home and project page) there are different ways to trigger it as well (! button at top, bug at bottom, mod + shift + .)
  • !
  • Bug bottom folder page
  • Bug bottom model render page
  • ❗ (Does not work on linux) mod + shift + .
• running app.{app,exe} path/to/file opens the kcl file (all oses)

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/default/cube.kcl

• running app.{app,exe} path/to/dir opens the directory and any kcl file in it or creates a main.kcl if empty (all oses)
  • Opens directory
  • Creates main file if empty

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/default

rm /home/kevin-nadro/Documents/zoo-modeling-app-projects/empty/main.kcl

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/empty

• If someone at the office, test you can hit make from the app and it works
• Test updating versions works
• until we have a performance test, sanity check that time to open file directly from command line is not shit
  • xx.xx

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/Documents/zoo-modeling-app-projects/default/chassis-with-rails.kcl

• Electron tests: ensure we can open files that in tauri we would not be able to Electron tests: ensure we can open files that in tauri we would not be able to #3455
  • Opening file
  • Editing file

/home/kevin-nadro/Downloads/Zoo\ Modeling\ App-0.25.5-x86_64-linux.AppImage /home/kevin-nadro/smoke-test-file-permissions-linux/main.kcl

• Electron test: upgrading app should not cause user to have to auth again Electron test: upgrading app should not cause user to have to auth again #3469 issues/3469
• Electron: logging in and authing works Electron: logging in and authing works #3465
• ❗ (Does not work on Linux) double click to open files from finder opens new window each time (mac/windows)
• export from the app works and opens a file picker ux (needs to be manual since we override for playwright)
• Move around and sketch rapidly: is interaction still snappy?

Notes

• Opening a path to a directory that doesn't exist, hangs, should we create the directory?

Loading project at startup: /home/kevin-nadro/Documents/zoo-modeling-app-projects/empty-2
Error: ENOENT: no such file or directory, stat '/home/kevin-nadro/Documents/zoo-modeling-app-projects/empty-2'
[42821:0930/142409.906556:ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 1 times!
Checking for update
Update for version 0.25.5 is not available (latest version: 0.25.4, downgrade is disallowed).

• Cannot figure out mod + shift + . on linux for core dump
• Double clicking files on linux does not work.

[Irev-Dev]

for

running app.{app,exe} path/to/file opens the kcl file (all oses)
running app.{app,exe} path/to/dir opens the directory and any kcl file in it or creates a main.kcl if empty (all oses)

on Macos

/Applications/Zoo\ Modeling\ App.app/Contents/MacOS/Zoo\ Modeling\ App /path/to/file
# Likely something like this
/Applications/Zoo\ Modeling\ App.app/Contents/MacOS/Zoo\ Modeling\ App /Users/<yourUser>/Documents/zoo-modeling-app-projects/projectName/fileName.kcl

[lf94]

I'm moving @jessfraz 's original post here just in case - it's been replaced with a more appropriate manual test tasklist.

---

### Manual Tasks
- [ ] Mash keyboard in code editor
- [ ] https://github.com/KittyCAD/modeling-app/issues/3373
- [ ] coredump will open github issue with stuff populated (from home and project page) there are different ways to trigger it as well (! button at top, bug at bottom, `mod + shift + .`)
- [ ] running app.{app,exe} path/to/file opens the kcl file (all oses)
- [ ] running app.{app,exe} path/to/dir opens the directory and any kcl file in it or creates a main.kcl if empty (all oses)
- [ ] If someone at the office, test you can hit `make` from the app and it works
- [ ] Test updating versions works
- [ ] until we have a performance test, sanity check that time to open file directly from command line is not shit
- [ ] https://github.com/KittyCAD/modeling-app/issues/3455
- [ ] https://github.com/KittyCAD/modeling-app/issues/3469
- [ ] https://github.com/KittyCAD/modeling-app/issues/3465
- [ ] double click to open files from finder opens new window each time (mac/windows)
- [ ] export from the app works and opens a file picker ux (needs to be manual since we override for playwright)
- [ ] Ensure that a loading toast appears when testing the auto-updater, saying the download has begun
- [ ] https://github.com/KittyCAD/modeling-app/issues/4038
- [ ] Move around and sketch rapidly: is interaction still snappy?

also relevant #3459