Bucket list of various things that could be done better/differently

[bartekpacia]

This issue is somewhat inspired by flutter/flutter#24722. Subtasks might be broken into individual issues later on.

Orchestra

• There's no clear system for commands that generate output (as in: artifacts, e.g. files, screen recordings)
• Orchestra should not be concerned with filesystem operations (such as creating files taken with takeScreenshot or startRecording). Instead it should write to okio.Sinks.
  • Whenever Orchestra wants to write to such a sink, it should request it by calling commandWillGenerateOutput: (subpath, commandType) -> Sink.
  • additionally, I think its output should land in ~/.maestro/tests/<id>, not in the project repo root (which is not even Maestro workspace).
  • well akshually, it shouldn't even know what a file system is, this seems like too much responsibility. It should write output to Sinks.
• There's code duplication between MaestroCommandRunner and TestSuiteInteractor:
  • takeDebugScreenshot()
  • Orchestra instance creation and callbacks setup

Permeating whole codebase

• Lack of clear definitions of some terms across the codebase
  • test vs flow? "Flow run", "test suite run" but TestSuiteInteractor?
  • FlowStatus, TestExecutionSummary
  • what actually is a driver?
    • Is this a thing that runs on the device or on the host?
    • On iOS side of things, I feel like terms "XCTest driver" and "XCTest Runner" are used interchangeably and it confuses me
• We should use Okio's FileSystem instead of stuff from Java, to allow for mocking/testing and no dependency on JVM.
• TestExecutionSummary is only created when running in two different ways:
  • (it's returned directly by TestSuiteInteractor.runTestSuite())
  • but the summary report is still generated (in both single and multi mode)
• HtmlTestSuiteReporter is hard to test and iterate on. action item: create a dummy TestExecutionSummary so that HTML can be easily generated with Gradle

config

• MaestroConfig should be renamed to FlowConfig

One-offs

• TestDebugReporter.logSystemInfo() and DebugLogStore.logSystemInfo() are duplicated
• Remove binary app artifacts from the repo #1822
• Maestro starts up too slow and takes too long to exit on CTRL+C
• • package in maestro-client is called maestro. Should be maestro.client.
• In generated HTML report, Boostrap should be replaced with Tailwind
• some DeviceInfo props should be renamed:
  • DeviceInfo.heightGrid -> DeviceInfo.heightPoints
  • DeviceInfo.widthGrid -> DeviceInfo.widthPoints
• Maestro.screenshotBasedTap() and Maestro.hierarchyBasedTap() look like their names have been accidentally swapped

---

Action items

• Introduce terms for Maestro CLI running modes: single mode, multi mode (aka single test mode, multi test mode)
  • in the longer term, strive to de-duplicate code paths for them
• Decide on either "test" or "flow" name and use it consistently