Update to 1028

Add "watch" configuration in addition to "hooks". This is for monitoring
changes to things like files or database config and triggering scripts
in response.

text/1028-cli-hooks.rst

@@ -28,6 +28,9 @@ There is also a longstanding need to implement some kind of mechanism for
 creating data fixtures. Unlike the migration hooks, fixtures are more likely
 to be run at project initialization or after a branch wipe command.
 
+Also, there is a need to watch for file changes (schema, queries) and
+potentially respond to these by running some scripts.
+
 
 Hooks Configuration
 ===================
@@ -122,23 +125,107 @@ means that the hooks associated with applying schema changes must be executed
 for these commands as well.
 
 
+Watch Configuration
+===================
+
+Sometimes in a Gel project there's a need to respond to some file changes: run
+a migration due to schema change or run code generators due to query file
+change. The CLI already supports watching the schema for changes and applying
+them, but we can generalize this mechanism and make it more flexible, much
+like the hooks that respond to CLI commands.
+
+The idea is to have a mechanism for specifying what changes you want to watch
+and a script that gets triggered by those changes. Unlike hooks, there are no
+"before" and "after" triggers here. Only one kind of response is possible:
+trigger when a change to the watched entity is detected.
+
+The watch configuration will be described in the ``gel.toml`` project manifest
+file in the ``[watch]`` table. We will introduce a ``[watch.files]`` sub-table
+for watching file system changes and triggering scripts. Potentially, we will
+also later introduce a ``[watch.gel-config]`` sub-table for monitoring changes
+to the various database config values and responding to them.
+
+The general structure of watch tables is going to use the key (potentially
+quoted) to specify *what is being watched* and the corresponding value to
+specify the script that will be triggered by changes.
+
+The ``gel watch`` command (without needing further options) would then be used
+to start the watch process and monitor whatever is specified in the
+``gel.toml``.
+
+The output of the scripts will then appear in the same TTY as the ``gel
+watch`` command. We may also consider setting up a logfile (because that
+creates a record which survives closing of terminals or reboots) as an
+additional convenience feature. This can be specified in the general
+``[watch]`` section as ``logfile="<path-to-logfile>"``.
+
+We may want to setup a debouncer so that we can delay before triggering the
+scripts on a sequence of changes. This is mostly to reduce unnecessary
+multiple triggers for the same watched entity. The debouncer delay may be
+configured in the ``[watch]`` section as ``deboucer-delay=<integer>`` with the
+value being the delay in milliseconds.
+
+Another consequence of executing triggered scripts in the background is that
+sometimes the changes are invalid in some way and the script will fail. This
+is considered part of normal operation (e.g. syntax error in a saved query
+file) and the scripts should be such that failure does not create some
+non-recoverable state.
+
+
+Files
+-----
+
+The ``[watch.files]`` table should contain keys that are interpreted as file
+system paths. They can also contain common glob patterns (such as provided by
+`this library <https://docs.rs/globset/latest/globset/#syntax>`_).
+
+The paths should be valid in the underlying file system (therefore using ``/``
+for Linux and ``\`` for Windows, etc.). The relative file paths are assumed to
+start at the project root.
+
+The values corresponding to the keys are strings that are going to be executed
+as shell commands, much like for the hooks.
+
+All watch scripts will use the project root directory as the execution
+directory. They are executed using ``/bin/sh`` on all platforms. On Windows,
+the scripts are always executed in WSL.
+
+An example of this configuration::
+
+    [watch.files]
+    "queries/*.edgeql"="npx @edgedb/generate queries"
+
+
+CLI Interactions
+----------------
+
+Currently ``gel watch`` is not relying on the ``[watch.files]`` config, but
+going forward it would be nice to generalize it using the watch mechanism
+instead of having special treatment for it. We may need to introduce a special
+migration command though that mimics the behavior of ``gel watch`` in terms of
+applying schema changes without writing a migration step for each such change.
+Perhaps a ``gel migration try`` command that would attempt to apply changes
+based on the current schema files without writing a corresponding migration
+file.
+
+With that CLI change we could make ``gel watch`` command create the following
+entry in ``gel.toml``::
+
+    [watch.files]
+    "dbschema/**.gel"="gel migration try"
+
+
 Design Considerations
 =====================
 
 It makes sense to follow a convention of filling out the ``[hooks]``
 table in order of execution priority from highest to lowest::
 
     [hooks]
-    project.init.after=[
-      "setup_dsn.sh"
-    ]
-    branch.wipe.after=[]
-    branch.switch.after=[
-      "setup_dsn.sh"
-    ]
-    migration.apply.after=[
-      "gel-orm sqlalchemy --mod compat --out compat"
-    ]
+    project.init.after="setup_dsn.sh"
+    branch.wipe.after=""
+    branch.switch.after="setup_dsn.sh"
+    migration.apply.after="gel-orm sqlalchemy --mod compat --out compat"
 
 The order in which hook *keys* appear does not impact their priority (we don't
 want people getting subtle bugs due to different key order). It would simply
@@ -157,4 +244,12 @@ could use whatever you would have used if you ran the scripts by hand from the
 shell. For example, to get the instance name ``gel project info
 --instance-name``. The idea is that the scripts wouldn't be relying on any
 hook-specific magic and you could run them (and thus debug them) by hand with
-the identical effects.
+the identical effects.
+
+We don't want to provide a list of scripts to run for hooks and watch
+triggers. This is because with a list of scripts we need to specify what
+happens when some scripts fail. Should the rest of the list be executed or
+aborted. Under different circumstances different approaches would make sense
+and we would need to implement all these interaction variants. Instead any
+such complexity can be handled inside a singe script that the trigger
+references.