Overhaul of loadup scripts (#1699)

* Make medley.sh and its associated scripts POSIX compliant - i.e., debashify them

* Added config file for medley script, medley now reads from config file and prepends arguemnts from file to the copmmand line arguments

* WIP. Updates to medley.sh scripts.

* WIP.  More on medley.sh and friends update.

* WIP. Medley redo

* WIP.  Debugging new medley scripts

* Renamed medley.sh/medley.command to be medley_main.sh.  Added code to compile single medley.sh/medley.command script by inlining all of the source'd medley_*.sh files.

* Add temp fix for cygwin Issue #1685

* Minor fixup to medley_utils.sh; take debug code out out of run_medley

* Add README to medley directory to explain how to compile medley.sh (medley.command).

* Ooops. This time really adding the README file to the medley directory explaining how to compile medley.sh (medley.command)

* Update loadup- scripts to use updated medley scripts rather than run-medley

* Fix default setting of $config_file in medley_configfile.sh

* Redo medley compile to pick up last commikt

* Fixing how maiko exe is found and sysout argument error processing - both issues discovered testing on MAcOS

* In medley_configfile, replace echo with printf %s because echo - does not work in zsh

* Supress config file on loadups calls to Medley

* Add oldschool support (use original run-medley) to loadup scripts; improve FAILURE detection so loadup-all won't proceed once one of the components fails

* Add in medley_args.sh add -prog as synonym to --maikoprog to aid in loadup scripts; in medley_run.sh script try to get a good exit code for call to maiko, especially useful for loadup scripts

* Run loadup scripts thru shellcheck and update as necessary to make Posix compliant

* Get rid of -nt comparisons in loadup-setup.sh because they are not posix-complaint.  They were not really needed anyway.

* Removing (for now) use of lde exit codes to decide FAILURE case in loadup-setup.sh since exit codes from lde apperar to be inverted on MacOS.

* Update medley man page.  Add - functionality to more args is medley_args.sh

* Compile medley.sh with changes from last commit

* Ooops.  Left medley_args.sh changes out of last commit.  Rectifying here.

* Added support for LDEKEYBOARDTYPE to medley_run to match run-medley

* Add to medley.sh: auto numbered id's and titles with id's inserted

* Cleanup some shellcheck issues in medley_main.sh

* fix maiko args -nh-xxx.  were -nethub-xxxx. In medley_run.sh

* Overhaul handling of pass-on args to manage the quoting issues prevelant in the previous implementation

* Cleanup minor shellcheck issues in medley_*.sh scripts

* Add underscore as character allowed in ids - makes things clearer when id used with +

* Add a self-numbering id to medley calls in loadup scripts

* Put workaround in medley_run.sh for Issue #1702 - issues with sysout arg processing in Maiko

* Oops.  messed up LDESRCSYSOUT in last commit.  should be LDESOURCESYSOUT

* compile medley.sh

Author: fghalasz

docs/man-page/man_medley.html

@@ -41,6 +41,10 @@ Flags
 -z, \-\-man
 : Show the man page for medley
 
+-c [*FILE* | -], \-\-config [*FILE* | -]
+: Use *FILE* as the config file for this run of Medley. See information on *CONFIG FILE* below.  If *FILE* is "-",
+then suppress the use of a config file for this run of Medley. 
+
 -f, \-\-full
 : Start Medley from the standard "full" sysout. full.sysout includes a complete Interlisp and CommonLisp environment
 with a standard set of development tools.  It does not include any of the applications built using Medley.
@@ -57,88 +61,122 @@ applications including Notecards, Rooms and CLOS.  It also includes pre-installe
 documentation.
 (See *SYSOUT_FILE* below for more information on starting sysouts.)
 
--e, \-\-interlisp (relevent only when \-\-apps is specified)
-: Make the initial Exec window within Medley be an Interlisp Exec.  Default is to start in an XCL Exec.
-
--n, \-\-noscroll
-: Ordinarily Medley displays scroll bars to enable the user to pan the Medley virtual display within the
+-u, \-\-continue
+:  Nullify any prior setting of the sysout file (e.g., from the config file) - causing Medley to start from
+the virtual memory file resulting from the previous invocation (with the same values for --id and --logindir),
+if any.  If there is no matching virtual memory file, Medley will start from the full.sysout (see -f/--full above).
+Equivalent to "-y -".  (See *SYSOUT FILE* section below.) 
+
+-y [*SYSOUT_FILE* | -], \-\-sysout [*SYSOUT-FILE* | -]
+: Start Medley from the specified *SYSOUT-FILE*.  This is an alternative to specifying the *SYSOUT-FILE*
+as the last argument on the command line (but before any *PASS_ON_ARGS*).  It can be used to specify the
+*SYSOUT-FILE* in the config file (see information on *CONFIG FILE* below). If *SYSOUT-FILE* is "-", then
+any prior setting of the sysout file (e.g., from the config file) is nullified (see -u/--continue above).
+(See *SYSOUT FILE* section below.) 
+
+-e [+ | -], \-\-interlisp [+ | -]
+: If value is "+" or no value, make the initial Exec window within Medley be an Interlisp Exec.
+If value is "-", make the initial Exec window be the default XCL Exec.
+This flag applies only when the --apps flag is used.
+
+-n [+ | -], \-\-noscroll [+ | -]
+: Medley ordinarily displays scroll bars to enable the user to pan the Medley virtual display within the
 Medley window.  This is true even when the entire virtual display fits within the window.  Specifying
-\-\-noscroll turns off the scroll bars.  Note: If \-\-noscroll is specified and the virtual screen is larger
+"-n +" (--noscroll +) turns off scroll bars.  Specifying "-n -" (--scroll -) turns on scroll bars.
+Specifying -n (--noscroll) with no value is equivalent to specifying "--noscroll +".  Default
+is scroll bars off. Note: If scroll bars are off and the virtual screen is larger
 than the window, there will be no way to pan to the non-visible parts of the virtual display.
 
--g *WxH*, \-\-geometry *WxH*
+-g [*WxH* | -], \-\-geometry [*WxH* | -]
 : Sets the size of the X Window (or VNC window) that Medley runs in to be Width x Height. (Full X Windows
-geomtery specification with +X+Y is not currently supported).  If \-\-geometry is not specified but \-\-screensize is,
+geomtery specification with +X+Y is not currently supported).
+If a value of "-" is given, geometry is set to the default value.
+If \-\-geometry is not specified but \-\-screensize is,
 then the window size will be determined based on the \-\-screensize values and the \-\-noscroll flag.  If neither
 \-\-geometry nor \-\-screensize is provided, then the window size is set to 1440x900 if \-\-noscroll is set and 1462x922
-if \-\-noscroll is not set.
+if \-\-noscroll is not set.  (Also see note below under *CONFIG FILE* on the use of geometry and screensize
+in config files.)
 
--s *WxH*, \-\-screensize *WxH*
+-s [*WxH* | -], \-\-screensize [*WxH* | -]
 : Sets the size of the virtual display as seen from Medley's point of view.
+If a value of "-" is given, screensize is set to the default value.
 The Medley window is an unscaled viewport onto this virtual display. If \-\-screensize is not specified but
 \-\-geometry is, then the virtual display size will be set so that the entire virtual display fits into the given
 window geometry.  If neither \-\-screensize nor \-\-geometry is provided, then the screen size is set to 1440x900.
+(Also see note below under *CONFIG FILE* on the use of geometry and screensize in config files.)
+
+-ps [*N* | -], --pixelscale [*N* | -]    \*\* **Applicable only when display is SDL-based (e.g., on Windows/Cygwin)** \*\*
+: Sets the pixel scaling factor to *N*. If value of "-" is given, the pixel scale factor is set to its default of 1.
 
--t *STRING*, \-\-title *STRING*
-: Use STRING as title of Medley window.  Ignored when when the \-\-vnc flag is set or when running on Windows (Docker)
-installations.
+-t [*STRING* | -], \-\-title [*STRING* | -]
+: Use STRING as title of Medley window.  If the value of "-" is given, sets the title to its default value ("Medley Interlisp").
+Ignored when when the \-\-vnc flag is set.
 
--d *:N*, \-\-display *:N*    \*\* **Not applicable to Windows (Docker) installations** \*\*
-~ Use X display :N.  Defaults to the value of $DISPLAY.  This flag is ignored when the \-\-vnc flag is set as
-well as on Windows (Docker) installations.
+-d [*:N* | -], \-\-display [*:N* | -]
+: Use X display :N.  If value is "-", reset display to its default value. Default value is the value of $DISPLAY. 
+On platforms that support X Windows as well as SDL, the value of -d (--display) should
+be set to "SDL" to select using SDL instead of X Windows.  This flag is ignored on the Windows/Cygwin platform and when the \-\-vnc flag is
+set on Windows System for Linux.
 
--v, \-\-vnc    \*\* **Applicable only to WSL installations** \*\*
-: Use a VNC window running on the Windows side instead of an X window.
-The VNC window will folllow the Windows desktop scaling setting allowing
+-v [+ | -] , \-\-vnc [+ | -]    \*\* **Applicable only to WSL installations** \*\*
+: If value is "+" or no value is given, then use a VNC window running on the Windows side instead of an X window.  If value is "-", then do not
+use a VNC window, relying instead on a standard X Window.
+A VNC window will folllow the Windows desktop scaling setting allowing
 for much more usable Medley on high resolution displays.  On WSL, X windows
 do not scale well.  This flag is always set for WSL1 installations.
 
 -i [*ID_STRING* | - | \-\-], \-\-id [*ID_STRING* | - | \-\-]
-: Use ID_STRING as the id for this run of Medley, iunless ID_STRING is "-" or "\-\-". 
-If ID_STRING is "-", then use the basename of $MEDLEYDIR as the id.
-If ID_STRING is "\-\-", then use the basename of the parent directory of $MEDLEYDIR as the id.
+: Use ID_STRING as the id for this run of Medley, unless ID_STRING is "-", "\-\-", or "\-\-\-". 
+If ID_STRING is "-", then reset the id to "default" (e.g., if it was previously set in the
+config file).  If ID_STRING is "\-\-", then use the basename of $MEDLEYDIR as the id.
+If ID_STRING is "\-\-\-", then use the basename of the parent directory of $MEDLEYDIR as the id.
 Only one instance of Medley with a given id can run at a time.
 The id is used to distinguish the virtual memory stores so that multiple
 instances of Medley can run simultaneously.  Default id is "default".
 
--m *N*, \-\-mem *N*
-: Set Medley to run in *N* MB of virtual memory.  Defaults to 256MB.
+-m [*N* | -], \-\-mem [*N* | -]
+: Set Medley to run in *N* MB of virtual memory.  Defaults to 256MB. If a value of "-" is given, resets
+to default value.
 
--p *FILE*, \-\-vmem *FILE*
+-p [*FILE* | -], \-\-vmem [*FILE* | -]
 : Use FILE as the Medley virtual memory (vmem) store.  FILE must be writeable by the current user.
 Care must be taken not to use the same vmem FILE for two instances of Medley running simultaneously.
 The \-\-id flag will not protect against vmem collisions when the \-\-vmem flag is used.
-Default is to store the vmem in LOGINDIR/vmem/lisp_XXX.virtualmem, where XXX is the id of this
-Medley run (see \-\-id flag above).  See \-\-logindir below for setting of LOGINDIR. On Windows (Docker) installations, *FILE* is specified in the Medley file system, not the host Windows file system.
+If the value "-" is given, then resets the vmem file to the default.
+Default is to store the vmem in LOGINDIR/vmem/lisp_III.virtualmem, where III is the id of this
+Medley run (see \-\-id flag above).  See \-\-logindir below for setting of LOGINDIR. 
 
 -r \[*FILE* | -], \-\-greet \[*FILE* | -]
 : Use FILE as the Medley greetfile, unless FILE is "-" in which case
 Medley will start up without using a greetfile. The default Medley greetfile
 is $MEDLEYDIR/greetfiles/MEDLEYDIR-INIT, except when the \-\-apps flag is used
-in which case it is $MEDLEYDIR/greetfiles/APPS-INIT. On Windows (Docker) installations, *FILE* is
+in which case it is $MEDLEYDIR/greetfiles/APPS-INIT. On Windows/Cygwin installations, *FILE* is
 specified in the Medley file system, not the host Windows file system.
 
--x \[*DIR* | -], \-\-logindir \[*DIR* | -]    \*\* **On Linux and WSL installations** \*\*
-: Use DIR as LOGINDIR in Medley, unless DIR is "-", in which case use
-\$MEDLEYDIR/logindir.  DIR (or \$MEDLEYDIR/logindir) must be writeable by the current user.
+-x \[*DIR* | - | --], \-\-logindir \[*DIR* | - | --]
+: Use DIR as LOGINDIR in Medley.  If the value is "--", use
+\$MEDLEYDIR/logindir as LOGINDIR.
+If a value of "-" is given, then reset LOGINDIR to its default value.
+DIR (or \$MEDLEYDIR/logindir) must be writeable by the current user.
 LOGINDIR defaults to \$HOME/il.  LOGINDIR is used by Medley as the working directory on start-up
-and where it loads any "personal" initialization file from.
-
--x \[*DIR* | -], \-\-logindir \[*DIR* | -]    \*\* **On Windows (Docker) installations** \*\*
-: Map DIR in the Windows host file system to /home/medley/il in the Medley
-file system (in the Docker container).  LOGINDIR is always /home/medley/il from Medley's standpoint.  The "-" value is not valid in this case.
+and where it loads any "personal" initialization file from. On Windows/Cygwin installations, *FILE* is
+specified in the Medley file system, not the host Windows file system.
 
--u, \-\-update    \*\* **Windows (Docker) installations only** \*\*
-: Before running Medley, do a pull to retrieve the latest interlisp/medley docker image from Docker Hub.
+-nh *Host:Port:Mac:Debug*, \-\-nethub *Host:Port:Mac:Debug*
+: Set the parameters for using Nethub XNS networking.  *Host* is the full domain name of the nethub host.  *Port* is the port on *Host* that nethub is using.
+*Mac* is the Mac address that this instance of Medley should use when contacting the nethub host.  *Debug* is the level of nethub debug information 
+that should be printed on stdout (value is 0, 1, or 2).  A *Host* value is required and serves to turn nethub functionality on.  *Port*, *Mac* and *Debug*
+parameters are optional and will default if left off.  Finally, if any of the parameters have a value of "-", any previous setting (e.g., in a config file)
+for the parameter will be reset to the default value - which in the case of *Host* is the null string, turning nethub functionality off.
 
--b, \-\-background    \*\* **Windows (Docker) installations only** \*\*
-: Run Medley in background rather than foreground.
+-nf, -NF, --nofork
+: No fork.  Relevant only to the Medley loadup workflow.
 
--p *PORT*, \-\-port *PORT*    \*\* **Windows (Docker) installations only** \*\*
-: Use *PORT* as the port that VNC viewer uses to contact the VNC server within the Docker container.  Default is 5900.
+-prog *EXE*, --maikoprog *EXE*
+: Use *EXE* as the basename of the Maiko executable.  Relevant only to the Medley loadup workflow.
 
--w \[*DISTRO* | -], \-\-wsl \[*DISTRO* | -]    \*\* **Windows (Docker) installations only** \*\*
-: Run Medley in the context of the named WSL *DISTRO* instead of within Docker.  If *DISTRO* is "-", used the default WSL distro.  Equivalent to typing "wsl -d *DISTRO* medley ..." into a Command or Powershell window.
+--maikodir *DIR*
+: Use *DIR* as the directory containing the Maiko emulator.  For testing purposes only.
 
 
 Other Options
@@ -154,11 +192,33 @@ If no such virtual memory file exists, then Medley will start from the standard
 specified in the Medley file system, not the host Windows file system.
 
 *PASS_ON_ARGS*
-: All arguments after the "\-\-" flag, are passed unaltered to lde via run-medley.
+: All arguments after the "\-\-" flag, are passed unaltered to the Maiko emulator.
+
 
+CONFIG FILE
+===========
+A config file can be used to "pre-specify" any of the above command line arguments.
+The config file consists of command line arguments (flags or flag-value pairs), *one per line*.
+These arguments are read from the config file and prepended to the arguments actually given on
+the command line.  Since later arguments override earlier arguments, any argument actually given
+on the command line will override a conflicting argument given in the config file.
+
+Unless specified using the -c (--config) argument, the default config file will be $MEDLEYDIR/.medley_config,
+if it exists, and $HOME/.medley_config, otherwise.
+
+Specifying, "-c -" or "--config -" on the command line will suppress the use of config files for the
+current run of Medley.
 
-FILES
-=====
+*Note:* care must be taken when using -g (--geometry) and/or -s (--screensize) arguments in config files.
+If only one of these is specified, then the other is conputed.  But if both are specified, then the specified
+dimensions are used as given.  Unexpected results can arise if one is specified in the config file 
+but the other is specified on the command line. In this case, the two specified dimensions will be used as given.
+It will not be the case, as might be expected, that the dimension given in the config file will be overridden
+by a dimension computed from the dimension given on the command line.
+
+
+OTHER FILES
+===========
 
 \$HOME/il
 :  Default Medley LOGINDIR
@@ -181,4 +241,4 @@ See GitHub Issues: <https://github.com/Interlisp/medley/issues>
 COPYRIGHT
 =========
 
-Copyright(c) 2023 by Interlisp.org
+Copyright(c) 2023-2024 by Interlisp.org

docs/man-page/medley.1

@@ -18,6 +18,9 @@
 # LDEDESTSYSOUT         name for destination of SaveVM/LOGOUT
 # MEDLEYDIR             used by init file to set other path variables
 
+#for x in "$@"; do echo $x; done
+#exit
+
 inferred_medleydir=false
 
 if [ -z "$MEDLEYDIR" ] ; then

docs/man-page/medley.1.gz

@@ -1,27 +1,43 @@
 #!/bin/sh
+# shellcheck disable=SC2181
 
-if [ ! -x run-medley ] ; then
-    echo run from MEDLEYDIR
+if [ ! -h ./medley ] || [ ! -d ./lispusers ]
+then
+    echo "*** ERROR ***"
+    echo "You must run $(basename "$0") while the cwd is a Medley top-level directory."
+    echo "The cwd ($(pwd)) is not a Medley top-level directory."
+    echo "Exiting."
     exit 1
 fi
 
+# shellcheck source=./loadup-setup.sh
 . scripts/loadup-setup.sh
 
-if [ "$1" = "-apps" ]; then
+# look thru args looking to see if -apps, --apps, or -a was specified in args
+apps=true
+j=1
+jmax=$#
+while [ "$j" -le "$jmax" ]
+do
+  if [ "$(eval "printf %s \${${j}}")" = "-a" ] || \
+     [ "$(eval "printf %s \${${j}}")" = "-apps" ] || \
+     [ "$(eval "printf %s \${${j}}")" = "--apps" ]
+  then
     apps="./scripts/loadup-apps-from-full.sh"
-else
-    apps="true"
-fi
+    break
+  fi
+done
 
+#  Do loadup components
 ./scripts/loadup-init.sh && \
     ./scripts/loadup-mid-from-init.sh && \
     ./scripts/loadup-lisp-from-mid.sh && \
     ./scripts/loadup-full-from-lisp.sh && \
     ${apps} && \
     ./scripts/loadup-aux.sh && \
-    ./scripts/copy-all.sh $1
+    ./scripts/copy-all.sh "$1"
 
-if [ $? -eq 0 ];
+if [ $? -eq 0 ]
 then
   echo "+++++ loadup-all.sh: SUCCESS +++++"
 else

docs/man-page/medley.1.md

@@ -1,29 +1,38 @@
 #!/bin/sh
 
-if [ ! -x run-medley ] ; then
-    echo must run from MEDLEYDIR ;
-    exit 1 ;
+if [ ! -h ./medley ] || [ ! -d ./lispusers ]
+then
+    echo "*** ERROR ***"
+    echo "You must run $(basename "$0") while the cwd is a Medley top-level directory."
+    echo "The cwd ($(pwd)) is not a Medley top-level directory."
+    echo "Exiting."
+    exit 1
 fi
 
+# shellcheck source=./loadup-setup.sh
 . scripts/loadup-setup.sh
 
 loadup_start
 
-export ROOMSDIR=${MEDLEYDIR}/rooms
-export CLOSDIR=${MEDLEYDIR}/clos
+export ROOMSDIR="${MEDLEYDIR}/rooms"
+export CLOSDIR="${MEDLEYDIR}/clos"
 
-export NOTECARDSDIR=${MEDLEYDIR}/notecards
-if [ ! -e ${NOTECARDSDIR} ]; then
-    NOTECARDSDIR=$(cd ${MEDLEYDIR}/../ && pwd)/notecards
-    if [ ! -e ${NOTECARDSDIR} ]; then
-        NOTECARDSDIR=$(cd ${MEDLEYDIR}/../../ && pwd)/notecards
-        if [ ! -e ${NOTECARDSDIR} ]; then
+export NOTECARDSDIR="${MEDLEYDIR}/notecards"
+if [ ! -e "${NOTECARDSDIR}" ]
+then
+    NOTECARDSDIR=$(cd "${MEDLEYDIR}/../" && pwd)/notecards
+    if [ ! -e "${NOTECARDSDIR}" ]
+    then
+        NOTECARDSDIR=$(cd "${MEDLEYDIR}/../../" && pwd)/notecards
+        if [ ! -e "${NOTECARDSDIR}" ]
+        then
             NOTECARDSDIR=""
         fi
     fi
 fi
 
-if [ -z "${NOTECARDSDIR}" ]; then
+if [ -z "${NOTECARDSDIR}" ]
+then
     echo "Error: Cannot find the Notecards directory"
     echo "It should be located at ${MEDLEYDIR}/../notecards or"
     echo "${MEDLEYDIR}/../../notecards.  But its not."
@@ -62,6 +71,6 @@ SHH
 "
 EOF
 
-./run-medley ${scr} -loadup "${cmfile}" "${LOADUP_WORKDIR}/full.sysout"
+run_medley "${LOADUP_WORKDIR}/full.sysout"
 
 loadup_finish "apps.sysout" "apps.*"

run-medley

@@ -1,10 +1,15 @@
 #!/bin/sh
 
-if [ ! -f run-medley ] ; then
-    echo run from MEDLEYDIR
+if [ ! -h ./medley ] || [ ! -d ./lispusers ]
+then
+    echo "*** ERROR ***"
+    echo "You must run $(basename "$0") while the cwd is a Medley top-level directory."
+    echo "The cwd ($(pwd)) is not a Medley top-level directory."
+    echo "Exiting."
     exit 1
 fi
 
+# shellcheck source=./loadup-setup.sh
 . scripts/loadup-setup.sh
 
 loadup_start
@@ -26,6 +31,6 @@ cat >"${cmfile}" <<"EOF"
 "
 EOF
 
-./run-medley ${scr} -loadup "${cmfile}" "${LOADUP_WORKDIR}"/full.sysout
+run_medley "${LOADUP_WORKDIR}/full.sysout"
 
 loadup_finish "whereis.hash" "whereis.hash" "exports.all"

scripts/loadup-all.sh

@@ -1,10 +1,15 @@
 #!/bin/sh
 
-if [ ! -x run-medley ] ; then
-    echo run from MEDLEYDIR
+if [ ! -h ./medley ] || [ ! -d ./lispusers ]
+then
+    echo "*** ERROR ***"
+    echo "You must run $(basename "$0") while the cwd is a Medley top-level directory."
+    echo "The cwd ($(pwd)) is not a Medley top-level directory."
+    echo "Exiting."
     exit 1
 fi
 
+# shellcheck source=./loadup-setup.sh
 . scripts/loadup-setup.sh
 
 loadup_start
@@ -35,7 +40,7 @@ cat >"${cmfile}" <<"EOF"
 "
 EOF
 
-./run-medley ${scr} -loadup "${cmfile}" "${SYSOUT}"
+run_medley "${SYSOUT}"
 
 loadup_finish "fuller.database" "fuller*"