miscellaneous refinements

.github/workflows/build.yaml

@@ -9,7 +9,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest]
-        python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.6", "3.x"]
       max-parallel: 6
 
     steps:

README.rst

@@ -20,8 +20,8 @@ Emborg — Front-End to Borg Backup
     :target: https://pypi.python.org/pypi/emborg/
 
 :Author: Ken Kundert
-:Version: 1.38.1
-:Released: 2023-11-08
+:Version: 1.38.2
+:Released: 2024-01-01
 
 *Emborg* is a simple command line utility to orchestrate backups. It is built as 
 a front-end to Borg, a powerful and fast deduplicating backup program.  With 

doc/accessories.rst

@@ -46,3 +46,25 @@ from each run into a single accumulating log file.  To arrange this you can use
 
 This accumulates the log files as they are created to 
 ~/.local/share/emborg/{config_name}.log.nt.
+
+If your text editor is configured to use fold markers, you can configure *ntlog* 
+to add headers to the composite logfile that contain fold markers.  In doing so 
+you can collapse large log entries into a single line folds until they are 
+needed, at which point you can easily open the fold and examine the contents of 
+the log file.  Here is an example that adds headers with Vim fold markers to the 
+composite log file::
+
+    run_after_borg = [
+        [
+            'ntlog',
+            '--keep-for', '2w',
+            '--day', 'D MMMM YYYY  {{{{{{1',
+            '--entry', 'h:mm A  {{{{{{2',
+            '--fold-marker', '{{{{{{ ❬❬❬',
+            '/home/me/.local/share/emborg/{config_name}.log',
+        ],
+    ]
+
+If you use Vim, you can figure it to fold the composite log file with ``:set 
+foldmethod=marker``.  You can then open a fold using ``zo`` and close it with 
+``zc``.

doc/conf.py

@@ -49,14 +49,14 @@
 
 # General information about the project.
 project = u'emborg'
-copyright = u'2018-2023, Ken Kundert'
+copyright = u'2018-2024, Ken Kundert'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The full version, including alpha/beta/rc tags.
-release = '1.38.1'
+release = '1.38.2'
 # The short X.Y version.
 version = '.'.join(release.split('.')[0:2])
 

doc/configuring.rst

@@ -1091,7 +1091,10 @@ successfully completes.  These commands often recreate useful files that were
 deleted by the :ref:`run_before_backup <run_before_backup>` commands.
 
 May be specified as a list of strings or as a multi-line string with one command 
-per line (lines that begin with ``#`` are ignored).
+per line (lines that begin with ``#`` are ignored).  If given as a string, 
+a shell is used to run the command or commands.  If given as a list of strings, 
+a shell is not used, meaning that shell path and variable expansions, 
+redirections and pipelines are not available.
 
 The commands specified in *run_after_backup* are run each time an archive is 
 created whereas commands specified in *run_after_last_backup* are run only if 
@@ -1129,7 +1132,10 @@ starts the backup. These commands often delete large files that can be easily
 recreated from those files that are backed up.
 
 May be specified as a list of strings or as a multi-line string with one command 
-per line (lines that begin with ``#`` are ignored).
+per line (lines that begin with ``#`` are ignored).  If given as a string, 
+a shell is used to run the command or commands.  If given as a list of strings, 
+a shell is not used, meaning that shell path and variable expansions, 
+redirections and pipelines are not available.
 
 The commands specified in *run_before_backup* are run each time an archive is 
 created whereas commands specified in *run_before_first_backup* are run only if 
@@ -1157,7 +1163,10 @@ after the last one is run.  These can be used, for example, to mount and then
 unmount a remote repository, if such a thing is needed.
 
 May be specified as a list of strings or as a multi-line string with one command 
-per line (lines that begin with ``#`` are ignored).
+per line (lines that begin with ``#`` are ignored).  If given as a string, 
+a shell is used to run the command or commands.  If given as a list of strings, 
+a shell is not used, meaning that shell path and variable expansions, 
+redirections and pipelines are not available.
 
 
 .. _show_progress:

doc/index.rst

@@ -3,8 +3,8 @@
 Emborg — Front-End to Borg Backup
 =================================
 
-| Version: 1.38.1
-| Released: 2023-11-08
+| Version: 1.38.2
+| Released: 2024-01-01
 | Please report all bugs and suggestions on GitHub_.
 
 

doc/releases.rst

@@ -3,8 +3,14 @@ Releases
 
 Latest development release
 --------------------------
-| Version: 1.38.1
-| Released: 2023-11-08
+| Version: 1.38.2
+| Released: 2024-01-01
+
+
+1.39 (2024-??-??)
+-----------------
+- Add date of last check to output of info command.
+- Miscellaneous refinements.
 
 
 1.38 (2023-11-04)

emborg/__init__.py

@@ -1,4 +1,4 @@
-__version__ = "1.38.1"
-__released__ = "2023-11-08"
+__version__ = "1.38.2"
+__released__ = "2024-01-01"
 
 from .emborg import Emborg

emborg/collection.py

@@ -5,7 +5,7 @@
 # than for large collections.
 
 # License {{{1
-# Copyright (C) 2016-2023 Kenneth S. Kundert
+# Copyright (C) 2016-2024 Kenneth S. Kundert
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by

emborg/command.py

@@ -1028,7 +1028,9 @@ class DueCommand(Command):
 
         Examples:
             > emborg due
-            home: 9 hours since last backup.  4.6 days since last squeeze.
+            root backup completed 9 hours ago.
+            root squeeze completed 4.6 days ago.
+            root check completed 12 days ago.
 
             > emborg due -d0.5 -m "It has been {days:.1f} days since the last {action}."
             It has been 0.8 days since the last backup.
@@ -1077,7 +1079,7 @@ def gen_message(cmd):
                         codicil = cmdline["--message"],
                     )
             else:
-                return f"{config} {action} run {elapsed} ago."
+                return f"{config} {action} completed {elapsed} ago."
 
         def email_message(cmd):
             if not cmd:
@@ -2180,7 +2182,7 @@ def show_setting(name, desc):
                 try:
                     if is_str(v) and "{" in v and k not in settings.do_not_expand:
                         output(resolved(
-                            f'{leader}{render(settings.resolve(v), level=6)}'
+                            f'{leader}{render(settings.resolve(k, v), level=6)}'
                         ))
                 except Error:
                     pass

emborg/emborg.py

@@ -1,7 +1,7 @@
 # Emborg Settings
 
 # License {{{1
-# Copyright (C) 2018-2023 Kenneth S. Kundert
+# Copyright (C) 2018-2024 Kenneth S. Kundert
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -32,9 +32,11 @@
     display,
     done,
     errors_accrued,
+    full_stop,
     get_informer,
     indent,
     is_str,
+    is_collection,
     join,
     log,
     narrate,
@@ -310,11 +312,16 @@ def read_config(self, name=None, **kwargs):
             includes = Collection(settings.get(INCLUDE_SETTING))
             includes = [config] + list(includes.values())
 
+        # check file permissions
         if settings.get("passphrase"):
             if getmod(path) & 0o077:
                 warn("file permissions are too loose.", culprit=path)
                 codicil(f"Recommend running: chmod 600 {path!s}")
 
+        # add command name to settings so it can be used in expansions
+        if 'cmd_name' in kwargs:
+            settings['cmd_name'] = kwargs['cmd_name']
+
         self.settings.update(settings)
 
         # read include files, if any are specified
@@ -406,7 +413,7 @@ def value(self, name, default=""):
         value = self.settings.get(name, default)
         if not is_str(value) or name in self.do_not_expand:
             return value
-        return self.resolve(value)
+        return self.resolve(name, value)
 
     # get values {{{2
     def values(self, name, default=()):
@@ -420,23 +427,25 @@ def values(self, name, default=()):
         )
         if name in self.do_not_expand:
             return values
-        return [self.resolve(v) for v in values]
+        return [self.resolve(name, v) for v in values]
 
     # resolve {{{2
-    def resolve(self, value):
+    def resolve(self, name, value):
         """Expand any embedded names in value"""
 
         # escape any double braces
         try:
             value = value.replace("{{", r"\b").replace("}}", r"\e")
         except AttributeError:
+            if is_collection(value):
+                return [self.resolve(name, v) for v in value]
             if isinstance(value, int) and not isinstance(value, bool):
                 return str(value)
             return value
 
         # expand names contained in braces
         try:
-            # build kywars in a way so that user can override values normally
+            # build kwargs in a way so that user can override values normally
             # provided by emborg itself in settings file.
             kwargs = dict(
                 host_name = hostname,
@@ -447,8 +456,10 @@ def resolve(self, value):
             resolved = value.format(**kwargs)
         except KeyError as e:
             raise Error("unknown setting.", culprit=e)
+        except ValueError as e:
+            raise Error(full_stop(e), codicil=name)
         if resolved != value:
-            resolved = self.resolve(resolved)
+            resolved = self.resolve(name, resolved)
 
         # restore escaped double braces with single braces
         return resolved.replace(r"\b", "{").replace(r"\e", "}")
@@ -648,9 +659,11 @@ def run_user_commands(self, setting):
         for i, cmd in enumerate(self.values(setting)):
             narrate(f"staging {setting}[{i}] command.")
             try:
-                Run(cmd, "SoEW")
+                Run(cmd, "SoEW" if is_str(cmd) else 'soEW')
             except Error as e:
-                e.reraise(culprit=(setting, i, cmd.split()[0]))
+                if is_str(cmd):
+                    cmd = cmd.split()
+                e.reraise(culprit=(setting, i, cmd[0]))
 
         # the following two statements are only useful from run_before_borg
         self.settings[setting] = []  # erase the setting so it is not run again
@@ -898,11 +911,11 @@ def __enter__(self):
         if not data_dir.exists():
             # data dir does not exist, create it
             data_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
-        self.date_file = data_dir / self.resolve(DATE_FILE)
+        self.date_file = data_dir / self.resolve('DATE_FILE', DATE_FILE)
         self.data_dir = data_dir
 
         # perform locking
-        lockfile = self.lockfile = data_dir / self.resolve(LOCK_FILE)
+        lockfile = self.lockfile = data_dir / self.resolve('LOCK_FILE', LOCK_FILE)
             # This must be outside if statement because of breaklock command.
             # It want to remove lock file even though it does not require exclusivity.
 
@@ -942,10 +955,10 @@ def __enter__(self):
         # open logfile
         # do this after checking lock so we do not overwrite logfile
         # of emborg process that is currently running
-        self.logfile = data_dir / self.resolve(LOG_FILE)
+        self.logfile = data_dir / self.resolve('LOG_FILE', LOG_FILE)
         log_command = self.log_command and "no-log" not in self.emborg_opts
         if log_command:
-            self.prev_logfile = data_dir / self.resolve(PREV_LOG_FILE)
+            self.prev_logfile = data_dir / self.resolve('PREV_LOG_FILE', PREV_LOG_FILE)
             rm(self.prev_logfile)
             if self.logfile.exists():
                 mv(self.logfile, self.prev_logfile)

emborg/help.py

@@ -2,7 +2,7 @@
 # Output a help topic.
 
 # License {{{1
-# Copyright (C) 2018-2023 Kenneth S. Kundert
+# Copyright (C) 2018-2024 Kenneth S. Kundert
 #
 # This program is free software: you can redistribute it and/or modify it under
 # the terms of the GNU General Public License as published by the Free Software